Skip to content

Commit 85ad986

Browse files
authored
docs: create guides for version control and continuous integration (#356)
Creates guides for using dbc in version control and with CI. Incidentally syncs up some of the index.md files and mkdocs.yml since they were slightly out of sync. Closes #164 Closes #342
1 parent 4b3d2bf commit 85ad986

8 files changed

Lines changed: 175 additions & 1 deletion

File tree

docs/concepts/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,4 @@ Check out the following pages for conceptual information on ADBC fundamentals:
2525
There's also a concept page for dbc's driver list functionality:
2626

2727
- [Driver List](./driver_list.md)
28+
- [Driver Registry](./driver_registry.md)
Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
<!--
2+
Copyright 2026 Columnar Technologies Inc.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
-->
16+
17+
# Continuous Integration
18+
19+
dbc works well in non-interactive environments such as on continuous integration (CI) platforms. You may also want to read through our [Version Control](./version_control.md) guide as these two concepts are related.
20+
21+
## GitHub Actions
22+
23+
We recommend using the [columnar-tech/setup-dbc](https://github.com/columnar-tech/setup-dbc) action if you're using [GitHub Actions](https://docs.github.com/en/actions) for CI.
24+
25+
As an example, here's a workflow that automatically installs all drivers listed in your [driver list](../concepts/driver_list.md) before running your tests:
26+
27+
```yaml
28+
name: Test
29+
on: [push]
30+
31+
jobs:
32+
test:
33+
runs-on: ubuntu-latest
34+
steps:
35+
- uses: actions/checkout@v6
36+
37+
# Note: Automatically installs drivers specified in dbc.toml
38+
- uses: columnar-tech/setup-dbc@v1
39+
40+
- name: Run tests
41+
run: pytest ...
42+
```
43+
44+
See the [columnar-tech/setup-dbc README](https://github.com/columnar-tech/setup-dbc) for usage information and more examples.
45+
46+
## Other CI Systems
47+
48+
To use dbc with other CI systems, we recommend using our command line installers because they will always install the latest version of dbc for whatever platform you run them on.
49+
50+
As an example for you to adapt to your system, here's a GitHub Actions workflow that installs and makes dbc available without using [columnar-tech/setup-dbc](https://github.com/columnar-tech/setup-dbc):
51+
52+
{% raw %}
53+
```yaml
54+
name: Test
55+
on: [push]
56+
57+
jobs:
58+
test:
59+
runs-on: ${{ matrix.os }}
60+
61+
strategy:
62+
matrix:
63+
os: [ubuntu-latest, windows-latest, macos-latest]
64+
65+
steps:
66+
- uses: actions/checkout@v6
67+
68+
- name: Install dbc (Linux, macOS)
69+
if: runner.os != 'Windows'
70+
run: |
71+
curl -LsSf https://dbc.columnar.tech/install.sh | sh
72+
73+
- name: Install dbc (Windows)
74+
if: runner.os == 'Windows'
75+
run: |
76+
powershell -ExecutionPolicy ByPass -c "irm https://dbc.columnar.tech/install.ps1 | iex"
77+
78+
- name: Add dbc to PATH (Linux, macOS)
79+
if: runner.os != 'Windows'
80+
shell: bash
81+
run: echo "$HOME/.local/bin" >> $GITHUB_PATH
82+
83+
- name: Add dbc to PATH (Windows)
84+
if: runner.os == 'Windows'
85+
shell: pwsh
86+
run: |
87+
Join-Path $env:USERPROFILE ".local\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
88+
89+
- name: Run tests
90+
run: pytest ...
91+
```
92+
{% endraw %}

docs/guides/index.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,3 +23,6 @@ For detailed guides on using dbc, see the following pages:
2323
- [Using a Driver List](./driver_list.md)
2424
- [Installing a Driver Manager](./driver_manager.md)
2525
- [Python Notebooks](./python_notebooks.md)
26+
- [Private Drivers](./private_drivers.md)
27+
- [Continuous Integration](./continuous_integration.md)
28+
- [Version Control](./version_control.md)

docs/guides/installing.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -270,6 +270,10 @@ Installed some_driver 1.0.0 to /Users/user/Library/Application Support/ADBC/Driv
270270

271271
Make note of the name "some_driver" printed above as this will be the name to use when loading the driver with a [Driver Manager](../concepts/driver_manager.md). i.e., `dbapi.connect(driver="some_driver")`.
272272

273+
## GitHub Actions
274+
275+
To use dbc with [GitHub Actions](https://docs.github.com/en/actions), we recommend the official [`columnar-tech/setup-dbc`](https://github.com/columnar-tech/setup-dbc) action. See the [Continuous Integration](./continuous_integration.md) guide for more detail and examples.
276+
273277
## Uninstalling Drivers
274278

275279
You can uninstall a driver with the `dbc uninstall` subcommand.

docs/guides/version_control.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
<!--
2+
Copyright 2026 Columnar Technologies Inc.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
-->
16+
17+
# Version Control
18+
19+
When using dbc in projects where version control software such as [git](https://git-scm.com) is being used, we recommend the following:
20+
21+
- Use a [driver list](../concepts/driver_list.md) to record drivers and their version constraints instead of installing drivers manually with [`dbc install`](./installing.md)
22+
- Track `dbc.toml` with version control and always use `dbc sync` to install drivers after checkout
23+
- To maximize reproducibility, also track [`dbc.lock`](./driver_list.md#lockfile)
24+
- Don't track installed driver directories with version control, use `dbc.toml` instead
25+
26+
## Example Workflow
27+
28+
To help illustrate how this works in practice, see the example below for how to use dbc when collaborating with git. This assumes both developers have already [installed](../getting_started/installation.md) dbc.
29+
30+
Developer 1 sets up dbc with the drivers their project needs:
31+
32+
```console
33+
# Create a driver list file
34+
$ dbc init
35+
36+
# Add the mysql and sqlite drivers to it. Constrain sqlite's version.
37+
$ dbc add mysql "sqlite<2"
38+
added mysql to driver list
39+
added sqlite to driver list with constraint <2
40+
use `dbc sync` to install the drivers in the list
41+
42+
# Install the drivers from dbc.toml
43+
$ dbc sync
44+
✓ mysql-0.1.0
45+
✓ sqlite-1.11.0
46+
Done!
47+
48+
# Start tracking dbc.toml with git
49+
$ git add dbc.toml
50+
51+
# Commit and push
52+
$ git commit -m "Create dbc.toml"
53+
$ git push
54+
```
55+
56+
Developer 2 then clones the repository and uses `dbc sync`:
57+
58+
```console
59+
$ git clone example/repo
60+
61+
# Install the drivers from dbc.toml
62+
$ dbc sync
63+
✓ mysql-0.1.0
64+
✓ sqlite-1.11.0
65+
Done!
66+
```
67+
68+
Now, at this point, both Developer 1 and Developer 2 have the same set of drivers available on their systems.

docs/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -923,7 +923,7 @@ dbc is the command-line tool for installing and managing [ADBC](https://arrow.ap
923923
- Create reproducible environments with [driver list](concepts/driver_list.md) files
924924
- Cross-platform: Runs on macOS, Linux, and Windows
925925
- Installable with pip, Docker, and more (See [Installation](./getting_started/installation.md))
926-
- Works great in CI/CD environments
926+
- Works great in CI/CD environments (See [Continuous Integration](./guides/continuous_integration.md))
927927

928928
## Help
929929

docs/reference/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,3 +20,4 @@ limitations under the License.
2020
- [Config level](./config_level.md)
2121
- [Driver List](./driver_list.md)
2222
- [Supported Platforms](./supported_platforms.md)
23+
- [Analytics](./analytics.md)

mkdocs.yml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,8 @@ nav:
4747
- Driver Managers: guides/driver_manager.md
4848
- Python Notebooks: guides/python_notebooks.md
4949
- Private Drivers: guides/private_drivers.md
50+
- Continuous Integration: guides/continuous_integration.md
51+
- Version Control: guides/version_control.md
5052
- Concepts:
5153
- concepts/index.md
5254
- Driver: concepts/driver.md
@@ -147,6 +149,9 @@ plugins:
147149
- guides/driver_list.md: Using driver lists for reproducible setups
148150
- guides/driver_manager.md: Working with driver managers
149151
- guides/python_notebooks.md: Using dbc in Python notebooks
152+
- guides/private_drivers.md: Using private drivers
153+
- guides/continuous_integration.md: Using dbc in CI/CD pipelines
154+
- guides/version_control.md: Using dbc with version control
150155
Concepts:
151156
- concepts/driver.md: What is an ADBC driver
152157
- concepts/driver_manager.md: Understanding driver managers

0 commit comments

Comments
 (0)