BUILDING.md

Building the DuckDB documentation

Table of contents

• Building the DuckDB documentation
  • Table of contents
  • Using a local Jekyll installation
    • Prerequisites
    • Serving the site using a local Jekyll installation
  • Using Docker
    • Prerequisites
    • Serving the site from Docker
  • With a Dev Container
  • Generating the search index
  • Updating the release calendar
  • Syntax highlighter
  • Troubleshooting
    • Jekyll doesn't work on Windows
    • Cannot install dependency
    • Jekyll fails
    • Jekyll Content Not Rendering Properly
    • Bundle update fails
    • Bundle install fails
    • ERROR bad URI

The site is built using Jekyll used by GitHub Pages.

Using a local Jekyll installation

Prerequisites

1. The site is built using Jekyll. For instructions on setting up Ruby for Jekyll, please visit the Installation page of Jekyll. To install a specific Ruby version, we recommend using chruby.

2. Install Jekyll and the other required Ruby dependencies using Bundler:

   bundle install

   For more details on setting up Jekyll, consult GitHub's instructions.

3. Create a Python virtual env and install requirements:

   python3 -m venv venv
   source venv/bin/activate
   pip install -r requirements.txt

Serving the site using a local Jekyll installation

To serve this website, run:

scripts/serve-latest.sh

Visit http://localhost:4000/docs/ to browse the website.

Note that to save time on building, the serve-latest.sh script only deploys the current/lts versions and excludes the legacy versions. To serve the full website with old versions included, run:

scripts/serve.sh

Using Docker

Prerequisites

Install Docker.

Serving the site from Docker

For portability, we provide a Docker image.

First, build the image using:

scripts/docker-build.sh

Serve the website (latest only, archives excluded) with:

scripts/docker-serve-latest.sh

To browse the website, visit http://localhost:4000/docs/.

Serve the full website with:

scripts/docker-serve.sh

To stop the container, run:

scripts/docker-stop.sh

With a Dev Container

If you are using a Dev Container, click the green Code button to the top right to open a new codespace with this repository initialized.

Generating the search index

To generate the search index, run:

scripts/install-dependencies.sh
scripts/generate-search-index.sh

Updating the release calendar

The release calendar is updated automatically by CI. To manually update the release calendar, run:

python scripts/get_calendar.py

Syntax highlighter

We use a fork of the Rouge syntax highligher, which is extended with keywords not in standard SQL (e.g., RETURNING, ASOF). This is automatically installed by Bundler.

To text updates to rouge locally, edit the Gemfile:

-gem "rouge", git: "https://github.com/duckdb/rouge.git", branch: "duckdb"
+gem "rouge", git: "/path/to/your/local/rouge/directory"

Then, run:

bundle install

Troubleshooting

Jekyll doesn't work on Windows

If you are on Windows, run these two commands to ensure Jekyll works:

gem uninstall eventmachine --force
gem install eventmachine --platform ruby

Cannot install dependency

The following error occurs:

posix-spawn.c:226:27: error: incompatible function pointer types passing 'int (VALUE, VALUE, posix_spawn_file_actions_t *)' (aka 'int (unsigned long, unsigned long,

The workaround is to run the following bundle command:

bundle config set --global build.posix-spawn "--with-cflags=-Wno-error=incompatible-function-pointer-types"

Jekyll fails

After upgrading Ruby, Jekyll fails with the following error message:

/opt/homebrew/opt/ruby/bin/bundler:25:in `load': cannot load such file -- /opt/homebrew/lib/ruby/gems/3.3.0/gems/bundler-2.5.4/exe/bundler (LoadError)
	from /opt/homebrew/opt/ruby/bin/bundler:25:in `<main>'

The solution is to run the following commands in the repository:

gem install bundler
bundle install

If this workaround is not sufficient, you likely have to upgrade your Bundler version. To do so, run:

rm Gemfile.lock
bundle install

Jekyll Content Not Rendering Properly

Dynamic page regeneration might not apply to includes, railroad diagrams, or other content. To force regeneration of a page or include so that it is rendered in your localhost browser, perform the following steps:

1. Stop the terminal session where you ran the Jekyll build command, i.e., scripts/serve-latest.sh.
2. Run git add myfile.md for your particular page.
3. Stop Jekyll and run rm -rf _site to cleanup the previously generated file.
4. If the site is still not regenerating, run git clean to cleanup the repository but be careful not to loose any unstaged/uncommitted changes.
5. Run the scripts/serve-latest.sh from your terminal again.
6. If the page your editing is open in your browser, it should refresh automatically. If this doesn't happen, refresh manually. The changes from your local branch should be visible now.

Bundle update fails

Bundle update fails with the following error message:

bundle update

Git error: command `git fetch --force --quiet
/opt/homebrew/lib/ruby/gems/3.3.0/cache/bundler/git/minima-4abf4ea566b1c7c640342d1bbff5586f3c10dd05 --depth 1
1d5286cf9a1aae34078420d183d560dd673d98b5` in directory /opt/homebrew/lib/ruby/gems/3.3.0/bundler/gems/minima-1d5286cf9a1a has failed.
fatal: '/opt/homebrew/lib/ruby/gems/3.3.0/cache/bundler/git/minima-4abf4ea566b1c7c640342d1bbff5586f3c10dd05' does not appear to be a git
repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

To resolve this, clean the Jekyll gem cache:

rm -rf /opt/homebrew/lib/ruby/gems/3.3.0/cache/

Bundle install fails

Bundle install fails with the following error message:

bundle install

The running version of Bundler (2.5.11) does not match the version of the specification installed for it (2.5.18). This can be caused by
reinstalling Ruby without removing previous installation, leaving around an upgraded default version of Bundler. Reinstalling Ruby from
scratch should fix the problem.

The solution, according to a Stack Overflow answer, is to run:

gem update --system

ERROR bad URI

The following error occurs when trying to access the locally built website via HTTPS (https://localhost:4000/).

[2024-09-17 12:15:36] ERROR bad URI `����\x12`�\x17\x03�L\x00\x00\x14�'.
[2024-09-17 12:15:36] ERROR bad Request-Line ...

This happens frequently with browsers looking to force an HTTPS connection such as Safari. The solution is to use an HTTP connection (http://localhost:4000) and optionally try another browser (e.g., Chrome or Firefox).