url-tracker

Track versioned snapshots of URL content with Babashka.

When fetched content changes, url-tracker writes a new snapshot file, updates a -latest.json symlink, and optionally runs a per-target hook.

Features

• Versioned JSON snapshots with UTC timestamped filenames
• Change detection (no new file if content is unchanged)
• Multiple targets via config file
• Per-target hook command on updates
• Hook environment includes both old and new snapshot paths:
  • OLD (previous snapshot, empty on first version)
  • NEW (newly created snapshot)
• Configurable with shell-smith (CLI, env vars, YAML)

Requirements

• Babashka (bb)
• curl

Installation

Install with bbin:

bbin install io.github.200ok-ch/url-tracker

After that, run it from anywhere:

url-tracker --help

Usage

url-tracker <config-file> [--dry-run] [--verbose]
url-tracker [--url <url> --directory <directory> --prefix <prefix> --hook <command>] [--dry-run] [--verbose]
url-tracker (-h | --help)

Options:

• --url <url>: URL to fetch
• --directory <directory>: where snapshots are stored
• --prefix <prefix>: filename prefix
• --hook <command>: command run when a new version is detected
• --dry-run: print actions without writing files
• -v, --verbose: print detailed progress (URL, latest snapshot, fetch time, hook command, hook env)

You can also pass a config file path positionally:

url-tracker ./configs/url-tracker.yml

In positional config-file mode, targets without directory default to the directory that contains the config file.

Snapshot Naming

New snapshots are written as:

<directory>/<prefix>-<UTC_TIMESTAMP>-v<version>.json

Example:

snapshots/api/api-20260227T092214Z-v2.json

Latest symlink:

<directory>/<prefix>-latest.json

Configuration

Create url-tracker.yml in the working directory to define multiple targets.

targets:
  - url: https://example.com/a.json
    directory: snapshots/a
    prefix: a
    hook: ./notify-a.sh

  - url: https://example.com/b.json
    directory: snapshots/b
    prefix: b
    hook: ./notify-b.sh

You can still run a single target directly from CLI:

./url-tracker \
  --url "https://example.com/data.json" \
  --directory "snapshots/data" \
  --prefix "data" \
  --hook "./on-change.sh"

If your config is not in the current working directory, pass its path directly:

url-tracker /path/to/url-tracker.yml

Run as a Cron Job

For periodic tracking, run url-tracker from cron. This example runs every 15 minutes and writes logs to a file:

*/15 * * * * cd /path/to/project && /usr/local/bin/url-tracker >> /var/log/url-tracker.log 2>&1

Tips:

• Use absolute paths for both url-tracker and log files.
• If you rely on url-tracker.yml in the project directory, keep the cd /path/to/project && ... part.
• Edit crontab with crontab -e and confirm with crontab -l.

Hook Contract

If hook is set for a target and a new version is created, the command is executed with bash -lc.

Environment variables available to the hook:

• OLD_FILE: previous snapshot path (empty on first snapshot)
• NEW_FILE: new snapshot path
• TRACK_URL_URL: target URL
• TRACK_URL_DIRECTORY: target directory
• TRACK_URL_PREFIX: target prefix
• TRACK_URL_NEW_FILE: new snapshot path (same value as NEW_FILE)
• OLD_VERSION: previous numeric version (0 on first snapshot)
• NEW_VERSION: new numeric version

Example hook command:

--hook "sh -c 'diff -u "$OLD_FILE" "$NEW_FILE" || true'"

Notes

• On first run for a target, version starts at v1.
• If fetched content is unchanged from <prefix>-latest.json, no new snapshot is created.
• Use --dry-run to verify config and target processing without writing files.