|
| 1 | +# Analytics plan |
| 2 | + |
| 3 | +## Intention |
| 4 | + |
| 5 | +Cmdr is approaching its first public release. We need to understand user behavior so we can make good product decisions, |
| 6 | +and track downloads so we know adoption momentum. We want to do this while keeping our privacy-friendly brand intact. |
| 7 | + |
| 8 | +Currently, the privacy policy mentions PostHog and in-app analytics, but **neither is actually implemented**. Downloads |
| 9 | +go directly to GitHub Releases with no tracking. This plan closes that gap. |
| 10 | + |
| 11 | +## Guiding principles |
| 12 | + |
| 13 | +- **Privacy first**: No PII, no individual tracking, no cookies where avoidable. |
| 14 | +- **Self-hosted where practical**: Umami on our VM, PostHog only where self-hosting is impractical (in-app product |
| 15 | + analytics with funnels/retention). |
| 16 | +- **Minimal footprint**: No heavy SDKs. HTTP POST calls where possible. |
| 17 | +- **Honest privacy policy**: Only claim what we actually do. Update it to match reality. |
| 18 | + |
| 19 | +## Architecture overview |
| 20 | + |
| 21 | +``` |
| 22 | +┌─────────────────┐ ┌──────────────────┐ ┌─────────────────────┐ |
| 23 | +│ getcmdr.com │ │ Cmdr desktop app │ │ Download redirect │ |
| 24 | +│ (Astro) │ │ (Tauri) │ │ (CF Worker) │ |
| 25 | +│ │ │ │ │ │ |
| 26 | +│ Umami script ──┼──► │ PostHog HTTP ────┼──► │ Logs version + │ |
| 27 | +│ (cookieless) │ │ capture API │ │ arch + country │ |
| 28 | +│ │ │ (anonymous) │ │ then 302 → GitHub │ |
| 29 | +└────────┬────────┘ └────────┬──────────┘ └──────────┬──────────┘ |
| 30 | + │ │ │ |
| 31 | + ▼ ▼ ▼ |
| 32 | + Umami server PostHog cloud Cloudflare KV or |
| 33 | + (self-hosted VM) (free tier) Analytics Engine |
| 34 | +``` |
| 35 | + |
| 36 | +## Milestone 1: Website analytics with Umami |
| 37 | + |
| 38 | +### Why Umami instead of PostHog for the website |
| 39 | + |
| 40 | +- Already self-hosted on the VM for other projects — zero additional cost. |
| 41 | +- Cookieless by default — no cookie banner needed, simpler privacy policy. |
| 42 | +- Lightweight script (~2 KB) vs PostHog's SDK — fits the static Astro site better. |
| 43 | +- Good enough for website analytics (page views, referrers, geo, UTM params). |
| 44 | +- Full REST API: websites can be created, events sent, and stats queried programmatically via |
| 45 | + `POST /api/websites`, `POST /api/send`, `GET /api/websites/:id/stats`, etc. No GUI interaction required for setup. |
| 46 | + |
| 47 | +### Steps |
| 48 | + |
| 49 | +1. **Register getcmdr.com in Umami** via `POST /api/websites` (or via the GUI, either works). Note the website ID. |
| 50 | +2. **Add the Umami tracking script** to the website's `<head>` in `apps/website/src/layouts/Layout.astro`: |
| 51 | + ```html |
| 52 | + <script defer src="https://{umami-host}/script.js" data-website-id="{website-id}"></script> |
| 53 | + ``` |
| 54 | + Use an env var for the Umami host and website ID so it's not hardcoded. |
| 55 | +3. **Track download clicks** as custom events on the download button in `apps/website/src/components/Download.astro`. |
| 56 | + Umami supports `data-umami-event` attributes: |
| 57 | + ```html |
| 58 | + <a href="..." data-umami-event="download" data-umami-event-version="0.5.0" data-umami-event-arch="aarch64"> |
| 59 | + ``` |
| 60 | + This gives us download intent by version and architecture, with geo from Umami automatically. |
| 61 | +4. **Disable in dev mode**: Only include the script when `import.meta.env.PROD` is true, or use a feature flag env |
| 62 | + var. Remember: `withGlobalTauri: true` in dev mode is a security risk for external scripts. |
| 63 | +5. **Verify**: Check the Umami dashboard (or query `GET /api/websites/:id/stats`) to confirm data flows. |
| 64 | + |
| 65 | +## Milestone 2: Download tracking with geo via Cloudflare Worker |
| 66 | + |
| 67 | +### Why a separate endpoint |
| 68 | + |
| 69 | +- Umami's download button events tell us **intent** (clicked download). A redirect endpoint tells us **actual |
| 70 | + downloads** (the browser fetched the .dmg). |
| 71 | +- Cloudflare Workers have access to `request.cf.country`, `request.cf.city`, `request.cf.continent` for free — no |
| 72 | + external geo IP service needed. |
| 73 | +- We already run a CF Worker for the license server. This can live alongside it or as a separate Worker. |
| 74 | + |
| 75 | +### Design |
| 76 | + |
| 77 | +`GET https://dl.getcmdr.com/download/:version/:arch` (or a path under the license server domain) |
| 78 | + |
| 79 | +1. Extract `version`, `arch` from the path. |
| 80 | +2. Read `request.cf.country` and `request.cf.continent`. |
| 81 | +3. Log the event to **Cloudflare Analytics Engine** (free, built into Workers) or increment a counter in **KV**. |
| 82 | +4. Respond with `302 → https://github.com/vdavid/cmdr/releases/download/v{version}/Cmdr_{version}_{arch}.dmg`. |
| 83 | + |
| 84 | +#### Decision: Analytics Engine vs KV |
| 85 | + |
| 86 | +- **Analytics Engine** (recommended): purpose-built for this. Write events with dimensions (version, arch, country), |
| 87 | + query with SQL via the dashboard or API. No aggregation logic needed. Free tier: 100K events/day. |
| 88 | +- **KV**: simpler but requires manual aggregation (composite keys like `downloads:0.5.0:aarch64:SE`). Gets messy for |
| 89 | + querying. Better for simple counters only. |
| 90 | + |
| 91 | +Analytics Engine is the better fit unless the free tier becomes a concern (unlikely at launch scale). |
| 92 | + |
| 93 | +### Steps |
| 94 | + |
| 95 | +1. **Decide**: separate Worker (`dl.getcmdr.com`) or new route in the license server. I'd lean toward a new route |
| 96 | + in the license server (`/download/:version/:arch`) to avoid another deployment target. But a separate subdomain |
| 97 | + is cleaner for CDN caching and separation of concerns. |
| 98 | +2. **Implement the redirect endpoint** in Hono: |
| 99 | + ```ts |
| 100 | + app.get('/download/:version/:arch', async (c) => { |
| 101 | + const { version, arch } = c.req.param() |
| 102 | + const country = c.req.raw.cf?.country ?? 'unknown' |
| 103 | + // Log to Analytics Engine or KV |
| 104 | + const dmgName = `Cmdr_${version}_${arch}.dmg` |
| 105 | + return c.redirect(`https://github.com/vdavid/cmdr/releases/download/v${version}/${dmgName}`, 302) |
| 106 | + }) |
| 107 | + ``` |
| 108 | +3. **Enable Analytics Engine** in `wrangler.toml` (if using it): |
| 109 | + ```toml |
| 110 | + [[analytics_engine_datasets]] |
| 111 | + binding = "DOWNLOADS" |
| 112 | + ``` |
| 113 | +4. **Update the website download links** in `Download.astro` to point to the redirect endpoint instead of directly |
| 114 | + to GitHub. |
| 115 | +5. **Update `latest.json`** generation in the release workflow if it contains direct GitHub URLs. |
| 116 | +6. **Test**: download via the new URL, verify the redirect works and the event is logged. |
| 117 | + |
| 118 | +## Milestone 3: In-app product analytics with PostHog |
| 119 | + |
| 120 | +### Why PostHog for in-app (not Umami) |
| 121 | + |
| 122 | +- PostHog has funnels, retention, cohorts, and feature flags — Umami doesn't. |
| 123 | +- PostHog's free tier (1M events/month) is more than enough for a desktop app's launch phase. |
| 124 | +- We only need PostHog's HTTP capture API (`POST https://us.i.posthog.com/capture`), no SDK. |
| 125 | + |
| 126 | +### Design principles for in-app telemetry |
| 127 | + |
| 128 | +- **Anonymous by default**: Use a random `distinct_id` generated on first launch, stored locally. Not tied to email, |
| 129 | + license key, or any PII. Rotate it if the user resets the app. This keeps the "no individual tracking" promise. |
| 130 | +- **Opt-out available**: Add a toggle in Settings. Respect it immediately. |
| 131 | +- **Events, not sessions**: We're a desktop app, not a web app. Track discrete actions, not page views. |
| 132 | +- **No content**: Never include file names, paths, AI prompts, or user content in events. |
| 133 | + |
| 134 | +### What to track (starter set) |
| 135 | + |
| 136 | +| Event | Properties | Why | |
| 137 | +|---|---|---| |
| 138 | +| `app_launched` | `version`, `os_version`, `arch` | Understand active user base | |
| 139 | +| `feature_used` | `feature` (for example, `copy`, `network_browse`, `search`, `command_palette`) | Know which features matter | |
| 140 | +| `settings_changed` | `setting_key` (not the value) | Understand what people customize | |
| 141 | +| `update_installed` | `from_version`, `to_version` | Track update adoption | |
| 142 | +| `license_activated` | `license_type` (`personal`/`commercial`) | Understand conversion | |
| 143 | + |
| 144 | +Keep the event list small. Each event should answer a specific product question. |
| 145 | + |
| 146 | +### Steps |
| 147 | + |
| 148 | +1. **Create a PostHog project** (cloud, free tier). Get the project API key. |
| 149 | +2. **Implement a telemetry module** in the Svelte frontend (`$lib/telemetry/`): |
| 150 | + - `telemetry.svelte.ts`: reactive state for opt-in/opt-out, `distinct_id` generation and persistence. |
| 151 | + - `capture.ts`: thin wrapper around `fetch('https://us.i.posthog.com/capture', ...)`. No SDK dependency. |
| 152 | + - Respect the opt-out setting. If opted out, `capture()` is a no-op. |
| 153 | +3. **Add the opt-out toggle** in Settings, under a "Privacy" or "Telemetry" section. |
| 154 | +4. **Instrument the starter events** listed above. |
| 155 | +5. **Disable in dev mode**: Don't send events when running `pnpm dev`. Use an env var or check `import.meta.env.DEV`. |
| 156 | +6. **Verify**: Check PostHog dashboard for incoming events. |
| 157 | + |
| 158 | +### Open question: Rust-side events? |
| 159 | + |
| 160 | +Some events (like file operation counts) might be easier to capture from Rust. Options: |
| 161 | +- Emit them as Tauri events, let the frontend capture and forward to PostHog. |
| 162 | +- Or call PostHog's HTTP API directly from Rust (adds a dependency: `reqwest` or similar). |
| 163 | + |
| 164 | +Recommendation: start with frontend-only. Add Rust-side capture later if needed. Simpler to maintain one telemetry |
| 165 | +path. |
| 166 | + |
| 167 | +## Milestone 4: Update the privacy policy |
| 168 | + |
| 169 | +The privacy policy currently mentions PostHog for the website and in-app analytics, but neither exists. After |
| 170 | +implementing the above: |
| 171 | + |
| 172 | +### Changes needed |
| 173 | + |
| 174 | +1. **Website analytics section**: Replace PostHog with Umami. Mention it's self-hosted and cookieless. |
| 175 | +2. **Remove or shrink the cookies section**: Umami doesn't set cookies. Only Paddle might (for checkout). |
| 176 | +3. **In-app analytics section**: Keep the current language about aggregate stats. Add that PostHog is the provider. |
| 177 | + Emphasize the anonymous `distinct_id` and opt-out toggle. |
| 178 | +4. **Download tracking**: Add a brief mention that we track download counts by version and country. No PII involved. |
| 179 | +5. **Data processors list**: Remove PostHog from website, add it under in-app. Add a note that website analytics |
| 180 | + are self-hosted (no third-party processor). |
| 181 | +6. **Update `lastUpdated` date**. |
| 182 | + |
| 183 | +### Net effect on privacy posture |
| 184 | + |
| 185 | +- **Better**: Self-hosted, cookieless website analytics. Fewer third-party processors for the website. |
| 186 | +- **Honest**: In-app analytics now actually exist and match what the policy says. |
| 187 | +- **Transparent**: Opt-out toggle gives users control. |
| 188 | + |
| 189 | +## Task list |
| 190 | + |
| 191 | +### Milestone 1: Website analytics (Umami) |
| 192 | +- [x] Register getcmdr.com in Umami (API or GUI) |
| 193 | +- [x] Add Umami script to `Layout.astro` with env var config, disabled in dev |
| 194 | +- [x] Add download event tracking to `Download.astro` |
| 195 | +- [x] Add env vars to `.env.example` and deployment config |
| 196 | +- [x] Set `PUBLIC_UMAMI_HOST` and `PUBLIC_UMAMI_WEBSITE_ID` in production deployment env |
| 197 | +- [ ] Verify data flows in Umami dashboard |
| 198 | + |
| 199 | +### Milestone 2: Download tracking (CF Worker) |
| 200 | +- [ ] Decide: new route in license server vs separate Worker |
| 201 | +- [ ] Enable Analytics Engine in `wrangler.toml` |
| 202 | +- [ ] Implement `/download/:version/:arch` redirect endpoint |
| 203 | +- [ ] Update website download links to use the redirect |
| 204 | +- [ ] Update release workflow if `latest.json` needs changes |
| 205 | +- [ ] Test end-to-end: download via redirect, verify event logged |
| 206 | + |
| 207 | +### Milestone 3: In-app analytics (PostHog) |
| 208 | +- [ ] Create PostHog project, get API key |
| 209 | +- [ ] Implement `$lib/telemetry/` module (capture, opt-out, anonymous ID) |
| 210 | +- [ ] Add opt-out toggle in Settings |
| 211 | +- [ ] Instrument starter events |
| 212 | +- [ ] Disable in dev mode |
| 213 | +- [ ] Verify events in PostHog dashboard |
| 214 | + |
| 215 | +### Milestone 4: Privacy policy update |
| 216 | +- [ ] Rewrite website analytics section (Umami, self-hosted, cookieless) |
| 217 | +- [ ] Rewrite in-app analytics section (PostHog, anonymous, opt-out) |
| 218 | +- [ ] Add download tracking mention |
| 219 | +- [ ] Update data processors list |
| 220 | +- [ ] Shrink cookies section |
| 221 | +- [ ] Update `lastUpdated` date |
| 222 | +- [ ] Run checks: `./scripts/check.sh --check website-prettier --check website-build` |
0 commit comments