Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/rich-queens-marry.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"trackio": minor
---

feat:Avoid HF token leaks in static snapshots
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,8 @@ trackio.sync(

This uploads your local project database to a new or existing Space. The Space will display all your logged experiments and metrics, and if a custom frontend is configured or passed explicitly it will be deployed there too.

Static Trackio Spaces (`sdk="static"`) are read-only browser-only snapshots, so their snapshot data must be public. Use the default Gradio Space for private dashboards; `sdk="static"` does not support `private=True`.

**Example workflow:**

```py
Expand Down
7 changes: 5 additions & 2 deletions docs/source/cli_commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ Deploy as a static Space (reads from an HF Bucket, no server needed):
trackio sync --project "my-project" --space-id "username/space_id" --sdk static
```

Static Spaces serve data directly from the browser and therefore require public snapshot data. `--sdk static --private` is not supported; use the default Gradio SDK for private dashboards.

Sync all projects that have unsynced data to their configured Spaces:

```sh
Expand All @@ -51,7 +53,7 @@ trackio sync --all
| `--space-id` | The HF Space ID to sync to (e.g. `username/space_id`). If not provided, uses the previously-configured Space |
| `--all` | Sync all projects with unsynced data |
| `--sdk` | `gradio` (default) for a live server, or `static` for a read-only bucket-backed Space |
| `--private` | Make the Space private if creating a new one |
| `--private` | Make the Space private if creating a new Gradio Space. Not supported with `--sdk static` |
| `--force` | Overwrite the existing database without prompting |

## Freeze Command
Expand All @@ -73,10 +75,11 @@ trackio freeze --space-id "username/my-space" --project "my-project" --new-space
| `--space-id` | The source Gradio Space ID (required) |
| `--project` | The project to freeze (required) |
| `--new-space-id` | The destination static Space ID. Defaults to `{space_id}_static` |
| `--private` | Make the new static Space private |
| `--private` | Not supported for static frozen snapshots |

> **Note:** The source must be a Gradio Space with a bucket mounted at `/data`. If the destination Space already exists and is not a Trackio static Space, `freeze` will refuse to overwrite it.
> The frozen Space is a snapshot. Later metrics synced to the original Gradio Space do not appear in the frozen static Space unless you run `freeze` again.
> Static frozen snapshots require public destination data, so `trackio freeze --private` is not supported.

## List Commands

Expand Down
4 changes: 2 additions & 2 deletions docs/source/deploy_embed.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ trackio.sync(project="my-project", space_id="username/space_id", sdk="static")
trackio sync --project "my-project" --space-id "username/space_id" --sdk static
```

Static Spaces are lightweight and free — they serve a read-only dashboard backed by Parquet files in an HF Bucket.
Static Spaces are lightweight and free — they serve a read-only dashboard backed by Parquet files in an HF Bucket. Because the dashboard runs entirely in the browser, static Trackio Spaces require public snapshot data and do not support `private=True`. Use the default Gradio Space (`sdk="gradio"`) for private dashboards.

## Freezing a Space Snapshot

Expand All @@ -61,6 +61,7 @@ trackio freeze --space-id "username/my-space" --project "my-project"
This creates a new static Space (by default named `{space_id}_static`) containing a snapshot of the project's data from the source Space's bucket. The original Space is not modified.

Note that`freeze()` is a one-time snapshot. If new metrics are later uploaded to the original Gradio Space, the frozen static Space will not update automatically.
The source Gradio Space can use a private bucket, but the frozen static snapshot is public data. `freeze(private=True)` is not supported; use a Gradio Space if the frozen dashboard must stay private.

You can customize the destination:

Expand All @@ -69,7 +70,6 @@ trackio.freeze(
space_id="username/my-space",
project="my-project",
new_space_id="username/my-snapshot",
private=True,
)
```

Expand Down
64 changes: 64 additions & 0 deletions tests/unit/test_deploy.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
import io
import json
from types import SimpleNamespace
from unittest.mock import patch

import pytest

from trackio import deploy
from trackio.bucket_storage import _list_bucket_file_paths
from trackio.frontend_config import ResolvedFrontend
Expand Down Expand Up @@ -133,3 +136,64 @@ def test_deploy_as_static_space_uploads_resolved_frontend(tmp_path, monkeypatch)
assert any(
call["folder_path"] == str(frontend_dir) for call in fake_api.uploaded_folders
)


def _make_static_deploy_env(tmp_path, monkeypatch):
frontend_dir = tmp_path / "static-frontend"
frontend_dir.mkdir()
(frontend_dir / "index.html").write_text("<!doctype html>")

fake_api = _FakeHfApi()
monkeypatch.setattr(deploy.huggingface_hub, "HfApi", lambda: fake_api)
monkeypatch.setattr(deploy.huggingface_hub, "create_repo", lambda *a, **k: None)
monkeypatch.setattr(
deploy,
"resolve_frontend_dir",
lambda frontend_dir=None, announce=False: ResolvedFrontend(
path=frontend_dir.resolve(),
source="argument",
is_custom=True,
),
)
return fake_api, frontend_dir


def _get_uploaded_config(fake_api):
config_upload = next(
item
for item in fake_api.uploaded_files
if item["path_in_repo"] == "config.json"
)
return json.loads(config_upload["payload"])


def test_deploy_as_static_space_config_omits_hf_token(tmp_path, monkeypatch):
fake_api, frontend_dir = _make_static_deploy_env(tmp_path, monkeypatch)

deploy.deploy_as_static_space(
"abidlabs/static-space",
None,
"demo-project",
bucket_id="abidlabs/static-bucket",
hf_token="hf_supersecrettoken",
frontend_dir=frontend_dir,
)

config = _get_uploaded_config(fake_api)
assert "hf_token" not in config
assert "token" not in config
assert "hf_supersecrettoken" not in json.dumps(config)


def test_deploy_as_static_space_rejects_private_true(tmp_path, monkeypatch):
_, frontend_dir = _make_static_deploy_env(tmp_path, monkeypatch)

with pytest.raises(ValueError, match="private=True is not supported"):
deploy.deploy_as_static_space(
"abidlabs/static-space",
None,
"demo-project",
bucket_id="abidlabs/static-bucket",
private=True,
frontend_dir=frontend_dir,
)
55 changes: 38 additions & 17 deletions trackio/deploy.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import tempfile
import threading
import time
import warnings
from collections import Counter
from importlib.resources import files
from pathlib import Path
Expand Down Expand Up @@ -902,12 +903,18 @@ def deploy_as_static_space(
if on_spaces():
return

if private is True:
raise ValueError(
"private=True is not supported for static Trackio Spaces. Static Spaces "
"run entirely in the browser, so their snapshot data must be public. "
"Use sdk='gradio' for a private dashboard."
)
hf_api = huggingface_hub.HfApi()

try:
huggingface_hub.create_repo(
space_id,
private=private,
private=False,
space_sdk="static",
repo_type="space",
exist_ok=True,
Expand All @@ -918,7 +925,7 @@ def deploy_as_static_space(
huggingface_hub.login(add_to_git_credential=False)
huggingface_hub.create_repo(
space_id,
private=private,
private=False,
space_sdk="static",
repo_type="space",
exist_ok=True,
Expand Down Expand Up @@ -960,8 +967,13 @@ def deploy_as_static_space(
config["bucket_id"] = bucket_id
if dataset_id is not None:
config["dataset_id"] = dataset_id
Comment thread
abidlabs marked this conversation as resolved.
if hf_token and private:
config["hf_token"] = hf_token
if hf_token is not None:
warnings.warn(
"`hf_token` is ignored by deploy_as_static_space() for static Space "
"deployment and will be removed in a future release.",
DeprecationWarning,
stacklevel=2,
)

Comment thread
abidlabs marked this conversation as resolved.
_retry_hf_write(
"Static Space config upload",
Expand Down Expand Up @@ -1017,7 +1029,8 @@ def sync(
private (`bool`, *optional*):
Whether to make the Space private. If None (default), the repo will be
public unless the organization's default is private. This value is ignored
if the repo already exists.
if the repo already exists. Not supported with ``sdk="static"`` because
static Trackio dashboards read snapshot data directly from the browser.
force (`bool`, *optional*, defaults to `False`):
If `True`, overwrite the existing database without prompting for confirmation.
If `False`, prompt the user before overwriting an existing database.
Expand All @@ -1038,6 +1051,12 @@ def sync(
"""
if sdk not in ("gradio", "static"):
raise ValueError(f"sdk must be 'gradio' or 'static', got '{sdk}'")
if sdk == "static" and private is True:
raise ValueError(
"private=True is not supported for static Trackio Spaces. Static Spaces "
"run entirely in the browser, so their snapshot data must be public. "
"Use sdk='gradio' for a private dashboard."
)
bucket_id_was_explicit = bucket_id is not None

if space_id is None:
Expand All @@ -1064,18 +1083,16 @@ def _do_sync():

if sdk == "static":
if dataset_id is not None:
upload_dataset_for_static(project, dataset_id, private=private)
hf_token = huggingface_hub.utils.get_token() if private else None
upload_dataset_for_static(project, dataset_id, private=False)
deploy_as_static_space(
space_id,
dataset_id,
project,
private=private,
hf_token=hf_token,
private=False,
frontend_dir=frontend_dir,
)
elif bucket_id is not None:
create_bucket_if_not_exists(bucket_id, private=private)
create_bucket_if_not_exists(bucket_id, private=False)
upload_project_to_bucket_for_static(project, bucket_id)
print(
f"* Project data uploaded to bucket: https://huggingface.co/buckets/{bucket_id}"
Expand All @@ -1085,8 +1102,7 @@ def _do_sync():
None,
project,
bucket_id=bucket_id,
private=private,
hf_token=huggingface_hub.utils.get_token() if private else None,
private=False,
frontend_dir=frontend_dir,
)
else:
Expand Down Expand Up @@ -1165,15 +1181,21 @@ def freeze(
The ID for the new static Space. If not provided, defaults to
`"{space_id}_static"`.
private (`bool`, *optional*):
Whether to make the new Space private. If None (default), the repo
will be public unless the organization's default is private.
Not supported. Frozen static dashboards read snapshot data directly
from the browser, so the destination snapshot must be public.
bucket_id (`str`, *optional*):
The ID of the HF Bucket for the new static Space's data storage.
If not provided, one is auto-generated from the new Space ID.

Returns:
`str`: The Space ID of the newly created static Space.
"""
if private is True:
raise ValueError(
"private=True is not supported for frozen static Trackio Spaces. Static "
"Spaces run entirely in the browser, so their snapshot data must be "
"public. Use a Gradio Space if the frozen dashboard must stay private."
)
space_id, _, _ = preprocess_space_and_dataset_ids(space_id, None, None)

try:
Expand Down Expand Up @@ -1214,7 +1236,7 @@ def freeze(
except RepositoryNotFoundError:
pass

create_bucket_if_not_exists(bucket_id, private=private)
create_bucket_if_not_exists(bucket_id, private=False)
export_from_bucket_for_static(source_bucket_id, bucket_id, project)
print(
f"* Project data uploaded to bucket: https://huggingface.co/buckets/{bucket_id}"
Expand All @@ -1224,8 +1246,7 @@ def freeze(
None,
project,
bucket_id=bucket_id,
private=private,
hf_token=huggingface_hub.utils.get_token() if private else None,
private=False,
frontend_dir=frontend_dir,
)
return new_space_id
24 changes: 6 additions & 18 deletions trackio/frontend/src/lib/staticApi.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,6 @@ function resolveUrl(filename) {
return `https://huggingface.co/datasets/${config.dataset_id}/resolve/main/${filename}`;
}

function authHeaders() {
if (config.hf_token) {
return { Authorization: `Bearer ${config.hf_token}` };
}
return {};
}

export async function initialize(cfg) {
config = cfg;
}
Expand All @@ -41,28 +34,25 @@ export function getReadOnlySource() {

async function getMetricsData() {
if (metricsData) return metricsData;
metricsData = await readParquet(resolveUrl("metrics.parquet"), authHeaders());
metricsData = await readParquet(resolveUrl("metrics.parquet"));
return metricsData;
}

async function getSystemData() {
if (systemData) return systemData;
systemData = await readParquet(
resolveUrl("aux/system_metrics.parquet"),
authHeaders(),
);
systemData = await readParquet(resolveUrl("aux/system_metrics.parquet"));
return systemData;
}

async function getConfigsData() {
if (configsData) return configsData;
configsData = await readParquet(resolveUrl("aux/configs.parquet"), authHeaders());
configsData = await readParquet(resolveUrl("aux/configs.parquet"));
return configsData;
}

async function getRunsJson() {
if (runsData) return runsData;
const resp = await fetch(resolveUrl("runs.json"), { headers: authHeaders() });
const resp = await fetch(resolveUrl("runs.json"));
if (!resp.ok) {
runsData = [];
return runsData;
Expand All @@ -73,7 +63,7 @@ async function getRunsJson() {

async function getSettingsJson() {
if (settingsData) return settingsData;
const resp = await fetch(resolveUrl("settings.json"), { headers: authHeaders() });
const resp = await fetch(resolveUrl("settings.json"));
if (!resp.ok) {
settingsData = {};
return settingsData;
Expand Down Expand Up @@ -435,7 +425,6 @@ export async function getProjectFiles() {
if (config.bucket_id) {
const resp = await fetch(
`https://huggingface.co/api/buckets/${config.bucket_id}/tree?prefix=media/files/&recursive=true`,
{ headers: authHeaders() },
);
if (!resp.ok) {
fileListData = [];
Expand All @@ -453,7 +442,6 @@ export async function getProjectFiles() {

const resp = await fetch(
`https://huggingface.co/api/datasets/${config.dataset_id}`,
{ headers: authHeaders() },
);
if (!resp.ok) {
fileListData = [];
Expand Down Expand Up @@ -497,7 +485,7 @@ export async function fetchMediaBlob(path) {
const url = resolveUrl(`media/${relative}`);
if (blobCache.has(url)) return blobCache.get(url);

const resp = await fetch(url, { headers: authHeaders() });
const resp = await fetch(url);
if (!resp.ok) return url;
const blob = await resp.blob();
const blobUrl = URL.createObjectURL(blob);
Expand Down
Loading