Skip to content

Commit dc593a9

Browse files
dpark01claude
andcommitted
Update docs with Phase 0-1 completion status and learnings
AGENT_CONTEXT.md: - Add current status table showing Phase 0-1 complete - Note viral-baseimage is not local (use GitHub API) - Document micromamba learnings (symlink location, PATH for non-interactive) - Add decision log entries for license, base image, feature branch cleanup - Document what was done in Phase 0 and Phase 1 - Add reference repositories section MONOREPO_IMPLEMENTATION_PLAN.md: - Mark Phase 0 as complete with checkmarks - Mark Phase 1 as complete with completed items list Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 09295a9 commit dc593a9

2 files changed

Lines changed: 76 additions & 17 deletions

File tree

AGENT_CONTEXT.md

Lines changed: 57 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,22 @@
22

33
> This document provides background context for the Claude Code agent that will implement the monorepo migration. Read this alongside `MONOREPO_IMPLEMENTATION_PLAN.md`.
44
5+
---
6+
7+
## CURRENT STATUS
8+
9+
| Phase | Status | Notes |
10+
|-------|--------|-------|
11+
| Phase 0: Prepare repo | ✅ COMPLETE | Legacy archived, branches cleaned, secrets configured |
12+
| Phase 1: Foundation | ✅ COMPLETE | pyproject.toml, baseimage, CI workflow created |
13+
| Phase 2: Migrate viral-core | 🔲 NOT STARTED | Next step |
14+
| Phase 3: Migrate derivatives | 🔲 NOT STARTED | assemble, phylo, classify |
15+
| Phase 4: Finalize | 🔲 NOT STARTED | mega image, docs, badges |
16+
17+
**Next action:** Push to origin, then start Phase 2 (migrate viral-core with git history preservation).
18+
19+
---
20+
521
## Overview
622

723
You are helping migrate 5 separate git repositories into a single monorepo at `github.com/broadinstitute/viral-ngs`. This is a substantial refactoring project that will modernize the codebase's CI/CD, Docker builds, and Python packaging.
@@ -12,13 +28,13 @@ All repositories are located at `/Users/dpark/dev/`:
1228

1329
| Repository | Path | Description |
1430
|------------|------|-------------|
15-
| viral-baseimage | `/Users/dpark/dev/viral-baseimage` | Base Docker image with conda/python |
31+
| viral-baseimage | NOT LOCAL (use GitHub API) | Base Docker image with conda/python |
1632
| viral-core | `/Users/dpark/dev/viral-core` | Core utilities, illumina, read_utils, tools/ |
1733
| viral-classify | `/Users/dpark/dev/viral-classify` | Metagenomics, taxonomy filtering |
1834
| viral-assemble | `/Users/dpark/dev/viral-assemble` | Genome assembly tools |
1935
| viral-phylo | `/Users/dpark/dev/viral-phylo` | Phylogenetic analysis |
2036

21-
**Target repository:** `/Users/dpark/dev/viral-ngs` (this directory - currently dormant)
37+
**Target repository:** `/Users/dpark/dev/viral-ngs` (this directory - monorepo in progress)
2238

2339
## Codebase Architecture
2440

@@ -199,12 +215,15 @@ cache-to: type=registry,ref=quay.io/broadinstitute/viral-ngs:cache-core-amd64,mo
199215
200216
## Micromamba Migration Notes
201217
202-
From viral-baseimage PR #29, key lessons:
218+
From viral-baseimage PR #29 and read-qc-tools repo, key lessons:
203219
204220
1. **API differences**: `conda config --add X` → `micromamba config append X`
205221
2. **No default Python**: Must explicitly install Python in base image
206-
3. **Symlink trick**: Create `conda` and `mamba` symlinks to `micromamba`
207-
4. **Activation**: Set `MAMBA_DOCKERFILE_ACTIVATE=1`
222+
3. **Symlink trick**: Create `conda` and `mamba` symlinks pointing to `/usr/bin/micromamba`
223+
4. **Activation**: Set `ARG MAMBA_DOCKERFILE_ACTIVATE=1` for RUN commands
224+
5. **PATH for non-interactive**: Must explicitly add `/opt/conda/bin` to PATH with `ENV` statement - the mambaorg entrypoint only sets PATH for interactive shells
225+
6. **Base image**: Use `mambaorg/micromamba:2.4.0-ubuntu24.04` as base (simpler than building from scratch)
226+
7. **Symlink location**: In mambaorg image, micromamba is at `/usr/bin/micromamba`, NOT `/usr/local/bin/`
208227

209228
## Test Strategy
210229

@@ -261,13 +280,16 @@ These decisions have been made and should not be revisited:
261280
|----------|--------|-----------|
262281
| Python packaging | `viral-ngs` pip package | Modern, cleaner imports |
263282
| Import style | `from viral_ngs.util import file` | Standard package pattern |
264-
| Docker registry | Single repo with flavor tags | Modern pattern like python:slim |
265-
| Multi-arch | Native parallel builds | 10x faster than QEMU |
283+
| Docker registry | Quay.io + ghcr.io | Redundancy, both registries |
284+
| Multi-arch | QEMU via docker/setup-qemu-action | Simpler than native runners for now |
266285
| Cache backend | Registry on Quay.io | GHA 10GB limit too small |
267286
| Default branch | `main` | Modern convention |
268287
| Code coverage | CodeCov | Replacing Coveralls |
269288
| Retire tools | kaiju, diamond | Reduce complexity |
270289
| Consolidate | kraken2, krona to main env | Try, keep multi-env if fails |
290+
| Feature branch cleanup | `quay.expires-after=10w` label | No API token needed |
291+
| License | MIT | Replacing BSD-3-Clause |
292+
| Base image | `mambaorg/micromamba:2.4.0-ubuntu24.04` | Simpler than custom build |
271293

272294
## Common Pitfalls to Avoid
273295

@@ -290,6 +312,33 @@ When working on this migration:
290312
## Next Steps
291313

292314
1. Read `MONOREPO_IMPLEMENTATION_PLAN.md` for the detailed task list
293-
2. Start with Phase 0 (prepare viral-ngs repo)
315+
2. **Phase 2**: Migrate viral-core with git history preservation using `git filter-repo`
294316
3. Work through phases sequentially
295317
4. Verify each phase before moving to the next
318+
319+
## What Was Done in Phase 0 & 1
320+
321+
### Phase 0 (Complete)
322+
- Created `archive/legacy-monolith` tag and `archive/legacy` branch
323+
- Deleted 148 stale feature branches (ct-*, is-*, sy-*, etc.)
324+
- Deleted all legacy files (783 files, 83K+ lines)
325+
- Replaced BSD-3-Clause license with MIT
326+
- Renamed `master` to `main`, set as default branch
327+
- Closed all 53 legacy issues with migration notice
328+
- Configured secrets: `QUAY_USERNAME`, `QUAY_TOKEN`, `CODECOV_TOKEN`
329+
- Updated repo description and topics
330+
331+
### Phase 1 (Complete)
332+
- Created `pyproject.toml` with setuptools-scm
333+
- Created `src/viral_ngs/__init__.py` and `py.typed`
334+
- Created `docker/Dockerfile.baseimage` using mambaorg/micromamba
335+
- Created `docker/install-conda-deps.sh` for unified dependency installation
336+
- Created `.github/workflows/docker.yml` for multi-arch builds
337+
- Verified baseimage builds locally with Python 3.12 and conda/mamba symlinks
338+
339+
## Reference Repositories
340+
341+
When implementing Docker patterns, refer to:
342+
- `broadinstitute/read-qc-tools` - Simple micromamba-based Dockerfile pattern
343+
- `broadinstitute/viral-baseimage` branch `ct-reduce-google-sdk-install-size` - PR #29 for micromamba migration ideas
344+
- `broadinstitute/viral-core` - Current Dockerfile and install scripts for reference

MONOREPO_IMPLEMENTATION_PLAN.md

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -18,39 +18,49 @@
1818

1919
---
2020

21-
## Phase 0: Prepare Existing viral-ngs Repository
21+
## Phase 0: Prepare Existing viral-ngs Repository ✅ COMPLETE
2222

2323
**Goal:** Clean up the dormant repo for reuse.
2424

2525
### Tasks
2626

27-
1. **Audit current state**
28-
- [ ] Review any open issues/PRs (close or migrate if relevant)
29-
- [ ] Check branch structure (identify branches to archive/delete)
30-
- [ ] Review any existing GitHub Actions workflows
31-
- [ ] Check for secrets/settings that need updating
27+
1. **Audit current state**
28+
- [x] Review any open issues/PRs (closed 53 issues with migration notice)
29+
- [x] Check branch structure (deleted 148 stale feature branches)
30+
- [x] Review any existing GitHub Actions workflows (none existed, Travis CI removed)
31+
- [x] Check for secrets/settings that need updating
3232

33-
2. **Archive old content**
33+
2. **Archive old content**
3434
```bash
3535
git tag archive/legacy-monolith
3636
git branch archive/legacy
3737
git push origin archive/legacy-monolith archive/legacy
3838
```
3939

40-
3. **Update repository settings**
40+
3. **Update repository settings**
4141
- [x] Set default branch to `main`
4242
- [x] Add secrets: `QUAY_USERNAME`, `QUAY_TOKEN`, `CODECOV_TOKEN`
4343
- [x] Update repository description/topics
4444
- [x] Close all legacy issues with migration notice
4545
- [ ] Branch protection rules (deferred to Phase 4)
4646
- [x] GitHub Packages/ghcr.io (auto-enables on first push, uses GITHUB_TOKEN)
47+
- [x] Replaced BSD-3-Clause license with MIT
4748

4849
---
4950

50-
## Phase 1: Set Up Monorepo Foundation
51+
## Phase 1: Set Up Monorepo Foundation ✅ COMPLETE
5152

5253
**Goal:** Establish fresh structure with modern tooling.
5354

55+
### Completed Items
56+
- [x] Created `pyproject.toml` with setuptools-scm versioning
57+
- [x] Created `src/viral_ngs/__init__.py` with version detection
58+
- [x] Created `src/viral_ngs/py.typed` marker for type hints
59+
- [x] Created `docker/Dockerfile.baseimage` using `mambaorg/micromamba:2.4.0-ubuntu24.04`
60+
- [x] Created `docker/install-conda-deps.sh` for unified dependency installation
61+
- [x] Created `.github/workflows/docker.yml` for multi-arch builds to Quay.io + ghcr.io
62+
- [x] Verified baseimage builds locally (Python 3.12, conda/mamba symlinks work)
63+
5464
### Directory Structure
5565

5666
```

0 commit comments

Comments
 (0)