Skip to content

Commit ad1a83f

Browse files
jeremymanningjeremymanning
authored
Merge pull request #46 from ContextLab/copilot/add-solutions-for-all-problems
Add bulk solving capability to generate AI solutions for all missing problems
2 parents 19c8126 + 73f33aa commit ad1a83f

File tree

6 files changed

+857
-3
lines changed

6 files changed

+857
-3
lines changed

‎.github/workflows/bulk_solver.yml‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
name: Bulk Solve Missing LeetCode Problems
2+
3+
on:
4+
workflow_dispatch: # Manual trigger only
5+
inputs:
6+
batch_size:
7+
description: 'Number of problems to solve per run (default: 50)'
8+
required: false
9+
default: '50'
10+
start_index:
11+
description: 'Start from problem at this index in the missing list (0-based, default: 0)'
12+
required: false
13+
default: '0'
14+
15+
jobs:
16+
bulk-solve:
17+
runs-on: ubuntu-latest
18+
if: github.repository_owner == 'ContextLab' # only run if on the ContextLab (source) repository!
19+
20+
permissions:
21+
contents: write # Required to push commits back to the repository
22+
23+
steps:
24+
- name: Checkout repository
25+
uses: actions/checkout@v3
26+
with:
27+
fetch-depth: 0 # Fetch all history for proper git operations
28+
29+
- name: Set up Python
30+
uses: actions/setup-python@v4
31+
with:
32+
python-version: '3.11'
33+
34+
- name: Install dependencies
35+
run: |
36+
python -m pip install --upgrade pip
37+
pip install requests openai httpx
38+
39+
- name: Identify missing problems
40+
id: find_missing
41+
run: |
42+
# Extract problem numbers from README
43+
grep -oP '\| \[(\d+)\]' README.md | grep -oP '\d+' | sort -n | uniq > /tmp/readme_problems.txt
44+
45+
# Find problems missing gpt5-mini.md
46+
python3 << 'EOF'
47+
from pathlib import Path
48+
49+
with open('/tmp/readme_problems.txt', 'r') as f:
50+
readme_problems = set(int(line.strip()) for line in f if line.strip())
51+
52+
# Check which problems have gpt5-mini.md
53+
problems_dir = Path("problems")
54+
with_ai = set()
55+
56+
for item in problems_dir.iterdir():
57+
if item.is_dir() and item.name.isdigit():
58+
gpt5_file = item / "gpt5-mini.md"
59+
if gpt5_file.exists():
60+
with_ai.add(int(item.name))
61+
62+
# Find problems missing AI solutions
63+
missing_ai = sorted(readme_problems - with_ai)
64+
65+
# Save missing problems
66+
with open('/tmp/missing_ai_solutions.txt', 'w') as f:
67+
for prob in missing_ai:
68+
f.write(f"{prob}\n")
69+
70+
print(f"Total problems in README: {len(readme_problems)}")
71+
print(f"Problems with AI solutions: {len(with_ai)}")
72+
print(f"Problems missing AI solutions: {len(missing_ai)}")
73+
EOF
74+
75+
# Set output for GitHub Actions
76+
MISSING_COUNT=$(wc -l < /tmp/missing_ai_solutions.txt)
77+
echo "total_missing=$MISSING_COUNT" >> $GITHUB_OUTPUT
78+
79+
- name: Create batch file
80+
run: |
81+
START_INDEX=${{ github.event.inputs.start_index }}
82+
BATCH_SIZE=${{ github.event.inputs.batch_size }}
83+
84+
# Extract the batch of problems to solve
85+
tail -n +$((START_INDEX + 1)) /tmp/missing_ai_solutions.txt | head -n $BATCH_SIZE > /tmp/batch_problems.txt
86+
87+
BATCH_COUNT=$(wc -l < /tmp/batch_problems.txt)
88+
echo "Processing batch: $BATCH_COUNT problems starting from index $START_INDEX"
89+
echo "Problems in this batch:"
90+
cat /tmp/batch_problems.txt
91+
92+
- name: Run bulk solver
93+
env:
94+
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
95+
run: |
96+
python3 auto_solver.py --bulk /tmp/batch_problems.txt
97+
98+
- name: Commit and push changes
99+
uses: stefanzweifel/git-auto-commit-action@v4
100+
with:
101+
commit_message: "Bulk solve LeetCode problems (batch starting at index ${{ github.event.inputs.start_index }})"
102+
branch: main
103+
file_pattern: problems/*/gpt5-mini.md

‎BULK_SOLVER_README.md‎

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
# Bulk Solution Generation for Missing LeetCode Problems
2+
3+
This document describes the tools and processes for systematically adding AI-generated solutions to all LeetCode problems in the repository.
4+
5+
## Problem Statement
6+
7+
The repository tracks 548 LeetCode problems (as listed in README.md), but many lack AI-generated solutions:
8+
- **76 problems** have `gpt5-mini.md` files (AI-generated solutions)
9+
- **472 problems** are missing AI-generated solutions
10+
11+
Some problems have only human-generated solutions (e.g., `jeremymanning.md`), and we want to add AI solutions for all of them.
12+
13+
## Tools Provided
14+
15+
### 1. `identify_missing.py`
16+
17+
Analyzes the repository to identify which problems are missing AI-generated solutions.
18+
19+
**Usage:**
20+
```bash
21+
python3 identify_missing.py
22+
```
23+
24+
**What it does:**
25+
- Scans README.md to find all problem numbers
26+
- Checks for existence of `problems/*/gpt5-mini.md` files
27+
- Reports which problems are missing AI solutions
28+
- Creates batch files for processing (50 problems per batch)
29+
- Saves results to temp directory and batch files
30+
31+
### 2. `auto_solver.py` (Enhanced)
32+
33+
The existing auto_solver script now supports three modes:
34+
35+
**Mode 1: Daily problem (default)**
36+
```bash
37+
python3 auto_solver.py
38+
```
39+
40+
**Mode 2: Specific problem by ID**
41+
```bash
42+
python3 auto_solver.py 123
43+
```
44+
45+
**Mode 3: Bulk solve from file**
46+
```bash
47+
# Set your OpenAI API key
48+
export OPENAI_API_KEY="your-api-key-here"
49+
50+
# Process a batch of problems
51+
python3 auto_solver.py --bulk /tmp/batch_001.txt
52+
```
53+
54+
**What it does:**
55+
- Fetches problem details from LeetCode API
56+
- Generates solutions using GPT-5-mini
57+
- Saves solutions as `problems/{id}/gpt5-mini.md`
58+
- Handles rate limiting (2 seconds between requests in bulk mode)
59+
- Reports success/failure for each problem
60+
61+
### 3. GitHub Actions Workflow: `bulk_solver.yml`
62+
63+
Automated workflow for bulk solving problems via GitHub Actions.
64+
65+
**Location:** `.github/workflows/bulk_solver.yml`
66+
67+
**How to use:**
68+
1. Go to the repository's Actions tab
69+
2. Select "Bulk Solve Missing LeetCode Problems"
70+
3. Click "Run workflow"
71+
4. Configure:
72+
- **batch_size**: Number of problems to solve (default: 50)
73+
- **start_index**: Which problem to start from (default: 0)
74+
5. Click "Run workflow" to start
75+
76+
**What it does:**
77+
- Automatically identifies missing AI solutions
78+
- Processes a batch of problems using auto_solver.py
79+
- Commits and pushes generated solutions
80+
- Can be run multiple times with different start_index values
81+
82+
**Example workflow runs:**
83+
- Run 1: start_index=0, batch_size=50 → solves problems 1-50
84+
- Run 2: start_index=50, batch_size=50 → solves problems 51-100
85+
- Run 3: start_index=100, batch_size=50 → solves problems 101-150
86+
- etc.
87+
88+
## Workflow for Adding All Missing Solutions
89+
90+
### Option A: Using GitHub Actions (Recommended)
91+
92+
1. **Initial Analysis**
93+
```bash
94+
python3 identify_missing.py
95+
```
96+
This shows you how many problems need solutions (currently 472).
97+
98+
2. **Trigger Workflow Runs**
99+
- Go to Actions → "Bulk Solve Missing LeetCode Problems"
100+
- Run workflow with: start_index=0, batch_size=50
101+
- Wait for completion, then run again with: start_index=50, batch_size=50
102+
- Repeat until all problems are processed (10 runs total for 472 problems)
103+
104+
3. **Monitor Progress**
105+
- Check the Actions tab for workflow status
106+
- Solutions will be automatically committed to main branch
107+
108+
### Option B: Using Local Script (If you have OpenAI API key)
109+
110+
1. **Setup**
111+
```bash
112+
# Install dependencies
113+
pip install requests openai httpx
114+
115+
# Set API key
116+
export OPENAI_API_KEY="your-key-here"
117+
```
118+
119+
2. **Generate batch files**
120+
```bash
121+
python3 identify_missing.py
122+
```
123+
124+
3. **Process batches**
125+
```bash
126+
# Process first batch
127+
python3 auto_solver.py --bulk /tmp/batch_001.txt
128+
129+
# Commit changes
130+
git add problems/*/gpt5-mini.md
131+
git commit -m "Add AI solutions for batch 1"
132+
git push
133+
134+
# Continue with remaining batches...
135+
python3 auto_solver.py --bulk /tmp/batch_002.txt
136+
# ... and so on
137+
```
138+
139+
## Solution Format
140+
141+
All AI-generated solutions follow this format:
142+
143+
```markdown
144+
# [Problem {ID}: {Title}]({URL})
145+
146+
## Initial thoughts (stream-of-consciousness)
147+
[Initial analysis and approach]
148+
149+
## Refining the problem, round 2 thoughts
150+
[Refined approach, edge cases, complexity analysis]
151+
152+
## Attempted solution(s)
153+
\`\`\`python
154+
[Complete Python solution]
155+
\`\`\`
156+
- [Notes about approach and complexity]
157+
```
158+
159+
## Security Notes
160+
161+
- SSL certificate verification is disabled by default to match the existing `auto_solver.py` pattern (required for environments behind proxies with self-signed certificates)
162+
- To enable SSL verification in production, set the environment variable: `HTTPX_VERIFY=true`
163+
- All scripts have been scanned with CodeQL and found no security vulnerabilities
164+
- API keys are passed via environment variables and never hard-coded
165+
166+
## Technical Details
167+
168+
### API Rate Limiting
169+
- The script includes 2-second delays between requests
170+
- This prevents overwhelming the LeetCode or OpenAI APIs
171+
- For 472 problems, expect ~15-20 minutes per batch of 50
172+
173+
### Error Handling
174+
- Problems that fail to fetch are skipped and logged
175+
- Failed problems are reported at the end
176+
- You can re-run with just the failed problems
177+
178+
### Network Requirements
179+
- Requires access to `leetcode.com` (for problem details)
180+
- Requires access to OpenAI API (for solution generation)
181+
- GitHub Actions environment has these by default
182+
183+
## Monitoring Progress
184+
185+
After running batches, you can check progress:
186+
187+
```bash
188+
# Count AI solutions
189+
find problems -name "gpt5-mini.md" | wc -l
190+
191+
# Re-run analysis
192+
python3 identify_missing.py
193+
```
194+
195+
## Notes
196+
197+
- The `auto_solver.py` script now supports three modes:
198+
1. **Daily mode** (default): `python3 auto_solver.py`
199+
2. **Single problem**: `python3 auto_solver.py 123`
200+
3. **Bulk mode**: `python3 auto_solver.py --bulk file.txt`
201+
- This bulk solver reuses the existing auto_solver logic, avoiding code duplication
202+
- Both daily and bulk modes use the same GPT-5-mini model and format
203+
- Solutions are idempotent - safe to re-run if a problem already has gpt5-mini.md

‎IMPLEMENTATION_SUMMARY.md‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# Implementation Complete ✅
2+
3+
## Summary
4+
5+
Successfully implemented a comprehensive solution to add AI-generated solutions for all missing LeetCode problems in the repository.
6+
7+
## What Was Done
8+
9+
### 1. Enhanced Existing auto_solver.py
10+
- Added support for three modes (daily, single problem, bulk)
11+
- Reused existing logic to avoid code duplication
12+
- Added `fetch_problem_by_id()` function
13+
- Added bulk processing with rate limiting
14+
- Preserved backward compatibility with daily problem solving
15+
16+
### 2. Created Analysis Tool (identify_missing.py)
17+
- Scans README.md to find all 548 problems
18+
- Identifies problems missing gpt5-mini.md files
19+
- Found 472 problems needing AI solutions (76 already have them)
20+
- Creates batch files for processing (50 problems each)
21+
- Cross-platform compatible using tempfile module
22+
23+
### 3. Created GitHub Actions Workflow
24+
- Manual trigger workflow for bulk solving
25+
- Configurable batch size and start index
26+
- Automatically commits generated solutions
27+
- Uses enhanced auto_solver.py with --bulk flag
28+
29+
### 4. Comprehensive Documentation
30+
- **BULK_SOLVER_README.md** - Technical documentation
31+
- **QUICKSTART.md** - Simple guide for maintainers
32+
- Usage examples and troubleshooting
33+
34+
## Current State
35+
36+
- **548 total problems** in README
37+
- **76 problems** with AI solutions
38+
- **472 problems** need AI solutions
39+
40+
## How to Use
41+
42+
### GitHub Actions (Recommended)
43+
1. Go to Actions tab
44+
2. Select "Bulk Solve Missing LeetCode Problems"
45+
3. Run workflow with default settings (batch_size=50, start_index=0)
46+
4. After completion, run again with start_index=50
47+
5. Repeat 10 times total to complete all 472 problems
48+
49+
### Local Execution
50+
```bash
51+
# Analyze what's missing
52+
python3 identify_missing.py
53+
54+
# Process a batch
55+
export OPENAI_API_KEY="your-key"
56+
python3 auto_solver.py --bulk /tmp/batch_001.txt
57+
```
58+
59+
## Quality Assurance
60+
61+
- ✅ All code reviewed and feedback addressed
62+
- ✅ CodeQL security scan: 0 vulnerabilities
63+
- ✅ No code duplication (reused auto_solver.py)
64+
- ✅ Cross-platform compatible
65+
- ✅ Rate limiting implemented
66+
- ✅ Error handling and reporting
67+
- ✅ SSL verification configurable
68+
69+
## Files Changed
70+
71+
- `auto_solver.py` - Enhanced with bulk mode support
72+
- `identify_missing.py` - New analysis tool
73+
- `.github/workflows/bulk_solver.yml` - New workflow
74+
- `BULK_SOLVER_README.md` - Technical documentation
75+
- `QUICKSTART.md` - User guide
76+
77+
## Next Steps
78+
79+
Repository maintainers can now:
80+
1. Trigger the GitHub Actions workflow to start generating solutions
81+
2. Process all 472 problems in approximately 3-4 hours total
82+
3. Each workflow run takes ~15-20 minutes for 50 problems
83+
4. Monitor progress via the Actions tab
84+
85+
All solutions will be automatically committed to the main branch with the same format and quality as existing AI solutions.

0 commit comments

Comments
 (0)
⚡