Skip to content

Commit 0e4fe23

Browse files
committed
feat: add performance metrics and provider comparison to README
1 parent fb00798 commit 0e4fe23

File tree

1 file changed

+119
-0
lines changed

1 file changed

+119
-0
lines changed

README.md

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -288,6 +288,125 @@ ollama pull nomic-embed-text
288288
}
289289
```
290290

291+
## 📈 Performance
292+
293+
The plugin is built for speed with a Rust native module. Here are typical performance numbers (Apple M1):
294+
295+
### Parsing (tree-sitter)
296+
297+
| Files | Chunks | Time |
298+
|-------|--------|------|
299+
| 100 | 1,200 | ~7ms |
300+
| 500 | 6,000 | ~32ms |
301+
302+
### Vector Search (usearch)
303+
304+
| Index Size | Search Time | Throughput |
305+
|------------|-------------|------------|
306+
| 1,000 vectors | 0.7ms | 1,400 ops/sec |
307+
| 5,000 vectors | 1.2ms | 850 ops/sec |
308+
| 10,000 vectors | 1.3ms | 780 ops/sec |
309+
310+
### Database Operations (SQLite with batch)
311+
312+
| Operation | 1,000 items | 10,000 items |
313+
|-----------|-------------|--------------|
314+
| Insert chunks | 4ms | 44ms |
315+
| Add to branch | 2ms | 22ms |
316+
| Check embedding exists | <0.01ms | <0.01ms |
317+
318+
### Batch vs Sequential Performance
319+
320+
Batch operations provide significant speedups:
321+
322+
| Operation | Sequential | Batch | Speedup |
323+
|-----------|------------|-------|---------|
324+
| Insert 1,000 chunks | 38ms | 4ms | **~10x** |
325+
| Add 1,000 to branch | 29ms | 2ms | **~14x** |
326+
| Insert 1,000 embeddings | 59ms | 40ms | **~1.5x** |
327+
328+
Run benchmarks yourself: `npx tsx benchmarks/run.ts`
329+
330+
## 🎯 Choosing a Provider
331+
332+
Use this decision tree to pick the right embedding provider:
333+
334+
```
335+
┌─────────────────────────┐
336+
│ Do you have Copilot? │
337+
└───────────┬─────────────┘
338+
┌─────┴─────┐
339+
YES NO
340+
│ │
341+
┌───────────▼───────┐ │
342+
│ Codebase < 1k │ │
343+
│ files? │ │
344+
└─────────┬─────────┘ │
345+
┌─────┴─────┐ │
346+
YES NO │
347+
│ │ │
348+
▼ │ │
349+
┌──────────┐ │ │
350+
│ Copilot │ │ │
351+
│ (free) │ │ │
352+
└──────────┘ │ │
353+
▼ ▼
354+
┌─────────────────────────┐
355+
│ Need fastest indexing? │
356+
└───────────┬─────────────┘
357+
┌─────┴─────┐
358+
YES NO
359+
│ │
360+
▼ ▼
361+
┌──────────┐ ┌──────────────┐
362+
│ Ollama │ │ OpenAI or │
363+
│ (local) │ │ Google │
364+
└──────────┘ └──────────────┘
365+
```
366+
367+
### Provider Comparison
368+
369+
| Provider | Speed | Cost | Privacy | Best For |
370+
|----------|-------|------|---------|----------|
371+
| **Ollama** | Fastest | Free | Full | Large codebases, privacy-sensitive |
372+
| **GitHub Copilot** | Slow (rate limited) | Free* | Cloud | Small codebases, existing subscribers |
373+
| **OpenAI** | Medium | ~$0.0001/1K tokens | Cloud | General use |
374+
| **Google** | Fast | Free tier available | Cloud | Medium-large codebases |
375+
376+
*Requires active Copilot subscription
377+
378+
### Setup by Provider
379+
380+
**Ollama (Recommended for large codebases)**
381+
```bash
382+
ollama pull nomic-embed-text
383+
```
384+
```json
385+
{ "embeddingProvider": "ollama" }
386+
```
387+
388+
**OpenAI**
389+
```bash
390+
export OPENAI_API_KEY=sk-...
391+
```
392+
```json
393+
{ "embeddingProvider": "openai" }
394+
```
395+
396+
**Google**
397+
```bash
398+
export GOOGLE_API_KEY=...
399+
```
400+
```json
401+
{ "embeddingProvider": "google" }
402+
```
403+
404+
**GitHub Copilot**
405+
No setup needed if you have an active Copilot subscription.
406+
```json
407+
{ "embeddingProvider": "github-copilot" }
408+
```
409+
291410
## ⚠️ Tradeoffs
292411

293412
Be aware of these characteristics:

0 commit comments

Comments
 (0)