feat: Implement optional version handling and improve CLI

This commit introduces support for storing and querying documentation without an explicit version, alongside significant improvements to version handling logic and CLI usability.

**Core Changes:**

*   **Optional Versions:** Modified database schema and services (`DocumentManagementService`, `DocumentRetrieverService`) to handle optional versions. `null` or `undefined` versions are now treated internally as an empty string (`""`) to represent unversioned documents.
*   **Schema Update:** Changed `documents.version` column to `TEXT NOT NULL DEFAULT ''`.
*   **Service Layer:** Updated methods in services to accept optional version parameters and perform normalization (`null`/`undefined` -> `""`). `listLibraries` now filters out the internal empty string version.
*   **Scraping Validation:** Implemented stricter version validation in `ScrapeTool`. It now accepts full semver (`1.2.3`), pre-release (`1.2.3-beta`), and partial (`1`, `1.2` coerced to `1.0.0`, `1.2.0`) versions, or no version (for unversioned). Ranges (`1.x`) and invalid formats are rejected during scraping.
*   **Version Finding:** Refactored `DocumentManagementService.findBestVersion` to return both the `bestMatch` (semver, with fallback to older versions) and `hasUnversioned` (boolean). Updated `FindVersionTool` and the `find-version` CLI command to provide more informative output based on this result.
*   **Search Tool:** Fixed a bug in `SearchTool` where the entire result object from `findBestVersion` was incorrectly passed to `searchStore`.
*   **CLI Consistency:** Updated `scrape`, `search`, and `find-version` CLI commands to use a consistent optional `--version` flag instead of positional arguments for the version.
*   **Documentation:** Updated `README.md` with a project summary, detailed explanations of the new version handling logic, and updated CLI command examples.
*   **Testing:** Added and updated tests to cover optional version handling, fallback logic, and validation, including fixing previous test failures.

Author: arabold

README.md

@@ -1,5 +1,7 @@
 # docs-mcp-server MCP Server
 
+This project provides a Model Context Protocol (MCP) server designed to scrape, process, index, and search documentation for various software libraries and packages. It fetches content from specified URLs, splits it into meaningful chunks using semantic splitting techniques, generates vector embeddings using OpenAI, and stores the data in an SQLite database. The server utilizes `sqlite-vec` for efficient vector similarity search and FTS5 for full-text search capabilities, combining them for hybrid search results. It supports versioning, allowing documentation for different library versions (including unversioned content) to be stored and queried distinctly. The server exposes MCP tools for scraping (`scrape_docs`), searching (`search_docs`), listing indexed libraries (`list_libraries`), and finding appropriate versions (`find_version`). A companion CLI (`docs-mcp`) is also included for local management and interaction.
+
 A MCP server for fetching and searching 3rd party package documentation
 
 ## CLI Usage
@@ -13,16 +15,118 @@ docs-mcp --help
 # Show help for a specific command
 docs-mcp scrape --help
 docs-mcp search --help
+docs-mcp find-version --help
+```
+
+### Scraping Documentation (`scrape`)
+
+Scrapes and indexes documentation from a given URL for a specific library.
+
+```bash
+docs-mcp scrape <library> <url> [options]
+```
+
+**Options:**
+
+- `-v, --version <string>`: The specific version to associate with the scraped documents.
+  - Accepts full versions (`1.2.3`), pre-release versions (`1.2.3-beta.1`), or partial versions (`1`, `1.2` which are expanded to `1.0.0`, `1.2.0`).
+  - If omitted, the documentation is indexed as **unversioned**.
+- `-p, --max-pages <number>`: Maximum pages to scrape (default: 100).
+- `-d, --max-depth <number>`: Maximum navigation depth (default: 3).
+- `-c, --max-concurrency <number>`: Maximum concurrent requests (default: 3).
+- `--ignore-errors`: Ignore errors during scraping (default: true).
+
+**Examples:**
+
+```bash
+# Scrape React 18.2.0 docs
+docs-mcp scrape react --version 18.2.0 https://react.dev/
+
+# Scrape React docs without a specific version (indexed as unversioned)
+docs-mcp scrape react https://react.dev/
+
+# Scrape partial version (will be stored as 7.0.0)
+docs-mcp scrape semver --version 7 https://github.com/npm/node-semver
+
+# Scrape pre-release version
+docs-mcp scrape mylib --version 2.0.0-rc.1 https://mylib.com/docs
+```
+
+### Searching Documentation (`search`)
+
+Searches the indexed documentation for a library, optionally filtering by version.
+
+```bash
+docs-mcp search <library> <query> [options]
 ```
 
-### Version Handling
+**Options:**
+
+- `-v, --version <string>`: The target version or range to search within.
+  - Supports exact versions (`18.0.0`), partial versions (`18`), or ranges (`18.x`).
+  - If omitted, searches the **latest** available indexed version.
+  - If a specific version/range doesn't match, it falls back to the latest indexed version _older_ than the target.
+  - To search **only unversioned** documents, explicitly pass an empty string: `--version ""`. (Note: Omitting `--version` searches latest, which _might_ be unversioned if no other versions exist).
+- `-l, --limit <number>`: Maximum number of results (default: 5).
+- `-e, --exact-match`: Only match the exact version specified (disables fallback and range matching) (default: false).
+
+**Examples:**
+
+```bash
+# Search latest React docs for 'hooks'
+docs-mcp search react 'hooks'
+
+# Search React 18.x docs for 'hooks'
+docs-mcp search react --version 18.x 'hooks'
+
+# Search React 17 docs (will match 17.x.x or older if 17.x.x not found)
+docs-mcp search react --version 17 'hooks'
+
+# Search only React 18.0.0 docs
+docs-mcp search react --version 18.0.0 --exact-match 'hooks'
+
+# Search only unversioned React docs
+docs-mcp search react --version "" 'hooks'
+```
+
+### Finding Available Versions (`find-version`)
+
+Checks the index for the best matching version for a library based on a target, and indicates if unversioned documents exist.
+
+```bash
+docs-mcp find-version <library> [options]
+```
+
+**Options:**
+
+- `-v, --version <string>`: The target version or range. If omitted, finds the latest available version.
+
+**Examples:**
+
+```bash
+# Find the latest indexed version for react
+docs-mcp find-version react
+
+# Find the best match for react version 17.x
+docs-mcp find-version react --version 17.x
+
+# Find the best match for react version 17.0.0 (may fall back to older)
+docs-mcp find-version react --version 17.0.0
+```
+
+### Listing Libraries (`list-libraries`)
+
+Lists all libraries currently indexed in the store.
+
+```bash
+docs-mcp list-libraries
+```
 
-This server supports partial version matching, selecting the best available version based on these rules:
+### Version Handling Summary
 
-- If no version is specified, the latest indexed version is used.
-- If a full version (e.g., `1.2.3`) is specified, that exact version is used, if available.
-- If a partial version (e.g., `1.2`) is specified, the latest matching version (e.g., `1.2.5`) is used.
-- If the specified version (full or partial) is not found, the server will attempt to find the closest preceding version.
+- **Scraping:** Requires a specific, valid version (`X.Y.Z`, `X.Y.Z-pre`, `X.Y`, `X`) or no version (for unversioned docs). Ranges (`X.x`) are invalid for scraping.
+- **Searching/Finding:** Accepts specific versions, partials, or ranges (`X.Y.Z`, `X.Y`, `X`, `X.x`). Falls back to the latest older version if the target doesn't match. Omitting the version targets the latest available. Explicitly searching `--version ""` targets unversioned documents.
+- **Unversioned Docs:** Libraries can have documentation stored without a specific version (by omitting `--version` during scrape). These can be searched explicitly using `--version ""`. The `find-version` command will also report if unversioned docs exist alongside any semver matches.
 
 ## Development
 

src/cli.ts

@@ -33,17 +33,19 @@ async function main() {
       .version("1.0.0");
 
     program
-      .command("scrape <library> <version> <url>")
+      .command("scrape <library> <url>") // Remove <version> as positional
       .description("Scrape and index documentation from a URL")
+      .option("-v, --version <string>", "Version of the library (optional)") // Add optional version flag
       .option("-p, --max-pages <number>", "Maximum pages to scrape", "100")
       .option("-d, --max-depth <number>", "Maximum navigation depth", "3")
       .option("-c, --max-concurrency <number>", "Maximum concurrent page requests", "3")
       .option("--ignore-errors", "Ignore errors during scraping", true)
-      .action(async (library, version, url, options) => {
+      .action(async (library, url, options) => {
+        // Update action parameters
         const result = await tools.scrape.execute({
           url,
           library,
-          version,
+          version: options.version, // Get version from options
           options: {
             maxPages: Number.parseInt(options.maxPages),
             maxDepth: Number.parseInt(options.maxDepth),
@@ -55,24 +57,29 @@ async function main() {
       });
 
     program
-      .command("search <library> <version> <query>")
+      .command("search <library> <query>") // Remove <version> as positional
       .description(
         "Search documents in a library. Version matching examples:\n" +
-          "  - search react 18.0.0 'hooks' -> matches docs for React 18.0.0 or earlier versions\n" +
-          "  - search react 18.0.0 'hooks' --exact-match -> only matches React 18.0.0\n" +
-          "  - search typescript 5.x 'types' -> matches any TypeScript 5.x.x version\n" +
-          "  - search typescript 5.2.x 'types' -> matches any TypeScript 5.2.x version",
+          "  - search react --version 18.0.0 'hooks' -> matches docs for React 18.0.0 or earlier versions\n" +
+          "  - search react --version 18.0.0 'hooks' --exact-match -> only matches React 18.0.0\n" +
+          "  - search typescript --version 5.x 'types' -> matches any TypeScript 5.x.x version\n" +
+          "  - search typescript --version 5.2.x 'types' -> matches any TypeScript 5.2.x version",
+      )
+      .option(
+        "-v, --version <string>", // Add optional version flag
+        "Version of the library (optional, supports ranges)",
       )
       .option("-l, --limit <number>", "Maximum number of results", "5")
       .option(
         "-e, --exact-match",
         "Only use exact version match (e.g., '18.0.0' matches only 18.0.0, not 17.x.x) (default: false)",
         false,
       )
-      .action(async (library, version, query, options) => {
+      .action(async (library, query, options) => {
+        // Update action parameters
         const result = await tools.search.execute({
           library,
-          version,
+          version: options.version, // Get version from options
           query,
           limit: Number.parseInt(options.limit),
           exactMatch: options.exactMatch,
@@ -89,17 +96,22 @@ async function main() {
       });
 
     program
-      .command("find-version <library> [targetVersion]")
+      .command("find-version <library>") // Remove [targetVersion] positional
       .description("Find the best matching version for a library")
-      .action(async (library, targetVersion) => {
-        const version = await tools.findVersion.execute({
+      .option(
+        "-v, --version <string>", // Add optional version flag
+        "Target version to match (optional, supports ranges)",
+      )
+      .action(async (library, options) => { // Update action parameters
+        const versionInfo = await tools.findVersion.execute({
           library,
-          targetVersion,
+          targetVersion: options.version, // Get version from options
         });
-        if (!version) {
-          throw new Error("No matching version found");
+        // findVersion.execute now returns a string, handle potential error messages within it
+        if (!versionInfo) { // Should not happen with current tool logic, but good practice
+          throw new Error("Failed to get version information");
         }
-        console.log(version);
+        console.log(versionInfo); // Log the descriptive string from the tool
       });
 
     await program.parseAsync();

src/mcp/index.ts

@@ -47,7 +47,7 @@ export async function startServer() {
       {
         url: z.string().url().describe("URL of the documentation to scrape"),
         library: z.string().describe("Name of the library"),
-        version: z.string().describe("Version of the library"),
+        version: z.string().optional().describe("Version of the library"),
         maxPages: z
           .number()
           .optional()