Implement AI search success tracking telemetry for user interaction analytics

[Copilot]

Adds comprehensive telemetry tracking to measure AI search effectiveness and user engagement patterns in VS Code's search functionality.

Overview

This PR implements a telemetry framework that tracks how users interact with AI search results compared to traditional text search, providing data-driven insights for improving the AI search experience.

Key Features

📊 Search Session Tracking

• Session Management: New SearchSessionTracker class manages search session lifecycle and metrics
• Query Patterns: Tracks search behavior including query count, AI usage, and trigger methods
• Success Measurement: Determines session success based on user interaction with results

🎯 Result Interaction Analytics

• Click Tracking: Enhanced onFocus() method captures when users click on search results
• AI vs Text Classification: Differentiates between AI and traditional search results using isAIContributed property
• Performance Metrics: Measures time-to-click, result ranking, and engagement patterns

📈 Telemetry Events

aiSearchResultClick - Fired when users interact with search results:

{
  resultType: 'ai' | 'text',           // Source of the result
  resultRank: number,                  // Position in results list (1-based)
  queryLength: number,                 // Length of search query
  searchTriggerType: 'auto' | 'manual' | 'runOnEmpty',
  timeToClick: number,                 // Milliseconds from search completion
  totalAIResults: number,              // Total AI results available
  totalTextResults: number             // Total text results available
}

searchSessionComplete - Fired when search sessions end:

{
  sessionDuration: number,             // Total session time in milliseconds
  queryCount: number,                  // Number of queries in session
  aiSearchTriggered: boolean,          // Whether AI search was used
  aiResultsUsed: boolean,              // Whether user clicked AI results
  sessionSuccessful: boolean,          // Whether session ended with file opened
  finalResultType: 'ai' | 'text' | 'none'
}

🔒 Privacy & Compliance

• All telemetry follows VS Code's GDPR guidelines with SystemMetaData classification
• No personal information or file content is collected
• Purpose-driven data collection for FeatureInsight and PerformanceAndHealth
• Graceful error handling ensures telemetry never impacts user experience

Integration Points

The implementation hooks into existing search infrastructure:

• Search Triggers: onQueryTriggered(), requestAIResults()
• Completion Tracking: onSearchComplete()
• Result Interaction: Enhanced onFocus() method
• Lifecycle Management: View disposal and session cleanup

Testing

Added comprehensive unit tests in searchTelemetry.test.ts covering:

• Session lifecycle management
• Query and result tracking
• Telemetry event generation
• Edge cases and error conditions

Benefits

This telemetry enables data-driven improvements to AI search:

• Engagement Analysis: Compare AI vs text search effectiveness
• UX Optimization: Identify successful interaction patterns
• Feature Development: Guide future AI search enhancements
• Performance Monitoring: Track search success rates and user satisfaction

The implementation is lightweight, privacy-compliant, and provides actionable insights without impacting search performance or user experience.

Created from VS Code via the GitHub Pull Request extension.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.