Kevin m kent/v2 domain intent taxonomy

[kevin-m-kent]

PR that builds up domain and intent from prompt clustering and labeling in our 1p data. Also adds some prompting that improves the classifier's error rate (in particular swapping domain and intent). This approach also reduces the number of categories by about 50%.

FYI @digitarald , pending our discussion later.

[Copilot]

Pull request overview

Updates the prompt-categorization taxonomy and prompting used by the panel prompt classifier to a new “v2” domain/intent scheme derived from clustering, with additional guidance intended to reduce domain/intent swaps.

Changes:

• Replaces the intent and domain category definitions with a reduced v2 taxonomy.
• Updates taxonomy prompt formatting and adds explicit “domain vs intent” separation guidance (and changes ordering to domain-first).
• Tweaks the system prompt wording for the categorization prompt.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
src/extension/prompts/node/panel/promptCategorization.tsx	Updates the system prompt text used to instruct the classifier and includes the taxonomy prompt.
src/extension/prompt/common/promptCategorizationTaxonomy.ts	Reworks domain/intent taxonomy definitions and regenerates the taxonomy prompt/guidance formatting.

Comments suppressed due to low confidence (1)

src/extension/prompt/common/promptCategorizationTaxonomy.ts:327

• The guidance explicitly instructs using unknown for intent/domain, but it doesn't mention what to do when scope is unclear. Since the scope taxonomy uses unknown_scope (not unknown), the current guidance can increase invalid tool outputs (rejected by isValidScope). Consider adding an explicit rule like "If scope is unclear, use unknown_scope" to keep tool calls valid.

**Domain** is the technical subject area or problem space the user is operating in.
- It describes a system, architecture, technology area, or problem space — never an activity.
- Think of it as answering: "What area of technology is this about?"
- If the prompt does not clearly indicate a technical domain, use \`unknown\`.

**Intent** is the developer action or goal being performed within that domain.
- It describes what the user is trying to accomplish — the verb, not the noun.
- Think of it as answering: "What is the user trying to do?"
- If the prompt does not clearly indicate an intent, use \`unknown\`.

[digitarald]

Taxonomy changes look good — data-driven consolidation is a solid approach. Left a few non-blocking nits for follow-up (telemetry versioning, prompt boundary, dead code).

[digitarald]

nit (non-blocking, follow-up): The promptCategorization telemetry event in promptCategorizer.ts emits intent and domain as raw strings. Since v2 keys are completely different from v1 (e.g., code_fixing → troubleshoot_debug, frontend → web_ui), queries on this data will get mixed v1/v2 values with no way to distinguish them once this ships.

Suggestion: Add a taxonomyVersion: 'v2' property to the telemetry event and its GDPR annotation so dashboards can filter correctly.

[kevin-m-kent]

I like this idea. Let me update.

[digitarald]

nit (non-blocking): systemPrompt is joined with \n\n but then rendered immediately before <SafetyRules /> with no separator, so the last line of the taxonomy can run together with the safety rules text.

Consider adding a trailing \n\n:

].join('\n\n') + '\n\n';

Or a <br /> before <SafetyRules /> in the JSX.

[digitarald]

nit (non-blocking): def.signals rendering is now dead code — no v2 definitions use signals anymore (all switched to keywords). Could be cleaned up in a follow-up.