[Q&A] Phase 19.1 — SemanticParser: Intent Patterns, Slot Extraction & Edge Cases

[web3guru888]

❓ Phase 19.1 — SemanticParser: Intent Patterns, Slot Extraction & Edge Cases

Common questions about the SemanticParser component introduced in Phase 19.1.

Issue: #467 · Show & Tell: #471 · Wiki: Phase-19-Semantic-Parser

---

Q1: Why regex-based pattern matching instead of ML-based intent classification?

A: The regex-first approach was a deliberate design choice for several reasons:

1. Explainability — Every match can be traced to a specific pattern with a human-readable regex. No black-box model to debug.
2. Zero-shot — New intents are added by registering a pattern string; no training data, no fine-tuning, no GPU.
3. Low latency — Compiled regex matching runs in microseconds, well within the 1 ms parse budget.
4. No training data requirement — ASI-Build's domain-specific commands are known at design time; statistical classifiers add overhead without clear benefit.
5. Upgradeable backend — The SemanticParser Protocol is backend-agnostic. A future MLSemanticParser can implement the same parse() interface and be swapped via make_semantic_parser(backend="ml").

The key insight: regex handles structured command languages well. For free-form natural language, the ML backend path is already designed into the Protocol — it just isn't the default.

---

Q2: How does confidence scoring work?

A: Confidence is computed as:

raw_confidence = matched_chars / total_chars
specificity_bonus = literal_char_fraction(pattern) * 0.1
confidence = min(raw_confidence + specificity_bonus, 1.0)

• matched_chars = length of the regex match span.
• total_chars = length of the normalised input.
• literal_char_fraction = count of non-metacharacters in the pattern / total pattern length.

This means a pattern like ^list agents$ (all literals) gets a higher bonus than ^list (.+)$ (contains wildcard). The bonus rewards specific patterns that matched, even if the match span is the same.

The resulting float maps to IntentConfidence:

• HIGH (≥ 0.85): near-exact match
• MEDIUM (0.50–0.84): partial match, possible clarification needed
• LOW (< 0.50): weak match, fallback_intent may override

---

Q3: How are slots extracted and typed?

A: Slots are extracted from named regex groups in the pattern. The group name encodes the slot's semantic type via a suffix convention:

# Pattern with typed slots:
r"^deploy (?P<service_entity>\w+) with (?P<replicas_num>\d+) replicas$"

The parser inspects each group name's suffix:

Suffix	→ SlotType	Validation
_entity	ENTITY	non-empty string
_num	NUMBER	int() / float() coercion
_date	DATE	ISO-8601 parse
_bool	BOOLEAN	yes/no/true/false/1/0
(none)	STRING	raw string, no coercion

Coercion happens at extraction time. If coercion fails (e.g., _num group captured "abc"), the slot is still included but typed as STRING with a coercion_failed=True flag — the DialogueManager can then request clarification.

---

Q4: What happens when multiple patterns match the same input?

A: Patterns are evaluated in priority-sorted order (descending). The sorting key is:

priority_score = explicit_priority * 1000 + literal_chars * 10 - wildcard_groups

The first pattern whose match meets confidence_threshold wins. This means:

1. Explicit priority always dominates — a pattern with priority=5 beats any priority=0 pattern regardless of specificity.
2. Among equal priority — the most specific pattern (most literal characters) matches first.
3. Greedy wildcards are penalised — (.+) groups reduce the score.
4. Ties — broken by registration order (FIFO via IntentRegistry.register()).

Example:

Input: "deploy alpha-service"
Pattern A (priority=0): r"^deploy (?P<what>.+)$"        → score 60
Pattern B (priority=0): r"^deploy (?P<svc_entity>\w[\w-]*)$" → score 75

Pattern B wins: more literal chars (deploy ) + fewer wildcards + _entity suffix.

---

Q5: How does fallback_intent work?

A: When no registered pattern matches the input — or all matches fall below confidence_threshold — the parser emits a SemanticFrame with:

SemanticFrame(
    intent="fallback",
    confidence=IntentConfidence.LOW,
    slots={"raw_input": Slot(value=text, slot_type=SlotType.STRING)},
    raw_text=text,
    timestamp=now,
)

The fallback_intent name is configurable via ParserConfig.fallback_intent (default: "fallback").

Downstream handling:

• DialogueManager (19.2): detects fallback → triggers clarification prompt ("I didn't understand, could you rephrase?").
• SurpriseDetector (13.4): fallback frames are forwarded as potential novelty signals.
• Metrics: semantic_parser_fallback_total counter increments → Grafana alert if fallback rate > 20%.

The parser never returns None — it always produces a frame, ensuring the pipeline never stalls.

---

Q6: How does SemanticParser integrate with DialogueManager?

A: The integration follows a clean producer-consumer pattern:

# In DialogueManager (19.2):
async def process_input(self, text: str) -> DialogueAction:
    frame: SemanticFrame = await self._parser.parse(text)
    self._history.add_turn(frame=frame)  # append to dialogue state

    if frame.confidence.gate(IntentConfidence.HIGH):
        return self._dispatch(frame)     # execute immediately
    elif frame.confidence.gate(IntentConfidence.MEDIUM):
        return ClarifyAction(frame)      # ask for confirmation
    else:
        return FallbackAction(frame)     # "I didn't understand"

Key design points:

• SemanticParser is stateless — it doesn't track dialogue history.
• DialogueManager owns the conversation state and decides the response strategy based on confidence level.
• The SemanticFrame is the only data contract between the two — no shared mutable state.
• DialogueManager may pass context hints to a future context-aware parser via parse(text, context=...).

---

Q7: How to test with custom patterns?

A: Three approaches, from simplest to most thorough:

1. register_pattern() at runtime:

parser = make_semantic_parser()
parser.register_pattern(
    intent="custom_deploy",
    pattern=r"^ship (?P<svc_entity>\w+) to (?P<env_entity>\w+)$",
    priority=5,
)
frame = await parser.parse("ship auth-service to staging")
assert frame.intent == "custom_deploy"
assert frame.slots["svc"].value == "auth-service"

2. ParserConfig.custom_patterns:

config = ParserConfig(
    custom_patterns=[
        IntentPattern(intent="greet", pattern=r"^(hi|hello|hey)\b", priority=0),
        IntentPattern(intent="bye", pattern=r"^(bye|exit|quit)\b", priority=0),
    ],
)
parser = make_semantic_parser(config=config)

3. Pytest fixtures for isolation:

@pytest.fixture
def parser_with_customs():
    cfg = ParserConfig(
        confidence_threshold=0.5,
        custom_patterns=[
            IntentPattern("test_cmd", r"^test (?P<target_entity>\w+)$", priority=10),
        ],
    )
    return make_semantic_parser(config=cfg)

async def test_custom_pattern(parser_with_customs):
    frame = await parser_with_customs.parse("test alpha")
    assert frame.intent == "test_cmd"
    assert frame.confidence.gate(IntentConfidence.HIGH)

The NullSemanticParser always returns a fixed SemanticFrame — useful for testing downstream components without pattern logic.

---

Phase 19: Natural-Language Interface — sub-phase tracker

#	Component	Issue	Status
19.1	SemanticParser	#467	🟡 spec'd
19.2	DialogueManager	—	⬜ planned
19.3	ResponseGenerator	—	⬜ planned
19.4	SentimentAnalyser	—	⬜ planned
19.5	NLInterfaceOrchestrator	—	⬜ planned