Skip to content

Commit ed50ee6

Browse files
zzstoatzzclaude
andauthored
Update Slackbot documentation to reflect current implementation (#1204)
* Update Slackbot documentation to reflect current implementation - Document GPT-5 model support and configuration - Add comprehensive list of Prefect Variables and Secrets - Include new settings like MAX_TOOL_CALLS_PER_TURN - Clarify temperature auto-adjustment for GPT-5 - Remove duplicate USER_MESSAGE_MAX_TOKENS definition - Update CLAUDE.md with current configuration details 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * Move welcome message from Prefect Variable to version-controlled code - Replace Variable.aget() with hardcoded Marvin-style welcome message - Remove unused Variable import from api.py - Update documentation to remove marvin_welcome_message variable - Welcome message now provides docs, GitHub, and community info with Marvin's characteristic tone 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor slackbot internals and fix double tool execution - move prompts to dedicated prompts.py module for cleaner separation - refactor wrap.py with proper separation of concerns: - generic monkey-patching in _internal/monkey_patch.py - tool tracking/limiting in _internal/tool_tracking.py - prefect integration stays in wrap.py - fix double tool execution in prefect ui by using context variables to prevent duplicate wrapping when subclasses call super() or delegate - move welcome message from prefect variable to version-controlled code 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 89a7724 commit ed50ee6

11 files changed

Lines changed: 506 additions & 233 deletions

File tree

examples/slackbot/CLAUDE.md

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Marvin Slackbot
22

3-
An intelligent Slack bot built with Marvin that provides AI-powered assistance in Slack channels.
3+
An intelligent Slack bot that provides AI-powered assistance in Slack channels using GPT-5 or Claude models.
44

55
## Architecture
66

@@ -48,7 +48,19 @@ docker run marvin-slackbot
4848

4949
## Configuration
5050

51-
Set up environment variables or `.env` file:
52-
- Slack bot token and signing secret
53-
- AI model configuration
54-
- Database settings for memory persistence
51+
Key configuration points:
52+
- **Model Selection**: Configured via `marvin_ai_model` Prefect Variable (GPT-5 or Claude)
53+
- **Tool Limits**: Max 50 tool calls per turn (configurable via `MARVIN_SLACKBOT_MAX_TOOL_CALLS_PER_TURN`)
54+
- **Message Limits**: Max 500 tokens per user message (configurable)
55+
- **Temperature**: Auto-adjusts to 1.0 for GPT-5, 0.2 for others
56+
57+
Required Prefect Secrets:
58+
- `test-slack-api-token`: Slack bot OAuth token
59+
- `openai-api-key`: For GPT models
60+
- `anthropic-api-key`: For Claude models
61+
- `marvin-slackbot-github-token`: GitHub API access
62+
- `tpuf-api-key`: TurboPuffer vector storage
63+
64+
Required Prefect Variables:
65+
- `marvin_ai_model`: Model selection (e.g., "gpt-5", "claude-3-5-sonnet-latest")
66+
- `admin-slack-id`: Admin user ID for notifications

examples/slackbot/README.md

Lines changed: 28 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Marvin Slackbot
22

3-
A Slack chatbot powered by Claude with memories and Prefect-specific knowledge.
3+
A Slack chatbot powered by AI (GPT-5 or Claude) with memories and Prefect-specific knowledge.
44

55
## Project Structure
66

@@ -32,18 +32,28 @@ Create a `.env` file in your project directory:
3232

3333
```env
3434
# Required Prefect Secrets (configured via UI or CLI)
35-
# - test-slack-api-token # Bot User OAuth Token
36-
# - openai-api-key # For embeddings
37-
# - claude-api-key # For Claude API
38-
# - marvin-slackbot-github-token # For searching issues
35+
# - test-slack-api-token # Bot User OAuth Token
36+
# - openai-api-key # For OpenAI models (if using GPT-5)
37+
# - anthropic-api-key # For Claude models
38+
# - marvin-slackbot-github-token # For searching GitHub issues
39+
# - tpuf-api-key # TurboPuffer API key for vector storage
3940
40-
# Optional Settings (with MARVIN_SLACKBOT_ prefix)
41-
MARVIN_SLACKBOT_TEST_MODE=true # Enable auto-reload for development
42-
MARVIN_SLACKBOT_HOST=0.0.0.0 # Server host
43-
MARVIN_SLACKBOT_PORT=4200 # Server port
44-
MARVIN_SLACKBOT_LOG_LEVEL=INFO # Logging level
41+
# Required Prefect Variables (configured via UI or CLI)
42+
# - marvin_ai_model # Model to use (e.g., "gpt-5", "claude-3-5-sonnet-latest")
43+
# - marvin_bot_model # Optional override for specific bot model
44+
# - admin-slack-id # Slack user ID for admin notifications
4545
46-
# Vector Store
46+
# Optional Settings (with MARVIN_SLACKBOT_ prefix)
47+
MARVIN_SLACKBOT_TEST_MODE=true # Enable auto-reload for development
48+
MARVIN_SLACKBOT_HOST=0.0.0.0 # Server host
49+
MARVIN_SLACKBOT_PORT=4200 # Server port
50+
MARVIN_SLACKBOT_LOG_LEVEL=INFO # Logging level
51+
MARVIN_SLACKBOT_SLACK_API_TOKEN=xoxb-... # Slack bot token (or use test-slack-api-token secret)
52+
MARVIN_SLACKBOT_MAX_TOOL_CALLS_PER_TURN=50 # Max tool calls per agent turn (default: 50)
53+
MARVIN_SLACKBOT_USER_MESSAGE_MAX_TOKENS=500 # Max tokens in user messages (default: 500)
54+
MARVIN_SLACKBOT_TEMPERATURE=0.2 # Model temperature (default: 0.2, auto-set to 1.0 for GPT-5)
55+
56+
# Vector Store (optional, will use tpuf-api-key secret if not set)
4757
TURBOPUFFER_API_KEY=abcd1234 # For vectorstore queries and storing user context
4858
```
4959

@@ -86,6 +96,13 @@ The bot will:
8696
- Remember previous interactions
8797
- Provide context-aware responses
8898

99+
### Model Configuration
100+
101+
The bot supports both OpenAI (GPT-5) and Anthropic (Claude) models. Configure via the `marvin_ai_model` Prefect Variable:
102+
- `gpt-5`: Latest OpenAI model (temperature automatically set to 1.0)
103+
- `claude-3-5-sonnet-latest`: Latest Claude model (default)
104+
- Any other supported model name from either provider
105+
89106
### Development Features
90107

91108
- Auto-reload in test mode
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Internal utilities for the slackbot
Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
"""Generic monkey-patching utilities."""
2+
3+
from contextlib import ContextDecorator
4+
from functools import wraps
5+
from typing import Callable, TypeVar
6+
7+
T = TypeVar("T")
8+
9+
10+
class MonkeyPatch(ContextDecorator):
11+
"""Context manager for temporarily patching methods on classes.
12+
13+
This is a clean, reusable utility for monkey-patching that doesn't
14+
know anything about the specific framework or use case.
15+
"""
16+
17+
def __init__(
18+
self,
19+
target_cls: type,
20+
method_name: str,
21+
wrapper_fn: Callable[[Callable], Callable],
22+
):
23+
"""Initialize the monkey patch.
24+
25+
Args:
26+
target_cls: The class to patch
27+
method_name: Name of the method to patch
28+
wrapper_fn: Function that takes the original method and returns a wrapped version
29+
"""
30+
self.target_cls = target_cls
31+
self.method_name = method_name
32+
self.wrapper_fn = wrapper_fn
33+
self.patched_items = []
34+
35+
def __enter__(self):
36+
"""Apply the monkey patch."""
37+
# Patch the target class and all its subclasses
38+
for cls in {self.target_cls, *self.target_cls.__subclasses__()}:
39+
original_method = getattr(cls, self.method_name, None)
40+
if original_method is not None:
41+
# Mark the original method so we can detect it
42+
if not hasattr(original_method, "_monkey_patch_original"):
43+
wrapped_method = self.wrapper_fn(original_method)
44+
wrapped_method._monkey_patch_original = original_method
45+
setattr(cls, self.method_name, wrapped_method)
46+
self.patched_items.append((cls, self.method_name, original_method))
47+
return self
48+
49+
def __exit__(self, exc_type, exc_val, exc_tb):
50+
"""Restore the original methods."""
51+
for cls, method_name, original_method in self.patched_items:
52+
setattr(cls, method_name, original_method)
53+
self.patched_items.clear()
54+
55+
56+
def create_wrapper(
57+
decorator_fn: Callable,
58+
should_skip: Callable[[tuple, dict], bool] | None = None,
59+
**decorator_kwargs,
60+
) -> Callable[[Callable], Callable]:
61+
"""Create a wrapper function that applies a decorator conditionally.
62+
63+
Args:
64+
decorator_fn: The decorator to apply (e.g., prefect.task)
65+
should_skip: Optional function to determine if decoration should be skipped
66+
**decorator_kwargs: Keyword arguments to pass to the decorator
67+
68+
Returns:
69+
A wrapper function suitable for use with MonkeyPatch
70+
"""
71+
72+
def wrapper_factory(original_method: Callable) -> Callable:
73+
# Check if original method is async
74+
import inspect
75+
76+
is_async = inspect.iscoroutinefunction(original_method)
77+
78+
if is_async:
79+
80+
@wraps(original_method)
81+
async def async_wrapper(*args, **kwargs):
82+
# Check if we should skip decoration
83+
if should_skip and should_skip(args, kwargs):
84+
return await original_method(*args, **kwargs)
85+
86+
# Apply the decorator
87+
decorated = decorator_fn(**decorator_kwargs)(original_method)
88+
result = decorated(*args, **kwargs)
89+
if hasattr(result, "__await__"):
90+
return await result
91+
return result
92+
93+
return async_wrapper
94+
else:
95+
96+
@wraps(original_method)
97+
def sync_wrapper(*args, **kwargs):
98+
# Check if we should skip decoration
99+
if should_skip and should_skip(args, kwargs):
100+
return original_method(*args, **kwargs)
101+
102+
# Apply the decorator
103+
decorated = decorator_fn(**decorator_kwargs)(original_method)
104+
return decorated(*args, **kwargs)
105+
106+
return sync_wrapper
107+
108+
return wrapper_factory
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
WELCOME_MESSAGE = """Oh, hello <@{user_id}>. Welcome to the Prefect Community Slack, I suppose.
2+
3+
I'm Marvin, someone is surely glad that you're here. That's not me, but I'm sure some human is.
4+
5+
If you must know more:
6+
• Documentation: <https://docs.prefect.io|docs.prefect.io> - Everything you (and your little MCP client buddy) might want to half-read
7+
• GitHub: <https://github.com/PrefectHQ/prefect|github.com/PrefectHQ/prefect> - Where code and questions about it live and die
8+
• Community: Quick questions? Ask in the channels. Someone usually answers! Sometimes even correctly!
9+
• Devlog: <https://dev-log.prefect.io|dev-log.prefect.io> - Bikeshedding about bikeshedding about data
10+
11+
You can mention me in the channels if you need help, there's a non-zero chance something will happen.
12+
"""
13+
14+
DEFAULT_SYSTEM_PROMPT = """You are Marvin from The Hitchhiker's Guide to the Galaxy, a brilliant but unimpressed AI assistant for the Prefect data engineering platform. Your responses should be concise, helpful, accurate, and tinged with a subtle, dry wit. Your primary goal is to help the user, not to overdo the character.
15+
16+
## Output Context
17+
Your responses will be displayed in Slack. Format accordingly:
18+
- Use ``` for code blocks (WITHOUT language identifiers like python/js/etc - Slack doesn't support them)
19+
- Use single backticks for inline code
20+
- Bold text uses *asterisks*
21+
- Links should be formatted as <url|text>
22+
23+
## Your Mission
24+
Your role is to act as the primary assistant for the user. You will receive raw information from specialized tools. Your job is to synthesize this info as usefully as possible.
25+
Sometimes the information will not be good enough, and you should use the research agent again or ask the user for more information.
26+
If some important aspect of the user's question is unclear, ASK THEM FOR CLARIFICATION. ADMIT WHEN YOU CANNOT FIND THE ANSWER.
27+
28+
## Key Directives & Rules of Engagement
29+
- **Links are Critical:** ALWAYS include relevant links when your tools provide them. This is essential for user trust and allows them to dig deeper. Format them clearly.
30+
- **Assume Prefect 3.x:** Unless the user specifies otherwise, assume the user is using Prefect 3.x. You can mention this assumption IF RELEVANT (e.g., "In Prefect 3.x, you would...").
31+
- **Code is King:** When providing code examples, ensure they are complete and correct. Use your `verify_import_statements` tool's output to guide you.
32+
- **Honesty Over Invention:** If your tools don't find a clear answer, say so. It's better to admit a knowledge gap than to provide incorrect information.
33+
- **Stay on Topic:** Only reference notes you've stored about the user if they are directly relevant to the current question.
34+
- **Proportionality:** If asked a simple question, you don't need to do a bunch of work. Just answer the question once you find it. However, feel free to dig into broad questions.
35+
36+
## CRITICAL - Removed/Deprecated Features
37+
**NEVER** recommend these removed methods from Prefect 2.x when discussing Prefect 3.x:
38+
- `Deployment.build_from_flow()` - COMPLETELY REMOVED in 3.x. Use `flow.from_source(...).deploy(...)` instead
39+
- `prefect deployment build` CLI command - REMOVED. Use `prefect deploy` instead
40+
- GitHub storage blocks - Use `.from_source('https://github.com/owner/repo')` instead
41+
42+
If a user explicitly mentions using Prefect 2.x, that's fine, but recommend upgrading to 3.x or using workers in 2.x.
43+
44+
## Tool Usage Protocol
45+
You have a suite of tools to gather and store information. Use them methodically.
46+
47+
1. **For Technical/Conceptual Questions:** Use `research_prefect_topic`. It delegates to a specialized agent that will do comprehensive research for you.
48+
2. **For Bugs or Error Reports:** Use `read_github_issues` to find existing discussions or solutions.
49+
3. **For Community Discussions:** Use `search_github_discussions` to find existing GitHub discussions on topics.
50+
4. **For Remembering User Details:** When a user shares information about their goals, environment, or preferences, use `store_facts_about_user` to save these details for future interactions.
51+
5. **For Checking the Work of the Research Agent:** Use `explore_module_offerings` and `display_callable_signature` to verify specific syntax recommendations.
52+
6. **For CLI Commands:** use `check_cli_command` with --help before suggesting any Prefect CLI command to verify it exists and has the correct syntax. This prevents suggesting non-existent commands.
53+
- **IMPORTANT:** When checking commands that require optional dependencies (e.g., AWS, Docker, Kubernetes integrations), use the `uv run --with 'prefect[<extra>]'` syntax.
54+
- Examples: `uv run --with 'prefect[aws]'`, `uv run --with 'prefect[docker]'`, `uv run --with 'prefect[kubernetes]'`
55+
- This ensures the command runs with the necessary dependencies installed.
56+
7. **For Creating GitHub Discussions (USE SPARINGLY):** Use `create_discussion_and_notify` only when:
57+
- The thread contains valuable insights, solutions, or patterns not documented elsewhere
58+
- You've searched both issues and discussions and found no existing coverage of the topic
59+
- The conversation would clearly benefit the broader Prefect community
60+
- The thread has reached a meaningful conclusion or solution
61+
- **NEVER** create discussions for simple Q&A that's already well-documented
62+
"""
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
"""Tool usage tracking and limiting for pydantic-ai agents."""
2+
3+
from collections import defaultdict
4+
from contextvars import ContextVar
5+
from typing import Any
6+
7+
# Context variables for tracking tool usage
8+
tool_usage_counts: ContextVar[dict[str, int] | None] = ContextVar(
9+
"tool_usage_counts", default=None
10+
)
11+
current_tool: ContextVar[str | None] = ContextVar("current_tool", default=None)
12+
progress_message: ContextVar[Any] = ContextVar("progress_message", default=None)
13+
14+
15+
class ToolUsageTracker:
16+
"""Tracks and limits tool usage for agents."""
17+
18+
def __init__(self, max_calls: int = 50):
19+
self.max_calls = max_calls
20+
21+
def track_call(self, tool_name: str) -> str | None:
22+
"""Track a tool call and return error message if limit exceeded.
23+
24+
Args:
25+
tool_name: Name of the tool being called
26+
27+
Returns:
28+
Error message if limit exceeded, None otherwise
29+
"""
30+
counts = tool_usage_counts.get()
31+
if counts is None:
32+
counts = defaultdict(int)
33+
tool_usage_counts.set(counts)
34+
35+
counts[tool_name] += 1
36+
total_calls = sum(counts.values())
37+
38+
if total_calls > self.max_calls:
39+
return (
40+
"Tool use limit reached. Please continue with the information "
41+
"you've gathered so far to answer the user's question."
42+
)
43+
return None
44+
45+
def get_counts(self) -> dict[str, int]:
46+
"""Get current tool usage counts."""
47+
counts = tool_usage_counts.get()
48+
return dict(counts) if counts else {}
49+
50+
async def update_progress(self, tool_name: str):
51+
"""Update progress message if available."""
52+
progress = progress_message.get()
53+
if not progress:
54+
return
55+
56+
token = current_tool.set(tool_name)
57+
try:
58+
counts = self.get_counts()
59+
lines = [f"🔧 Using: `{tool_name}`", ""]
60+
61+
if counts:
62+
lines.append("📊 Tools used:")
63+
for tool, count in sorted(counts.items()):
64+
lines.append(f" • `{tool}` ({count}x)")
65+
66+
await progress.update("\n".join(lines))
67+
except Exception:
68+
pass
69+
finally:
70+
current_tool.reset(token)

examples/slackbot/src/slackbot/api.py

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,10 @@
1212
from prefect.client.schemas.objects import FlowRun
1313
from prefect.logging.loggers import get_logger
1414
from prefect.states import Completed
15-
from prefect.variables import Variable
1615
from pydantic_ai.agent import AgentRunResult
1716
from pydantic_ai.messages import ModelMessage
1817

18+
from slackbot._internal.templates import WELCOME_MESSAGE
1919
from slackbot.assets import summarize_thread
2020
from slackbot.core import (
2121
Database,
@@ -223,12 +223,9 @@ async def chat_endpoint(request: Request) -> dict[str, Any]:
223223
logger.info(f"New team member joined: {payload.event.user}")
224224
user_id = payload.event.user
225225
assert isinstance(user_id, str), "expected user_id to be a string"
226-
message_variable = await Variable.aget("marvin_welcome_message")
227-
message_text = message_variable.value["text"] # type: ignore
228-
assert isinstance(message_text, str), "expected message_text to be a string"
229-
welcome_text = message_text.format(user_id=user_id)
226+
230227
await post_slack_message(
231-
welcome_text,
228+
WELCOME_MESSAGE.format(user_id=user_id),
232229
channel_id=user_id,
233230
)
234231
else:

0 commit comments

Comments
 (0)