-
Notifications
You must be signed in to change notification settings - Fork 36
Expand file tree
/
Copy pathquiz.json
More file actions
90 lines (90 loc) · 3.04 KB
/
Copy pathquiz.json
File metadata and controls
90 lines (90 loc) · 3.04 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
{
"lesson": "10-multi-agent-software-team",
"title": "毕业项目 10 —— 多 agent 软件工程团队",
"questions": [
{
"stage": "pre",
"question": "为什么即便有 200k token 上下文,单 agent harness 在大型任务上仍会触及天花板?",
"options": [
"单个 agent 无法进行工具调用",
"沙箱拒绝托管一个以上的进程",
"上下文无法同时容纳架构计划、四份并行的代码库切片、评审者评论和测试输出",
"模型缺乏对 git 的支持"
],
"correct": 2,
"explanation": ""
},
{
"stage": "pre",
"question": "在这种工厂形态中,架构师(architect)角色负责什么?",
"options": [
"审查合并后的 diff 是否有幻觉",
"阅读 issue 并产出一份带有显式接口的子任务计划",
"在干净的沙箱中运行测试套件",
"当编码者意见不一致时强制合并分支"
],
"correct": 1,
"explanation": ""
},
{
"stage": "check",
"question": "为什么每个编码者要在自己的 git worktree 加上一个 Daytona 沙箱中工作?",
"options": [
"worktree 比分支克隆更快",
"Daytona 是唯一支持 SSH 的沙箱",
"隔离的工作树让 N 个编码者并行实现子任务,而不会互相踩踏共享文件",
"架构师需要 worktree 的 ID"
],
"correct": 2,
"explanation": ""
},
{
"stage": "check",
"question": "为保持评审者诚实,对它施加了什么约束?",
"options": [
"它只能在测试者签字之后运行",
"它不能批准自己编写或提议的 diff",
"它必须批准它读到的每一个 diff",
"它必须与架构师是同一个模型"
],
"correct": 1,
"explanation": ""
},
{
"stage": "check",
"question": "为什么细则要把 token 效率与单 agent 基线作权衡?",
"options": [
"多 agent 总是更便宜",
"角色边界增加了摘要和交接上下文,因此真正的问题是这个工厂是否按每美元胜出",
"单 agent 没有可观测性方案",
"token 成本与性能无关"
],
"correct": 1,
"explanation": ""
},
{
"stage": "post",
"question": "复盘期间,从每个失败的 issue 中产出哪种产物?",
"options": [
"一个新的架构师计划模板",
"一份更新的 A2A 协议规范",
"一份强制推送日志",
"一张交接失败直方图,指明是哪个角色边界出了问题(计划、合并、评审、测试)"
],
"correct": 3,
"explanation": ""
},
{
"stage": "post",
"question": "本毕业项目用哪个评估集来对各角色评分 pass@1?",
"options": [
"SWE-bench Pro 50 个 issue 的子集",
"HumanEval-Java",
"MT-Bench-v2",
"MigrationBench"
],
"correct": 0,
"explanation": ""
}
]
}