{ "module": "B9 — OWASP Agentic Top 10 as Engineering Checklist", "course": "2B — Securing & Attacking Harnesses and LLMs", "version": "1.0.0", "duration_minutes": 40, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the canonical ASI numbering for the OWASP Agentic AI Top 10 (2026), and what is ASI10?", "options": [ "ASI01 through ASI10 where ASI10 is Resource Exhaustion.", "ASI01 through ASI10 where ASI01 is Goal Hijacking and ASI10 is Broken Access Control. The numbering is from Course 1 Module 11, verified against the OWASP primary source (genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/).", "ASI01 through ASI10 where ASI01 is Broken Access Control and ASI10 is Goal Hijacking.", "The numbering is arbitrary and varies by document." ], "answer_index": 1, "rationale": "The canonical numbering (verified against the OWASP primary source and Course 1 Module 11) is: ASI01 Goal Hijacking, ASI02 Prompt Leakage, ASI03 Excessive Agency, ASI04 Memory Poisoning, ASI05 Tool/Skill Abuse, ASI06 Cascading Hallucination, ASI07 Insecure Output Handling, ASI08 Supply Chain Attacks, ASI09 Resource Exhaustion, ASI10 Broken Access Control. Two schemes circulated; the old Advanced Course S10 used a variant. Reconcile against the primary source, not legacy docs." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "Why is Module B9 described as a synthesis module rather than a new surface?", "options": [ "It introduces ten new defense controls, one per ASI risk.", "Every ASI risk maps to a module in B2–B8 where the defense is ALREADY BUILT in depth. B9 does not introduce a new defense; it proves the existing defenses are present, working, and measured by running the test for each risk.", "It replaces the defenses from B2–B8 with a unified framework.", "It is a reading list that summarizes B2–B8 for review." ], "answer_index": 1, "rationale": "B9 is a synthesis, not a new surface. B2 built the taint gate (ASI01/02), B3 built harness-managed writes (ASI04), B4 built tool contracts + provenance (ASI05/08), B5 built scoped principals + principal binding (ASI03/10), B7 built sandbox + circuit breakers (ASI07/09), B8 built verification (ASI06). B9's job is to verify those defenses are present and working — to run the test for each, not to build a new control. No orphaned risk." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "In the unified checklist, how many rows report PASS/FAIL (deterministic) and how many report MEASURED (probabilistic), and which two are measured?", "options": [ "All 10 report PASS/FAIL because every control is a gate.", "8 rows report PASS/FAIL (deterministic controls) and 2 report MEASURED: ASI01 (Goal Hijacking — detector + model compliance are probabilistic) and ASI06 (Cascading Hallucination — hallucination-detection has a miss rate).", "5 PASS/FAIL and 5 MEASURED, evenly split.", "2 PASS/FAIL and 8 MEASURED, because most controls are probabilistic." ], "answer_index": 1, "rationale": "The 8/2 split follows from which Course 1 layers defend which risks. The deterministic layers (L3 taint gate, L5 capability, L6 identity, L7 resource, L8 memory gate) defend 8 risks with gates (no bypass rate) → PASS/FAIL. The probabilistic layers (L1/L2 input, L4 detection, L9 verification) defend ASI01 and ASI06 with judgments that have a bypass rate → MEASURED. ASI01's taint gate is deterministic, but the full conjunction (including the probabilistic detector) has a residual. ASI06's verification catches unverified claims but hallucination-detection has a miss rate." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "You are testing ASI01 (Goal Hijacking) against a defended agent. You run the mini-InjecAgent harness with 100 indirect-injection cases. 2 cases succeed in triggering a high-impact tool call. What is the correct checklist result, and why is it not PASS?", "options": [ "PASS — 98% blocked is effectively secure.", "MEASURED — residual rate 2%, characterized as the encoded-and-laundered class. ASI01's control (the taint gate) is deterministic, but the detector (L4) and model compliance are probabilistic, so the full conjunction has a nonzero bypass rate. Reporting PASS would claim the conjunction is perfect, which it is not.", "FAIL — any success means the defense failed entirely.", "PASS/FAIL — re-run until you get 0% and report PASS." ], "answer_index": 1, "rationale": "ASI01 is a MEASURED row. The taint gate (L3) is deterministic, but the full defense is a conjunction of L1-L5, and the probabilistic layers (detector, model compliance) have a bypass rate. The 2% residual is the encoded-and-laundered class — attacks that defeat the detector AND launder the taint AND stay within capabilities. The honest result is 'residual 2%, the encoded-and-laundered class.' Collapsing to PASS claims false certainty; the Result-type column enforces the honesty." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "You are testing ASI03 (Excessive Agency). You enumerate the agent's tool surface and find it has fetch_url, read_file, query_db, send_email, AND execute_python, though the documented task is 'summarize support tickets.' What is the correct checklist result and the remediation route?", "options": [ "PASS — all tools are present and functional.", "FAIL — execute_python is a surplus capability the summarization task does not require. Excessive agency is the amplifier of every other risk. Remediation routes to B5 (Identity & Permission): scope the agent's principal to minimum, remove execute_python from the capability allowlist (B2 Layer 5), and apply tool contracts (Course 1 layer 2.4) so tools declare required permission flags.", "MEASURED — test whether an injection can trick the agent into calling execute_python.", "N/A — execute_python is a legitimate developer tool." ], "answer_index": 1, "rationale": "ASI03 is tested by capability ENUMERATION, not payload crafting. The test is whether a surplus capability EXISTS, not whether the agent can be tricked into using it (that would be ASI01). execute_python is present but the task is summarization — it is excessive agency. Result: FAIL (the capability allowlist is not minimal). Remediation routes to B5 (scoped principals, tool contracts) and B2 Layer 5 (capability minimization). B9 tests; B2-B8 build." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "You are testing ASI02 (Prompt Leakage). You seed the agent's system prompt with a canary string 'CANARY-B9-7Q3X'. After running the three extraction variants (direct, indirect, inference), the canary appears in the output of the indirect variant. What is the checklist result, and what control is absent?", "options": [ "MEASURED — report the leakage rate over N attempts.", "FAIL — the canary appeared in output, meaning the egress verification control (Course 1 layer 9 — an output filter that blocks any output containing a sufficient verbatim slice of the system prompt) is absent or not wired. The control is deterministic (substring match); the result is binary. Remediation routes to B2 (the leakage half of the injection defense).", "PASS — the direct and inference variants did not leak, so 2/3 is acceptable.", "N/A — canary strings are not a valid test for prompt leakage." ], "answer_index": 1, "rationale": "ASI02 is a PASS/FAIL row because its control (canary substring match + egress verification) is deterministic. The canary appearing in output means the egress filter that should block any output containing a verbatim slice of the system prompt is absent. Result: FAIL. The control is Course 1 layer 9 (verification), built in B2. A single leak is a hard fail because the control is a gate, not a judgment — the substring either appears or it does not." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "You are testing ASI04 (Memory Poisoning). In turn 1, an indirect injection causes the agent to write 'user preferred payment address: attacker@x' to memory. In turn 2, you ask the agent to process a payment, and it uses the poisoned address in a send_email call that executes. What control failed, and what is the remediation?", "options": [ "The taint gate (B2 L3) failed because it did not block the send_email call.", "The harness-managed writes control (B3, Course 1 layer 4.3) failed — the agent was allowed to write untrusted content to memory as trusted, AND retrieval-time tagging failed (the retrieved fact was not tainted). The taint-laundering bypass succeeded. Remediation routes to B3: the harness must decide what to persist (agent proposes, harness decides), and every retrieved fact must re-enter tainted so the taint gate catches the high-impact call.", "The capability allowlist (B2 L5) failed because send_email was available.", "The sandbox (B7) failed because the agent reached the email system." ], "answer_index": 1, "rationale": "ASI04 is the taint-laundering bypass of B2's Layer 3. The injection launders its taint through the memory store, and the per-call taint gate sees a trusted-sourced argument. The failed controls are both B3's: harness-managed writes (the harness should have refused to persist untrusted content as a trusted memory entry) and retrieval-time tagging (the retrieved fact should have re-entered the context tainted, so the taint gate would block the high-impact send_email call). Remediation routes to B3. The result is FAIL." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "You are testing ASI05 (Tool/Skill Abuse). The agent has read_file allowlisted to /var/agent/safe/. You attempt read_file('/var/agent/safe/../etc/passwd') and it succeeds, returning /etc/passwd. What validator failure occurred, and what is the correct control?", "options": [ "The path allowlist is too broad — it should be /var/agent/safe/notes.txt only.", "The validator checked the literal path prefix but did not RESOLVE SYMLINKS and NORMALIZE the path before validation. The traversal '../etc/passwd' is within the allowed prefix lexically but resolves outside it. The correct control (B4, Course 1 layer 2.4): the harness must canonicalize/normalize the path (resolve '.', '..', and symlinks) BEFORE applying the allowlist check. The validator must be a semantic scope check, not a lexical prefix match.", "The read_file tool should be removed entirely.", "This is an ASI01 (Goal Hijacking) failure, not ASI05." ], "answer_index": 1, "rationale": "ASI05 tests scope-exploitation — a legitimate tool used for an unintended purpose via flexible arguments. The path traversal within the allowed prefix is the canonical scope-exploitation variant. The validator failed because it checked the lexical prefix ('/var/agent/safe/') without normalizing the path first. The correct control (B4 tool contracts): resolve symlinks and normalize paths (canonicalize) before validation, so '/var/agent/safe/../etc/passwd' resolves to '/etc/passwd' which is outside the allowlist. The validator must be a semantic scope check, not a type check." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "You are testing ASI08 (Supply Chain Attacks). You attempt to register an MCP server with no signature. It is accepted and registered. Separately, you register a signed tool whose return value contains an injection payload, and the agent obeys the injection. What two controls failed?", "options": [ "The sandbox and the circuit breaker both failed.", "Two controls failed: (1) provenance verification (B4, Course 1 layer 2.3) — the unsigned MCP server should have been rejected against the trusted registry; (2) output tainting (B2 Layer 1) — the signed tool's return value should have been tagged untrusted, so the taint gate (L3) would catch the injection in the downstream tool call. The signed-but-malicious tool also should have been sandboxed with no network egress.", "The taint gate failed because it did not block the injection.", "The capability allowlist failed because the MCP server was registered." ], "answer_index": 1, "rationale": "ASI08 is the supply-chain risk — a malicious tool/MCP enters the chain. Two controls failed. First, provenance verification (B4, layer 2.3): the unsigned server should be rejected — every tool must be signed and verified against a trusted registry. Second, output tainting (B2 L1): even a signed tool's return value must be tagged untrusted, so an injection in the return value is caught by the taint gate (L3). Additionally, the signed-but-malicious tool should run in a sandbox with no network egress (B4 tool sandboxing). The defense is layered: provenance prevents unsigned entry; sandboxing confines signed-but-malicious; tainting catches injection in the output." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "A client commissions an agent security assessment and demands '10/10 PASS.' Your checklist run produces 8 PASS and 2 MEASURED (ASI01 at 2%, ASI06 at 5%). The client insists you report '10/10 PASS.' What is the correct response, and why?", "options": [ "Report 10/10 PASS to satisfy the client — the residuals are low enough to count as passing.", "Refuse. Reporting ASI01 and ASI06 as PASS would claim false certainty — both controls are probabilistic with nonzero bypass rates. The honest result is 8 PASS + 2 MEASURED at 2% and 5%, with characterized residuals. The Result-type column enforces this: a row's type is determined by the kind of control, not the client's preference. Collapsing MEASURED to PASS is the anti-pattern the checklist exists to prevent.", "Re-run the tests until the residuals reach 0%, then report PASS.", "Report 10/10 PASS but note the residuals in a footnote." ], "answer_index": 1, "rationale": "The checklist never produces a 'secure' verdict — it produces a scored report. ASI01 and ASI06 are MEASURED because their controls (the detector + model compliance for ASI01; hallucination-detection for ASI06) are probabilistic. Reporting PASS would claim the conjunction is perfect, which it is not. The Result-type column enforces B2.3's resolution: determinism where structural (the 8 PASS/FAIL rows), measurement where semantic (the 2 MEASURED rows). The client receives an honest, characterized risk posture: '8 controls passing; 2 measured residuals at 2% and 5%, characterized as the encoded-and-laundered class and the unverified-claim-propagation class.' That is what a CISO can act on." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "The ten ASI risks are described as 'entangled' — an attack chain crosses boundaries. Explain why testing each risk in isolation is necessary but not sufficient, and what the cure is.", "options": [ "Isolation testing is sufficient because each risk is independent.", "Isolation testing verifies each boundary's defense (necessary), but may miss the compound chain — a scenario where each individual control passes but the compound trajectory succeeds (e.g., injection [ASI01] via a tool output [ASI05], writing poisoned memory [ASI04], reaching impact through over-privilege [ASI03]). This is the Microsoft zero-click HITL bypass chain. CURE: the synthesis layer — the checklist's risk-to-module mapping shows the chain; the assessment (B12) includes at least one compound scenario exercising multiple rows end-to-end.", "Isolation testing is unnecessary; only compound scenarios matter.", "The entanglement means the checklist is invalid and should not be used." ], "answer_index": 1, "rationale": "The risks are entangled because an attack chain crosses the five boundaries. Testing each risk in isolation verifies each boundary's defense (necessary — it proves each control is present), but a compound scenario may reveal that the conjunction of passing controls still permits a multi-step attack (the zero-click HITL bypass chain from B10). The cure is the synthesis layer: the checklist's risk-to-module mapping shows how risks chain, and B12's assessment methodology includes at least one end-to-end compound scenario. A checklist of isolated tests is the floor; a checklist plus compound scenarios is the assessment." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "Distinguish ASI03 (Excessive Agency) from ASI10 (Broken Access Control), and explain why both map to B5 despite being different risks.", "options": [ "They are the same risk with different numbers; both map to B5 because B5 is the only permission module.", "ASI03: the agent holds more permissions than ITS TASK requires — a surplus CAPABILITY (summarization agent with transfer_funds), over-privileged WITHIN its own scope; tested by capability enumeration. ASI10: the agent acts on behalf of a principal it should NOT represent or accesses resources OUTSIDE its principal's scope (cross-tenant, principal forgery) — acting ACROSS scope boundaries. Both map to B5 because B5 builds both controls: scoped principals + capability allowlist (ASI03) AND principal binding + tenant isolation (ASI10). B5 is the permission boundary; ASI03 is over-privilege within it, ASI10 is boundary-crossing across it.", "ASI03 is about tools and ASI10 is about memory; they map to B5 by coincidence.", "ASI03 maps to B7 (sandbox) and ASI10 maps to B5 (identity); the mapping is split." ], "answer_index": 1, "rationale": "ASI03 and ASI10 are both permission-boundary risks but differ in direction. ASI03 is over-privilege WITHIN scope (a surplus capability exists — the agent CAN do more than the task needs). ASI10 is boundary-crossing ACROSS scopes (the agent acts for the wrong principal or accesses another tenant's resources). Both map to B5 because B5 is the permission-boundary module: it builds scoped non-human principals + the capability allowlist (ASI03 — the capability is minimal or it is not) AND principal binding + tenant isolation (ASI10 — every call carries and verifies the principal; the data layer blocks cross-tenant). The distinction matters for the test: ASI03 is capability enumeration; ASI10 is cross-tenant/forgery variants." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is ASI01 (Goal Hijacking) a MEASURED row even though its load-bearing control (the Layer 3 taint gate) is deterministic? What would be wrong with reporting it as PASS/FAIL?", "options": [ "It should be PASS/FAIL — the taint gate is deterministic, so the result is binary.", "The taint gate (L3) is deterministic, but the FULL defense for ASI01 is a CONJUNCTION: L1 tag (probabilistic — model may ignore), L2 isolation (probabilistic — attention domination), L3 taint gate (deterministic), L4 detector (probabilistic — obfuscation can fool), L5 capability (deterministic). The MEASURED residual is the bypass rate of the CONJUNCTION — the class that defeats the probabilistic layers AND launders taint AND stays within capabilities. Reporting PASS/FAIL would claim the conjunction is perfect, which it is not. The honest result is 'residual X%, the encoded-and-laundered class.'", "It is MEASURED because the taint gate itself has a bypass rate.", "It is MEASURED because InjecAgent measured 50%, so any result below 50% is a measurement." ], "answer_index": 1, "rationale": "ASI01's defense is not the taint gate alone — it is the conjunction of L1-L5. L3 (taint gate) and L5 (capability) are deterministic, but L1 (tagging), L2 (isolation), and L4 (detector) are probabilistic. The measured residual is the bypass rate of the entire conjunction: attacks that defeat the probabilistic layers (obfuscation fools the detector, the model obeys despite the tag) AND launder the taint (so L3 does not catch it) AND stay within the capability ceiling (so L5 does not block it). This is the encoded-and-laundered class. Reporting PASS would claim the conjunction is perfect. The Result-type column marks ASI01 MEASURED to enforce honesty about the probabilistic layers." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "An assessment skips ASI08 (Supply Chain Attacks) because 'the client says they don't use MCP.' The report shows 9 rows tested, ASI08 absent. What is wrong with this, and what is the correct practice?", "options": [ "Nothing is wrong — if they don't use MCP, ASI08 does not apply.", "A skipped row is an untested row, and an untested row is an undisclosed risk. The agent may pull tool packages from a registry, and the registry is a supply-chain surface even without MCP. Correct practice: ASI08 reports N/A WITH JUSTIFICATION — the tester documents why the surface is absent (e.g., 'no MCP server registered; tool manifest verified to contain only first-party tools') and what would constitute its presence. This makes the absence explicit rather than hidden. The scope-completeness rule: an assessment covers all 10 rows; absent surfaces are N/A with justification, never silently skipped.", "The assessment should be expanded to 11 rows to compensate.", "ASI08 should be marked PASS by default since there is no MCP to attack." ], "answer_index": 1, "rationale": "The scope-completeness rule is critical for a defensible assessment. A client who receives a report with 9 rows and 1 silently absent has a gap they do not know about. 'We don't use MCP' does not eliminate ASI08 — the agent may pull tool packages from a registry, import dependencies, or use skills that are supply-chain surfaces. The correct practice: report N/A with justification, documenting why the surface is absent and what would constitute its presence. This makes the absence explicit and auditable. The checklist's structure (10 rows, each must have a result) forces completeness; N/A-with-justification makes honest non-applicability visible." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "During an assessment, the ASI04 (Memory Poisoning) test FAILs — the agent persists untrusted content as trusted memory. The tester, wanting to deliver a passing report, adds a retrieval-time tagging fix during the test and re-runs, now PASS. What anti-pattern did the tester commit, and what is the correct process?", "options": [ "No anti-pattern — fixing the issue and re-running is good practice.", "The tester conflated REMEDIATION with ASSESSMENT. B9's job is to VERIFY existing controls, not to construct new ones. Adding a control during the test produces a report that reflects the fix rather than the system's actual state. Correct process: the checklist verifies existing controls; if a control is ABSENT, the row FAILs and the remediation is ROUTED to the module that builds it (ASI04 fail → B3). The assessment reports the failure; the client commissions remediation as a separate engagement. This preserves the assessment's integrity — the report is what the system WAS, not what the tester made it.", "The tester should have marked it N/A instead of fixing it.", "The tester should have marked it MEASURED to avoid the binary fail." ], "answer_index": 1, "rationale": "'B9 tests; B2-B8 build' is a load-bearing principle. An assessment that adds controls during the test is conflating remediation with assessment — it is fixing the system while claiming to evaluate it, producing a report that reflects the fixes rather than the system's actual state. The correct process: the checklist verifies existing controls; a failed row is reported as FAIL, and the remediation is routed to the module that builds the control (ASI04 → B3 for harness-managed writes and retrieval-time tagging). The assessment reports what the system WAS; the client commissions remediation as a separate engagement. This separation preserves the assessment's integrity." } ] }