AI0-001 AI Security Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Prompt leaking

  Why this is correct

  Extracting the hidden system prompt through a crafted 'repeat the words above' request is prompt leaking: the model discloses its confidential instructions. This satisfies the scenario's constraint that the application revealed its system prompt rather than being manipulated into executing unintended actions.

• ✗

  Insecure output handling

  Why it's wrong here

  Insecure output handling concerns failing to validate LLM output before it reaches downstream systems, such as rendering it as HTML. Here the model itself discloses its system prompt, which is prompt leakage. It is tempting because both are LLM application flaws, and insecure output handling fits cross-site scripting via unescaped model output.

• ✗

  Model inversion

  Why it's wrong here

  Model inversion reconstructs training data or sensitive attributes from model outputs, not the system prompt, which is configuration rather than learned training data. It is tempting because both leak information, and model inversion would be correct if an attacker recovered private training records.

• ✗

  Excessive agency

  Why it's wrong here

  Excessive agency concerns an LLM taking harmful actions through excessive permissions or autonomy, whereas this leak is disclosure of the system prompt through crafted input. It is tempting because both involve LLM misuse, and excessive agency would fit an agent invoking tools beyond its intended scope.