NCA-GENL Trustworthy AI Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Fine-tune the model on a dataset of adversarial prompts to teach it to ignore malicious instructions.

  Why it's wrong here

  Fine-tuning on adversarial prompts can improve robustness but is not a complete solution and requires significant effort. Attackers constantly develop new injection techniques, so a static fine-tuned defense may be bypassed. Moreover, fine-tuning is a form of retraining and may not be feasible for a deployed system; it is not a runtime mitigation like input or output rails.

• ✗

  Increase the model's temperature to make its responses less predictable and harder for attackers to exploit.

  Why it's wrong here

  Higher temperature increases randomness, which can make the model more susceptible to generating unexpected and potentially harmful content. It does not prevent prompt injection; in fact, it could make the model's behavior less stable and harder to control. This measure does not address the attack vector and may worsen security.

• ✗

  Restrict the model's access to the internet to prevent it from fetching external malicious content.

  Why it's wrong here

  Restricting internet access prevents the model from retrieving external content, but prompt injection attacks often come through user input or uploaded documents, not the internet. In a document summarization system, the malicious instructions could be embedded in the contract text itself. Therefore, this measure does not address the primary attack vector.

• ✓

  Implement output filtering that redacts any confidential legal terms or entity names before displaying the summary.

  Why this is correct

  Output filtering acts as a safety net: even if a prompt injection succeeds in manipulating the model, the filter scans the generated summary for sensitive legal terms or entity names and redacts them. This ensures that confidential information does not reach the user, providing defense in depth against injection attacks.

• ✓

  Use an input rail to classify and block prompts that contain instructions attempting to override system directives.

  Why this is correct

  An input rail can analyze user prompts for patterns indicative of prompt injection, such as phrases like 'ignore previous instructions' or attempts to exfiltrate the system prompt. By blocking such prompts before they reach the model, the rail prevents the attack from influencing the model's behavior, directly mitigating the risk of confidential data leakage.