CCAR-F Context and Reliability Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Send the same prompt to all available Claude model versions on every request and store the responses, then let a downstream voting service select the majority answer for the audit record.

  Why it's wrong here

  Querying every model version on each request multiplies cost and latency for a contract review workload and does not create reproducibility, because the set of versions itself changes over time. Majority voting across heterogeneous models also produces a synthesized output that no single model generated, which is harder to defend in an audit than a recorded deterministic request. It neither simplifies SDK logic nor cleanly adopts new improvements.

• ✗

  Pin the model to a dated snapshot identifier such as claude-3-5-sonnet-20240620 for all production traffic and treat any model upgrade as a code change reviewed by compliance.

  Why it's wrong here

  Pinning to a dated snapshot does give reproducibility, but it blocks the team's stated goal of adopting newer model improvements without redeploying code. Every upgrade becomes a code change, which contradicts the requirement. It also does not address avoiding provider-specific SDK logic, since the same Anthropic client code still constructs the request. The scenario explicitly wants both properties, and this choice sacrifices one of them.

• ✗

  Freeze the system prompt and temperature in a configuration file so that identical inputs always yield identical outputs, and rely on that determinism instead of recording the model identifier.

  Why it's wrong here

  Freezing prompt and temperature reduces variance but does not guarantee identical outputs, because sampling remains stochastic and, more importantly, the underlying model weights can change when an alias points to a new snapshot. Without recording the resolved model identifier, an auditor cannot reconstruct which model produced a past response. This approach also does not provide any mechanism for adopting newer improvements deliberately.

• ✓

  Route every request through a versioned internal gateway that records the exact model identifier and full request payload, and let the gateway map a logical alias such as claude-3-5-sonnet-latest to a concrete dated snapshot at request time.

  Why this is correct

  A versioned gateway gives the audit trail by persisting the resolved model identifier and full payload, satisfying reproducibility. Mapping a logical alias to a concrete dated snapshot at request time lets the team rotate the alias to a newer snapshot by configuration change rather than code change. Because the gateway owns provider-specific request construction, application code stays free of SDK-specific logic, matching all three stated constraints.