AAIA AI Operations Practice Test 4 — 15 Questions

Question 1

Which of the following is a key component of 'MLOps Maturity'?

Accepted Answer

Automated pipeline triggers. High maturity involves full CI/CD, automated testing, and active model monitoring.

Answer

Manual data entry

Answer

Single engineer silo

Answer

No documentation

Question 2

When deploying an AI model on a public cloud, what is the best practice for 'Secret Management' of API keys?

Accepted Answer

Using a dedicated Secret Vault service. Secrets must be stored in a dedicated vault (e.g., AWS Secrets Manager, Azure Key Vault) and injected at runtime.

Answer

Printing to logs

Answer

Hardcoding in the script

Answer

Storing in a Git repo

Question 3

In the context of 'AI Pipelines', what is the role of a 'Model Registry'?

Accepted Answer

Version control for model artifacts. The Model Registry provides a central repository to store, version, and manage the lifecycle of trained models.

Answer

Storing training logs

Answer

Monitoring compute costs

Answer

Optimizing SQL queries

Question 4

Which of the following is an example of an 'operational metric' for an AI system?

Accepted Answer

Request latency. Operational metrics (system health) include latency, error rate, and throughput.

Answer

F1 Score

Answer

ROC AUC

Answer

Training accuracy

Question 5

Why must you conduct a 'Bias Audit' on a model that has been fine-tuned on new data?

Accepted Answer

Fine-tuning can introduce new biases. Fine-tuning updates a model's weights on new data, and if that data reflects historical or sampling bias, the model will absorb and potentially amplify it. A bias audit after fine-tuning verifies that the update did not introduce or worsen disparate outcomes across protected groups. This is a governance and fairness requirement, not a performance optimization.

Answer

It speeds up inference

Answer

It fixes hardware bugs

Answer

It reduces model size

Question 6

During an audit of an LLM pipeline, you discover that 'System Prompt' versioning is missing. What is the operational risk?

Accepted Answer

Unpredictable model behavior changes. Without system prompt versioning, changes to the instructions can cause unpredictable behavior changes that cannot be reverted.

Answer

Higher cloud bills

Answer

Increased latency

Answer

Database corruption

Question 7

Which TWO are standard practices for securing AI model endpoints?

Accepted Answer

Mutual TLS (mTLS). Mutual TLS (mTLS) is correct because it enforces bidirectional certificate-based authentication, ensuring both the client and the AI model endpoint verify each other's identity before any inference request is processed, which prevents unauthorized or spoofed clients from reaching the model. API Rate Limiting is correct because it caps the number of requests per client or API key over a defined window, mitigating abuse, credential-stuffing, denial-of-service, and cost-exhaustion attacks against expensive model inference calls. Unprotected endpoints do not belong because exposing a model without authentication or encryption allows anyone to invoke it and exfiltrate data or consume resources. Storing logs on public S3 does not belong because publicly readable buckets leak prompts, responses, and potentially sensitive training or user data. Public internet access does not belong because unrestricted exposure of a model endpoint dramatically expands the attack surface instead of restricting access to trusted networks or identities.

Answer

Unprotected endpoints

Answer

Storing logs on public S3

Answer

Public internet access

Question 8

Which THREE documents are typically required for an AI Audit trail?

Accepted Answer

Model training logs. Model training logs (A) are required because they document the datasets, hyperparameters, training runs, and timestamps that establish how a model was built, which is essential for reproducibility and traceability in an AI audit trail. Model validation reports (B) are required because they record performance metrics, test results, bias/fairness evaluations, and sign-offs that demonstrate the model was assessed against acceptance criteria before use. Deployment approval records (C) are required because they provide the governance evidence of who authorized the release, when, and under what conditions, completing the chain of accountability from development to production. The remaining options do not belong: an employee vacation calendar (D) and cafeteria lunch receipts (E) are unrelated administrative records with no bearing on model provenance, validation, or deployment authorization.

Answer

Employee vacation calendar

Answer

Cafeteria lunch receipts

Question 9

What is the primary function of a 'Version Control System' (like Git) in an MLOps team?

Accepted Answer

Managing code changes. To manage code changes, enable collaboration, and allow rollbacks.

Answer

Executing Python scripts

Answer

Storing raw video

Answer

Managing databases

Question 10

What is the role of an 'A/B Test' in AI production?

Accepted Answer

Comparing two models in production. A/B testing allows comparing the performance of a new model against an existing one with real user traffic.

Answer

Encrypting data

Answer

Reducing training time

Answer

Deleting unused models

Question 11

What is the purpose of 'Containerization' (e.g., Docker) in MLOps?

Accepted Answer

Ensuring environment consistency. Ensures the environment (dependencies, OS) is consistent across development, testing, and production.

Answer

Encrypting the model

Answer

Reducing data size

Answer

Increasing model accuracy

Question 12

In SageMaker, what does 'Model Monitor' specifically detect?

Accepted Answer

Data and model drift. It monitors for data drift and model quality degradation in production.

Answer

Database uptime

Answer

Model size

Answer

Employee attendance

Question 13

An auditor finds that a model's 'input feature importance' has changed significantly after a retrain. What is the most appropriate action?

Accepted Answer

Investigate source data and feature logic. Investigate if the change is due to a shift in data or a feature engineering error.

Answer

Ignore the change

Answer

Delete the model

Answer

Upgrade the hardware

Question 14

Which TWO are common types of 'AI Model Drift'?

Accepted Answer

Concept drift. Concept drift (A) is a correct answer because it describes the situation where the statistical relationship between input features and the target variable changes over time, so a model's learned mapping becomes stale and its predictions degrade. Data drift (B) is also correct because it refers to a change in the distribution of the input features (also called covariate shift) relative to the training data, which is one of the most common forms of model drift monitored in production ML systems. The remaining options are not recognized categories of AI model drift: temperature drift (C) is not an ML drift type (temperature is a sampling parameter in LLMs, not a drift class), while CPU drift (D) and memory drift (E) are hardware/infrastructure phenomena unrelated to changes in data or concept distributions.

Answer

Temperature drift

Answer

CPU drift

Answer

Memory drift

Question 15

Which THREE tasks are required for a 'Model Decommissioning' workflow?

Accepted Answer

Terminate serving infrastructure. Decommissioning requires archival, notification, and resource cleanup.

Answer

Ask developer for coffee

Answer

Delete all company records