AI0-001 AI Infrastructure and Technologies — 20 Questions

Question 1

An organization uses Azure Machine Learning to manage the ML lifecycle. They want to automatically retrain a model when new data arrives in Azure Blob Storage. Which Azure service should they integrate with Azure ML to trigger retraining?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Azure Event Grid. Azure Event Grid is the correct service because it provides a native event-driven architecture that can react to Blob Storage events (e.g., BlobCreated) and route them directly to Azure Machine Learning workspaces via webhooks or event subscriptions. This allows automatic retraining pipelines to be triggered as soon as new data lands in the storage container, without polling or custom code.

Answer

Azure Logic Apps

Answer

Azure Data Factory

Answer

Azure Functions

Question 2

A financial services company needs to deploy an ML model for loan approval that must be explainable to regulators. The model is a gradient boosting ensemble. They need to track experiments, log model parameters, and serve the model with explanations. Which THREE tools from the MLOps ecosystem should they use?

Accepted Answer

Weights & Biases. Weights & Biases (W&B) is correct because it provides experiment tracking, hyperparameter logging, and model versioning, which are essential for the regulatory requirement of explainability and auditability. It integrates directly with gradient boosting frameworks like XGBoost and LightGBM to log parameters and metrics, enabling reproducible ML pipelines.

Answer

Apache Kafka

Answer

Docker Compose

Question 3

A data scientist wants to build a proof-of-concept chatbot using a large language model. They need to choose a cloud AI platform that provides easy access to pre-trained models via API, with built-in safety filters and prompt engineering tools. Which TWO platforms are best suited?

Accepted Answer

Azure OpenAI Service. Azure OpenAI Service (B) is correct because it provides direct API access to pre-trained models like GPT-4 with built-in content safety filters (e.g., Azure AI Content Safety) and integrated prompt engineering tools (e.g., Prompt Flow in Azure Machine Learning). This makes it ideal for quickly building a proof-of-concept chatbot with safety guardrails.

Answer

Amazon Bedrock

Answer

Amazon SageMaker

Answer

Hugging Face Hub

Question 4

A team wants to deploy a large language model on edge devices with limited memory and compute. They need to reduce model size by at least 50% while preserving accuracy. Which combination of techniques is most effective?

Accepted Answer

Apply INT8 quantization and weight pruning. INT8 quantization reduces the precision of weights and activations from 32-bit to 8-bit, cutting memory usage by approximately 75% for those tensors, while weight pruning removes redundant connections, often achieving over 50% size reduction with minimal accuracy loss when combined. Together, they directly address the constraints of edge devices by shrinking the model footprint and computational requirements without requiring a complete architecture redesign.

Answer

Distill the model into a smaller architecture without quantization or pruning

Answer

Use FP32 precision and increase batch size

Answer

Use FP16 quantization and add more layers

Question 5

A company uses a vector database to store embeddings for a RAG application. Users report that some queries return irrelevant results. Which adjustment is most likely to improve relevance?

Accepted Answer

Change the similarity metric from cosine to dot product and use a different embedding model. Switching from cosine similarity to dot product and using a different embedding model can improve relevance because the choice of similarity metric must align with the embedding model's training objective. Many modern embedding models (e.g., text-embedding-ada-002) are optimized for dot product or cosine similarity, but if the current model was trained for cosine and the queries are not normalized, dot product may better capture magnitude and direction. A different model may also produce higher-quality embeddings that better represent semantic relationships, directly addressing irrelevant results.

Answer

Reduce the chunk size of documents

Answer

Switch from an HNSW index to a flat index

Answer

Increase the top-k retrieval count

Question 6

Which AWS service would a developer use to integrate a pre-built foundation model into an application via API, without managing underlying infrastructure?

Accepted Answer

Amazon Bedrock. Amazon Bedrock is a fully managed service that provides access to pre-built foundation models (FMs) from providers like AI21 Labs, Anthropic, Cohere, Meta, and Stability AI via a unified API. It eliminates the need to manage underlying infrastructure such as servers, GPUs, or scaling, allowing developers to integrate FMs into applications with minimal operational overhead.

Answer

Amazon EC2

Answer

Amazon SageMaker

Answer

AWS Lambda

Question 7

An organization must ensure that an AI model deployed on an IoT device meets stringent latency requirements. The model is currently in FP32 and runs at 200ms per inference on the device; the target is 50ms. Which technique will provide the greatest latency reduction with the least accuracy loss?

Accepted Answer

Quantize the model to INT8. Quantizing the model from FP32 to INT8 reduces the precision of weights and activations, which directly decreases memory bandwidth and computational load. On IoT devices with limited resources, this typically yields a 2-4x speedup, bringing the 200ms inference time close to the 50ms target, while INT8 quantization often retains over 90% of the original accuracy when using calibration techniques.

Answer

Apply weight pruning to remove 50% of parameters

Answer

Switch from TensorFlow Lite to Core ML

Answer

Distill the model into a smaller architecture

Question 8

Which open-source framework is commonly used for building, training, and deploying machine learning models and provides high-level APIs like Keras?

Accepted Answer

TensorFlow. TensorFlow is the correct answer because it is the open-source framework that provides high-level APIs like Keras for building, training, and deploying machine learning models. Keras, now integrated as tf.keras, offers a user-friendly interface for rapid prototyping while TensorFlow handles the underlying computation graph, distributed training, and model serving via TensorFlow Serving.

Answer

Hugging Face Transformers

Answer

scikit-learn

Answer

PyTorch

Question 9

A machine learning engineer needs to deploy a PyTorch model for real-time inference with low latency. The model uses custom operators that are not supported by standard ONNX conversion. Which deployment approach is MOST appropriate?

Accepted Answer

Deploy the model using TorchServe with a custom handler. TorchServe is the native serving solution for PyTorch models and supports custom operators through custom handlers, allowing you to implement arbitrary preprocessing, inference, and postprocessing logic in Python. This approach avoids the need for ONNX conversion entirely, which is critical when custom operators are not supported by the ONNX standard. It also provides built-in features like model versioning, batching, and metrics for low-latency real-time inference.

Answer

Use TensorFlow Serving with a saved model format

Answer

Wrap the model in a Flask app and deploy on a VM

Answer

Convert the model to ONNX and serve with ONNX Runtime

Question 10

A team is building a retrieval-augmented generation (RAG) pipeline. They need to store embeddings of company documents and perform fast similarity searches. Which data store is BEST suited for this task?

Accepted Answer

Pinecone. Pinecone is a purpose-built vector database designed for storing and querying high-dimensional embeddings with fast approximate nearest neighbor (ANN) search. In a RAG pipeline, embeddings of company documents must be retrieved quickly to feed relevant context to the LLM, and Pinecone’s optimized indexing (e.g., HNSW or IVF) and serverless scaling make it the ideal choice for this task.

Answer

Snowflake

Answer

Apache Kafka

Answer

Amazon S3

Question 11

A team is deploying a model on a cloud-based ML service and needs to handle variable traffic patterns with automatic scaling based on request latency. They want to minimize costs during low traffic. Which endpoint configuration should they use?

Accepted Answer

Configure automatic scaling with a target tracking metric based on latency. The ML service's automatic scaling with a target tracking metric based on latency allows the endpoint to dynamically adjust the number of instances in response to real-time request latency, ensuring cost efficiency during low traffic while maintaining performance during spikes. This approach uses a predefined or custom metric (e.g., a metric like invocations per instance) to trigger scaling policies, minimizing over-provisioning and idle costs.

Answer

Use a serverless function with the ML service

Answer

Provision a fixed number of instances with multi-model endpoints

Answer

Use a single large instance to handle peak traffic

Question 12

A data scientist is deploying a model on edge devices using TensorFlow Lite. The model currently uses FP32 precision. Which TWO techniques can reduce the model size and improve inference speed without significant accuracy loss? (Choose TWO.)

Accepted Answer

Post-training quantization to INT8. Post-training quantization to INT8 reduces model size by converting FP32 weights and activations to 8-bit integers, which also speeds up inference on edge devices by leveraging integer-optimized hardware. This technique typically preserves accuracy within 1–2% of the original FP32 model, making it suitable for deployment on resource-constrained devices.

Answer

Use a larger batch size during inference

Answer

Increase the number of layers

Answer

Convert to FP16 precision

Question 13

A company is building a multi-modal AI application that processes text, images, and audio. They need a unified platform to store embeddings for all modalities, perform hybrid search (vector + metadata filtering), and scale to millions of vectors. Which THREE services are suitable for this purpose? (Choose THREE.)

Accepted Answer

Weaviate. Weaviate is a purpose-built vector database that natively supports multi-modal embeddings (text, images, audio) through its vectorizer modules and provides hybrid search combining vector similarity with metadata filtering (e.g., using GraphQL or REST APIs). It is designed to scale to millions of vectors with built-in sharding and replication, making it a strong fit for the described unified platform.

Answer

Amazon S3

Answer

Snowflake

Question 14

A team is using Kubeflow to orchestrate ML workflows on Kubernetes. They need to ensure reproducibility, track experiments, and share models across the organization. Which THREE components or tools should they integrate? (Choose THREE.)

Accepted Answer

Kubeflow Pipelines. Kubeflow Pipelines is a core component of Kubeflow that enables the definition, deployment, and management of end-to-end ML workflows on Kubernetes. It provides a platform for building reproducible pipelines by capturing the entire workflow as a directed acyclic graph (DAG) of containerized steps, ensuring that each run can be exactly recreated. This directly addresses the team's need for reproducibility and orchestration within their existing Kubernetes environment.

Answer

Apache Airflow

Answer

Weights & Biases

Question 15

A data science team is deploying a deep learning model for real-time inference on edge devices with limited power and memory. Which model optimisation technique would be MOST effective for reducing latency and memory footprint while maintaining acceptable accuracy?

Accepted Answer

Apply quantisation to convert weights from FP32 to INT8. Quantization reduces the precision of model weights from 32-bit floating point (FP32) to 8-bit integer (INT8), which directly cuts memory usage by 75% and accelerates inference on edge devices by leveraging integer arithmetic. This technique is specifically designed for resource-constrained environments where power and memory are limited, and it typically preserves accuracy within 1-2% of the original model.

Answer

Use a larger batch size during inference

Answer

Train the model for more epochs to improve convergence

Answer

Increase the number of layers to improve feature extraction

Question 16

A company uses AWS SageMaker to train a large language model. The training job fails with an out-of-memory error. The team is already using the largest available GPU instance. Which step should the team take to resolve the issue without modifying the model architecture?

Accepted Answer

Enable gradient accumulation in the training script. Gradient accumulation allows the model to simulate a larger batch size by accumulating gradients over several forward/backward passes before performing an optimizer step. This reduces per-step memory usage because the gradients are stored and averaged rather than requiring the entire batch to be loaded into GPU memory at once. Since the team cannot change the instance type or model architecture, enabling gradient accumulation is the correct approach to resolve the out-of-memory error.

Answer

Increase the learning rate to converge faster

Answer

Switch to a CPU-based instance

Answer

Reduce the number of attention heads in the model

Question 17

A financial institution requires that all AI model predictions be explainable and auditable for regulatory compliance. Which model serving approach should be used to meet these requirements?

Accepted Answer

Deploy the model as a containerised microservice with REST API and log all request/response pairs. Logging all request/response pairs provides a complete audit trail, which is essential for regulatory compliance in financial institutions. Containerized microservices with REST APIs are stateless and can be easily integrated with centralized logging systems (e.g., ELK stack) to capture every prediction for explainability and review. This approach ensures that model decisions are transparent and can be traced back to specific inputs, satisfying both explainability and auditability requirements.

Answer

Use gRPC streaming for lower latency

Answer

Export the model to ONNX format and run on a dedicated inference server

Answer

Deploy the model on edge devices to avoid centralised logging

Question 18

A company is implementing a retrieval-augmented generation (RAG) pipeline using a vector database. They notice that the retrieved documents often lack relevance to the query. Which adjustment would MOST improve retrieval quality?

Accepted Answer

Use a better embedding model fine-tuned on domain-specific data. Retrieval quality in a RAG pipeline is fundamentally determined by the semantic alignment between query embeddings and document embeddings. A domain-specific fine-tuned embedding model captures the unique terminology, context, and relationships within the company's data, producing vector representations that are far more relevant than those from a generic model. This directly improves the similarity search results in the vector database, leading to higher-quality retrieved documents.

Answer

Increase the chunk size of documents

Answer

Switch from cosine similarity to Euclidean distance

Answer

Reduce the number of retrieved documents from 5 to 3

Question 19

A team is using Hugging Face Transformers to serve an LLM via a REST API. They notice high latency during inference. The model is deployed on a single GPU. Which optimisation would reduce inference latency WITHOUT changing the model architecture?

Accepted Answer

Use model quantisation to FP16. FP16 quantization reduces the memory footprint and computational load of the model by using half-precision floating-point numbers, which allows the GPU to process more operations per second and reduces memory bandwidth usage. This directly lowers inference latency without altering the model's architecture, making it the correct choice for a single-GPU deployment.

Answer

Add more GPUs and distribute the model

Answer

Increase the batch size to process multiple requests simultaneously

Answer

Switch from GPU to CPU

Question 20

A healthcare AI startup must store and query high-dimensional embeddings of medical records for a RAG system. They need low-latency similarity search at scale. Which database should they choose?

Accepted Answer

Pinecone. Pinecone is a fully managed vector database optimized for high-dimensional embeddings and low-latency similarity search at scale. It provides built-in indexing (e.g., HNSW), automatic sharding, and serverless scaling, making it ideal for RAG systems that require fast approximate nearest neighbor (ANN) queries on medical record embeddings.

Answer

Amazon S3

Answer

pgvector

Answer

BigQuery