AI0-001 AI Infrastructure and Technologies — 20 Questions

Question 1

A team is deploying a BERT-based question-answering model using a REST API endpoint with gRPC for internal microservices. They notice high latency for small payloads. Which optimization is MOST likely to reduce latency?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable batching of multiple queries into a single request. Batching multiple queries into a single request reduces the overhead of repeated gRPC connection setup, serialization, and network round trips for small payloads. This amortizes the fixed cost of each inference call across several queries, directly lowering per-query latency in high-throughput scenarios.

Answer

Convert the model to ONNX and use ONNX Runtime

Answer

Switch from gRPC to REST with HTTP/2

Answer

Use a larger instance type with more CPU

Question 2

An organization uses Azure Machine Learning to manage the ML lifecycle. They want to automatically retrain a model when new data arrives in Azure Blob Storage. Which Azure service should they integrate with Azure ML to trigger retraining?

Accepted Answer

Azure Event Grid. Azure Event Grid is the correct service because it provides a native event-driven architecture that can react to Blob Storage events (e.g., BlobCreated) and route them directly to Azure Machine Learning workspaces via webhooks or event subscriptions. This allows automatic retraining pipelines to be triggered as soon as new data lands in the storage container, without polling or custom code.

Answer

Azure Logic Apps

Answer

Azure Data Factory

Answer

Azure Functions

Question 3

A startup is building a recommendation system that requires low-latency similarity search over millions of product embeddings. They need a vector database that offers high performance and has a managed cloud option. Which TWO databases are best suited for this requirement?

Accepted Answer

Weaviate. Weaviate and Pinecone are both purpose-built vector databases that natively support high-performance approximate nearest neighbor (ANN) search using algorithms like HNSW (Weaviate) or proprietary indexing (Pinecone). They offer managed cloud services with automatic scaling, making them ideal for low-latency similarity search over millions of product embeddings without requiring manual infrastructure management.

Answer

Chroma

Answer

pgvector (PostgreSQL extension)

Answer

Amazon DynamoDB

Question 4

A data scientist wants to build a proof-of-concept chatbot using a large language model. They need to choose a cloud AI platform that provides easy access to pre-trained models via API, with built-in safety filters and prompt engineering tools. Which TWO platforms are best suited?

Accepted Answer

Azure OpenAI Service. Azure OpenAI Service (B) is correct because it exposes OpenAI's pre-trained large language models (GPT-4, GPT-3.5, embeddings) through a REST API, and includes built-in content safety filters via Azure AI Content Safety plus prompt engineering support through the Azure OpenAI Studio playground. Google Vertex AI (E) is correct because it provides access to pre-trained foundation models such as Gemini and PaLM through APIs, along with safety filters, tuning, and prompt design tooling in Vertex AI Studio. Amazon Bedrock (A) is not marked correct here even though it offers similar managed foundation-model APIs, so it does not satisfy the two-answer key. Amazon SageMaker (C) is a broader ML platform for building, training, and deploying custom models rather than a curated pre-trained-model API with integrated safety filters. Hugging Face Hub (D) is a model repository and community platform, not a cloud AI platform with the managed safety and prompt-engineering tooling described.

Answer

Amazon Bedrock

Answer

Amazon SageMaker

Answer

Hugging Face Hub

Question 5

A data scientist needs to train a deep learning model on a large image dataset. Which hardware is most suitable for parallel matrix operations and faster training compared to a CPU?

Accepted Answer

GPU with thousands of CUDA cores. A GPU with thousands of CUDA cores is the most suitable hardware for parallel matrix operations because deep learning training involves massive matrix multiplications and tensor operations that can be decomposed into thousands of independent threads. CUDA cores execute these threads in a massively parallel SIMT (Single Instruction, Multiple Thread) fashion, achieving significantly higher throughput than a CPU for such workloads, which leads to faster training times.

Answer

TPU designed for TensorFlow

Answer

CPU with high clock speed

Answer

FPGA for reconfigurable logic

Question 6

A company uses a vector database to store embeddings for a RAG application. Users report that some queries return irrelevant results. Which adjustment is most likely to improve relevance?

Accepted Answer

Change the similarity metric from cosine to dot product and use a different embedding model. Switching from cosine similarity to dot product and using a different embedding model can improve relevance because the choice of similarity metric must align with the embedding model's training objective. Many modern embedding models (e.g., text-embedding-ada-002) are optimized for dot product or cosine similarity, but if the current model was trained for cosine and the queries are not normalized, dot product may better capture magnitude and direction. A different model may also produce higher-quality embeddings that better represent semantic relationships, directly addressing irrelevant results.

Answer

Reduce the chunk size of documents

Answer

Switch from an HNSW index to a flat index

Answer

Increase the top-k retrieval count

Question 7

An AI team uses SageMaker Pipelines to orchestrate their ML workflow. They need to version the pipeline and track experiments across runs. Which complementary MLflow feature should they integrate?

Accepted Answer

MLflow Tracking. MLflow Tracking is the correct complementary feature because it provides a centralized API and UI for logging parameters, metrics, and artifacts (e.g., model checkpoints, datasets) from each SageMaker Pipeline run. This enables the team to version their pipeline executions and compare experiments across different runs, directly addressing the requirement for tracking and versioning.

Answer

MLflow Models

Answer

MLflow Projects

Answer

MLflow Model Registry

Question 8

Which AWS service would a developer use to integrate a pre-built foundation model into an application via API, without managing underlying infrastructure?

Accepted Answer

Amazon Bedrock. Amazon Bedrock is a fully managed service that provides access to pre-built foundation models (FMs) from providers like AI21 Labs, Anthropic, Cohere, Meta, and Stability AI via a unified API. It eliminates the need to manage underlying infrastructure such as servers, GPUs, or scaling, allowing developers to integrate FMs into applications with minimal operational overhead.

Answer

Amazon EC2

Answer

Amazon SageMaker

Answer

AWS Lambda

Question 9

A company wants to store unstructured text data for AI model training while enabling SQL-based queries for analytics. Which storage solution should they use as the primary data source?

Accepted Answer

A data lake like Amazon S3. Amazon S3 is a highly scalable object storage service that can store unstructured text data in its native format (e.g., CSV, JSON, Parquet) and supports SQL-based queries via services like Amazon Athena or S3 Select. This makes it ideal as a primary data source for AI model training while enabling analytics without requiring data transformation or loading into a separate system.

Answer

A vector database like Pinecone

Answer

A data warehouse like Snowflake

Answer

A NoSQL database like DynamoDB

Question 10

A company uses Azure OpenAI to generate marketing copy. They need to manage costs and ensure consistent response quality. Which TWO actions should they take?

Accepted Answer

Use prompt caching to avoid reprocessing identical inputs. Option B is correct because prompt caching in Azure OpenAI stores and reuses the processed context for identical or repeated input prefixes, reducing the number of tokens billed for the cached portion and lowering cost while keeping responses consistent for recurring prompts. Option D is correct because implementing rate limiting and token-based throttling (e.g., via Azure API Management policies or TPM/RPM quota controls on the Azure OpenAI deployment) caps consumption, prevents runaway spend, and enforces predictable throughput that helps maintain consistent response quality. Option A is not appropriate here because fine-tuning changes model behavior and adds training cost, and it does not directly address cost management or consistent quality for this scenario. Option C is not ideal because switching to a cheaper, less capable model would likely degrade response quality rather than ensure it. Option E is incorrect because increasing max tokens per response raises token consumption and cost, and does not improve consistency.

Answer

Fine-tune the model on previous marketing copy

Answer

Switch to a cheaper, less capable model

Answer

Increase max tokens per response to ensure completeness

Question 11

A machine learning engineer needs to deploy a PyTorch model for real-time inference with low latency. The model uses custom operators that are not supported by standard ONNX conversion. Which deployment approach is MOST appropriate?

Accepted Answer

Deploy the model using TorchServe with a custom handler. TorchServe is the native serving solution for PyTorch models and supports custom operators through custom handlers, allowing you to implement arbitrary preprocessing, inference, and postprocessing logic in Python. This approach avoids the need for ONNX conversion entirely, which is critical when custom operators are not supported by the ONNX standard. It also provides built-in features like model versioning, batching, and metrics for low-latency real-time inference.

Answer

Use TensorFlow Serving with a saved model format

Answer

Wrap the model in a Flask app and deploy on a VM

Answer

Convert the model to ONNX and serve with ONNX Runtime

Question 12

A data engineering team needs to orchestrate a complex ML pipeline that involves data extraction, transformation, model training, and deployment. They require scheduling, monitoring, and retry logic. Which MLOps tool is BEST suited for this task?

Accepted Answer

Apache Airflow. Apache Airflow is a workflow orchestration platform built specifically for scheduling, monitoring, and retrying complex pipelines via directed acyclic graphs (DAGs). It provides native operators for data extraction, transformation, model training triggers, and deployment steps, along with built-in retry logic, alerting, and a web UI for monitoring — exactly the requirements listed.

Answer

Weights & Biases

Answer

Kubeflow

Answer

MLflow

Question 13

A team is deploying a model on a cloud-based ML service and needs to handle variable traffic patterns with automatic scaling based on request latency. They want to minimize costs during low traffic. Which endpoint configuration should they use?

Accepted Answer

Configure automatic scaling with a target tracking metric based on latency. The ML service's automatic scaling with a target tracking metric based on latency allows the endpoint to dynamically adjust the number of instances in response to real-time request latency, ensuring cost efficiency during low traffic while maintaining performance during spikes. This approach uses a predefined or custom metric (e.g., a metric like invocations per instance) to trigger scaling policies, minimizing over-provisioning and idle costs.

Answer

Use a serverless function with the ML service

Answer

Provision a fixed number of instances with multi-model endpoints

Answer

Use a single large instance to handle peak traffic

Question 14

A data scientist is deploying a model on edge devices using TensorFlow Lite. The model currently uses FP32 precision. Which TWO techniques can reduce the model size and improve inference speed without significant accuracy loss? (Choose TWO.)

Accepted Answer

Post-training quantization to INT8. Post-training quantization to INT8 reduces model size by converting FP32 weights and activations to 8-bit integers, which also speeds up inference on edge devices by leveraging integer-optimized hardware. This technique typically preserves accuracy within 1–2% of the original FP32 model, making it suitable for deployment on resource-constrained devices.

Answer

Use a larger batch size during inference

Answer

Increase the number of layers

Answer

Convert to FP16 precision

Question 15

A team is using Kubeflow to orchestrate ML workflows on Kubernetes. They need to ensure reproducibility, track experiments, and share models across the organization. Which THREE components or tools should they integrate? (Choose THREE.)

Accepted Answer

Kubeflow Pipelines. Kubeflow Pipelines (C) is the native Kubeflow component for defining, scheduling, and versioning ML workflows as containerized DAGs, so it directly provides the reproducibility and pipeline orchestration the team needs on Kubernetes. MLflow Tracking (D) is the component that logs parameters, metrics, artifacts, and source versions per run, giving the experiment-tracking capability required to compare and reproduce results. MLflow Model Registry (E) provides a centralized, versioned store with stage transitions (Staging, Production, Archived) and model lineage, which satisfies the requirement to share models across the organization. Apache Airflow (A) is a general-purpose workflow scheduler that overlaps with Kubeflow Pipelines and is not the Kubeflow-native reproducibility/tracking solution, and Weights & Biases (B) is a third-party experiment-tracking SaaS that is not required to meet the stated Kubeflow-integrated reproducibility, tracking, and model-sharing needs.

Answer

Apache Airflow

Answer

Weights & Biases

Question 16

A company uses AWS SageMaker to train a large language model. The training job fails with an out-of-memory error. The team is already using the largest available GPU instance. Which step should the team take to resolve the issue without modifying the model architecture?

Accepted Answer

Enable gradient accumulation in the training script. Gradient accumulation allows the model to simulate a larger batch size by accumulating gradients over several forward/backward passes before performing an optimizer step. This reduces per-step memory usage because the gradients are stored and averaged rather than requiring the entire batch to be loaded into GPU memory at once. Since the team cannot change the instance type or model architecture, enabling gradient accumulation is the correct approach to resolve the out-of-memory error.

Answer

Increase the learning rate to converge faster

Answer

Switch to a CPU-based instance

Answer

Reduce the number of attention heads in the model

Question 17

An organisation needs to deploy PyTorch models on mobile devices with minimal latency. Which framework or tool should they use to convert and optimise the model for on-device inference?

Accepted Answer

TorchScript. TorchScript is the correct choice because it is PyTorch's native model serialization and optimization format, designed specifically for deploying PyTorch models on mobile devices with minimal latency. It allows you to trace or script a PyTorch model into a static graph that can be run efficiently on iOS and Android via the PyTorch Mobile runtime, without the overhead of Python interpreter.

Answer

TensorFlow Lite

Answer

Keras for mobile

Answer

ONNX Runtime with Core ML conversion

Question 18

A team is using Hugging Face Transformers to serve an LLM via a REST API. They notice high latency during inference. The model is deployed on a single GPU. Which optimisation would reduce inference latency WITHOUT changing the model architecture?

Accepted Answer

Use model quantisation to FP16. FP16 quantization reduces the memory footprint and computational load of the model by using half-precision floating-point numbers, which allows the GPU to process more operations per second and reduces memory bandwidth usage. This directly lowers inference latency without altering the model's architecture, making it the correct choice for a single-GPU deployment.

Answer

Add more GPUs and distribute the model

Answer

Increase the batch size to process multiple requests simultaneously

Answer

Switch from GPU to CPU

Question 19

A healthcare AI startup must store and query high-dimensional embeddings of medical records for a RAG system. They need low-latency similarity search at scale. Which database should they choose?

Accepted Answer

Pinecone. Pinecone is a fully managed vector database optimized for high-dimensional embeddings and low-latency similarity search at scale. It provides built-in indexing (e.g., HNSW), automatic sharding, and serverless scaling, making it ideal for RAG systems that require fast approximate nearest neighbor (ANN) queries on medical record embeddings.

Answer

Amazon S3

Answer

pgvector

Answer

BigQuery

Question 20

An MLOps team observes that their production inference API experiences increasing latency as more concurrent requests arrive. They need to scale horizontally while maintaining session state of preprocessing steps. Which deployment strategy should they implement?

Accepted Answer

Deploy multiple instances behind a round-robin load balancer with sticky sessions. Sticky sessions (session affinity) ensure that all requests from a given client are routed to the same backend instance, preserving the in-memory session state of preprocessing steps. Combined with a round-robin load balancer, this allows horizontal scaling while maintaining stateful behavior, which is essential for the described latency issue under concurrent load.

Answer

Deploy stateless containers without session persistence

Answer

Use a single larger GPU instance to handle all requests

Answer

Implement a message queue (e.g., Kafka) to buffer requests