AI Infrastructure and Technologies practice questions

Azure Logic Apps

Why wrong: Logic Apps can trigger workflows but is less integrated with ML pipelines than Event Grid for storage events.

Azure Data Factory

Why wrong: Data Factory is for data movement and orchestration, not real-time event triggering.

Azure Functions

Why wrong: Functions can run code on events, but Event Grid is the native event routing service that connects Blob Storage to Azure ML.

Apache Kafka

Why wrong: Kafka is a streaming platform, not an MLOps tool for tracking/model serving.

Docker Compose

Why wrong: Docker Compose is for container orchestration, not ML experiment tracking.

Amazon Bedrock

Why wrong: Bedrock is also a good choice, but the question asks for TWO; Azure OpenAI and Vertex AI are more commonly paired.

Amazon SageMaker

Why wrong: SageMaker is a full ML platform but does not offer pre-trained LLM APIs directly; Bedrock does.

Hugging Face Hub

Why wrong: Hugging Face Hub hosts models but is not a cloud platform with managed API and safety filters like the others.

Distill the model into a smaller architecture without quantization or pruning

Why wrong: Distillation alone may not achieve 50% reduction; combining with quantization/pruning is more effective.

Use FP32 precision and increase batch size

Why wrong: FP32 is full precision, not reducing size; larger batch size increases memory usage.

Use FP16 quantization and add more layers

Why wrong: FP16 reduces size modestly, but adding layers increases size, contradicting the goal.

Reduce the chunk size of documents

Why wrong: Chunk size can affect context but is not the most direct lever for relevance.

Switch from an HNSW index to a flat index

Why wrong: Index type affects speed, not necessarily relevance.

Increase the top-k retrieval count

Why wrong: More results may include less relevant ones, worsening precision.

Amazon EC2

Why wrong: EC2 is raw compute, requiring manual setup.

Amazon SageMaker

Why wrong: SageMaker is a full ML platform, not specifically for serverless FM access.

AWS Lambda

Why wrong: Lambda runs code but does not provide pre-built FM APIs.

Apply weight pruning to remove 50% of parameters

Why wrong: Pruning reduces model size but may not yield 4x latency improvement; speedup depends on sparsity support.

Switch from TensorFlow Lite to Core ML

Why wrong: Changing runtime helps but does not reduce model computation; quantization is more impactful.

Distill the model into a smaller architecture

Why wrong: Distillation can reduce size but may require retraining and may not achieve the exact speedup.

Hugging Face Transformers

Why wrong: Hugging Face focuses on transformer models, not a general-purpose framework with Keras.

scikit-learn

Why wrong: scikit-learn is for traditional ML algorithms, not deep learning.

PyTorch

Why wrong: PyTorch is a deep learning framework but does not include Keras by default.

Use TensorFlow Serving with a saved model format

Why wrong: TensorFlow Serving does not support PyTorch models directly.

Wrap the model in a Flask app and deploy on a VM

Why wrong: While possible, it lacks production features like batching, monitoring, and autoscaling.

Convert the model to ONNX and serve with ONNX Runtime

Why wrong: ONNX conversion fails for custom operators, making this approach unfeasible.

Snowflake

Why wrong: Snowflake is a data warehouse for structured queries, not vector search.

Apache Kafka

Why wrong: Kafka is a streaming platform, not a storage and retrieval system for embeddings.

Amazon S3

Why wrong: S3 is object storage, not optimized for vector similarity search.

Use a serverless function with the ML service

Why wrong: Lambda is for serverless compute but not designed for real-time model serving with scaling based on latency.

Provision a fixed number of instances with multi-model endpoints

Why wrong: Fixed instances do not scale automatically, potentially wasting resources or causing underprovisioning.

Use a single large instance to handle peak traffic

Why wrong: This is overprovisioned during low traffic and may still be insufficient during spikes.

Use a larger batch size during inference

Why wrong: Larger batch size increases memory usage, not suitable for edge devices.

Increase the number of layers

Why wrong: Adding layers increases model size and latency, opposite of the goal.

Convert to FP16 precision

Why wrong: FP16 reduces size by half, but INT8 is more effective; however, the question asks for TWO and FP16 is less impactful than INT8 and pruning.

Amazon S3

Why wrong: S3 is object storage, not a vector database.

Snowflake

Why wrong: Snowflake is a data warehouse, not optimized for vector search.

Apache Airflow

Why wrong: Airflow is an alternative orchestrator, not typically integrated with Kubeflow.

Weights & Biases

Why wrong: W&B is a separate experiment tracking tool; could be used but not the typical Kubeflow integration.

Use a larger batch size during inference

Why wrong: Larger batch sizes are for throughput, not latency reduction on a single inference; edge devices often process one request at a time.

Train the model for more epochs to improve convergence

Why wrong: More epochs improve accuracy but do not reduce model size or latency; they may even lead to overfitting.

Increase the number of layers to improve feature extraction

Why wrong: Adding layers increases model size and latency, the opposite of what edge deployment requires.

Increase the learning rate to converge faster

Why wrong: Learning rate adjustments do not address memory constraints and may harm convergence.

Switch to a CPU-based instance

Why wrong: CPU instances would be significantly slower and unlikely to solve memory issues for large models.

Reduce the number of attention heads in the model

Why wrong: This modifies the model architecture, which the team wants to avoid.

Use gRPC streaming for lower latency

Why wrong: gRPC is about performance, not auditability; it does not inherently provide explainability or logging.

Export the model to ONNX format and run on a dedicated inference server

Why wrong: ONNX focuses on interoperability, not on logging or explainability.

Deploy the model on edge devices to avoid centralised logging

Why wrong: Edge deployment makes auditing harder, not easier; regulatory compliance typically requires centralised logging.

Increase the chunk size of documents

Why wrong: Larger chunks may contain more noise and reduce precision.

Switch from cosine similarity to Euclidean distance

Why wrong: Distance metric choice has less impact than embedding quality; cosine is standard for text.

Reduce the number of retrieved documents from 5 to 3

Why wrong: Reducing count may omit relevant documents but does not improve relevance of each retrieved item.

Add more GPUs and distribute the model

Why wrong: Adding more GPUs and distributing the model primarily addresses memory constraints for models too large for a single GPU, or increases overall throughput for handling many concurrent requests. For a model already fitting on one GPU, distributing it across multiple devices introduces inter-GPU communication overhead, which can actually increase inference latency for a single forward pass. This option is tempting as more hardware often seems like a universal performance solution, and it would be the correct choice if the objective was to serve an extremely large model or maximise the number of simultaneous inferences.

Increase the batch size to process multiple requests simultaneously

Why wrong: This can improve throughput but not per-request latency; it may even increase latency for the first request if batching is synchronous.

Switch from GPU to CPU

Why wrong: CPU would be much slower for LLM inference.

Amazon S3

Why wrong: S3 is object storage, not optimised for vector similarity search.

pgvector

Why wrong: pgvector is an extension for PostgreSQL; while capable, it may not provide the same low-latency performance as Pinecone at very large scale.

BigQuery

Why wrong: BigQuery is a data warehouse for analytics, not vector search.