Hard Difficulty Questions

Prompt engineer a large foundation model with few-shot examples in Vertex AI Studio

Why it fails: Few-shot prompting conditions the model at inference only; it cannot durably adjust weights to a 500-example taxonomy, so classification drifts across contracts. It suits rapid prototyping or low-volume tasks where labelled data is scarce and near-perfect consistency is not required.

Use the RAG Engine to retrieve similar clauses and ask the model to classify

Why it fails: Retrieval returns semantically similar clauses but does not teach the model the taxonomy's label boundaries, so classification stays inconsistent across the 500 labelled contracts. RAG suits grounding answers in source documents, such as citing clause text. Supervised fine-tuning on the labelled examples is the approach that fixes the taxonomy.

Use a larger foundation model without fine-tuning and rely on its pre-trained knowledge

Why it fails: Pre-trained knowledge cannot enforce a bespoke taxonomy; without examples the model invents its own clause categories, so accuracy stays inconsistent. It tempts when the task is generic summarisation or drafting, where broad world knowledge suffices and no labelled data exists.

Use A/B testing between versions and manually select the best performer

Why it fails: A/B testing requires manual intervention and does not automatically promote based on thresholds.

Manually track version IDs and deploy the latest version

Why it fails: Manual tracking is not automated and can lead to deploying untested versions.

Deploy all versions to a single endpoint and route traffic manually

Why it fails: Manual routing is not automated and increases risk of deploying poor versions.

Switch to a different foundation model like Stable Diffusion.

Why it fails: Switching models is a major change; the issue is likely with fine-tuning approach.

Use a larger batch size during fine-tuning.

Why it fails: Larger batch size can help generalization but is not specifically for preventing pattern repetition.

Further increase the resolution of training images to 5120x3840.

Why it fails: Higher resolution may worsen overfitting and increase compute.

Vertex AI Explainable AI

Why it fails: Explainable AI provides feature attributions, not audit logging of predictions.

VPC Service Controls

Why it fails: VPC Service Controls prevent data exfiltration, not audit logging.

Vertex AI Feature Store

Why it fails: Feature Store manages feature data, not prediction logging.

Gemini Ultra

Why it fails: Gemini Ultra is the largest, most capable cloud-hosted variant, so it cannot execute offline on an Android device's constrained hardware. It is tempting because Ultra delivers the highest reasoning quality, and would suit demanding cloud inference where latency and device memory are not constraints.

Gemini Pro

Why it fails: Gemini Pro is a cloud-hosted model requiring network access, so it cannot run inference locally on an Android device offline. It is tempting because Pro offers strong general reasoning, and would be the right pick for server-side or online workloads needing higher capability than Flash.

Gemini Flash

Why it fails: Gemini Flash is a cloud-hosted model served via API, so it cannot perform offline inference on an Android device without connectivity. It is tempting because Flash is optimised for low latency and cost, and would be the right choice for high-throughput online tasks rather than on-device execution.

Use the largest available model to ensure accuracy

Why it fails: Larger models cost more; choose the smallest model that meets accuracy needs.

Include multiple few-shot examples of JSON in every prompt

Why it fails: Few-shot examples increase token count and cost.

Migrate all GenAI workloads to a single on-premises server to reduce cloud costs

Why it fails: Consolidating onto one on-premises server removes cloud elasticity, capacity for 50,000 employees, and access to current models, so it cannot balance control with innovation. It is tempting as a cost-capping measure, and on-premises hosting would be correct where data residency or air-gapped operation forbids public cloud.

Mandate all teams use a single model (e.g., Gemini) via a centralized Vertex AI endpoint with usage quotas

Why it fails: A single mandated model on one Vertex AI endpoint with quotas centralises governance and cost, but it blocks teams from selecting task-appropriate models, which the stem's innovation requirement demands. It is tempting because standardisation genuinely solves fragmentation, and it would be correct where consistency outweighs model diversity.

Allow teams to continue using their own models but require them to submit monthly cost reports

Why it fails: Monthly cost reports give visibility after spend occurs but leave each team's models, cloud accounts, and output quality fragmented, so governance stays reactive. It is tempting because reporting is low-friction and preserves autonomy, and it would be correct where teams already share a governance framework and only chargeback is missing.

Increase the batch size for batch requests

Why it fails: Batching amortises per-request overhead but each request still bills for its full prompt tokens, so repetitive boilerplate cost is unchanged. It is tempting because batch processing lowers unit cost, and would be correct for high-volume, latency-tolerant workloads rather than for trimming redundant prompt content.

Switch to a smaller model size

Why it fails: A smaller model reduces per-token pricing but still processes the same repetitive boilerplate tokens, so the dominant cost driver remains. It is tempting because model choice is a standard cost lever, and would be correct where output quality tolerates reduced capability rather than where input prompts are bloated.

Ignore the cost increase as it will stabilize

Why it fails: Costs scale with prompt tokens, so repetitive boilerplate keeps inflating spend rather than stabilising on its own. It is tempting because some workloads plateau once traffic settles, and would be correct only where the cost rise is genuinely transient, not driven by persistent prompt bloat.

Build a retrieval-augmented generation (RAG) system in Vertex AI Agent Builder

Why it fails: RAG is for answering questions over documents, not for structured extraction of specific clauses. Agent Builder is more suited for conversational interfaces.

Use the Gemini API directly with the raw PDF files as input

Why it fails: Gemini API can accept PDFs, but without preprocessing, the model may not correctly interpret complex layouts, leading to errors. Cost may also be higher for large PDFs.

Fine-tune a Gemini model on 100 annotated contracts and run inference on all contracts

Why it fails: 100 contracts are too few to handle layout variations; fine-tuning is expensive and may not generalize well to diverse document formats.

Using only English prompts

Why it fails: Restricting prompts to English does not remove bias; training data skew persists regardless of prompt language, and non-English users simply receive degraded service. It is tempting because standardising inputs appears to control variability, and English-only prompting would be reasonable for a deliberately English-scoped deployment, not for bias mitigation.

Increasing model size

Why it fails: Scaling parameters does not target the biased correlations embedded in training data; larger models can reproduce or amplify them with greater fluency. It is tempting because bigger models score higher on many benchmarks, and increasing size would be correct when the constraint is general capability or reasoning quality rather than fairness.

Persistent Disk

Why it fails: Persistent Disk is block storage attached to a single VM, offering limited IOPS and no parallel multi-reader access, so it cannot sustain the throughput TPU training demands. It is tempting because it is simple and familiar for single-instance workloads, and it would be correct for a lone VM needing a boot or scratch volume.

Cloud Filestore

Why it fails: Filestore is a managed NFS file service for shared POSIX access, not a parallel throughput tier; it cannot match the aggregate bandwidth TPU pods demand when streaming sharded training data. It is tempting because shared filesystems suit many general workloads, and Filestore would be the right pick for shared home directories or small-scale training.

Cloud Spanner

Why it fails: Spanner is a globally distributed relational database tuned for transactional consistency and SQL queries, so it cannot deliver the parallel object throughput that TPU training data loading requires. It is tempting because it scales horizontally, and Spanner would be correct for an application needing strongly consistent, multi-region OLTP storage.

Use AutoML to train a new model directly from the dataset without fine-tuning.

Why it fails: AutoML is designed for training models from scratch with automated hyperparameter tuning and architecture search, not for fine-tuning an existing large language model. It would not leverage the proprietary dataset or the base model, resulting in a fundamentally different approach that does not meet the requirement of fine-tuning the existing LLM.

Deploy the existing pipeline on a Google Kubernetes Engine cluster and use Google Cloud Filestore for shared storage.

Why it fails: Deploying the existing pipeline on Google Kubernetes Engine with Filestore for shared storage requires significant operational overhead for cluster management and storage scaling. While possible, it does not provide the incremental sync capability of Vertex AI Managed Datasets, leading to higher data transfer costs and potential downtime during full data migrations. This is not the most optimized solution for minimizing downtime and cost.

Use Cloud Storage Transfer Service to move all data to Cloud Storage, then set up a Vertex AI custom training job to run the fine-tuning.

Why it fails: Cloud Storage Transfer Service is intended for one-time or scheduled batch transfers of data to Cloud Storage. It would require a full initial data transfer, incurring high costs and potential downtime, and does not natively support incremental syncing of new data for weekly fine-tuning. This approach lacks the integration with BigQuery and the incremental capabilities that Vertex AI Managed Datasets provide.

Enterprise Data Governance on Vertex AI

Why it fails: Enterprise Data Governance covers classification, lineage and policy controls over data assets; it does not configure the retrieval source used by grounding. It is tempting because governance sounds like it governs what the model may access, and it would be correct when the goal is cataloguing or policy enforcement across Vertex AI datasets.

VPC Service Controls

Why it fails: VPC Service Controls builds a perimeter around Google Cloud resources to block data exfiltration, but it does not choose which corpus a grounding request queries. It is tempting because it governs data boundaries, and it would be correct when the requirement is preventing data leaving a project rather than selecting internal knowledge bases.

Google Search Grounding

Why it fails: Google Search Grounding deliberately retrieves from the public web, which is the opposite of the stated requirement. It is tempting because it is the default grounding option and improves factual accuracy, and it would be correct when answers must reflect current public information rather than internal documents.

Private Google Access for on-premises connectivity

Why it fails: Private Google Access addresses network privacy, not chatbot deployment or compliance.

Dialogflow CX with Cloud Logging

Why it fails: Dialogflow CX is powerful but may not provide the same level of gen AI customization and auditability.

Cloud AI Platform Pipelines

Why it fails: Cloud AI Platform Pipelines is for ML pipeline orchestration, not chatbot deployment.

Codey

Why it fails: Codey is a family of code-generation models that produce or complete source code from natural-language prompts. It suits developer assistance tasks, but it outputs text code rather than video frames, so it cannot satisfy a text-to-video requirement.

Chirp

Why it fails: Chirp is a speech model family used for text-to-speech and speech-to-text conversion. It is correct when the requirement is synthesising or transcribing audio, but it produces audio output, not video, so it cannot generate a video from a text description.

Imagen

Why it fails: Imagen generates images from text prompts, not video. It is the right choice for text-to-image tasks such as creating still artwork or product visuals, but generating a video from a description requires a video-generation model, which Imagen does not provide.

The safety filters are too aggressive; relax them

Why it fails: Aggressive safety filters suppress harmful or toxic output; they do not generate the sophisticated-versus-basic vocabulary split, so relaxing them removes guardrails without addressing the skew. Safety filters are tempting because they are the visible content control, and tuning them is correct when outputs are wrongly blocked as unsafe.

The temperature parameter is set too low; increase it to introduce more randomness

Why it fails: Low temperature makes sampling deterministic and consistent; it does not create the systematic high-end versus budget vocabulary disparity, and raising it adds randomness rather than removing stereotype. Temperature tuning is tempting as a quick creativity dial, and is correct when outputs are repetitive or need varied phrasing.

The model architecture is biased; switch to a different base model

Why it fails: The skew stems from biased training data and prompt or label associations, not the architecture itself; swapping base models leaves the underlying data imbalance intact. Replacing the model is tempting because it appears to reset behaviour, and would be right where a specific architecture demonstrably cannot meet capability requirements.

Increase the number of training steps

Why it fails: More training steps lengthen the run and increase compute charges, worsening both symptoms. It is tempting because extra steps can improve convergence when a model is undertrained, but the stem describes an already costly run, so the lever needed is reducing iterations, not adding them.

Increase the batch size

Why it fails: A larger batch size reduces step count but raises per-step memory, often forcing smaller models or gradient accumulation, which can slow convergence and worsen quality. It suits throughput-bound training on abundant accelerator memory, not fine-tuning where quality must hold.

Use a higher learning rate

Why it fails: Raising the learning rate risks divergence or degraded quality, and does not itself cut the number of passes over data. It is tempting because a larger step size can shorten convergence in some runs, but that trades stability for speed rather than reducing the training workload.

Use Vertex AI Grounding with a public dataset and disable safety filters

Why it fails: Disabling safety filters removes the very controls that block medical advice, and public datasets are not trusted medical sources. Grounding should point to a curated, authoritative corpus such as a licensed medical knowledge base, with safety filters enabled to suppress advice-giving responses.

Enable safety filters only, without grounding

Why it fails: Safety filters alone block harmful categories but cannot ground responses in trusted medical sources, so citations and factual accuracy go unverified. Grounding against an authoritative medical corpus is also required. Filters-only suits general content moderation where source attribution is not a requirement.

Fine-tune the model on a curated medical dataset and disable safety filters for faster responses

Why it fails: Fine-tuning shapes tone and domain vocabulary but cannot guarantee refusal of medical advice, and disabling safety filters removes that guardrail entirely. Safety filters must stay enabled; grounding in trusted medical sources supplies verifiable citations. Fine-tuning suits adapting style to a curated dataset, not enforcing safety policy.

The system must be explainable using chain-of-thought reasoning

Why it fails: Chain-of-thought explainability is not an EU AI Act obligation; the Act mandates risk management, logging, human oversight and technical documentation instead. It tempts because transparency is a genuine Act principle, and chain-of-thought prompting is a real technique for debugging model reasoning — but it satisfies no statutory high-risk requirement.

The system must achieve a minimum accuracy of 90% on validation data

Why it fails: The EU AI Act sets no numeric accuracy threshold such as 90%; it requires appropriate accuracy, robustness and cybersecurity levels declared in documentation. A fixed percentage is a tempting proxy for performance obligations but is not a statutory requirement.

The system must be trained on data that is representative of the target population

Why it fails: Representative training data addresses bias and is expected under data governance, but the Act's high-risk obligations centre on risk management, technical documentation, logging, human oversight and conformity assessment. Representativeness is a data-quality principle, not the mandatory requirement tested here.

A single TPU v5e VM with multiple accelerators

Why it fails: A single TPU v5e VM with multiple accelerators cannot scale across nodes, so it fails the multi-node requirement for highest compute density. It is correct for smaller single-host training or inference workloads that fit within one VM.

A cluster of A100 GPU VMs connected via standard networking

Why it fails: A100 VMs over standard networking lack the dedicated high-bandwidth interconnect that multi-node transformer training demands, so inter-node communication latency stays high. It is tempting because A100 clusters genuinely suit single-node or loosely coupled GPU workloads, where that interconnect is unnecessary. The scenario's explicit latency-minimisation requirement rules it out.

Cloud Run jobs with GPU acceleration

Why it fails: Cloud Run jobs with GPU acceleration run containerised batch tasks and provide no multi-node interconnect fabric, so inter-node latency cannot be minimised. They are correct for short-lived, event-driven or batch container workloads rather than tightly coupled distributed training.