CCNA AI Technologies And Controls Questions

Differential privacy helps ensure that the model does not memorize specific training data samples, which is a common target for poisoning attacks aimed at steering model output.

Versioning enables rapid rollback to a known good state if a new version is found to be insecure or perform poorly.

Auditing for fairness requires checking data representation and measuring model performance across different demographic subgroups.

An interface VPC endpoint (AWS PrivateLink) allows private connectivity to SageMaker endpoints from within a VPC.

AI Red Teaming identifies vulnerabilities by actively attempting to exploit the model through adversarial prompts.

Homomorphic encryption allows computation on encrypted data, enabling a provider to train or run a model on data without ever decrypting it.

The SageMaker Model Registry allows for version tracking, metadata management, and approval states to control deployment.

A 'Sandboxed Execution Environment' for agents prevents the AI from accessing the host file system or network, limiting the impact of an injection.

Model validation ensures that the model meets performance and safety requirements, minimizing the risk of unexpected behavior.

Adding noise (differential privacy) to the outputs or rate-limiting makes the model's decision boundaries difficult to map precisely.

Model inversion is mitigated by reducing the information content of model outputs and limiting query volume.

Microsoft Defender for Cloud leverages 'AI Security Posture Management' to identify anomalous prompts that resemble injection patterns.

The Fast Gradient Sign Method (FGSM) is a classic white-box attack technique used to test evasion vulnerability by introducing imperceptible noise.

Outlier detection (or anomaly detection) is the primary mechanism to identify malicious, poisoned data points that deviate from the expected distribution of the training set.

SageMaker Model Monitor is designed to detect data drift and concept drift by comparing real-time inference data against the baseline training statistics.

An AI-specific incident response plan must address the unique nature of AI, including specialized stakeholders and specific technical triggers.

Unauthorized data access via AI agents often results from 'Broken Access Control' where the AI inherits privileges it should not possess.

Supply chain security involves securing the training data, the model architecture, and the software dependencies.

To detect security threats like prompt injection, you must enable diagnostic logging for the Azure OpenAI service specifically for the 'RequestResponse' category, which captures the interaction payload.

A blind spot attack exploits areas in the input space where the model's confidence is high, but its performance is actually incorrect due to poor training coverage.

Differential privacy ensures that an individual's data cannot be inferred from the aggregate model output by injecting statistical noise during training.

AWS IAM (Identity and Access Management) is the primary tool for defining permissions for Bedrock API operations.

Data lineage and cryptographic hashing ensure that the training dataset has not been altered maliciously.

Public models may contain hidden backdoors (poisoning) or have been trained on sensitive/copyrighted data that could cause legal or compliance issues.

Secrets management services (AWS Secrets Manager, HashiCorp Vault) are the secure way to manage credentials.

API keys should never be hardcoded; they should be stored in secrets managers like AWS Secrets Manager or Azure Key Vault.

AWS Bedrock Guardrails allow for PII entity redaction or blocking; the configuration must be set to filter both incoming prompts and outgoing model responses to ensure comprehensive protection.

Adding adversarial noise to an image (e.g., a sticker) causes the model to misclassify it, which is the definition of evasion.

Model extraction is mitigated by rate limiting, output perturbation (noise), and limiting the detail/granularity of the output returned to the API.

Implementing a CAPTCHA or sophisticated WAF challenge for high-volume users is a standard way to differentiate human traffic from bots.

Adversarial training involves augmenting the training set with adversarial examples, hardening the model against evasion (inference-time) attacks.

HITL involves human review of model decisions before they are finalized or acted upon, especially for high-stakes decisions.

The EU AI Act requires that high-risk systems be designed with human-in-the-loop (HITL) capabilities and adequate logging to ensure transparency and accountability.

Azure OpenAI Content Safety provides text filtering and PII masking.

Encryption at rest ensures that even if the storage media is stolen or accessed unauthorized, the data remains unreadable.

Model monitoring tools track statistical properties of features and predictions to identify drift.

Egress filtering (outbound rules) ensures that the model environment cannot communicate with unauthorized external servers, preventing data exfiltration.

Differential privacy and output perturbation are the standard methods to reduce the probability of sensitive data leakage through inference queries.

Vertex AI Model Garden and Metadata are used for lineage tracking, but specifically for provenance and audit, the Vertex AI ML Metadata service allows tracking of dataset versions and training pipeline execution history.

Isolation is the first step to prevent further damage or exfiltration of sensitive data.

The Vertex AI 'Safety Settings' and 'Content filtering' allow practitioners to define thresholds for harassment, hate speech, and PII leakage.

Access to Model Monitor requires permissions to both the infrastructure (S3) and the specific model monitoring job definitions.

A Model Card is a documentation artifact that provides transparency into the model's intended use, limitations, and training data.

Model signing uses digital signatures to verify that the model artifact has not been tampered with between the training environment and the production environment.

An LLM gateway acts as a proxy to enforce security policies, redact PII, and prevent prompt injection before requests reach the model.

The Prompt Shield feature in Azure AI Content Safety specifically targets user-to-model prompt injection and jailbreak attacks.

The IBM watsonx.ai Governance module uses FactSheets to automatically capture and document the lifecycle, lineage, and validation status of models.

Rate limiting is a standard control to prevent resource exhaustion and model scraping.

Azure OpenAI requires Entra ID (Azure AD) for authentication and Azure Monitor/Log Analytics for auditing request/response patterns.

Model stealing involves querying a model to extract enough information to replicate its functionality, leading to the loss of proprietary intellectual property.

Centralized logging (e.g., CloudWatch, Azure Monitor) and anomaly detection are critical for identifying unexpected model interactions.

Data provenance allows auditors to trace the origin of data, ensuring no poisoned or copyrighted data entered the training pipeline.

IAM (Identity and Access Management) manages who can perform which actions on which resources (e.g., who can invoke a model).

Security Groups provide stateful packet filtering for ECS containers, restricting who can communicate with the model endpoint.

RAG (Retrieval-Augmented Generation) forces the model to base its answers on specific, provided documents, reducing hallucination risks.

The primary defenses involve rigorous input validation, context separation between instructions and data, and using specialized output filters.

Opacus is the official PyTorch library for training models with differential privacy.

Logging provides the audit trail necessary to investigate security incidents and understand how the system reached a specific output.

Data sanitization removes sensitive information (PII/PHI) and malicious patterns, ensuring that the model does not learn or leak private info.

Poisoning often results in degraded accuracy for specific targets (backdoors) or broader performance degradation that is detectable through validation testing.

Insecure Output Handling occurs when a model output is interpreted by a downstream system without sanitization. Output validation (canonicalization) is the primary defense.

Incident response involves identification (detection), containment (isolation), and eradication/recovery, followed by post-incident learning.

Securing the pipeline involves code signing, scanning dependencies for vulnerabilities, and using isolated container environments.

Digital signatures (using tools like Cosign or AWS KMS) verify that the model has not been tampered with since being signed.

This attack determines whether a specific data point was included in the training set, potentially compromising privacy if the data is sensitive.

Shadow AI refers to the use of AI tools without IT or security oversight, leading to data leaks and compliance violations.

The roles/aiplatform.user role allows usage, but 'aiplatform.endpoints.create' or specific restrictions on the Endpoint resource are needed to prevent public exposure.

The attack surface includes all points where an adversary can enter or extract data from the system, including API endpoints and training interfaces.

Denial of Wallet attacks involve high-cost, high-token-count requests. Monitoring the 'usage.total_tokens' or 'billing_tokens' field is critical for identifying this behavior.

Access controls include authentication (IAM), network restrictions (VPC/Security Groups), and fine-grained permissions for API operations.

A robust policy must define acceptable use cases and mandate security testing (like red teaming) for all AI assets.

Defensive distillation increases the model's robustness against small adversarial perturbations but can lead to reduced accuracy on clean data or increased training complexity.

A backdoor is a trigger (a specific pattern) embedded into the model during training so that it behaves normally until the trigger is present, at which point it performs a malicious action.

Access control for registries should follow the principle of least privilege and use identity federation.

Defending against poisoning requires cleaning the training data and using robust learning algorithms.