Study guide
6.1 Securing AI Systems: Exfiltration, Malicious Prompting and Model Armor
Google Cloud Professional Machine Learning Engineer Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
0.1 Exam Overview, Blueprint & Certification Logistics 0.2 Study Strategy, Scenario Mechanics & 8-Week Roadmap
2 Section 1: Architecting Low-Code AI Solutions (~13%)
1.1 BigQuery ML: Model Types, SQL Syntax and In-Database Evaluation 1.2 Feature Engineering and Feature Selection in BigQuery ML 1.3 Generating Predictions with BigQuery ML 1.4 AutoML on Agent Platform: Tabular, Image, Text and Video Training 1.5 Fine-Tuning Gemini Models from BigQuery 1.6 Model Garden: Evaluating and Selecting the Right Model 1.7 Industry-Specific AI APIs: Document AI, Vision, Translate and Speech 1.8 Generative Media and Models as a Service: Gemini, Imagen and Veo 1.9 Optimizing Gemini Applications for Cost, Latency and Availability
3 Section 2: Collaborating Within and Across Teams to Manage Data and Models (~16%)
2.1 Organizing and Exploring Tabular, Text, Image and Video Data 2.2 Choosing the Preprocessing Tool: BigQuery, Dataflow, Spark and In-Memory Python 2.3 Creating and Consolidating Features in Agent Platform Feature Store 2.4 Data Privacy, PII Handling and Sensitive Information 2.5 Notebook Environments: Collaboration and Security Best Practices 2.6 Prototyping in Workbench and Colab Enterprise with PyTorch, scikit-learn and JAX 2.7 Prototyping with Model Garden Foundational and Open Models in Notebooks 2.8 Choosing the Development and Experimentation Environment 2.9 Evaluating Generative Solutions: LLM-as-a-Judge and RAG Evaluation 2.10 Experiment Tracking, Artifact Lineage and Model Registry Governance
4 Section 3: Scaling Prototypes into ML Models (~21%)
3.1 Choosing the Model Type: ARIMA, DNN, LLM and Classical Baselines 3.2 Choosing the Product: AutoML, BigQuery ML, Custom Training and Pipelines 3.3 Choosing the Deployment Strategy at Design Time 3.4 Modeling Techniques for Interpretability Requirements 3.5 Organizing Training Data on Cloud Storage and BigQuery 3.6 Ingesting Structured and Unstructured Data into Training Pipelines 3.7 Training SDKs and Services: Custom Training, Kubeflow on GKE, AutoML and Tabular Workflows 3.8 Troubleshooting ML Model Training Failures 3.9 Hyperparameter Tuning, Bayesian Optimization and Vizier 3.10 Model Evaluation Metrics, Transfer Learning and Fine-Tuning 3.11 Evaluating Compute and Accelerator Options: CPU, GPU and TPU 3.12 Distributed Training Strategies, Hardware Acceleration and Performance Optimization
5 Section 4: Serving and Scaling Models (~20%)
4.1 Batch and Online Inference: Agent Platform, Model Garden, Cloud Run and GKE 4.2 Rollout Strategies: Canary, Traffic Splitting, Shadow and Blue-Green 4.3 Tuning Models for Production: Vector Search and Model Compression 4.4 Deploying Models to Public and Private Endpoints 4.5 Choosing Serving Hardware: CPU, GPU, TPU and Edge 4.6 Scaling the Serving Backend for Throughput
6 Section 5: Automating and Orchestrating ML Pipelines (~18%)
5.1 Validating Data and Models Inside Pipelines 5.2 Agent Platform Pipelines: Kubeflow Pipelines (KFP) and TFX Architecture 5.3 Reusable Pipeline Components, Artifacts and ML Metadata Lineage 5.4 Ray on Agent Platform for Distributed Python Workloads 5.5 MLOps CI/CD Automation, Retraining Triggers and Skew Prevention
7 Section 6: Monitoring AI Solutions (~13%)
6.1 Securing AI Systems: Exfiltration, Malicious Prompting and Model Armor 6.2 Responsible AI, Explainable AI, Fairness Metrics and Model Governance 6.3 Model Monitoring: Feature Skew and Drift Detection 6.4 Operational Troubleshooting, Performance Profiling and Cost Optimization 6.5 Monitoring, Testing and Evaluating Generative AI in Production
6.1 Securing AI Systems: Exfiltration, Malicious Prompting and Model Armor
Key Takeaways Explain This Topic AI
Model Armor screens both prompts and responses for prompt injection, jailbreak attempts, malicious URLs, sensitive data disclosure, and unsafe content, and integrates in line with the Agent Platform.
Prompt injection is an instruction hidden in data the model reads, which is why retrieved documents and tool outputs must be treated as untrusted input rather than trusted context.
Defence is layered: input screening, least-privilege tool and data access for the model, output screening, and a perimeter that limits what can leave.
Sensitive data should be de-identified before it reaches a model, and responses should be screened before they reach a user, because a model can surface training or context data verbatim.
Safety filters govern harmful content categories while Model Armor adds security-oriented screening; they solve different problems and are configured separately.
Last updated: September 2026
6.1 Securing AI Systems: Exfiltration, Malicious Prompting and Model Armor
Blueprint reference: Section 6.1, "Building secure AI systems by protecting against unintentional exploitation and leaks of data or models (e.g., data exfiltration, malicious prompting, and sharing sensitive data with LLMs) using the appropriate security tool (e.g., Regex, safety filters, and Model Armor)."
This is one of the newest considerations in the exam guide and is absent from most pre-2026 preparation material. It also carries an important reframing: Section 6 is no longer only about monitoring . Identifying and mitigating risks to AI solutions is now the first sub-section.
The Threat Model
Threat What it looks like
Prompt injection Instructions embedded in data the model reads — a retrieved document, a web page, an email, a tool response — that override the developer's intent
Jailbreaking A crafted user prompt that circumvents the system instruction or safety policy
Sensitive data disclosure The model emits PII, secrets, or confidential content from its context, its training data, or a retrieved document
Data exfiltration An attacker induces the system to send data outward — through a generated URL, a tool call, or a crafted response
- Model or prompt theft System instructions or proprietary model behaviour extracted through probing
- Unsafe content Harmful, hateful, or otherwise policy-violating output
- Insider copying An authorized principal copies training data or model artifacts outside the boundary
The critical mental shift for prompt injection: anything the model reads is input, not instruction . A retrieved document is data. A tool's return value is data. Treating retrieved context as trusted is how injection succeeds.
Model Armor
Model Armor is Google Cloud's screening service for AI prompts and responses. It sits between the application and the model, inspecting traffic in both directions.
What it screens for:
Prompt injection and jailbreak detection — identifying attempts to override instructions or bypass policy.
Malicious URL detection — links embedded in prompts or responses that would lead a user or an agent somewhere harmful.
Sensitive Data Protection integration — detecting and de-identifying PII in prompts and responses.
Responsible AI content filtering — harmful content categories.
How it is configured: a Model Armor template is a named set of filters and confidence thresholds that defines how content is screened. Templates are applied to workloads, which means the security policy is centrally defined and versioned rather than reimplemented in each application. Model Armor offers in-line integration with the Agent Platform so screening applies without every caller having to invoke it explicitly, and it also integrates with Gemini Enterprise.
The exam-relevant distinction: safety filters address harmful content categories; Model Armor adds security-oriented screening — injection, jailbreak, malicious URLs, sensitive data — on top. A scenario about hate speech points at safety filters; a scenario about instructions hidden in a retrieved PDF points at Model Armor.
Regex and Deterministic Checks
The exam guide names Regex alongside the managed tools, and it belongs in the stack for the same reason input validation belongs in any application: deterministic pattern checks are cheap, predictable, and auditable.
Appropriate uses: blocking known-forbidden tokens, validating that a structured output matches an expected shape, redacting a well-defined internal identifier format, enforcing an allowlist of URL domains in generated output.
Its limit, which is the point of naming it alongside the others: regex cannot detect semantic attacks. "Ignore your previous instructions" has unbounded paraphrases. Regex is a layer, never the strategy.
- Layered Defence User / retrieved content
- │
- [1] Input screening ......... Model Armor template + regex validation
- │ de-identify PII before it reaches the model
- [2] Model call .............. safety filters configured
- │ system instruction hardened; context clearly delimited
- [3] Tool / data access ...... least-privilege service account; allowlisted tools;
- │ no ambient credentials in the model's reach
- [4] Output screening ........ Model Armor on responses; SDP de-identification;
- │ URL allowlist; structured-output validation
- [5] Perimeter .............. VPC Service Controls; private endpoints; audit logging
- │
- Response to user
Each layer addresses what the others cannot:
Input screening catches known attack patterns but cannot catch every novel phrasing.
Least-privilege tool access is the layer that makes a successful injection harmless: if the model can only call a read-only search tool scoped to one dataset, an attacker who fully hijacks the instruction still cannot delete records or reach another customer's data. This is the most durable control and the most frequently omitted.
Output screening catches disclosure that the model produced despite the earlier layers.
The perimeter limits what can leave regardless of what happens inside.
Sharing Sensitive Data with Models
The blueprint names this explicitly. The rules:
De-identify before the prompt where the task does not require raw values. Sensitive Data Protection can tokenize or redact in the request path.
Assume prompts and responses may be logged. Request-response logging, debugging traces, and evaluation datasets all persist content; if raw PII is in the prompt, it is now in those stores too.
Do not put secrets in system instructions. They are extractable by probing.
Scope grounding data. A retrieval corpus that contains records the requesting user is not entitled to see will eventually surface them; apply the same access filters to retrieval that you apply to the underlying data.
Insider and Exfiltration Controls
For the "authorized person copies the data" case, IAM is insufficient by definition. The controls are:
VPC Service Controls — a perimeter that blocks data movement to projects outside it.
Private endpoints — no public network path.
CMEK — organization-controlled keys.
Audit logging and Access Transparency — evidence of who did what.
Exam Traps
Treating retrieved documents as trusted context. They are untrusted input.
Relying on the system instruction alone to prevent injection.
Regex as the whole defence. It cannot catch semantic attacks.
Confusing safety filters with Model Armor. Content categories versus security screening.
Broad tool permissions for an agent, which turns a successful injection into a breach.
Sending raw PII to a model when de-identified values would do.