Study guide
18.3 Responsible AI: Detecting & Monitoring Bias
Google Cloud Professional Machine Learning Engineer PMLE
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam Overview & the Agent Platform Map
1.1 PMLE Exam Facts, Policies & the June 2026 Blueprint 1.2 From Vertex AI to Gemini Enterprise Agent Platform: The Product Name Map
2 Chapter 2: Low-Code ML with BigQuery ML
2.1 Choosing, Training & Predicting with BigQuery ML Models 2.2 Feature Engineering & Feature Selection in BigQuery ML 2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery
3 Chapter 3: AutoML & Google Cloud AI APIs
3.1 Training Models with Agent Platform AutoML 3.2 Building with Document AI, Vision, Translation & Other AI APIs
4 Chapter 4: Foundation Models in Model Garden
4.1 Evaluating & Selecting Models in Model Garden 4.2 Building & Tuning Solutions with Gemini, Imagen, Veo & MaaS 4.3 Optimizing Gemini Applications for Cost, Latency & Availability
5 Chapter 5: Exploring, Preprocessing & Protecting Data
5.1 Organizing & Exploring Tabular, Text & Image Data 5.2 Choosing a Preprocessing Tool: BigQuery, Dataflow, Spark or Python 5.3 Creating & Consolidating Features in Agent Platform Feature Store 5.4 Protecting PII & Sensitive Data in ML Workflows
6 Chapter 6: Notebooks, Prototyping & Experiment Tracking
6.1 Workbench vs. Colab Enterprise: Secure, Collaborative Notebooks 6.2 Prototyping with PyTorch, scikit-learn, JAX & Model Garden in Notebooks 6.3 Experiment Environments, Tracking & Lineage with Experiments and ML Metadata
7 Chapter 7: Evaluating Predictive & Generative AI Solutions
7.1 Evaluation Metrics for Predictive Models 7.2 Evaluating Gen AI: Computation Metrics, Rubrics & LLM-as-a-Judge
8 Chapter 8: Choosing the Modeling Approach
8.1 Choosing the Model Type: ARIMA, Trees, DNNs & LLMs 8.2 Choosing the Product & Deployment Strategy 8.3 Modeling Techniques for Interpretability Requirements
9 Chapter 9: Training Data & Training SDKs
9.1 Organizing & Ingesting Training Data from Cloud Storage & BigQuery 9.2 Custom Training on Agent Platform: Containers, Jobs & Worker Pools 9.3 Kubeflow on GKE, AutoML & Tabular Workflows as Training Choices
10 Chapter 10: Troubleshooting, Hyperparameter Tuning & Fine-Tuning
10.1 Troubleshooting ML Training Failures 10.2 Hyperparameter Tuning on Agent Platform 10.3 Fine-Tuning Foundation Models & When to Tune
11 Chapter 11: Training Hardware & Distributed Training
11.1 Choosing CPUs, GPUs & TPUs for Training 11.2 Distributed Training: Data & Model Parallelism
12 Chapter 12: Deploying Models for Inference
12.1 Batch vs. Online Inference on Agent Platform 12.2 Serving Beyond Agent Platform Endpoints: Model Garden, Cloud Run & GKE 12.3 Prebuilt & Custom Serving Containers
13 Chapter 13: Model Versions, Rollouts & Inference Logic
13.1 Organizing & Versioning Models in Model Registry 13.2 Rollout Strategies: A/B Tests, Canaries & Traffic Splits 13.3 Inference Preprocessing & Postprocessing
14 Chapter 14: Scaling Online Serving
14.1 Serving Features Online with Feature Store 14.2 Public, Dedicated & Private Endpoints 14.3 Serving Hardware & Autoscaling for Throughput 14.4 Tuning Models for Production Training & Serving
15 Chapter 15: End-to-End ML Pipelines
15.1 Validating Data & Models in Pipelines 15.2 Agent Platform Pipelines with KFP: Components, Templates & Caching 15.3 Ensuring Consistent Preprocessing Between Training & Serving
16 Chapter 16: Orchestration with Managed Airflow & Ray
16.1 Orchestrating ML Workflows with Managed Service for Apache Airflow 16.2 Ray on Agent Platform for Distributed ML Workloads
17 Chapter 17: Automating Retraining & CI/CD/CT
17.1 Choosing a Retraining Policy 17.2 CI/CD/CT for ML with Cloud Build
18 Chapter 18: Securing AI Systems, Responsible AI & Explainability
18.1 Protecting AI Systems from Data & Model Exfiltration 18.2 Defending Gen AI Apps: Safety Filters, Model Armor & Regex Guardrails 18.3 Responsible AI: Detecting & Monitoring Bias 18.4 Model Explainability on Agent Platform
19 Chapter 19: Monitoring AI Solutions
19.1 Configuring Model Monitoring on Agent Platform 19.2 Training-Serving Skew, Data Drift, Concept Drift & Attribution Drift 19.3 Monitoring, Testing & Evaluating Gen AI Solutions in Production
18.3 Responsible AI: Detecting & Monitoring Bias
Key Takeaways Explain This Topic AI
Agent Platform data bias metrics include Difference in Population Size and Difference in Positive Proportions in True Labels, computed before training.
Agent Platform model bias metrics include Accuracy Difference, Difference in Positive Proportions in Predicted Labels, Recall Difference, Specificity Difference, and Difference in Ratio of Error Types.
In Agent Platform's fairness metrics, a positive value generally indicates potential bias favoring slice 1, zero indicates no difference, and a negative value favors slice 2.
Bias can enter through unrepresentative data, historical labels, proxy features, and deployment context, so fairness checks belong before training, before deployment, and in production monitoring.
For gen AI, responsible AI practice combines safety filters, evaluation of outputs across user groups, and human review of high-impact use cases.
Last updated: September 2026
The exam guide lists responsible AI practices (for example, monitoring for bias) as part of identifying risks to AI solutions. The exam expects you to recognize bias sources, pick suitable fairness metrics, and put checks into the lifecycle, not just to state principles.
Where Bias Comes From
- Source Example
- Representation A smile-detection dataset with far fewer older people
- Historical labels Past hiring decisions that favored one group become the "ground truth"
- Measurement Different data quality across regions (older devices, missing fields)
- Proxy features ZIP code standing in for protected attributes
- Aggregation One model for populations that behave differently
- Deployment context A model used for decisions it wasn't designed or validated for
Feedback loops A model's decisions shape future training data, such as fewer loans giving fewer repayment examples
Agent Platform Fairness Metrics
Agent Platform provides bias metrics (preview) comparing slice 1 (the group measured) against slice 2 (the comparison group). For most metrics, positive values suggest potential bias favoring slice 1 , zero means no difference , and negative values favor slice 2 . The evaluation pipeline components DetectDataBiasOp and DetectModelBiasOp compute them.
Data bias metrics (before training)
Example: 200 California applicants (60 accepted) and 100 Florida applicants (20 accepted).
- Metric Formula Example
- Difference in Population Size (n₁ − n₂) / (n₁ + n₂) (200 − 100) / 300 = 0.33
Difference in Positive Proportions in True Labels (DPPTL) , also called label imbalance positives₁/n₁ − positives₂/n₂ 60/200 − 20/100 = 0.10
Model bias metrics (after training)
Confusion matrices: California TP 50, FN 10, FP 20, TN 120. Florida TP 20, FN 0, FP 30, TN 50.
- Metric Compares Example
- Accuracy Difference (TP+TN)/n for each slice 170/200 − 70/100 = 0.15
- Difference in Positive Proportions in Predicted Labels (DPPPL) Predicted positives / n 70/200 − 50/100 = −0.15
- Recall Difference TP / (TP+FN) 50/60 − 20/20 = −0.17
- Specificity Difference TN / (TN+FP) 120/140 − 50/80 ≈ 0.23
Difference in Ratio of Error Types (treatment equality) FN/FP for each slice 10/20 − 0/30 = 0.50 . Both slices have 30 errors, but California's errors include missed acceptances while Florida's are all false acceptances
Reading the example: the model is more accurate for California (+0.15) but predicts acceptance more often for Florida (−0.15) and has higher recall for Florida (−0.17). Which gap matters depends on the harm. In admissions, wrongly rejecting qualified applicants (recall) may be the main concern.
Choosing Fairness Criteria
There is no single fairness metric. They can conflict, so choose based on the decision's harms:
- If the main harm is... Watch
- Qualified people being denied (missed opportunities) Recall difference (equal opportunity)
- Unqualified people being wrongly approved or flagged False positive rate or specificity difference
- Unequal selection rates across groups DPPPL (demographic parity)
- Overall quality gaps Accuracy difference plus per-slice precision and recall
Involve domain, legal, and affected stakeholders in picking metrics and acceptable thresholds.
Putting Fairness into the Lifecycle
- Stage Practice
- Problem framing Decide if ML is appropriate. Document intended use and misuse
- Data Check representation (Difference in Population Size) and label imbalance (DPPTL). Look for proxies
Training Reweight or resample underrepresented groups. Consider constraints or separate thresholds where lawful
- Evaluation Slice metrics and bias metrics as pipeline gates (Chapter 15)
- Deployment Canary monitoring by segment. Human review for high-impact decisions
- Production Monitor prediction distributions and outcomes by slice , and retrain or roll back when gaps widen
- Documentation Record data sources, evaluation slices, limitations, and approvals with the model version
- Mitigation Techniques
Collect more representative data for underrepresented groups. This is often the most effective fix.
Reweight or resample training examples, for example with AutoML's row weight column.
Remove or transform proxy features , and test whether performance gaps shrink.
Adjust decision thresholds per operating context where policy and law allow.
Human-in-the-loop review for borderline or high-impact predictions.
Explain predictions (Section 18.4) to find features driving unequal outcomes.
Monitoring Fairness in Production
Fairness at launch doesn't guarantee fairness later, because populations and behavior change.
Log the slice attributes needed for analysis (where lawful and privacy-approved) alongside predictions.
Schedule bias metrics on recent predictions and, once labels arrive, on outcomes. For example, run the model bias component monthly in a pipeline.
Alert on thresholds agreed with stakeholders, such as a recall difference beyond ±0.05.
Investigate root causes with slice-level feature distributions and explanations before retraining.
Record decisions (accepted trade-offs, mitigations) with the model version for accountability.
Responsible Gen AI
Safety filters and Model Armor reduce harmful outputs (Section 18.2).
Evaluate across user groups and languages , because quality and safety can vary by dialect or topic (Gen AI evals, Chapter 7).
Grounding and citations reduce fabricated claims that can mislead users.
Transparency: tell users they are interacting with AI and where content is AI-generated.
Human oversight for medical, legal, financial, and employment uses.