13.3 Inference Preprocessing & Postprocessing

Google Cloud Professional Machine Learning Engineer PMLE

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Exam Overview & the Agent Platform Map

1.1 PMLE Exam Facts, Policies & the June 2026 Blueprint 1.2 From Vertex AI to Gemini Enterprise Agent Platform: The Product Name Map

2 Chapter 2: Low-Code ML with BigQuery ML

2.1 Choosing, Training & Predicting with BigQuery ML Models 2.2 Feature Engineering & Feature Selection in BigQuery ML 2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery

3 Chapter 3: AutoML & Google Cloud AI APIs

3.1 Training Models with Agent Platform AutoML 3.2 Building with Document AI, Vision, Translation & Other AI APIs

4 Chapter 4: Foundation Models in Model Garden

4.1 Evaluating & Selecting Models in Model Garden 4.2 Building & Tuning Solutions with Gemini, Imagen, Veo & MaaS 4.3 Optimizing Gemini Applications for Cost, Latency & Availability

5 Chapter 5: Exploring, Preprocessing & Protecting Data

5.1 Organizing & Exploring Tabular, Text & Image Data 5.2 Choosing a Preprocessing Tool: BigQuery, Dataflow, Spark or Python 5.3 Creating & Consolidating Features in Agent Platform Feature Store 5.4 Protecting PII & Sensitive Data in ML Workflows

6 Chapter 6: Notebooks, Prototyping & Experiment Tracking

6.1 Workbench vs. Colab Enterprise: Secure, Collaborative Notebooks 6.2 Prototyping with PyTorch, scikit-learn, JAX & Model Garden in Notebooks 6.3 Experiment Environments, Tracking & Lineage with Experiments and ML Metadata

7 Chapter 7: Evaluating Predictive & Generative AI Solutions

7.1 Evaluation Metrics for Predictive Models 7.2 Evaluating Gen AI: Computation Metrics, Rubrics & LLM-as-a-Judge

8 Chapter 8: Choosing the Modeling Approach

8.1 Choosing the Model Type: ARIMA, Trees, DNNs & LLMs 8.2 Choosing the Product & Deployment Strategy 8.3 Modeling Techniques for Interpretability Requirements

9 Chapter 9: Training Data & Training SDKs

9.1 Organizing & Ingesting Training Data from Cloud Storage & BigQuery 9.2 Custom Training on Agent Platform: Containers, Jobs & Worker Pools 9.3 Kubeflow on GKE, AutoML & Tabular Workflows as Training Choices

10 Chapter 10: Troubleshooting, Hyperparameter Tuning & Fine-Tuning

10.1 Troubleshooting ML Training Failures 10.2 Hyperparameter Tuning on Agent Platform 10.3 Fine-Tuning Foundation Models & When to Tune

11 Chapter 11: Training Hardware & Distributed Training

11.1 Choosing CPUs, GPUs & TPUs for Training 11.2 Distributed Training: Data & Model Parallelism

12 Chapter 12: Deploying Models for Inference

12.1 Batch vs. Online Inference on Agent Platform 12.2 Serving Beyond Agent Platform Endpoints: Model Garden, Cloud Run & GKE 12.3 Prebuilt & Custom Serving Containers

13 Chapter 13: Model Versions, Rollouts & Inference Logic

13.1 Organizing & Versioning Models in Model Registry 13.2 Rollout Strategies: A/B Tests, Canaries & Traffic Splits 13.3 Inference Preprocessing & Postprocessing

14 Chapter 14: Scaling Online Serving

14.1 Serving Features Online with Feature Store 14.2 Public, Dedicated & Private Endpoints 14.3 Serving Hardware & Autoscaling for Throughput 14.4 Tuning Models for Production Training & Serving

15 Chapter 15: End-to-End ML Pipelines

15.1 Validating Data & Models in Pipelines 15.2 Agent Platform Pipelines with KFP: Components, Templates & Caching 15.3 Ensuring Consistent Preprocessing Between Training & Serving

16 Chapter 16: Orchestration with Managed Airflow & Ray

16.1 Orchestrating ML Workflows with Managed Service for Apache Airflow 16.2 Ray on Agent Platform for Distributed ML Workloads

17 Chapter 17: Automating Retraining & CI/CD/CT

17.1 Choosing a Retraining Policy 17.2 CI/CD/CT for ML with Cloud Build

18 Chapter 18: Securing AI Systems, Responsible AI & Explainability

18.1 Protecting AI Systems from Data & Model Exfiltration 18.2 Defending Gen AI Apps: Safety Filters, Model Armor & Regex Guardrails 18.3 Responsible AI: Detecting & Monitoring Bias 18.4 Model Explainability on Agent Platform

19 Chapter 19: Monitoring AI Solutions

19.1 Configuring Model Monitoring on Agent Platform 19.2 Training-Serving Skew, Data Drift, Concept Drift & Attribution Drift 19.3 Monitoring, Testing & Evaluating Gen AI Solutions in Production

13.3 Inference Preprocessing & Postprocessing

Key Takeaways Explain This Topic AI

Preprocessing can live in the model graph, in the serving container (for example, a custom prediction routine), or upstream in the application, and the choice affects training-serving consistency.

Batch inference instanceConfig can include or exclude input fields and send instances as JSON objects or arrays without custom container code.

Postprocessing turns raw model outputs into decisions: thresholds, label mapping, top-k filtering, calibration, and business rules.

Gen AI postprocessing validates structured outputs against a schema, checks grounding or citations, and filters unsafe or sensitive content before responses reach users.

Putting shared preprocessing inside the exported model or serving container avoids re-implementing it in each client application.

Last updated: September 2026

The exam guide lists developing solutions for inference preprocessing and postprocessing . The design question is where each transformation runs, because that decides latency, consistency with training, and how many teams must maintain the logic.

Where Preprocessing Can Run

Location Examples Pros Cons

Inside the model Keras preprocessing layers, a TensorFlow Transform graph exported with the model, BigQuery ML TRANSFORM Training and serving use identical logic. Clients send raw features Limited to operations the framework can express

Serving container Custom prediction routine preprocess() , custom container code, Triton ensembles Arbitrary Python logic (tokenizers, lookups) next to the model. One implementation for all clients Adds latency on every request. Code must match training

Upstream in the application or gateway API service validates and enriches the request, and fetches features from Feature Store Keeps the model server simple. Reuses app data Risk of skew if clients implement it differently

Data pipeline (batch/streaming) Dataflow MLTransform then RunInference , BigQuery SQL before batch scoring Scales for large volumes. Full-pass statistics computed once Not for synchronous low-latency calls

Rule: logic that must match training exactly should be inside the model or shared code used by both training and serving (Chapter 15). Business logic that changes often (display labels, promotion rules) belongs after the model, where it can change without retraining.

• Preprocessing Patterns
• Online inference

Validate inputs: types, ranges, and required fields. Reject bad requests with clear errors instead of producing garbage predictions.

Enrich with features the client doesn't have. For example, fetch customer_90d_spend from Feature Store online serving (Chapter 14).

Transform : scale, encode, and tokenize, using the same fitted artifacts (vocabularies, scalers) saved at training time.

Batch small requests on the server when the framework supports it (for example, TensorFlow Serving's batching configuration) to use GPUs efficiently.

Custom prediction routines (CPR)

A CPR Predictor separates stages cleanly:

• Method Typical work
• load(artifacts_uri) Load the model plus preprocessing artifacts, such as a scaler or tokenizer
• preprocess(request) Parse instances, validate, and transform
• predict(instances) Run the model
• postprocess(outputs) Map outputs to response JSON with labels, scores, and reasons

A custom Handler covers raw HTTP concerns such as unusual payload formats or headers.

Batch inference without custom code

instanceConfig on a batch inference job can:

excludedFields or includedFields : drop keys or extra columns (such as customerId ) from what the model sees, while keeping them in the output for joins.

instanceType : send each instance as a JSON object or array .

For heavier preprocessing, run a BigQuery or Dataflow step before the batch job, or use Dataflow RunInference so transformation and scoring run in one pipeline.

Postprocessing for Predictive Models

• Step Example
• Thresholding Convert probability 0.73 to "flag for review" using the business-chosen threshold (Chapter 7)
• Label mapping Class index 4 → "Water damage"
• Top-k and filtering Return the top 10 recommendations, excluding items already purchased or out of stock
• Calibration Adjust scores so a 0.8 score really means about 80% likelihood
• Business rules Never auto-decline a loan. Send declines to human review
• Explanations formatting Turn feature attributions into human-readable reason codes
• Output validation Clamp impossible values, such as negative delivery times
• Pre- and Postprocessing for Gen AI

Stage Examples

Pre Prompt templating with system instructions, retrieving grounding context (RAG), redacting PII from user input, screening prompts for injection (Model Armor, Chapter 18), trimming context to control tokens

Post Validating JSON against the response schema , checking that citations exist, filtering unsafe content, redacting sensitive data in responses, retry or fallback when validation fails

Example: An insurance claims assistant sends a claim note with retrieved policy clauses and a JSON schema. Postprocessing validates the JSON, confirms each cited clause ID exists, and redacts any account numbers before returning the result.

Testing Pre- and Postprocessing

Unit tests for each transformation, with fixed inputs and expected outputs.

Parity tests that run the training-time transformation and the serving-time transformation on the same records and require identical results.

Contract tests that send real client payloads, including malformed ones, to a staging endpoint.

Golden prediction sets : fixed inputs whose expected outputs are checked after every container or model update.

Performance and Reliability Considerations

Measure latency by stage. Tokenization or feature lookups can exceed model inference time.

Cache static lookups, such as vocabularies and reference tables, in memory at load() .

Keep containers stateless so autoscaling replicas behave identically.

Version preprocessing artifacts with the model version . A new vocabulary means a new model version.

Log inputs after preprocessing (where privacy allows), so skew monitoring compares what the model actually saw.

Worked Scenario

A mobile app sends raw text reviews for sentiment scoring, and several app teams call the endpoint.

Put tokenization and truncation in a CPR preprocess() , using the tokenizer saved at training time, so no app team re-implements it.

Return {"sentiment": "negative", "score": 0.91} from postprocess() , and keep the threshold configurable.

Fetch nothing upstream. Clients send only the review text and a request ID.