Study guide

2.2 Foundation Model Selection Criteria

Google Cloud Generative AI Leader Exam

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Gen AI Foundations & Core Concepts

1.1 Traditional AI vs. Machine Learning vs. Deep Learning vs. Generative AI 1.2 Supervised, Unsupervised, and Reinforcement Learning 1.3 Foundation Models, LLMs, and Diffusion Architectures 1.4 Structured, Semi-Structured, and Unstructured Data in AI 1.5 Data Quality, Data Accessibility, and Labeled vs. Unlabeled Data 1.6 The Four Gen AI Use-Case Patterns: Create, Summarize, Discover, Automate

2 Chapter 2: The Gen AI Lifecycle, Landscape & Model Selection

2.1 The Machine Learning and Gen AI Lifecycle 2.2 Foundation Model Selection Criteria 2.3 The Five Layers of the Gen AI Landscape

3 Chapter 3: Google’s Foundation Model Family

3.1 The Gemini Foundation Model Family 3.2 Gemma, Code Models, and Google's Open Approach 3.3 Imagen, Veo, and Digital Provenance with SynthID

4 Chapter 4: Why Google Cloud — The Enterprise Gen AI Advantage

4.1 Google's AI-First Approach and Comprehensive AI Ecosystem 4.2 An Enterprise-Ready AI Platform and Control Over Your Data 4.3 Democratizing AI Development: No-Code, Low-Code, and Pre-Trained APIs

5 Chapter 5: Google Cloud Infrastructure & the Agent Platform

5.1 Google Cloud AI Hypercomputer & Accelerators 5.2 Gemini Enterprise Agent Platform Architecture & MLOps 5.3 Agent Studio vs. Google AI Studio 5.4 Data Infrastructure & Vector Storage for Gen AI

6 Chapter 6: Google Gen AI Applications & Workplace Solutions

6.1 Gemini App and Gemini Advanced 6.2 Gemini for Google Workspace 6.3 Gemini Enterprise: The Front Door to AI at Work 6.4 NotebookLM and Gemini in Google Cloud

7 Chapter 7: Search, Agents & Customer Engagement

7.1 Agent Search & Grounded Enterprise Retrieval 7.2 Building Custom Agents on Agent Platform 7.3 Google Cloud's RAG Offerings and Grounding Services 7.4 Customer Engagement Suite & Specialized Cloud APIs

8 Chapter 8: Prompt Engineering Techniques

8.1 Fundamentals & Anatomy of Effective Prompts 8.2 Zero-Shot, Few-Shot, and Role Prompting 8.3 Advanced Prompting: Chain-of-Thought, ReAct, and Chaining

9 Chapter 9: Improving Model Output — Grounding, RAG, Tuning & Controls

9.1 Model Limitations, Hallucinations, and Mitigations 9.2 Grounding Sources: First-Party, Third-Party, and World Data 9.3 Grounding Patterns and Retrieval-Augmented Generation (RAG) 9.4 Fine-Tuning & Model Customization Strategies 9.5 Sampling Parameters, Output Controls & Safety Filters 9.6 Continuous Monitoring, Evaluation, and Model Lifecycle Management

10 Chapter 10: Business Strategy & Gen AI Adoption

10.1 Gen AI Solution Types and Choosing the Right One 10.2 Gen AI Use Case Identification & ROI Measurement 10.3 Enterprise Adoption, Change Management & Upskilling

11 Chapter 11: Secure AI & Responsible AI Governance

11.1 Google Secure AI Framework (SAIF) & Cloud Security 11.2 Responsible AI Principles, Fairness, and Privacy 11.3 Accountability, Explainability, and Transparency in AI Systems

2.2 Foundation Model Selection Criteria

Key Takeaways Explain This Topic AI

Model selection is a strategic multi-criteria balancing act across modality, reasoning capability, context window capacity, latency profiles, token economics, and enterprise data governance.

Selecting model size follows the principle of proportional capability: lightweight models (e.g., Gemini Flash, Gemma) deliver high speed and low cost for straightforward operational tasks, while frontier models (Gemini Pro) handle complex, multi-hop reasoning.

Million-token context windows (e.g., Gemini 3.1 Pro and Gemini 3.5 Flash with 1M tokens) fundamentally simplify enterprise architectures by enabling in-context document ingestion, native video analysis, and reduced reliance on complex chunking heuristics.

Prompt caching dramatically reshapes long-context economics, offering substantial discounts (up to 75%–80%) on input tokens when the same extensive context (e.g., codebases, legal repositories) is queried repeatedly.

Enterprise cloud contracts must guarantee that customer prompts, inputs, and generated outputs are never used to train the vendor's foundation models, backed by intellectual property indemnification.

Last updated: September 2026

Foundation Model Selection Criteria

Executive Summary: Selecting the right foundation model is one of the most consequential architectural decisions an enterprise technology leader will make. There is no single "best" model for all enterprise workloads. Instead, organizations must evaluate foundation models across seven technical and strategic dimensions: modality, reasoning capability, context window size, latency and throughput, cost structure, deployment topology, and data privacy governance. Matching model capabilities to specific workload requirements ensures optimal performance while avoiding wasteful cloud spending.

In the rapidly evolving foundation model landscape, leaders face a bewildering variety of models: proprietary frontier giants, lightweight high-speed models, open-weights releases, and domain-specialized systems. Attempting to use a massive frontier reasoning model for every operational query leads to exorbitant costs and unacceptable user latency. Conversely, deploying a small, under-parameterized model for complex multi-step policy synthesis results in frequent errors and hallucinations. A structured selection framework is essential.

The Foundation Model Selection Dimensions

To make objective, defensible architectural choices, enterprise architects and business leaders should evaluate foundation models against nine core dimensions, which together cover every criterion the official exam guide names (modality, context window, security, availability and reliability, cost, performance, fine-tuning, and customization): Foundation Model Selection Framework

  • ├── 1. Modality (Text, Vision, Audio, Video, Multimodal)
  • ├── 2. Size & Reasoning Tier (Frontier vs. Efficient / Small)
  • ├── 3. Context Window & Memory Retention (8k to 2M+ tokens)
  • ├── 4. Latency & Throughput (TTFT, Inter-token speed, Batch vs. Real-Time)
  • ├── 5. Cost Structure (Input tokens, Output tokens, Caching, Provisioned)
  • ├── 6. Deployment Topology (Managed API, Dedicated Endpoints, Edge / On-Device)
  • └── 7. Data Privacy, Compliance & Licensing (Indemnification, Zero-Data Retention)
  • 1. Modality: Text-Only vs. Native Multimodality

Early generative models were strictly unimodal (e.g., text-in, text-out). Modern enterprise foundation models, pioneered by Google's Gemini architecture, are natively multimodal from the ground up:

Unimodal / Text-Only Models: Well-suited for straightforward text transformation, translation, tabular data summarization, and classic NLP tasks. Often lower cost and simpler to deploy.

Multimodal Models (Text, Images, Audio, Video, Code): Natively process and cross-reference diverse data types within a single prompt. For example, Gemini 3 can ingest a 45-minute video recording of an industrial plant inspection alongside technical PDF blueprints and answer: "At what timestamp did the technician identify corrosion on valve B-12, and how does that compare to the maintenance manual specification?"

2. Model Size and Capability Tiers

Models generally fall into three distinct operational tiers:

Frontier / High-Reasoning Tier (e.g., Gemini 3.1 Pro): Designed for complex multi-step logical deduction, intricate code refactoring, multi-document synthesis, and complex mathematical or scientific analysis. These models have large parameter counts, produce high-quality reasoning, but operate with higher per-token costs and slightly longer latency.

Lightweight / Efficient Tier (e.g., Gemini 3.5 Flash): Optimized for high-frequency, low-latency operational workloads. Flash offers extraordinary speed, highly competitive reasoning for the vast majority of enterprise tasks (such as sentiment analysis, data extraction, chat routing, and first-draft summarization), and costs a fraction of frontier models.

Open-Weights / Edge Tier (e.g., Gemma 4 26B A4B / 31B Dense, Gemini Nano): Open-weights models that can be self-hosted on private virtual machines (Compute Engine, Google Kubernetes Engine) or run directly on client edge devices (smartphones, IoT gateways). Ideal for air-gapped environments, strict data sovereignty regulations, or zero-connectivity edge computing.

3. Context Window and Information Retention

The context window defines the maximum quantity of tokens (words, sub-words, image patches, audio seconds) a model can accept and process in a single interaction:

Standard Context Windows (4k – 32k tokens): Require complex chunking, vector embeddings, and multi-stage RAG architectures to feed small excerpts of documents to the model. Risk losing macro-level context or cross-document narrative relationships.

Extended Context Windows (128k tokens): Handle single books or moderate financial reports, easing RAG retrieval constraints.

Ultra-Long Context Windows (1 Million – 2 Million+ tokens): Pioneered by Gemini 3.1 Pro and Gemini 3.5 Flash. A 2-million-token context window allows an enterprise to ingest up to 1.5 hours of video, roughly 8 hours of audio, 60,000 lines of source code, or over 1.4 million words in a single prompt. Models with perfect "needle-in-a-haystack" retrieval can analyze entire enterprise codebases or multi-year legal archives without requiring external vector chunking.

4. Latency and Throughput Profiles

Application requirements dictate acceptable performance thresholds:

Time to First Token (TTFT): The time elapsed between sending the prompt and the model outputting its initial token. In real-time conversational agents, customer care chatbots, or interactive voice bots, a TTFT under 500–800 milliseconds is vital to prevent user frustration.

Inter-Token Latency (ITL) & Tokens per Second (TPS): The speed at which subsequent tokens are generated. High TPS is critical for code completion, document drafting, and reading aloud via text-to-speech.

Synchronous Interactive vs. Asynchronous Batch: Interactive user experiences demand lightweight models (e.g., Gemini Flash) with streaming enabled. Conversely, overnight batch extraction of 500,000 insurance claims can utilize frontier models via asynchronous batch prediction APIs, which often provide significant cost discounts (e.g., 50% discount on Agent Platform Batch Inference).

5. Cost Structure and Token Economics

Gen AI pricing is fundamentally consumption-based, calculated per million tokens (MTok):

Input Token Pricing: The cost to process the prompt, system instructions, and supplied context.

Output Token Pricing: The cost to generate the response. Output tokens are typically 3x to 4x more expensive than input tokens because generating tokens is autoregressive and computationally intensive (generating token N+1 requires re-evaluating preceding tokens).

Context Caching: Google Cloud Agent Platform allows developers to cache static, recurring context (such as a 500-page operational manual, medical repository, or system instruction) in memory. When subsequent queries reference this cached context, input token costs are reduced by up to 75% to 80% , and latency drops substantially.

Provisioned Throughput: For enterprise applications with sustained high traffic, organizations can purchase dedicated Provisioned Throughput (processing units) ensuring zero rate-limiting, consistent low latency, and predictable monthly billing instead of pay-as-you-go token metering.

6. Deployment Topology: Serverless API vs. Self-Hosted vs. Edge

Where does the model physically execute?

Managed Serverless API (Agent Platform Gemini API): Fully managed, automatically scalable, zero infrastructure management, pay-per-token or provisioned. Upgraded transparently by Google.

Self-Hosted Open-Weights (Gemma on GKE or Compute Engine): The enterprise provisions and manages virtual machines equipped with accelerators (e.g., NVIDIA L4/A100/H100 or Cloud TPU v5e). The enterprise has complete control over the serving stack (vLLM, TGI), custom quantization, and internal network perimeters, but incurs 24/7 infrastructure hosting bills regardless of traffic.

On-Device / Edge (Gemini Nano): Executes locally on mobile devices or edge silicon without internet connectivity. Guarantees complete data privacy on the client device and zero network latency, but is limited by the computational power of mobile processors.

7. Data Privacy, Compliance, and Commercial Licensing

Enterprise adoption hinges on legal and security protections:

Enterprise Data Boundary (Google Cloud Commitment): Google Cloud explicitly guarantees that customer prompts, generated responses, and proprietary data submitted to Agent Platform are never used to train Google's base foundation models , are never shared with other customers, and remain strictly encrypted at rest and in transit within the customer's tenant boundary.

Intellectual Property (IP) Indemnification: Major cloud providers offer copyright indemnification. Google Cloud provides a two-pronged indemnification covering both training data copyright claims and output-generation copyright claims (provided the customer uses standard safety guardrails and does not intentionally prompt for infringing material).

Data Residency and Compliance: Verification that data processing occurs within specified geographic regions (e.g., US-only, EU-only) to comply with GDPR, HIPAA, SOC 1/2/3, and ISO/IEC standards.

8. Availability and Reliability

The exam guide names availability and reliability as an explicit selection criterion, and it is the one business leaders most often skip. A model that is brilliant but unreachable during your peak hour is worthless to the business.

Evaluate four things:

Regional availability. Not every model is served in every region. A German insurer with a data-residency obligation cannot select a model that is only served in us-central1 , no matter how capable it is. Check regional endpoints before you commit a use case to a model.

Capacity guarantees. Pay-as-you-go shared capacity is subject to quota limits and throttling under load. Provisioned throughput reserves dedicated capacity for a committed term, converting a variable-latency service into a predictable one. Choose it for customer-facing and revenue-critical paths.

Published service level agreements (SLAs). Managed Agent Platform endpoints carry a contractual uptime SLA. A self-hosted open-weights model on your own GKE cluster carries whichever SLA your platform team can operate to — usually a lower one, and always at your own cost.

Model lifecycle and deprecation. Foundation models are versioned and retired on published schedules. Gemini 1.5 Pro and 1.5 Flash were retired in 2025, and any workload pinned to them had to migrate. Pin a stable version, subscribe to deprecation notices, and budget for periodic migration. This is a reliability question, not a housekeeping one.

9. Fine-Tuning and Customization Support

The final official criterion asks whether a model can be adapted to your business at all, and how far.

  • Customization level What it changes Model support required
  • Prompt engineering Instructions only; no weights change Every model

Grounding / RAG The context supplied at inference; no weights change Any model with a sufficient context window and tool support

Parameter-efficient fine-tuning (for example, LoRA) A small adapter layer on top of frozen weights Only models the provider exposes for tuning

  • Full fine-tuning All model weights Practically limited to open-weights models such as Gemma
  • Distillation Trains a smaller, cheaper model to imitate a larger one Requires provider tooling or open weights

Two decision rules follow directly:

If the business need is dynamic proprietary knowledge, customization support barely matters — you need grounding and RAG, which any capable model supports. Do not select a model for its tuning features when the real requirement is fresh facts.

If the business need is a fixed style, tone, domain vocabulary, or strict output format , tuning support becomes a hard filter, and it can eliminate an otherwise-superior model from consideration. Confirm tuning availability before the model shortlist is approved, not after.

Open-weights models such as Gemma 4 score highest on customization freedom because you hold the weights, at the cost of owning the serving infrastructure and its reliability. Managed proprietary models such as Gemini 3.1 Pro score highest on availability and operational simplicity, with customization bounded by what Google exposes. That trade — control versus operational burden — is the recurring exam framing.

The Nine Criteria at a Glance

  • Official criterion Core question Typical decisive signal
  • Modality What inputs and outputs must it handle? Any non-text input eliminates text-only models
  • Context window How much must it consider at once? Long documents or long sessions
  • Security Where does data go, and who can see it? Regulated data, residency, no-training guarantees

Availability and reliability Can we depend on it in production? Region coverage, provisioned throughput, SLA, deprecation

  • Cost What does it cost at real volume? Token price times realistic monthly volume
  • Performance Is it accurate and fast enough? Benchmarks on your task, plus latency budget
  • Fine-tuning Can we adapt the weights? Needed only for fixed style or format
  • Customization How else can we shape behavior? Prompting and grounding usually suffice

Warning

Common Pitfall: Ranking models by public leaderboard score alone. The exam consistently rewards the answer that weighs the business constraint — a residency requirement, a latency budget, a per-token cost ceiling, or a deprecation risk — over raw benchmark capability. The most capable model is frequently the wrong answer.

Enterprise Decision Matrix: Foundation Model Archetypes

Capability / Metric Frontier Model (e.g., Gemini 3.1 Pro) Efficient Model (e.g., Gemini 3.5 Flash) Open-Weights Model (e.g., Gemma 4 31B Dense) Edge Model (e.g., Gemini Nano)

Target Workload Multi-hop reasoning, complex code refactoring, multi-modal analysis Real-time chat, high-volume extraction, routing, general summarization Air-gapped deployments, custom on-prem clusters, specialized tuning Local mobile app intelligence, offline smart devices, privacy-first UX

Modality Support Multimodal (Text, Image, Audio, Video, PDF) Multimodal (Text, Image, Audio, Video, PDF) Primarily Text / Code (varies by variant) Text / Multimodal (optimized for mobile)

Context Window Up to 2,000,000 tokens Up to 1,000,000+ tokens Typically 8,192 tokens Typically 2,048 – 4,096 tokens

Latency (TTFT) Moderate (~1.0 – 2.0s) Ultra-Fast (<500ms) Dependent on self-hosted hardware Ultra-Fast (Zero network latency)

Relative Cost Premium (High) Cost-Effective (Low) Fixed Infrastructure Cost (GPU/TPU) Zero inference cloud cost (Client compute)

Hosting Model Fully Managed Agent Platform API Fully Managed Agent Platform API Customer-managed (GKE, VMs) or Agent Platform Model Garden Embedded on client hardware

IP Indemnification Full Google Enterprise Indemnification Full Google Enterprise Indemnification Governed by open model license terms Governed by OS / SDK terms

Concrete Business Scenario: Telecommunications Enterprise Strategy

A national telecommunications carrier is rolling out generative AI across two distinct customer operations:

Workload A: Real-Time Interactive Customer Care Voice Agent: Requires answering subscriber questions over the phone regarding billing statements and cellular roaming options. Latency must be under 600ms to maintain natural conversational cadence. The carrier selects Gemini 3.5 Flash due to its sub-second Time to First Token, low cost across millions of daily subscriber calls, and reliable conversational grounding.

Workload B: Complex Network Infrastructure Anomaly Root-Cause Analysis: Once per day, network operations engineers must correlate 500MB of multi-node telemetry logs, tower maintenance audio recordings, and microwave transmission circuit schematics to diagnose chronic regional outages. The carrier selects Gemini 3.1 Pro because of its native multimodal capability, 2-million-token context window (allowing whole log archives to be ingested simultaneously), and deep multi-hop causal reasoning.

Exam Tips & Common Pitfalls

Tip

Exam Tip: Understand the economic impact of Context Caching in Agent Platform. If a scenario asks how an organization can dramatically cut costs and latency when sending the same large set of legal documents, developer codebases, or reference manuals across thousands of recurring employee prompts every day, the correct solution is Context Caching , not continually resending full documents in each prompt.

Warning

Common Pitfall: Assuming that larger models are always superior. Deploying a frontier model (like Gemini Pro) for simple classification, routing, or short entity extraction introduces unnecessary latency, burns cloud budgets rapidly, and risks hitting API rate limits unnecessarily. Best practice is to use the smallest, fastest model that reliably meets the accuracy bar.

Loading diagram... Enterprise Foundation Model Selection Flowchart

Relative Cost Comparison Across Model Tiers and Context Caching (Indexed to Frontier Input = 100)