Study guide

2.2 Lifecycle Metrics Across the Six DevOps Phases

Microsoft Certified: Azure DevOps Engineer Expert AZ-400

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Flow of Work, Traceability & Work Tracking

1.1 Flow of Work, GitHub Flow & Feedback Cycles 1.2 Work Tracking with Azure Boards & GitHub Projects 1.3 End-to-End DevOps Traceability & Quality Tracking

2 Chapter 2: DevOps Metrics, Dashboards & Team Collaboration

2.1 DORA Metrics, Flow Metrics & Dashboard Design 2.2 Lifecycle Metrics Across the Six DevOps Phases 2.3 Work Item Queries, WIQL Charts & Analytics 2.4 Project Documentation: Wikis, Mermaid Diagrams & Release Notes 2.5 Webhooks, Service Hooks & Microsoft Teams ChatOps

3 Chapter 3: Git Branching Strategies & Pull Request Workflows

3.1 Git Branching Models & Strategy Design 3.2 Pull Request Workflows & Merge Strategies 3.3 Branch Policies, Quality Gates & Compliance Restrictions

4 Chapter 4: Repository Management, Optimization & Git Data Operations

4.1 Git LFS Architecture & .gitattributes Tracking 4.2 LFS History Migration, git-fat & File Locking 4.3 Git Repository Scaling, Scalar & Monorepo Governance 4.4 Repository Permissions & Git Tag Governance 4.5 Git Data Recovery & Sensitive-Data Scrubbing

5 Chapter 5: Package Management & Dependency Versioning

5.1 Selecting Package Tools: Azure Artifacts vs. GitHub Packages 5.2 Feed Scoping, Upstream Sources & Pipeline Authentication 5.3 Package Views, Immutability & Quality Promotion 5.4 Dependency Versioning: SemVer, CalVer & Automated Versioning 5.5 Pipeline Artifact Versioning & Artifact Storage Strategy

6 Chapter 6: Pipeline Infrastructure, Agents & Integration

6.1 Selecting a CI/CD Solution: Azure Pipelines vs. GitHub Actions 6.2 GitHub-Azure Pipelines Integration & Hybrid Migration 6.3 Hosted vs. Self-Hosted Agents & Runners 6.4 VMSS Elastic Pools, Container Jobs & Agent Governance 6.5 Pipeline Triggers, Filters & Concurrency Management

7 Chapter 7: YAML Pipeline Architecture & Reusable Elements

7.1 YAML Pipeline Anatomy: Stages, Jobs, Steps & Workspaces 7.2 Job Execution Order: DAG Dependencies, Parallelism & Conditions 7.3 Reusable Elements: Templates, Parameters & Variables 7.4 Variable Groups, Secret Variables & Key Vault Linking 7.5 YAML Environments, Checks & Deployment Approvals

8 Chapter 8: Testing Strategy, Quality Gates & Code Coverage

8.1 Test Strategy Design: Local, Unit & Integration Testing 8.2 Load, Stress & Performance Testing in Pipelines 8.3 Test Execution Tasks, Agents & Results Integration 8.4 Code Coverage Analysis & SonarQube Integration 8.5 Quality Gates, Release Gates & Governance Checks

9 Chapter 9: Modern Deployment Strategies & Zero-Downtime Delivery

9.1 Progressive Delivery: Blue-Green, Canary & Ring Deployments 9.2 Minimizing Downtime: Deployment Slots & Rolling Updates 9.3 Feature Flags with Azure App Configuration & Feature Manager 9.4 Designing the High-Priority Hotfix Path 9.5 Ordered Dependency Deployments, Resiliency & Rollback

10 Chapter 10: Artifact Deployment & Database DevOps

10.1 Containerized Application Delivery: ACR, AKS & Container Apps 10.2 Binary, Web App, Function & Scripted Deployments 10.3 Database DevOps: State-Based & Migration-Based Strategies

11 Chapter 11: Infrastructure as Code (IaC) & Azure Deployment Environments

11.1 Infrastructure as Code Strategy: Bicep, ARM & Terraform 11.2 Desired State Configuration & Azure Machine Configuration 11.3 Azure Deployment Environments (ADE) & Developer Self-Service

12 Chapter 12: Pipeline Maintenance, Health & Modernization

12.1 Pipeline Health, Flaky Tests & Reliability Engineering 12.2 Pipeline Optimization: Caching, Slicing & Concurrency 12.3 Artifact & Build Retention Governance 12.4 Modernizing CI/CD: Classic to YAML Migration

13 Chapter 13: Authentication, Authorization & Identity Governance

13.1 Microsoft Entra Service Principals & Managed Identities 13.2 Service Connections, Workload Identity & Personal Access Tokens 13.3 Azure DevOps Projects, Teams & Permission Evaluation 13.4 Access Levels, GitHub Roles & Outside Collaborators

14 Chapter 14: Secrets Management, Sensitive Data & DevSecOps Scanning

14.1 Azure Key Vault, Secret Variables & Secretless CI/CD 14.2 Security Scanning Strategy, GHAS & Secret Scanning 14.3 CodeQL SAST in Pipelines and Containers 14.4 Software Supply Chain, Container Scanning & Cloud Defender

15 Chapter 15: Telemetry Collection, Azure Monitor & Alerting

15.1 Azure Monitor Architecture, AMA & Data Collection Rules 15.2 Application Insights & Specialized Azure Monitor Insights 15.3 Alerting Rules, Action Groups & GitHub Repository Insights

16 Chapter 16: Telemetry Analysis, Distributed Tracing & SRE Practices

16.1 Distributed Tracing, Application Map & Exception Triage 16.2 KQL Log Analytics, SRE Observability & Error Budgets

2.2 Lifecycle Metrics Across the Six DevOps Phases

Key Takeaways Explain This Topic AI

Planning metrics track predictability - sprint commitment versus completion, backlog aging and scope churn - not raw output volume.

Development metrics centre on pull request cycle time, review latency and branch lifespan, with trunk-based teams targeting branch lifetimes under 24 to 48 hours.

Testing metrics combine pass rate, execution duration and flaky-test rate; test execution above 10 to 15 minutes measurably degrades pull request feedback loops.

Security metrics measure mean time to remediate a vulnerability and the proportion of pull requests scanned, not the absolute count of open alerts.

Operations metrics own reliability: mean time to restore, mean time between failures, change failure rate and error budget consumption.

Last updated: September 2026

2.2 Lifecycle Metrics Across the Six DevOps Phases

Achieving operational excellence in Azure DevOps requires comprehensive observability across the entire software delivery lifecycle. Rather than focusing solely on build pass rates or production uptime, mature DevOps organizations capture quantitative metrics across six interconnected phases: Planning , Development , Testing , Security , Delivery , and Operations . In Azure DevOps, work item data across these phases is queried, filtered, and alerted upon using Work Item Query Language (WIQL) .

  • The 6-Phase DevOps Lifecycle Metrics Framework +-------------+ +-------------+ +-------------+
  • | 1. PLANNING | --> | 2. DEVELOP | --> | 3. TESTING |
  • | Velocity | | PR Reviews | | Pass Rate |
  • | Say-Do | | Branch Life | | Flaky Tests |
  • +-------------+ +-------------+ +-------------+
  • ^ |
  • | v
  • +-------------+ +-------------+ +-------------+
  • | 6. OPERATE | <-- | 5. DELIVERY | <-- | 4. SECURITY |
  • | Availability| | Deploy Freq | | MTTD / MTTR |
  • | MTBF / MTTR | | Queue Wait | | Secret Leaks|
  • +-------------+ +-------------+ +-------------+
  • Phase 1: Planning Metrics

Planning metrics assess estimation accuracy, team capacity utilization, and backlog health.

Sprint Velocity: Total story points or work item units completed during a sprint iteration. Velocity should stabilize over 3–5 sprints to facilitate reliable capacity planning.

Commitment Reliability (Say-Do Ratio): The percentage of committed work at sprint planning that was successfully delivered at sprint close: Say-Do Ratio = ( Completed Story Points Committed Story Points ) × 100 \text{Say-Do Ratio} = \left( \frac{\text{Completed Story Points}}{\text{Committed Story Points}} \right) \times 100 Say-Do Ratio = ( Committed Story Points Completed Story Points ​ ) × 100 Healthy target: 80% to 90%. Consistent values under 70% indicate overcommitment or unmanaged interruptions; 100% every sprint often indicates sandbagging.

Scope Creep: Percentage of story points or work items added to the sprint iteration after the sprint start date.

Capacity Allocation: Ratio of capacity dedicated to new feature development versus technical debt remediation and defect resolution.

Phase 2: Development Metrics

Development metrics evaluate code collaboration, peer review engagement, and source control hygiene.

Pull Request (PR) Turnaround Time: Elapsed time from PR creation until final approval and merge.

Review Comment Density: Average number of review comments logged per PR or per 100 lines of code. Low comment density combined with instantaneous approvals signals rubber-stamping.

Branch Lifespan: Total hours or days that a feature branch exists before merging back into main . Trunk-based development targets branch lifespans under 24 to 48 hours.

Code Churn: Total lines added, modified, or deleted over time. High churn on recently refactored modules indicates design instability.

Phase 3: Testing & Quality Metrics

Testing metrics ensure software correctness and pipeline validation integrity.

Automated Test Pass Rate: Percentage of automated unit, integration, and UI tests that pass on each pipeline execution.

Code Coverage Trends: Percentage of code statements, branches, and lines exercised by automated test suites. Tracked via Cobertura or JaCoCo in Azure Pipelines.

Test Execution Duration: Wall-clock time required to run automated test suites. When test execution exceeds 10–15 minutes, CI feedback loops degrade, prompting test parallelization via sliced test agents ( slicing in VSTest).

Flaky Test Percentage: Proportion of automated tests that exhibit non-deterministic results (passing and failing on identical source code commits). Azure DevOps detects flaky tests natively in Test Analytics and allows quarantine policies.

Phase 4: Security Metrics (DevSecOps)

Security metrics measure vulnerability posture and response times across the codebase and dependencies.

Mean Time to Detect (MTTD): Average elapsed time from when a vulnerability or security defect is introduced into source code until detection by SAST, DAST, or dependency scanners (e.g., GitHub Advanced Security, Microsoft Defender for DevOps).

Mean Time to Remediate (MTTR - Security): Time elapsed from vulnerability identification to deployment of a verified fix in production.

Open Vulnerability Count by Severity: Active security debt categorized by CVSS severity (Critical, High, Medium, Low).

Secret Leakage Incidents: Frequency of detected plaintext API keys, passwords, or connection strings committed to repositories. Evaluated via Secret Scanning with push protection.

Phase 5: Delivery / Pipeline Metrics

Delivery metrics evaluate CI/CD automation efficiency and infrastructure throughput.

Pipeline Build Success Rate: Ratio of successful pipeline runs to total runs initiated.

Mean Pipeline Execution Duration: Average duration from pipeline trigger to artifact publication or deployment completion.

Deployment Queue Wait Time: Time pipeline jobs sit waiting for an available Microsoft-hosted or self-hosted agent.

Phase 6: Operations & Reliability Metrics

Operations metrics assess production stability, system availability, and incident handling.

Service Availability (Uptime): Percentage of time the system meets its operational availability Service Level Objective (SLO) (e.g., 99.95% uptime).

Mean Time Between Failures (MTBF): Average operational time elapsed between system outages or failures: MTBF = Total Operational Runtime Total Number of Outage Incidents \text{MTBF} = \frac{\text{Total Operational Runtime}}{\text{Total Number of Outage Incidents}} MTBF = Total Number of Outage Incidents Total Operational Runtime ​

Mean Time to Acknowledge (MTTA): Time from an automated alert firing in Azure Monitor until on-call engineers acknowledge the incident.

Incident Volume & Severity: Count of production incidents categorized by severity (Sev 1 through Sev 4).

Comprehensive Lifecycle Metrics Reference Table

Lifecycle Phase Core Metric Formula / Calculation Healthy Target Azure DevOps / Azure Tool

Planning Say-Do Ratio ( Completed / Committed ) × 100 (\text{Completed} / \text{Committed}) \times 100 ( Completed / Committed ) × 100 80% – 90% Azure Boards Sprint Analytics

Planning Scope Creep ( Added Points / Original Points ) × 100 (\text{Added Points} / \text{Original Points}) \times 100 ( Added Points / Original Points ) × 100 < 10% Sprint Burndown Widget

Development PR Turnaround Time Merge Timestamp − Creation Timestamp \text{Merge Timestamp} - \text{Creation Timestamp} Merge Timestamp − Creation Timestamp < 24 hours Azure Repos PR Analytics

Development Branch Lifespan Branch Deletion − Branch Creation \text{Branch Deletion} - \text{Branch Creation} Branch Deletion − Branch Creation < 48 hours Git Insights / Branch Policies

Testing Flaky Test Ratio ( Flaky Test Runs / Total Test Runs ) × 100 (\text{Flaky Test Runs} / \text{Total Test Runs}) \times 100 ( Flaky Test Runs / Total Test Runs ) × 100 < 1% Azure Test Plans / Test Analytics

Testing Code Coverage ( Covered Lines / Total Executable Lines ) × 100 (\text{Covered Lines} / \text{Total Executable Lines}) \times 100 ( Covered Lines / Total Executable Lines ) × 100 70% – 85% Publish Code Coverage Results task

Security Security MTTR Remediation Timestamp − Detection Timestamp \text{Remediation Timestamp} - \text{Detection Timestamp} Remediation Timestamp − Detection Timestamp Critical < 24h; High < 7d GitHub Advanced Security / Defender

Delivery Pipeline Success Rate ( Successful Runs / Total Runs ) × 100 (\text{Successful Runs} / \text{Total Runs}) \times 100 ( Successful Runs / Total Runs ) × 100 > 90% Azure Pipelines Analytics

Delivery Queue Wait Time Agent Job Start − Job Queued \text{Agent Job Start} - \text{Job Queued} Agent Job Start − Job Queued < 2 minutes Agent Pool Analytics

Operations MTBF Total Runtime / Failure Count \text{Total Runtime} / \text{Failure Count} Total Runtime / Failure Count Maximized Azure Monitor / Application Insights

Operations Service Availability ( Total Time − Downtime ) / Total Time × 100 (\text{Total Time} - \text{Downtime}) / \text{Total Time} \times 100 ( Total Time − Downtime ) / Total Time × 100 Exceeding SLO (e.g., 99.9%) Application Insights Availability Tests

Loading diagram... Six-Phase DevOps Lifecycle Metrics and Telemetry Feedback Loops