Study guide
4.5 Git Data Recovery & Sensitive-Data Scrubbing
Microsoft Certified: Azure DevOps Engineer Expert AZ-400
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Flow of Work, Traceability & Work Tracking
1.1 Flow of Work, GitHub Flow & Feedback Cycles 1.2 Work Tracking with Azure Boards & GitHub Projects 1.3 End-to-End DevOps Traceability & Quality Tracking
2 Chapter 2: DevOps Metrics, Dashboards & Team Collaboration
2.1 DORA Metrics, Flow Metrics & Dashboard Design 2.2 Lifecycle Metrics Across the Six DevOps Phases 2.3 Work Item Queries, WIQL Charts & Analytics 2.4 Project Documentation: Wikis, Mermaid Diagrams & Release Notes 2.5 Webhooks, Service Hooks & Microsoft Teams ChatOps
3 Chapter 3: Git Branching Strategies & Pull Request Workflows
3.1 Git Branching Models & Strategy Design 3.2 Pull Request Workflows & Merge Strategies 3.3 Branch Policies, Quality Gates & Compliance Restrictions
4 Chapter 4: Repository Management, Optimization & Git Data Operations
4.1 Git LFS Architecture & .gitattributes Tracking 4.2 LFS History Migration, git-fat & File Locking 4.3 Git Repository Scaling, Scalar & Monorepo Governance 4.4 Repository Permissions & Git Tag Governance 4.5 Git Data Recovery & Sensitive-Data Scrubbing
5 Chapter 5: Package Management & Dependency Versioning
5.1 Selecting Package Tools: Azure Artifacts vs. GitHub Packages 5.2 Feed Scoping, Upstream Sources & Pipeline Authentication 5.3 Package Views, Immutability & Quality Promotion 5.4 Dependency Versioning: SemVer, CalVer & Automated Versioning 5.5 Pipeline Artifact Versioning & Artifact Storage Strategy
6 Chapter 6: Pipeline Infrastructure, Agents & Integration
6.1 Selecting a CI/CD Solution: Azure Pipelines vs. GitHub Actions 6.2 GitHub-Azure Pipelines Integration & Hybrid Migration 6.3 Hosted vs. Self-Hosted Agents & Runners 6.4 VMSS Elastic Pools, Container Jobs & Agent Governance 6.5 Pipeline Triggers, Filters & Concurrency Management
7 Chapter 7: YAML Pipeline Architecture & Reusable Elements
7.1 YAML Pipeline Anatomy: Stages, Jobs, Steps & Workspaces 7.2 Job Execution Order: DAG Dependencies, Parallelism & Conditions 7.3 Reusable Elements: Templates, Parameters & Variables 7.4 Variable Groups, Secret Variables & Key Vault Linking 7.5 YAML Environments, Checks & Deployment Approvals
8 Chapter 8: Testing Strategy, Quality Gates & Code Coverage
8.1 Test Strategy Design: Local, Unit & Integration Testing 8.2 Load, Stress & Performance Testing in Pipelines 8.3 Test Execution Tasks, Agents & Results Integration 8.4 Code Coverage Analysis & SonarQube Integration 8.5 Quality Gates, Release Gates & Governance Checks
9 Chapter 9: Modern Deployment Strategies & Zero-Downtime Delivery
9.1 Progressive Delivery: Blue-Green, Canary & Ring Deployments 9.2 Minimizing Downtime: Deployment Slots & Rolling Updates 9.3 Feature Flags with Azure App Configuration & Feature Manager 9.4 Designing the High-Priority Hotfix Path 9.5 Ordered Dependency Deployments, Resiliency & Rollback
10 Chapter 10: Artifact Deployment & Database DevOps
10.1 Containerized Application Delivery: ACR, AKS & Container Apps 10.2 Binary, Web App, Function & Scripted Deployments 10.3 Database DevOps: State-Based & Migration-Based Strategies
11 Chapter 11: Infrastructure as Code (IaC) & Azure Deployment Environments
11.1 Infrastructure as Code Strategy: Bicep, ARM & Terraform 11.2 Desired State Configuration & Azure Machine Configuration 11.3 Azure Deployment Environments (ADE) & Developer Self-Service
12 Chapter 12: Pipeline Maintenance, Health & Modernization
12.1 Pipeline Health, Flaky Tests & Reliability Engineering 12.2 Pipeline Optimization: Caching, Slicing & Concurrency 12.3 Artifact & Build Retention Governance 12.4 Modernizing CI/CD: Classic to YAML Migration
13 Chapter 13: Authentication, Authorization & Identity Governance
13.1 Microsoft Entra Service Principals & Managed Identities 13.2 Service Connections, Workload Identity & Personal Access Tokens 13.3 Azure DevOps Projects, Teams & Permission Evaluation 13.4 Access Levels, GitHub Roles & Outside Collaborators
14 Chapter 14: Secrets Management, Sensitive Data & DevSecOps Scanning
14.1 Azure Key Vault, Secret Variables & Secretless CI/CD 14.2 Security Scanning Strategy, GHAS & Secret Scanning 14.3 CodeQL SAST in Pipelines and Containers 14.4 Software Supply Chain, Container Scanning & Cloud Defender
15 Chapter 15: Telemetry Collection, Azure Monitor & Alerting
15.1 Azure Monitor Architecture, AMA & Data Collection Rules 15.2 Application Insights & Specialized Azure Monitor Insights 15.3 Alerting Rules, Action Groups & GitHub Repository Insights
16 Chapter 16: Telemetry Analysis, Distributed Tracing & SRE Practices
16.1 Distributed Tracing, Application Map & Exception Triage 16.2 KQL Log Analytics, SRE Observability & Error Budgets
4.5 Git Data Recovery & Sensitive-Data Scrubbing
Key Takeaways Explain This Topic AI
git reflog records every HEAD movement locally, so a hard reset or a deleted branch can be recovered by checking out the recorded commit SHA.
Unreachable commits survive until garbage collection prunes them, typically after 14 to 30 days, which bounds the recovery window.
git revert creates a new inverse commit and is the only safe rollback for a branch other people have already pulled; git reset --hard rewrites shared history.
git rm does not remove a secret - the value remains in every historical commit and in every existing clone.
Remediation order is fixed: rotate and revoke the credential first, then rewrite history with git-filter-repo, force push, and require every consumer to re-clone.
Last updated: September 2026
4.5 Git Data Recovery & Sensitive-Data Scrubbing
Two failure modes look similar and demand opposite responses. A developer who destroys local work needs recovery , and Git almost always still holds the objects. A developer who commits a credential needs scrubbing plus rotation , because deletion alone leaves the secret in history and in every existing clone.
Git Disaster Recovery: Salvaging Lost Commits & History
A fundamental architectural strength of Git is that it rarely deletes any object immediately . When an engineer accidentally deletes a local branch, executes a git reset --hard , or loses work in a detached HEAD state, the commit objects remain intact inside the local object database as "dangling commits" until Git's garbage collection ( git gc ) purges them (typically after 14 to 30 days).
Tool 1: The Reference Log ( git reflog )
While git log displays the commit history of the current branch, git reflog tracks every single movement of the HEAD pointer on your local machine. Every commit, checkout, rebase, cherry-pick, and reset appends an entry to the reflog. # Inspect the chronological movement of HEAD
- git reflog
- Sample Reflog Output: 7f1a34d (HEAD -> main) HEAD@{0}: reset: moving to HEAD~2
- e4b219c HEAD@{1}: commit: Implement Stripe payment webhook
- 9a8c12b HEAD@{2}: commit: Add user profile database migration
- 3c4d5e6 HEAD@{3}: checkout: moving from feature/payment to main
- Recovery Scenario A: Recovering from an Accidental git reset --hard
A developer accidentally executes git reset --hard HEAD~2 , obliterating two commits from main .
Running git log shows those commits are gone.
The developer runs git reflog and sees that prior to the reset, HEAD was at commit e4b219c ( HEAD@{1} ).
- The developer restores the commits instantly: # Move the branch pointer back to the commit before the reset
- git reset --hard e4b219c
- Recovery Scenario B: Recovering an Accidentally Deleted Branch
A developer deletes a branch containing 3 days of unpushed work: git branch -D feature/analytics .
The developer runs git reflog and searches for the last commit on that branch: git reflog | grep feature/analytics
- # Output: 9a8c12b HEAD@{4}: commit: Complete event tracking pipeline
- Recreate the deleted branch pointing to that commit SHA: git branch feature/analytics 9a8c12b
- Tool 2: git cherry-pick for Selective Commit Salvage
git cherry-pick <commit-sha> applies the exact diff introduced by a specific commit from any branch and commits it cleanly onto the currently active branch.
Operational Scenario: An engineer accidentally commits a critical bug fix to the wrong branch (e.g., committed to an experimental sandbox branch instead of main ).
Remediation: Checkout main , run git cherry-pick <commit-sha> , verify via test suite, and push. Then checkout the sandbox branch and reset or revert the commit.
Safe Rollbacks on Public Branches: git revert vs. git reset --hard
Warning
AZ-400 Exam Rule: Never use git reset --hard on public, shared branches ( main , release/* )! Resetting moves the branch pointer backward, rewriting history. Pushing this change requires --force , which breaks the clones of every other developer on the team and violates compliance audit trails.
On any shared branch, always use git revert : # Safe forward-moving rollback: creates a NEW commit that applies the inverse diff
git revert 4c2a1e8
- # Reverting a merge commit (must specify mainline parent index, typically 1)
- git revert -m 1 9b3d2f1
git revert preserves the entire historical timeline, passes automated CI/CD branch policies without force-pushing, and provides transparent auditability.
Scrubbing Sensitive Data & Leaked Credentials
One of the most dangerous security incidents in DevOps occurs when a developer accidentally commits a private key, API secret, connection string, or .env file into a Git repository. The Invalidation of "git rm":
- Commit 1: Add appsettings.json (Contains Prod SQL Password!) ──► Blob written to .git/objects/
- │
Commit 2: Developer runs 'git rm appsettings.json' ──► Blob STILL EXISTS in Commit 1!
- (Accessible via git checkout)
- Why git rm Does NOT Delete Secrets
Running git rm password.txt and committing simply records that the file is not present in the new commit's tree. The secret remains permanently stored in the blob corresponding to the previous commit. Anyone cloning the repository or running git checkout HEAD~1 can inspect the plaintext secret.
The Modern Scrubbing Tool: git-filter-repo
Historically, teams used git filter-branch to rewrite history. However, git filter-branch is deprecated by the official Git project due to glacial performance, shell escaping vulnerabilities, and history corruption risks.
The modern, officially recommended Python tool is git-filter-repo (or the high-speed Java utility BFG Repo-Cleaner ). # Prerequisites: Install git-filter-repo via package manager
pip install git-filter-repo
Scrubbing Scenario 1: Purge an Entire Sensitive File from All History # Make a fresh mirror clone of the repository first
- git clone --mirror https://dev.azure.com/myorg/myproject/_git/myrepo
- cd myrepo.git
- # Completely delete credentials.json from every commit, tree, and tag
- git-filter-repo --path credentials.json --invert-paths
- Scrubbing Scenario 2: Redact a Specific Leaked Password Across All Files
Create a replacement file expressions.txt containing the leaked string and its replacement: password123===>REDACTED_SECRET
Server=tcp:prodsql.database.windows.net===>Server=tcp:localhost
Run git-filter-repo to replace matching strings across all files in all historical commits: git-filter-repo --replace-text expressions.txt
The Mandatory 4-Step Incident Remediation Protocol
Purging Git history with git-filter-repo is only one technical step in a complete security incident response. Exam AZ-400 strictly tests the end-to-end operational protocol when credentials leak: End-to-End Leaked Credential Remediation Lifecycle:
- Step 1: Credential Rotation (IMMEDIATE) ──► Invalidate leaked key in Azure Key Vault / Entra ID
- │
- Step 2: Rewrite Git History ──► Run git-filter-repo on a bare mirror clone
- │
- Step 3: Force-Push Cleaned History ──► git push origin --force --all --tags
- │
- Step 4: Mandatory Team Re-Clone ──► Delete local developer clones to prevent re-infection
Step 1: Immediate Credential Invalidation & Rotation (Highest Priority):
The instant a secret is pushed to a remote repository, assume it has been compromised by automated scraper bots or caching proxies.
Immediately navigate to Azure Key Vault, Microsoft Entra ID, or the affected database/service and rotate the secret, regenerate API keys, and revoke active tokens .
Exam Trap: Scrubbing the Git repository before rotating the key is an anti-pattern; attackers who already cloned the repository can exploit the active secret while you spend hours rewriting history!
Step 2: Rewrite History with git-filter-repo :
Execute git-filter-repo on a clean bare clone to excise the file or string from every commit and tag.
Step 3: Force-Push Rewritten History:
An authorized administrator temporarily enables Force push permissions in Azure Repos and pushes the rewritten refs: git push origin --force --all
git push origin --force --tags
Step 4: Coordinate Mandatory Team Re-Clones:
Instruct all developers to delete their local repository folders completely and perform a fresh git clone .
Why this is mandatory: If any developer runs git pull from their existing local repository, Git will merge the rewritten remote history with their old local history, resurrecting the deleted secret blobs back into the remote repository!