Study guide
8.3 Machine Learning and Prediction
FRM Part I Financial Risk Manager Study Guide Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
1.1 Current Exam Facts 1.2 Study Strategy & Domain Weights
2 Foundations: Risk Building Blocks & Governance
2.1 Building Blocks of Risk Management 2.2 How Firms Manage Financial Risk 2.3 Governance of Risk Management 2.4 Credit Risk Transfer Mechanisms
3 Foundations: Portfolio Models, Data & ERM
3.1 MPT and the CAPM 3.2 APT and Multifactor Models 3.3 Risk Data Aggregation & Reporting 3.4 Enterprise Risk Management
4 Foundations: Disasters, Crisis & Ethics
4.1 Learning from Financial Disasters 4.2 Anatomy of the Great Financial Crisis 4.3 GARP Code of Conduct
5 Quantitative Analysis: Probability Foundations
5.1 Fundamentals of Probability 5.2 Random Variables 5.3 Common Univariate Distributions 5.4 Multivariate Random Variables
6 Quantitative Analysis: Inference & Regression
6.1 Sample Moments 6.2 Hypothesis Testing 6.3 Linear Regression 6.4 Multiple Regression
7 Quantitative Analysis: Diagnostics & Time Series
7.1 Regression Diagnostics 7.2 Stationary Time Series 7.3 Non-Stationary Time Series 7.4 Returns, Volatility & Correlation
8 Quantitative Analysis: Simulation & Machine Learning
8.1 Simulation and Bootstrapping 8.2 Machine-Learning Methods 8.3 Machine Learning and Prediction
9 Financial Markets: Institutions & Derivatives Intro
9.1 Banks 9.2 Insurance Companies and Pension Plans 9.3 Fund Management 9.4 Introduction to Derivatives
10 Financial Markets: Clearing, Futures & Hedging
10.1 Exchanges and OTC Markets 10.2 Central Clearing 10.3 Futures Markets 10.4 Using Futures for Hedging
11 Financial Markets: FX, Forwards, Commodities & Options
11.1 Foreign Exchange Markets 11.2 Pricing Financial Forwards and Futures 11.3 Commodity Forwards and Futures 11.4 Options Markets
12 Financial Markets: Option Properties, Strategies & Rates
12.1 Properties of Options 12.2 Trading Strategies 12.3 Exotic Options 12.4 Properties of Interest Rates
13 Financial Markets: Bonds, MBS, Rate Futures & Swaps
13.1 Corporate Bonds 13.2 Mortgages and Mortgage-Backed Securities 13.3 Interest Rate Futures 13.4 Swaps
14 Valuation & Risk Models: VaR, Volatility & Ratings
14.1 Measures of Financial Risk 14.2 Calculating and Applying VaR 14.3 Measuring and Monitoring Volatility 14.4 External and Internal Credit Ratings
15 Valuation & Risk Models: Credit, OpRisk & Stress
15.1 Country Risk 15.2 Measuring Credit Risk 15.3 Operational Risk 15.4 Stress Testing
16 Valuation & Risk Models: Fixed-Income Valuation
16.1 Pricing, Discounting & Arbitrage 16.2 Interest Rates 16.3 Bond Yields and Return Calculations 16.4 Duration, Convexity & DV01
17 Valuation & Risk Models: Term Structure, Options & Greeks
17.1 Non-Parallel Term Structure Shifts 17.2 Binomial Trees 17.3 The Black-Scholes-Merton Model 17.4 Option Sensitivity Measures: The Greeks
8.3 Machine Learning and Prediction
Key Takeaways Explain This Topic AI
Linear and logistic regression remain core predictive baselines: linear for continuous targets, logistic for binary probabilities via a sigmoid link
Categorical predictors need encoding (dummy/one-hot or other schemes); omit one baseline category with an intercept to avoid perfect collinearity
Ridge shrinks coefficients with an L2 penalty; LASSO uses L1 and can zero out coefficients for sparse selection
Decision trees split on features; ensembles (bagging, random forests, boosting) reduce variance or bias relative to a single tree
KNN and SVM are geometric classifiers/regressors; neural nets stack nonlinear layers; confusion-matrix metrics (accuracy, precision, recall, specificity, F1) compare classification performance beyond raw error rate
Last updated: August 2026
Machine Learning and Prediction
QA–15 moves from ML taxonomy to the predictive toolkit : how models map features into forecasts, how penalties and trees control complexity, and how classification quality is scored when the cost of false alarms differs from missed events—central for credit default flags, fraud, and breach prediction.
- Linear Regression for Prediction
- In predictive mode, linear regression is still
- ŷ = β₀ + β₁ x₁ + … + βₚ xₚ
but success is judged by out-of-sample MSE, MAE, or economic P&L of decisions based on ŷ—not only by t-stats. OLS minimizes training sum of squared errors. Strengths: fast, interpretable partial effects (within the linear class), strong baseline. Weaknesses: cannot capture interactions/nonlinearities unless you engineer features; sensitive to collinearity without penalties.
Worked predictive check
Train MSE = 1.0, test MSE = 1.1 → decent generalization for a linear model. Train MSE = 0.2, test MSE = 2.5 → overfit or regime shift; do not ship on train metrics alone.
- Logistic Regression for Prediction
- For binary Y ∈ {0,1} (default or not), logistic regression models
- P(Y = 1 | x) = 1 / (1 + exp(−(β₀ + β′x))) = σ(β₀ + β′x)
where σ is the sigmoid. Coefficients are typically fit by maximum likelihood. Predictions can be reported as probabilities or thresholded (e.g., flag default if p̂ ≥ 0.5, or a risk-appetite cutoff like 0.10).
Log-odds interpretation: a unit rise in xⱼ shifts log-odds by βⱼ, holding other features fixed. For FRM prediction questions, emphasize calibration of probabilities and threshold choice under asymmetric costs—not only in-sample likelihood.
- Model Target Typical loss / fit
- Linear regression Continuous y Squared error
- Logistic regression Binary y Log-loss / likelihood
- Encoding Categorical Variables
Algorithms need numbers. Categorical features (rating bucket, sector, region) must be encoded:
One-hot / dummy encoding: create binary columns for categories. With an intercept, drop one baseline category to avoid the dummy variable trap (perfect multicollinearity).
Ordinal encoding: map ordered tiers (AAA=1 … C=7)—imposes numeric spacing that may be wrong.
Target / frequency encoding: replace category by historical mean y or frequency—powerful but leakage-prone if computed with full-sample including the row’s own y; compute on folds carefully.
Worked dummy example
Sector ∈ {Banks, Energy, Tech}. With intercept, include Banks and Energy dummies; Tech is baseline. Predicted mean for Tech uses β₀; Banks use β₀ + β_Banks. Including all three dummies plus intercept makes X′X singular for OLS.
Ridge Versus LASSO
When p is large or predictors are correlated, penalized linear/logistic models stabilize prediction.
Ridge (L2): minimize SSE + λ Σ βⱼ² (often not penalizing the intercept). Shrinks coefficients toward zero but rarely exactly to zero. Good for many correlated features (groups shrink together).
LASSO (L1): minimize SSE + λ Σ |βⱼ|. Shrinks and sets some coefficients exactly to zero —automatic feature selection. Can be unstable when predictors are highly correlated (picks one of a group arbitrarily).
Elastic net blends L1 and L2. λ (and mixing weights) are hyperparameters chosen by validation/cross-validation—not by maximizing in-sample R².
- Penalty Norm Exact zeros? Typical use
- Ridge L2 No Multicollinearity; dense signals
- LASSO L1 Yes Sparse selection
- Elastic net L1+L2 Sometimes Correlated sparse groups
- Worked penalty intuition
Unpenalized OLS coefficients on 50 noisy macro features: wild signs, test MSE high. Ridge with tuned λ shrinks everyone; test MSE drops. LASSO with tuned λ keeps 8 features and zeros 42; interpretability rises if the sparse set is stable across resamples.
Decision Trees
A decision tree (CART-style) recursively partitions the feature space with axis-aligned splits that minimize impurity (classification: Gini or entropy) or variance (regression). Prediction: average y (regression) or majority class / class probability (classification) in the terminal leaf.
Pros: captures nonlinearities and interactions; little need for rescaling; readable if shallow. Cons: high variance—small data changes can rewrite the tree; deep trees overfit.
Hyperparameters: max depth, min samples per leaf, min impurity decrease. Pruning or depth caps are the first defense against memorization.
Worked split sketch
Classify default. Root split: leverage > 0.45 versus ≤ 0.45. Left child (high leverage) splits on interest coverage < 2. Leaves report empirical default rates 25%, 8%, 3%, 1%. A new firm with leverage 0.50 and coverage 1.5 falls into the 25% leaf—transparent, but one deep tree may not generalize.
Ensembles: Bagging, Random Forests, Boosting
Bagging: train many models on bootstrap resamples; average predictions (or vote). Cuts variance of unstable learners (trees).
Random forest: bagged trees with random feature subsets at each split—decorrelates trees, often stronger than plain bagging.
Boosting (e.g., gradient boosting): add trees sequentially that focus on previous residuals/errors. Reduces bias aggressively; needs care with learning rate and depth to avoid overfit.
- Ensemble Mechanism Main benefit
- Bagging Average bootstrap trees Lower variance
- Random forest Bagging + feature randomness Lower variance, stronger
- Boosting Sequential residual fitting Lower bias (watch overfit)
Ensembles win many tabular prediction contests in credit and fraud—but monitoring, explainability (SHAP, etc.), and stability under regime shifts remain risk-management requirements.
K-Nearest Neighbors and Support Vector Machines
KNN: predict y using the K closest training points in feature space (majority vote or average). Nonparametric and simple; sensitive to scale and to the curse of dimensionality; prediction cost grows with training-set size.
SVM (support vector machine): finds a separating hyperplane with maximum margin between classes (soft margin allows violations with a penalty C). Kernel SVMs (RBF, polynomial) implicitly map features to richer spaces for nonlinear boundaries. Conceptually: focus on support vectors —points that define the margin—rather than fitting every observation equally.
Worked KNN sketch
K = 3, standardized features. New borrower distances to labeled neighbors: default, default, perform. Vote → predict default. If K = 1, a single noisy neighbor can flip the call—hence validation of K.
Neural Networks
A feed-forward neural net stacks layers: each layer computes linear combinations of inputs plus bias, then applies a nonlinear activation (ReLU, sigmoid, tanh). Depth and width create flexible function classes. Training typically minimizes log-loss or MSE via gradient methods (backpropagation, Adam, etc.).
Regularization: weight decay (L2), dropout, early stopping on validation loss, data augmentation where relevant. For tabular FRM-style problems, nets are not automatically better than boosted trees; they shine with large data and unstructured inputs (raw text with deep NLP, images). Exam point: nets approximate complex nonlinear maps but require careful validation and are harder to interpret than logistic regression.
Confusion Matrix and Comparison Metrics
For a binary classifier, count:
- Predicted + Predicted −
- Actual + TP FN
- Actual − FP TN
Definitions:
Accuracy = (TP + TN) / (TP + TN + FP + FN)—misleading under rare defaults.
Precision (positive predictive value) = TP / (TP + FP)—of flagged defaults, how many true?
Recall (sensitivity, true positive rate) = TP / (TP + FN)—of true defaults, how many caught?
Specificity = TN / (TN + FP)—of non-defaults, how many cleared?
F1 = harmonic mean of precision and recall = 2 · precision · recall / (precision + recall).
Worked comparison
1,000 loans, 50 true defaults (5% rate).
Model A: TP=30, FP=40, FN=20, TN=910.
Precision = 30/70 ≈ 0.43; Recall = 30/50 = 0.60; Accuracy = 940/1000 = 0.94.
Model B: TP=10, FP=5, FN=40, TN=945.
Precision = 10/15 ≈ 0.67; Recall = 10/50 = 0.20; Accuracy = 955/1000 = 0.955.
Model B has higher accuracy and precision but much worse recall—dangerous if missing defaults is costly. A naive “always predict non-default” gets 95% accuracy and zero recall. Always compare models on metrics aligned with the loss function , not accuracy alone.
ROC curves plot TPR (recall) versus FPR = FP/(FP+TN) across thresholds; AUC summarizes ranking quality. Precision–recall curves are often more informative under class imbalance.
Exam Synthesis
Choose linear/logistic baselines first; encode categoricals without the dummy trap; use ridge for shrinkage and LASSO for sparsity; prefer ensembles when a single tree overfits; remember KNN/SVM geometry and net flexibility; score classifiers with a confusion-matrix mindset when class imbalance and asymmetric costs matter. Prediction quality is an out-of-sample property—QA–15 rewards candidates who never confuse training fit with operational performance.