Chapter 9: Probability
Learn Machine Learning from very beginner to expert with detailed topic guidance, practical examples, practice exercises, and review questions.
What this chapter covers
This chapter contains 15 topics. Technical terms are followed by plain-language meanings in parentheses where they first appear. Code is included only when it naturally helps demonstrate the concept; architecture, workflow, governance, and comparison topics use practical scenarios instead.
9.1 Probability Fundamentals
Probability Fundamentals (a numerical description of how likely an event is). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small real-world project where Probability Fundamentals is the main idea. Identify the input information, the decision or transformation that occurs, and the result you would inspect to decide whether the method is working correctly.
Coding example
// Probability Fundamentals
const outcomes = [1, 0, 1, 1, 0, 1, 0, 1];
const successes = outcomes.reduce((sum, x) => sum + x, 0);
const probability = successes / outcomes.length;
const smoothed = (successes + 1) / (outcomes.length + 2);
console.log({ probability: probability.toFixed(3), smoothed: smoothed.toFixed(3) });Code explanation
- Each `1` represents an observed success and each `0` represents a non-success.
- Dividing the number of successes by the number of observations gives an empirical probability.
- The smoothed estimate adds one pseudo-success and one pseudo-failure so very small datasets are less extreme.
- Comparing the raw and smoothed results demonstrates how probabilistic estimates can change when prior information is introduced.
Expected result: Two probability estimates are printed for comparison.
Practice exercise
Create a second example for Probability Fundamentals. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
9.2 Sample Spaces
Sample Spaces (a practical concept used within the mathematical and statistical foundation). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small machine-learning project. Use Sample Spaces to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Sample Spaces
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Sample Spaces. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
9.3 Events
Events (a practical concept used within the mathematical and statistical foundation). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small machine-learning project. Use Events to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Events
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Events. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
9.4 Conditional Probability
Conditional Probability (a numerical description of how likely an event is). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small real-world project where Conditional Probability is the main idea. Identify the input information, the decision or transformation that occurs, and the result you would inspect to decide whether the method is working correctly.
Coding example
// Conditional Probability
const outcomes = [1, 0, 1, 1, 0, 1, 0, 1];
const successes = outcomes.reduce((sum, x) => sum + x, 0);
const probability = successes / outcomes.length;
const smoothed = (successes + 1) / (outcomes.length + 2);
console.log({ probability: probability.toFixed(3), smoothed: smoothed.toFixed(3) });Code explanation
- Each `1` represents an observed success and each `0` represents a non-success.
- Dividing the number of successes by the number of observations gives an empirical probability.
- The smoothed estimate adds one pseudo-success and one pseudo-failure so very small datasets are less extreme.
- Comparing the raw and smoothed results demonstrates how probabilistic estimates can change when prior information is introduced.
Expected result: Two probability estimates are printed for comparison.
Practice exercise
Create a second example for Conditional Probability. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
9.5 Independent Events
Independent Events (a practical concept used within the mathematical and statistical foundation). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small machine-learning project. Use Independent Events to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Independent Events
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Independent Events. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
9.6 Bayes' Theorem
Bayes' Theorem (a practical concept used within the mathematical and statistical foundation). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small machine-learning project. Use Bayes' Theorem to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Bayes' Theorem
const outcomes = [1, 0, 1, 1, 0, 1, 0, 1];
const successes = outcomes.reduce((sum, x) => sum + x, 0);
const probability = successes / outcomes.length;
const smoothed = (successes + 1) / (outcomes.length + 2);
console.log({ probability: probability.toFixed(3), smoothed: smoothed.toFixed(3) });Code explanation
- Each `1` represents an observed success and each `0` represents a non-success.
- Dividing the number of successes by the number of observations gives an empirical probability.
- The smoothed estimate adds one pseudo-success and one pseudo-failure so very small datasets are less extreme.
- Comparing the raw and smoothed results demonstrates how probabilistic estimates can change when prior information is introduced.
Expected result: Two probability estimates are printed for comparison.
Practice exercise
Create a small real-world example for Bayes' Theorem. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
9.7 Random Variables
Random Variables (a practical concept used within the mathematical and statistical foundation). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small machine-learning project. Use Random Variables to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Random Variables
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Random Variables. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
9.8 Probability Distributions
Probability Distributions (a numerical description of how likely an event is). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small real-world project where Probability Distributions is the main idea. Identify the input information, the decision or transformation that occurs, and the result you would inspect to decide whether the method is working correctly.
Coding example
// Probability Distributions
const outcomes = [1, 0, 1, 1, 0, 1, 0, 1];
const successes = outcomes.reduce((sum, x) => sum + x, 0);
const probability = successes / outcomes.length;
const smoothed = (successes + 1) / (outcomes.length + 2);
console.log({ probability: probability.toFixed(3), smoothed: smoothed.toFixed(3) });Code explanation
- Each `1` represents an observed success and each `0` represents a non-success.
- Dividing the number of successes by the number of observations gives an empirical probability.
- The smoothed estimate adds one pseudo-success and one pseudo-failure so very small datasets are less extreme.
- Comparing the raw and smoothed results demonstrates how probabilistic estimates can change when prior information is introduced.
Expected result: Two probability estimates are printed for comparison.
Practice exercise
Create a second example for Probability Distributions. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
9.9 Expected Value
Expected Value (a practical concept used within the mathematical and statistical foundation). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small machine-learning project. Use Expected Value to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Expected Value
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Expected Value. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
9.10 Variance
Variance (a measure of how spread out values are). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small machine-learning project. Use Variance to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Variance
const values = [12, 15, 11, 18, 14, 16];
const mean = values.reduce((sum, x) => sum + x, 0) / values.length;
const variance = values.reduce((sum, x) => sum + (x - mean) ** 2, 0) / values.length;
const std = Math.sqrt(variance);
console.log({ mean: mean.toFixed(2), std: std.toFixed(2) });Code explanation
- `values` is a tiny dataset that can be checked manually.
- The mean is the total divided by the number of observations.
- Variance measures average squared distance from the mean, and the square root of variance gives standard deviation.
- These summary values help you understand the scale and spread of data before choosing or evaluating a model.
Expected result: The mean and standard deviation are printed.
Practice exercise
Create a small real-world example for Variance. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
9.11 Covariance
Covariance (a measure of whether two variables tend to change together). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small machine-learning project. Use Covariance to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Covariance
const values = [12, 15, 11, 18, 14, 16];
const mean = values.reduce((sum, x) => sum + x, 0) / values.length;
const variance = values.reduce((sum, x) => sum + (x - mean) ** 2, 0) / values.length;
const std = Math.sqrt(variance);
console.log({ mean: mean.toFixed(2), std: std.toFixed(2) });Code explanation
- `values` is a tiny dataset that can be checked manually.
- The mean is the total divided by the number of observations.
- Variance measures average squared distance from the mean, and the square root of variance gives standard deviation.
- These summary values help you understand the scale and spread of data before choosing or evaluating a model.
Expected result: The mean and standard deviation are printed.
Practice exercise
Create a small real-world example for Covariance. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
9.12 Joint Probability
Joint Probability (a numerical description of how likely an event is). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small real-world project where Joint Probability is the main idea. Identify the input information, the decision or transformation that occurs, and the result you would inspect to decide whether the method is working correctly.
Coding example
// Joint Probability
const outcomes = [1, 0, 1, 1, 0, 1, 0, 1];
const successes = outcomes.reduce((sum, x) => sum + x, 0);
const probability = successes / outcomes.length;
const smoothed = (successes + 1) / (outcomes.length + 2);
console.log({ probability: probability.toFixed(3), smoothed: smoothed.toFixed(3) });Code explanation
- Each `1` represents an observed success and each `0` represents a non-success.
- Dividing the number of successes by the number of observations gives an empirical probability.
- The smoothed estimate adds one pseudo-success and one pseudo-failure so very small datasets are less extreme.
- Comparing the raw and smoothed results demonstrates how probabilistic estimates can change when prior information is introduced.
Expected result: Two probability estimates are printed for comparison.
Practice exercise
Create a second example for Joint Probability. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
9.13 Marginal Probability
Marginal Probability (a numerical description of how likely an event is). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small real-world project where Marginal Probability is the main idea. Identify the input information, the decision or transformation that occurs, and the result you would inspect to decide whether the method is working correctly.
Coding example
// Marginal Probability
const outcomes = [1, 0, 1, 1, 0, 1, 0, 1];
const successes = outcomes.reduce((sum, x) => sum + x, 0);
const probability = successes / outcomes.length;
const smoothed = (successes + 1) / (outcomes.length + 2);
console.log({ probability: probability.toFixed(3), smoothed: smoothed.toFixed(3) });Code explanation
- Each `1` represents an observed success and each `0` represents a non-success.
- Dividing the number of successes by the number of observations gives an empirical probability.
- The smoothed estimate adds one pseudo-success and one pseudo-failure so very small datasets are less extreme.
- Comparing the raw and smoothed results demonstrates how probabilistic estimates can change when prior information is introduced.
Expected result: Two probability estimates are printed for comparison.
Practice exercise
Create a second example for Marginal Probability. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
9.14 Maximum Likelihood
Maximum Likelihood (how compatible observed data is with a particular model or parameter value). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
Imagine a small machine-learning project. Use Maximum Likelihood to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Maximum Likelihood
const outcomes = [1, 0, 1, 1, 0, 1, 0, 1];
const successes = outcomes.reduce((sum, x) => sum + x, 0);
const probability = successes / outcomes.length;
const smoothed = (successes + 1) / (outcomes.length + 2);
console.log({ probability: probability.toFixed(3), smoothed: smoothed.toFixed(3) });Code explanation
- Each `1` represents an observed success and each `0` represents a non-success.
- Dividing the number of successes by the number of observations gives an empirical probability.
- The smoothed estimate adds one pseudo-success and one pseudo-failure so very small datasets are less extreme.
- Comparing the raw and smoothed results demonstrates how probabilistic estimates can change when prior information is introduced.
Expected result: Two probability estimates are printed for comparison.
Practice exercise
Create a small real-world example for Maximum Likelihood. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
9.15 Bayesian Probability
Bayesian Probability (a numerical description of how likely an event is). Within Chapter 9, this topic connects directly to the mathematical and statistical foundation. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
For a beginner, focus on the meaning before memorizing formulas or syntax. Work with a very small example, identify each quantity or step, and then connect it to the way a model learns from data. This makes later algorithms easier because the same ideas appear repeatedly in training, evaluation, and prediction.
Example
You begin with an initial belief about how common a condition is. After observing new evidence, Bayesian reasoning updates that belief to a new probability.
Coding example
// Bayesian Probability
const outcomes = [1, 0, 1, 1, 0, 1, 0, 1];
const successes = outcomes.reduce((sum, x) => sum + x, 0);
const probability = successes / outcomes.length;
const smoothed = (successes + 1) / (outcomes.length + 2);
console.log({ probability: probability.toFixed(3), smoothed: smoothed.toFixed(3) });Code explanation
- Each `1` represents an observed success and each `0` represents a non-success.
- Dividing the number of successes by the number of observations gives an empirical probability.
- The smoothed estimate adds one pseudo-success and one pseudo-failure so very small datasets are less extreme.
- Comparing the raw and smoothed results demonstrates how probabilistic estimates can change when prior information is introduced.
Expected result: Two probability estimates are printed for comparison.
Practice exercise
Create a second example for Bayesian Probability. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
Chapter 9 Review Questions and Answers
Q1. What is Probability Fundamentals?
Answer: Probability Fundamentals is a numerical description of how likely an event is. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q2. What is Sample Spaces?
Answer: Sample Spaces is a practical concept used within the mathematical and statistical foundation. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q3. What is Events?
Answer: Events is a practical concept used within the mathematical and statistical foundation. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q4. What is Conditional Probability?
Answer: Conditional Probability is a numerical description of how likely an event is. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q5. What is Independent Events?
Answer: Independent Events is a practical concept used within the mathematical and statistical foundation. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q6. What is Bayes' Theorem?
Answer: Bayes' Theorem is a practical concept used within the mathematical and statistical foundation. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q7. What is Random Variables?
Answer: Random Variables is a practical concept used within the mathematical and statistical foundation. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q8. What is Probability Distributions?
Answer: Probability Distributions is a numerical description of how likely an event is. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q9. What is Expected Value?
Answer: Expected Value is a practical concept used within the mathematical and statistical foundation. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q10. What is Variance?
Answer: Variance is a measure of how spread out values are. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q11. What is Covariance?
Answer: Covariance is a measure of whether two variables tend to change together. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q12. What is Joint Probability?
Answer: Joint Probability is a numerical description of how likely an event is. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q13. What is Marginal Probability?
Answer: Marginal Probability is a numerical description of how likely an event is. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q14. What is Maximum Likelihood?
Answer: Maximum Likelihood is how compatible observed data is with a particular model or parameter value. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q15. What is Bayesian Probability?
Answer: Bayesian Probability is a numerical description of how likely an event is. In this chapter, focus on the input, the method or decision, and the result that should be checked.