Machine Learning Chapter 1 - Foundations (Must-Have Basics)

Chapter 1 - Foundations (Must-Have Basics)

Machine Learning begins with strong mathematical foundations. Before a model can learn from data, it needs ways to represent numbers, measure change, describe uncertainty, and reduce error. This chapter introduces the main foundational ideas used throughout machine learning.

1.1 Algebra

Algebra is working with numbers and letters together. The letters represent unknown numbers. In algebra, a letter like x or y is called a variable. A variable is a symbol that represents an unknown number.

This means: unknown number plus 3 equals 7. To find x, subtract 3 from both sides.

x + 3 = 7
x = 7 - 3
x = 4

So the hidden number is 4.

Why Algebra is important in Machine Learning

Machine learning models use mathematical formulas to describe relationships between data.

Meaning:

• Size = house size
• 2000 = price per square unit
• 5000 = base price

If the size is 10, the computer calculates:

Price = 2000 × 10 + 5000
Price = 20000 + 5000
Price = 25000

So algebra helps computers predict values.

Example

Given:
y = 2x + 1
x = 3

Step 1 - replace x with 3
y = 2(3) + 1

Step 2 - multiply
y = 6 + 1

Step 3 - add
y = 7

Final answer: y = 7.

1.2 Linear Algebra (Matrices, Eigenvalues, SVD)

Linear Algebra is one of the most important parts of machine learning. It works with vectors and matrices, which are essential for data representation, transformations, and computations.

[ 80   2 ]
[120   3 ]
[150   4 ]

• Row = one observation, such as one house
• Column = one feature, such as size or bedrooms
• Machine learning datasets are often matrices

Why Linear Algebra matters in ML

• Represents datasets as matrices
• Supports multiplication and transformations
• Helps in PCA, Neural Networks, and Linear Regression

Eigenvalues

Eigenvalues measure the importance or magnitude of the main directions in data. Think of them as showing how much variance exists along a particular direction.

Imagine 2D data points forming an elongated cloud:

• The long axis of the cloud = largest variance = largest eigenvalue
• The short axis = smaller variance = smaller eigenvalue

SVD (Singular Value Decomposition)

SVD breaks a large matrix into smaller meaningful parts.

• Like breaking a song into voice, music, and background
• Used for compression
• Used for recommendation systems

All machine learning data is processed through matrix thinking, so Linear Algebra is a core foundation.

1.3 Probability

Probability is the measure of how likely an event is to happen. Its range is from 0 to 1.

• 0 = impossible
• 1 = certain

Where:

• Favorable outcomes = the results we want
• Total possible outcomes = every result that can happen

Example 1 - Rolling a die

Possible outcomes: 1, 2, 3, 4, 5, 6
Event: getting a 4

Favorable outcomes = 1
Total outcomes = 6

P(4) = 1 / 6
Probability = 0.1667 ≈ 16.7%

Example 2 - Tossing a coin

Outcomes: Heads, Tails
Event: getting Heads

Favorable outcomes = 1
Total outcomes = 2

P(Heads) = 1 / 2
Probability = 0.5 = 50%

Probability tells us how likely something is.

Why Probability is important in Machine Learning

Machine learning models often predict probabilities instead of only one final answer.

Spam email probability = 0.92

That means the model thinks there is a 92% chance the email is spam.

If 10 emails total and 7 are spam:
Probability(spam) = 7 / 10 = 70%

1.4 Statistics

Statistics is the science of collecting, organizing, analyzing, and interpreting data to learn from numbers and make decisions.

Purpose of Statistics

• Summarize large amounts of data
• Identify patterns and trends
• Make predictions or informed decisions

Suppose a teacher records student marks: 70, 85, 90, 60, 75. Statistics helps explain what these numbers mean.

Mean (Average)

Numbers: 4, 8, 6, 10, 12
Mean = (4 + 8 + 6 + 10 + 12) / 5
Mean = 40 / 5
Mean = 8

Variance

Variance measures how far values are spread from the mean.

Numbers: 4, 8, 6, 10, 12
Mean = 8

(4 - 8)² = 16
(8 - 8)² = 0
(6 - 8)² = 4
(10 - 8)² = 4
(12 - 8)² = 16

Sum = 40
Variance = 40 / 5 = 8

Standard Deviation

Standard deviation is the square root of variance and tells how far values are from the mean in the same units as the data.

Variance = 8
Standard Deviation = √8 ≈ 2.83

1.5 Calculus (Partial Derivatives, Gradients)

Calculus is the study of change. In machine learning, calculus is used to understand slopes, rates of change, and how model parameters should be updated.

Derivative

A derivative measures how fast something changes.

A derivative tells the slope at a specific point.

Integral

An integral measures accumulation, such as total area under a curve.

Partial Derivatives

A partial derivative tells how the output changes when only one variable changes and the others stay fixed. This matters because machine learning models usually depend on many variables.

Suppose loss:
L = (y_pred - y_true)²

Derivative with respect to y_pred:
dL/dy_pred = 2(y_pred - y_true)

This tells us how predictions should change to reduce error.

Gradient

A gradient shows how a function changes in many directions. Think of it as the slope in multi-dimensional space.

Mountain idea:

• Height = loss or error
• Position = model parameters
• Gradient = direction to move to reduce error

1.6 Optimization Basics

Optimization means finding the best solution from all possible options. In machine learning, this usually means minimizing error or maximizing performance.

Simple example

Suppose you compare several stores and choose the lowest price. That is optimization.

Loss (Error)

Loss measures how wrong a model prediction is compared to the real value.

Real value = 100
Predicted value = 90
Error = |100 - 90| = 10

• High loss = poor prediction
• Low loss = better prediction

Gradient Descent

Gradient Descent improves a model step by step by moving in the direction that reduces loss.

1. Look at the slope
2. Move downhill
3. Repeat
4. Stop when the error is very small

How the foundations work together in ML

1. Algebra uses equations
2. Linear Algebra represents data as matrices
3. Probability handles uncertainty
4. Statistics summarizes data
5. Calculus measures change
6. Optimization reduces error

Example: predicting house prices with machine learning.

• Store house features in a matrix
• Use statistics to understand the dataset
• Use algebra to describe the prediction rule
• Use probability when uncertainty matters
• Use gradients and optimization to improve the model

1.7 Summary

In this chapter, you learned the must-have basics for machine learning:

• Algebra helps describe relationships with equations
• Linear Algebra helps store and process data as vectors and matrices
• Probability helps work with uncertainty
• Statistics helps summarize and understand data
• Calculus explains change, slopes, and gradients
• Optimization improves models by reducing loss

These ideas form the base for more advanced machine learning topics.