Machine Learning Chapter 18: Linear Regression

What this chapter covers

This chapter contains 13 topics. Technical terms are followed by plain-language meanings in parentheses where they first appear. Code is included only when it naturally helps demonstrate the concept; architecture, workflow, governance, and comparison topics use practical scenarios instead.

18.1 Regression Problems

Regression Problems (predicting a continuous numerical value). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

Imagine a small machine-learning project. Use Regression Problems to decide what information is needed, what step happens next, and what result should be checked.

Coding example

// Regression Problems
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a small real-world example for Regression Problems. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.

18.2 Simple Linear Regression

Simple Linear Regression (a regression method that models the target as a weighted sum of input features). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

A model receives the size, age, and location of a house and predicts a numerical price such as $850,000.

Coding example

// Simple Linear Regression
const points = [[1, 2], [2, 4], [3, 5], [4, 8]];
const mean = arr => arr.reduce((a,b) => a+b, 0) / arr.length;
const xs = points.map(p => p[0]), ys = points.map(p => p[1]);
const mx = mean(xs), my = mean(ys);
const slope = xs.reduce((s,x,i) => s + (x-mx)*(ys[i]-my), 0) / xs.reduce((s,x) => s + (x-mx)**2, 0);
const intercept = my - slope * mx;
const predict = x => intercept + slope * x;

console.log({ slope: slope.toFixed(2), predictionAt5: predict(5).toFixed(2) });

Code explanation

The training data is stored as small `[input, target]` pairs.
The code calculates the slope from how input and target values vary together.
The intercept places the fitted line at the correct vertical position.
`predict(5)` applies the learned line to a new input, demonstrating how a fitted regression model produces a numeric prediction.

Expected result: The fitted slope and a prediction for input 5 are printed.

Practice exercise

Create a second example for Simple Linear Regression. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.

18.3 Multiple Linear Regression

Multiple Linear Regression (a regression method that models the target as a weighted sum of input features). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

A model receives the size, age, and location of a house and predicts a numerical price such as $850,000.

Coding example

// Multiple Linear Regression
const points = [[1, 2], [2, 4], [3, 5], [4, 8]];
const mean = arr => arr.reduce((a,b) => a+b, 0) / arr.length;
const xs = points.map(p => p[0]), ys = points.map(p => p[1]);
const mx = mean(xs), my = mean(ys);
const slope = xs.reduce((s,x,i) => s + (x-mx)*(ys[i]-my), 0) / xs.reduce((s,x) => s + (x-mx)**2, 0);
const intercept = my - slope * mx;
const predict = x => intercept + slope * x;

console.log({ slope: slope.toFixed(2), predictionAt5: predict(5).toFixed(2) });

Code explanation

The training data is stored as small `[input, target]` pairs.
The code calculates the slope from how input and target values vary together.
The intercept places the fitted line at the correct vertical position.
`predict(5)` applies the learned line to a new input, demonstrating how a fitted regression model produces a numeric prediction.

Expected result: The fitted slope and a prediction for input 5 are printed.

Practice exercise

Create a second example for Multiple Linear Regression. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.

18.4 Regression Equation

Regression Equation (predicting a continuous numerical value). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

A model receives the size, age, and location of a house and predicts a numerical price such as $850,000.

Coding example

// Regression Equation
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a second example for Regression Equation. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.

18.5 Coefficients

Coefficients (a practical concept used within supervised learning, evaluation, and ensemble methods). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

Imagine a small real-world project where Coefficients is the main idea. Identify the input information, the decision or transformation that occurs, and the result you would inspect to decide whether the method is working correctly.

Coding example

// Coefficients
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a second example for Coefficients. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.

18.6 Intercepts

Intercepts (a practical concept used within supervised learning, evaluation, and ensemble methods). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

Imagine a small real-world project where Intercepts is the main idea. Identify the input information, the decision or transformation that occurs, and the result you would inspect to decide whether the method is working correctly.

Coding example

// Intercepts
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a second example for Intercepts. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.

18.7 Least Squares

Least Squares (a practical concept used within supervised learning, evaluation, and ensemble methods). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

Imagine a small real-world project where Least Squares is the main idea. Identify the input information, the decision or transformation that occurs, and the result you would inspect to decide whether the method is working correctly.

Coding example

// Least Squares
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a second example for Least Squares. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.

18.8 Residuals

Residuals (a practical concept used within supervised learning, evaluation, and ensemble methods). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

Imagine a small machine-learning project. Use Residuals to decide what information is needed, what step happens next, and what result should be checked.

Coding example

// Residuals
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a small real-world example for Residuals. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.

18.9 R-Squared

R-Squared (a practical concept used within supervised learning, evaluation, and ensemble methods). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

Imagine a small machine-learning project. Use R-Squared to decide what information is needed, what step happens next, and what result should be checked.

Coding example

// R-Squared
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a small real-world example for R-Squared. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.

18.10 Adjusted R-Squared

Adjusted R-Squared (a practical concept used within supervised learning, evaluation, and ensemble methods). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

Imagine a small machine-learning project. Use Adjusted R-Squared to decide what information is needed, what step happens next, and what result should be checked.

Coding example

// Adjusted R-Squared
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a small real-world example for Adjusted R-Squared. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.

18.11 Assumptions

Assumptions (a practical concept used within supervised learning, evaluation, and ensemble methods). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

Imagine a small machine-learning project. Use Assumptions to decide what information is needed, what step happens next, and what result should be checked.

Coding example

// Assumptions
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a small real-world example for Assumptions. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.

18.12 Regression Diagnostics

Regression Diagnostics (predicting a continuous numerical value). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

Imagine a small machine-learning project. Use Regression Diagnostics to decide what information is needed, what step happens next, and what result should be checked.

Coding example

// Regression Diagnostics
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a small real-world example for Regression Diagnostics. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.

18.13 Predictions

Predictions (the output produced by a trained model for new input). Within Chapter 18, this topic connects directly to supervised learning, evaluation, and ensemble methods. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.

When using this idea, separate training data from evaluation data and compare performance on examples the model did not train on. Pay attention to assumptions, important settings, error patterns, and whether the method is appropriate for classification, regression, ranking, or probability estimation.

Example

A model first learns from past houses with known prices. After training, it receives a new house description and produces a predicted price.

Coding example

// Predictions
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);

console.log(results);

Code explanation

The sample starts with a small list of inputs so every result can be checked manually.
`transform()` represents the main operation for this topic in a deliberately simple form.
`map()` applies the same rule consistently to every item and returns a new result array.
Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.

Expected result: A transformed result is printed for each input value.

Practice exercise

Create a second example for Predictions. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.

Chapter 18 Review Questions and Answers

Q1. What is Regression Problems?

Answer: Regression Problems is predicting a continuous numerical value. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q2. What is Simple Linear Regression?

Answer: Simple Linear Regression is a regression method that models the target as a weighted sum of input features. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q3. What is Multiple Linear Regression?

Answer: Multiple Linear Regression is a regression method that models the target as a weighted sum of input features. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q4. What is Regression Equation?

Answer: Regression Equation is predicting a continuous numerical value. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q5. What is Coefficients?

Answer: Coefficients is a practical concept used within supervised learning, evaluation, and ensemble methods. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q6. What is Intercepts?

Answer: Intercepts is a practical concept used within supervised learning, evaluation, and ensemble methods. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q7. What is Least Squares?

Answer: Least Squares is a practical concept used within supervised learning, evaluation, and ensemble methods. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q8. What is Residuals?

Answer: Residuals is a practical concept used within supervised learning, evaluation, and ensemble methods. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q9. What is R-Squared?

Answer: R-Squared is a practical concept used within supervised learning, evaluation, and ensemble methods. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q10. What is Adjusted R-Squared?

Answer: Adjusted R-Squared is a practical concept used within supervised learning, evaluation, and ensemble methods. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q11. What is Assumptions?

Answer: Assumptions is a practical concept used within supervised learning, evaluation, and ensemble methods. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q12. What is Regression Diagnostics?

Answer: Regression Diagnostics is predicting a continuous numerical value. In this chapter, focus on the input, the method or decision, and the result that should be checked.

Q13. What is Predictions?

Answer: Predictions is the output produced by a trained model for new input. In this chapter, focus on the input, the method or decision, and the result that should be checked.