Chapter 59: AI Agents and Tool-Using Models
Learn Machine Learning from very beginner to expert with detailed topic guidance, practical examples, practice exercises, and review questions.
What this chapter covers
This chapter contains 12 topics. Technical terms are followed by plain-language meanings in parentheses where they first appear. Code is included only when it naturally helps demonstrate the concept; architecture, workflow, governance, and comparison topics use practical scenarios instead.
59.1 What Is an AI Agent?
What Is an AI Agent? (a system that observes, decides, and acts toward a goal). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
An AI agent receives a goal, decides which steps are needed, uses an available tool when necessary, checks the result, and continues until the task is complete.
Coding example
// What Is an AI Agent?
const tools = {
average: values => values.reduce((a,b)=>a+b,0)/values.length,
maximum: values => Math.max(...values)
};
const task = { tool: 'average', input: [4,7,9,10] };
const result = tools[task.tool](task.input);
console.log({ task, result });Code explanation
- The `tools` object acts as a small registry of allowed operations.
- The task explicitly names which tool should run and provides its input.
- The dispatcher selects the requested function and executes it.
- This pattern demonstrates controlled tool use and workflow orchestration without giving unrestricted access to arbitrary operations.
Expected result: The selected tool and its computed result are printed.
Practice exercise
Create a second example for What Is an AI Agent?. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
59.2 Agent Environments
Agent Environments (a system that observes, decides, and acts toward a goal). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
An AI agent receives a goal, decides which steps are needed, uses an available tool when necessary, checks the result, and continues until the task is complete.
Coding example
// Agent Environments
const tools = {
average: values => values.reduce((a,b)=>a+b,0)/values.length,
maximum: values => Math.max(...values)
};
const task = { tool: 'average', input: [4,7,9,10] };
const result = tools[task.tool](task.input);
console.log({ task, result });Code explanation
- The `tools` object acts as a small registry of allowed operations.
- The task explicitly names which tool should run and provides its input.
- The dispatcher selects the requested function and executes it.
- This pattern demonstrates controlled tool use and workflow orchestration without giving unrestricted access to arbitrary operations.
Expected result: The selected tool and its computed result are printed.
Practice exercise
Create a second example for Agent Environments. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
59.3 Planning
Planning (a practical concept used within generative, multimodal, and agent-based AI). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
Imagine a small machine-learning project. Use Planning to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Planning
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Planning. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
59.4 Reasoning Loops
Reasoning Loops (a practical concept used within generative, multimodal, and agent-based AI). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
Imagine a small machine-learning project. Use Reasoning Loops to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Reasoning Loops
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Reasoning Loops. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
59.5 Tool Calling
Tool Calling (a practical concept used within generative, multimodal, and agent-based AI). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
Imagine a small machine-learning project. Use Tool Calling to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Tool Calling
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Tool Calling. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
59.6 Function Calling
Function Calling (a practical concept used within generative, multimodal, and agent-based AI). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
Imagine a small machine-learning project. Use Function Calling to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Function Calling
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Function Calling. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
59.7 Memory Systems
Memory Systems (a practical concept used within generative, multimodal, and agent-based AI). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
Imagine a small machine-learning project. Use Memory Systems to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Memory Systems
const records = [3, 5, 7, 9, 11];
const transform = value => ({ input: value, output: value * 2 + 1 });
const results = records.map(transform);
console.log(results);Code explanation
- The sample starts with a small list of inputs so every result can be checked manually.
- `transform()` represents the main operation for this topic in a deliberately simple form.
- `map()` applies the same rule consistently to every item and returns a new result array.
- Use this pattern to focus on input, transformation, and output before replacing the toy rule with a more advanced method.
Expected result: A transformed result is printed for each input value.
Practice exercise
Create a small real-world example for Memory Systems. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
59.8 Retrieval Memory
Retrieval Memory (a practical concept used within generative, multimodal, and agent-based AI). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
Imagine a small machine-learning project. Use Retrieval Memory to decide what information is needed, what step happens next, and what result should be checked.
Coding example
// Retrieval Memory
const documents = [
'models learn patterns from data',
'graphs connect related entities',
'retrieval finds useful context'
];
const query = 'find useful data';
const words = s => new Set(s.toLowerCase().split(/\s+/));
const q = words(query);
const scored = documents.map((text,i)=>({i,text,score:[...words(text)].filter(w=>q.has(w)).length})).sort((a,b)=>b.score-a.score);
console.log(scored[0]);Code explanation
- The documents and query are converted into simple sets of lowercase words.
- Each document receives one point for every word it shares with the query.
- Sorting by score produces a basic relevance ranking.
- Real retrieval systems use stronger representations and indexes, but this tiny example makes the retrieval step visible.
Expected result: The highest-scoring document is printed.
Practice exercise
Create a small real-world example for Retrieval Memory. Write the input, the goal, the main steps, and the result you would check. Then list one limitation or mistake a beginner should watch for.
59.9 Agent Workflows
Agent Workflows (a system that observes, decides, and acts toward a goal). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
An AI agent receives a goal, decides which steps are needed, uses an available tool when necessary, checks the result, and continues until the task is complete.
Coding example
// Agent Workflows
const tools = {
average: values => values.reduce((a,b)=>a+b,0)/values.length,
maximum: values => Math.max(...values)
};
const task = { tool: 'average', input: [4,7,9,10] };
const result = tools[task.tool](task.input);
console.log({ task, result });Code explanation
- The `tools` object acts as a small registry of allowed operations.
- The task explicitly names which tool should run and provides its input.
- The dispatcher selects the requested function and executes it.
- This pattern demonstrates controlled tool use and workflow orchestration without giving unrestricted access to arbitrary operations.
Expected result: The selected tool and its computed result are printed.
Practice exercise
Create a second example for Agent Workflows. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
59.10 Multi-Agent Systems
Multi-Agent Systems (a system that observes, decides, and acts toward a goal). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
An AI agent receives a goal, decides which steps are needed, uses an available tool when necessary, checks the result, and continues until the task is complete.
Coding example
// Multi-Agent Systems
const tools = {
average: values => values.reduce((a,b)=>a+b,0)/values.length,
maximum: values => Math.max(...values)
};
const task = { tool: 'average', input: [4,7,9,10] };
const result = tools[task.tool](task.input);
console.log({ task, result });Code explanation
- The `tools` object acts as a small registry of allowed operations.
- The task explicitly names which tool should run and provides its input.
- The dispatcher selects the requested function and executes it.
- This pattern demonstrates controlled tool use and workflow orchestration without giving unrestricted access to arbitrary operations.
Expected result: The selected tool and its computed result are printed.
Practice exercise
Create a second example for Multi-Agent Systems. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
59.11 Agent Evaluation
Agent Evaluation (a system that observes, decides, and acts toward a goal). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
An AI agent receives a goal, decides which steps are needed, uses an available tool when necessary, checks the result, and continues until the task is complete.
Coding example
// Agent Evaluation
const values = [12, 15, 11, 18, 14, 16];
const mean = values.reduce((sum, x) => sum + x, 0) / values.length;
const variance = values.reduce((sum, x) => sum + (x - mean) ** 2, 0) / values.length;
const std = Math.sqrt(variance);
console.log({ mean: mean.toFixed(2), std: std.toFixed(2) });Code explanation
- `values` is a tiny dataset that can be checked manually.
- The mean is the total divided by the number of observations.
- Variance measures average squared distance from the mean, and the square root of variance gives standard deviation.
- These summary values help you understand the scale and spread of data before choosing or evaluating a model.
Expected result: The mean and standard deviation are printed.
Practice exercise
Create a second example for Agent Evaluation. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
59.12 Agent Safety
Agent Safety (a system that observes, decides, and acts toward a goal). Within Chapter 59, this topic connects directly to generative, multimodal, and agent-based AI. The important goal is to understand what information goes into the method, what transformation or decision happens, and what output should be checked.
The input may be images, sequences, graphs, interactions, rewards, or multiple modalities. Define the task and evaluation measure first, then check whether the representation and model architecture preserve the information needed for the final decision or generated output.
Example
An AI agent receives a goal, decides which steps are needed, uses an available tool when necessary, checks the result, and continues until the task is complete.
Coding example
// Agent Safety
const tools = {
average: values => values.reduce((a,b)=>a+b,0)/values.length,
maximum: values => Math.max(...values)
};
const task = { tool: 'average', input: [4,7,9,10] };
const result = tools[task.tool](task.input);
console.log({ task, result });Code explanation
- The `tools` object acts as a small registry of allowed operations.
- The task explicitly names which tool should run and provides its input.
- The dispatcher selects the requested function and executes it.
- This pattern demonstrates controlled tool use and workflow orchestration without giving unrestricted access to arbitrary operations.
Expected result: The selected tool and its computed result are printed.
Practice exercise
Create a second example for Agent Safety. Change one important condition or input, predict how the result should change, and explain why. Then identify one limitation or common mistake a beginner should watch for.
Chapter 59 Review Questions and Answers
Q1. What is What Is an AI Agent??
Answer: What Is an AI Agent? is a system that observes, decides, and acts toward a goal. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q2. What is Agent Environments?
Answer: Agent Environments is a system that observes, decides, and acts toward a goal. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q3. What is Planning?
Answer: Planning is a practical concept used within generative, multimodal, and agent-based AI. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q4. What is Reasoning Loops?
Answer: Reasoning Loops is a practical concept used within generative, multimodal, and agent-based AI. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q5. What is Tool Calling?
Answer: Tool Calling is a practical concept used within generative, multimodal, and agent-based AI. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q6. What is Function Calling?
Answer: Function Calling is a practical concept used within generative, multimodal, and agent-based AI. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q7. What is Memory Systems?
Answer: Memory Systems is a practical concept used within generative, multimodal, and agent-based AI. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q8. What is Retrieval Memory?
Answer: Retrieval Memory is a practical concept used within generative, multimodal, and agent-based AI. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q9. What is Agent Workflows?
Answer: Agent Workflows is a system that observes, decides, and acts toward a goal. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q10. What is Multi-Agent Systems?
Answer: Multi-Agent Systems is a system that observes, decides, and acts toward a goal. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q11. What is Agent Evaluation?
Answer: Agent Evaluation is a system that observes, decides, and acts toward a goal. In this chapter, focus on the input, the method or decision, and the result that should be checked.
Q12. What is Agent Safety?
Answer: Agent Safety is a system that observes, decides, and acts toward a goal. In this chapter, focus on the input, the method or decision, and the result that should be checked.