What are AI Agents?

In partnership with

Your Voice AI Agent, Just a Few Clicks Away

Want to have a calling assistant that takes calls 24/7 and performs routine tasks real-time booking and lead qualification but don’t know where to start? Browse through a library of ready-made templates of AI Agents – proven in Real-Life Scenarios & tailored to industries like real estate, healthcare, and more.

With features like real-time booking and lead qualification, you can get started fast—or even create and sell your own template to earn commission!

Explore the Templates Now

Compound AI
The first shift I want to discuss is the move from monolithic models to compound AI systems. Models on their own are limited by the data they've been trained on, which impacts what they know about the world and what tasks they can solve. They are also difficult to adapt. While you could tune a model, it would require an investment in data and resources.

To illustrate, let’s say I want to plan a vacation for the summer and find out how many vacation days I have available. If I feed that query into a model, it’s likely to give an incorrect answer because the model doesn’t know who I am and doesn’t have access to sensitive information about me.

While models on their own can be useful for tasks like summarizing documents, drafting emails, and creating reports, the real magic happens when we build systems around the model and integrate them into existing processes. To solve this vacation question, the model would need access to the database where my vacation data is stored. The query would then be processed to search the database, and the model would generate a sentence with the correct answer, such as, “Maya, you have ten days left in your vacation database.”

This is an example of a compound AI system, where problems are better solved by applying the principles of system design. In a system, there are multiple components, and systems are inherently modular. I can have a model, select from tuned models, large language models, or image generation models, but I can also incorporate programmatic components such as output verifiers, query processors, database searchers, and other tools.

This approach is easier to adapt and faster to solve than tuning a model. It allows for a more flexible system design and a quicker adaptation process.

Retrieval-Augmented Generation (RAG)
One of the most popular compound AI systems is retrieval-augmented generation (RAG). Let’s say I ask about the weather. If the system is designed to search my vacation policy database, it will fail because the query is unrelated to the vacation data. The path to answer a query is determined by the program’s control logic, which defines how the system should respond.

Agents and AI Logic Control
So where do agents come in? One way to control the logic of a compound AI system is by using a large language model (LLM). LLMs have made tremendous advancements in reasoning capabilities, and they can break down complex problems into manageable steps, creating a plan to solve them.

On one end of the spectrum, the system is told to think fast and act according to the given instructions. On the other end, the system is designed to think slowly, break down the problem, and adjust the plan as needed. By putting LLMs in charge of the system’s logic, we create an agentic approach, where the model not only executes tasks but reasons through the process and adapts.

Components of LLM Agents
The key components of LLM agents are:

1. Reasoning: The model comes up with a plan and thinks through each step.

2. Action: The model uses external tools (such as search engines, calculators, or other models) to gather information and act on it.

3. Memory: The system can store logs and retrieve them for future use. This can include past conversations or other relevant data that can improve personalization.

A popular way to configure agents is through ReACT, which combines reasoning and acting. When a user query is fed into the model, the model is instructed to think slowly, plan its actions, and execute them. If the first tool call results in an incorrect answer, the model can observe the error, adjust the plan, and retry until it finds the correct answer.

Concrete Example: Vacation Planning
Let’s return to my vacation example. I want to know how many two-ounce sunscreen bottles I should bring to Florida next month. This is a complex problem because I need to consider several factors: the number of vacation days, the expected hours in the sun, the recommended sunscreen dosage, and how much sunscreen fits into two-ounce bottles.

This is a complex system that can explore many possible paths to find a solution. Compound AI systems are inherently modular, which makes them well-suited to handle such complex problems.

The Future of AI Systems
In 2024, we’re going to see more AI systems becoming agent-based. These systems will offer a sliding scale of autonomy, allowing the designer to define the level of autonomy required for specific tasks. For narrow, well-defined problems, a programmatic approach may be more efficient. For more complex tasks, an agent approach will be beneficial, as it allows for greater flexibility and adaptability.

We are still in the early days of agent systems, but progress is happening rapidly. By combining system design with agentic behavior, we’re moving towards a future where AI can handle increasingly complex tasks, with humans providing oversight as needed.