Leveraging the Language Understanding in Your AI Agent

Most people building AI Agents are familiar with traditional "tool calling". This is where we describe a function, via its name and parameters, to the LLM powering our agent, and ask the agent to "invoke" this function when it needs some information or capability. The LLM actually just generates a text block that requests the function to be invoked. Then our agent framework actually calls the function and injects the result back into the LLM's context.

This works well, and the MCP (Model Context Protocol) system has created an ecosystem of thousands of "tools" that our agent can use to do useful work. Tool calling is very attractive because it fits our human understanding of how to do imperative programming.

But function calling is pretty limited, and in particular the ability for your LLM to generate input parameters to your function is inherently limited. If we go back and think about the language understanding of our LLM we can come up with some other approaches for accessing information.

Writing Code

One thing that current frontier LLMs excel at is writing code. In fact some researchers have shown that using LLMs to write function calls via code generation is more effective than using tool calling directly. Observing this, Hugging Face built CodeAgent into their smolagents library as a first-class citizen. Here is an example of using CodeAgent with a web-browsing tool:

import os
from smolagents import CodeAgent, WebSearchTool, InferenceClientModel
from smolagents import LiteLLMModel

model = LiteLLMModel(
    model_id="openai/gpt-4o-mini",
    temperature=0.2,
    api_key=os.environ["OPENAI_API_KEY"]
)

agent = CodeAgent(tools=[WebSearchTool()], model=model, stream_outputs=True)
agent.run("What is the latest news about the Premier League?")

and we can observe the code generation when we run the agent:

(sa) (base) scottp@MacBook-Air-76 sa % python agent.py
╭─────────────────────────── New run ───────────────────────────╮
│                                                               │
│ What is the latest news about the Premier League?             │
│                                                               │
╰─ LiteLLMModel - openai/gpt-4o-mini ───────────────────────────╯
━━━━━━━━━━━━━━━━━━━━━━━━━━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Thought: To find the latest news about the Premier League, I will
perform a web search using the query "latest news Premier League".
This should provide me with the most recent articles and updates
regarding the league.
─ Executing parsed code: ───────────────────────────────────────
  latest_news = web_search(query="latest news Premier League")
  print(latest_news)
────────────────────────────────────────────────────────────────
Execution logs:
## Search Results

[Premier League News - Latest News & Highlights - Sky Sports]
(https://www.skysports.com/premier-league-news)
Get the latest Premier League Football news, fixtures, results,
video and more from Sky Sports.

[Premier League Football - Latest news, results, stats & transfers - BBC]
(https://www.bbc.com/sport/football/premier-league)
The home of Premier League on BBC Sport online. Includes the latest
news stories, results, fixtures, video and audio.

This is pretty cool! I am a little skeptical that code generation performs better than tool calling as a general matter. And from playing with CodeAgent I find that it regularly generates code blocks which fail. Plus obviously there is some high security concern with running LLM-generated code without reviewing it, although for this purpose Hugging Face supports sandboxed code execution.

Using Text to SQL

One of the techniques we've been using with a lot of success at biztrip.ai is using SQL generation to access structured data. The classic approach to giving your Agent access to enterprise data is to give it some "lookup_info" or "search_for_data" tool. This works fine, but you have to strongly define all of the search patterns that your Agent can use.

Instead of this, we save enterprise data into a SQL database, and then give our Agent a generic "run_sql_query" command. This leverages the LLM's strong ability to generate SQL queries, and supports a huge range of data access patterns. The Agent can synthesize very exact filtering, fuzzy string matches, geo-based queries, and even extract "analytic" results by using aggregate functions, min/max, etc.

We have been using DuckDB (motherduck.com) as a data store because it is very lightweight, supports very expressive SQL, and our LLM "knows" its well-designed and well-documented syntax very well. (Contrast this with ClickHouse, which is an amazing data store but much less well known to the LLM.) You can even create DuckDB tables solely in memory and let your Agent query them directly from there.

These are just two examples of using the language part of your LLM more effectively to create smarter agents. In a future post we will talk about vector search for everything as a way to leverage LLM architecture to achieve very advanced results.

---

Originally published on LinkedIn.