@cloudflare/codemode

Cloudflare's Code Mode allows Large Language Models (LLMs) to generate and execute code for external tool calls. This enables LLMs to interact with APIs and services, moving beyond text generation to perform real-world tasks and build more autonomous AI agents.

intermediate10 min5 steps

The play

Define Your Tool Function
Create a Python function that acts as an external tool or API wrapper. This function should accept parameters that an LLM can infer from a user's prompt. For example, a weather fetching tool:
Simulate LLM-Generated Code
Imagine an LLM has processed a user request (e.g., "What's the weather in London?") and generated Python code to call your tool. This code will specify the tool's name and its arguments.
Execute the Generated Code
Use Python's `exec()` function to run the LLM-generated code string. This executes the tool call within your application's context. Ensure `globals()` is passed to allow the generated code to find your defined tool function.
Handle Dynamic Tool Calls
Demonstrate how the LLM can adapt to different prompts (e.g., "Tell me the weather in New York in Fahrenheit") by generating different tool calls. This highlights the dynamic problem-solving capability.
Integrate with an Actual LLM
In a real-world scenario, you would integrate this pattern with an actual LLM. The LLM's output parser would extract the `tool_name` and `tool_args` from its response (or the LLM directly generates the executable Python code), which you then safely execute.

Starter code

import json

# Define a sample 'tool' function that an LLM might call
def get_weather(location: str, unit: str = "celsius"):
    """Fetches current weather for a given location."""
    if location.lower() == "london":
        return {"location": "London", "temperature": "15", "unit": unit, "conditions": "cloudy"}
    elif location.lower() == "new york":
        return {"location": "New York", "temperature": "22", "unit": unit, "conditions": "sunny"}
    else:
        return {"location": location, "temperature": "unknown", "unit": unit, "conditions": "unavailable"}

# Simulate LLM-generated executable code (e.g., from prompt: "What's the weather in London?")
llm_generated_code = """
tool_name = "get_weather"
tool_args = {"location": "London", "unit": "celsius"}
result = globals()[tool_name](**tool_args)
print(json.dumps(result))
"""

# Execute the LLM-generated code
print("Executing LLM-generated code:")
exec(llm_generated_code, globals())

Source

Articlenpmjs.com