Function calling and tool use: a practical AI & ML guide

Function calling and tool use: a practical AI & ML guide

Large language models are increasingly capable at reasoning and planning, but they cannot directly interact with your systems—databases, APIs, payment processors, email services. Function calling and tool use bridges that gap, letting models request actions by describing what they need while your code maintains control over execution, validation, and permissions. This is how production AI systems fetch real data, make decisions, and trigger business logic safely.

Why function calling matters in production systems

In a typical chatbot or agent without tool use, the model generates text and that's it. You're stuck paraphrasing results or manually parsing responses. With function calling, the model understands it can invoke predefined tools—look up a customer order, calculate a discount, send a notification—and your backend orchestrates the execution. The model becomes an intelligent router; your code remains the gatekeeper.

We've built this pattern into e-commerce systems where agents answer customer questions by querying order history, inventory, and pricing in real time. We've used it in CRM workflows to automatically suggest next steps based on customer data. The difference between a static AI response and one that can act on live data is what separates a demo from production.

Describing tools with names and JSON schemas

Every tool your model can call must be explicitly defined. The model doesn't have magical access to your codebase; it reads a schema that tells it what's available, what arguments each tool expects, and what each argument means.

A typical tool definition includes a name, a human-readable description, and a JSON schema that specifies parameters. Here's an example for an e-commerce system:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_customer_orders",
            "description": "Fetch all orders for a customer. Returns order IDs, dates, statuses, and total amounts.",
            "parameters": {
                "type": "object",
                "properties": {
                    "customer_id": {
                        "type": "string",
                        "description": "The unique customer identifier"
                    },
                    "limit": {
                        "type": "integer",
                        "description": "Maximum number of orders to return. Default 10.",
                        "default": 10
                    }
                },
                "required": ["customer_id"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "apply_discount_code",
            "description": "Apply a discount code to an order. Returns the new total and breakdown.",
            "parameters": {
                "type": "object",
                "properties": {
                    "order_id": {
                        "type": "string",
                        "description": "The order to discount"
                    },
                    "code": {
                        "type": "string",
                        "description": "The discount code to apply"
                    }
                },
                "required": ["order_id", "code"]
            }
        }
    }
]

The schema is precise. The model reads it and learns exactly which tools exist, what inputs each needs, and which inputs are mandatory. Good descriptions help; vague ones confuse the model and lead to hallucinated arguments. Be specific about data types, valid ranges, and what the tool does.

The model chooses a tool and arguments

After you send the model a user query along with the tool definitions, the model analyzes the request and decides which tool to call. It doesn't execute anything—it responds with a structured message specifying the tool name and the arguments it wants to pass.

Here's what a model response looks like when using OpenAI's API format:

{
    "role": "assistant",
    "content": null,
    "tool_calls": [
        {
            "id": "call_abc123",
            "type": "function",
            "function": {
                "name": "get_customer_orders",
                "arguments": "{\"customer_id\": \"cust_42\", \"limit\": 5}"
            }
        }
    ]
}

Notice: the model isn't calling the function itself. It's saying "I want to call get_customer_orders with these arguments." Your code then decides whether to actually execute it. This separation of intent from execution is critical for security and auditability.

Handling the response in production code looks like this:

def process_tool_calls(response, user_id):
    results = []
    for tool_call in response.tool_calls:
        tool_name = tool_call.function.name
        arguments = json.loads(tool_call.function.arguments)
        
        # This is where YOUR code decides what happens
        if tool_name == "get_customer_orders":
            result = get_customer_orders(
                user_id=user_id,
                customer_id=arguments["customer_id"],
                limit=arguments.get("limit", 10)
            )
        elif tool_name == "apply_discount_code":
            result = apply_discount_code(
                user_id=user_id,
                order_id=arguments["order_id"],
                code=arguments["code"]
            )
        else:
            result = {"error": "Unknown tool"}
        
        results.append({
            "tool_call_id": tool_call.id,
            "result": result
        })
    
    return results

The model asks; your backend decides. This is where you enforce access control, rate limiting, and business logic.

Your code executes and returns results

Once you've validated the tool call, execute the underlying function and capture the result. In a ride-hailing system, this might query the database for nearby drivers. In a travel booking platform, it might call a third-party API for flight availability. The result is structured data that the model can understand.

Here's a realistic function that executes and returns structured feedback:

def get_customer_orders(user_id, customer_id, limit=10):
    # Verify the user has permission to view this customer
    if not user_has_access(user_id, customer_id):
        return {"error": "Access denied", "status": 403}
    
    # Query the database
    orders = db.query(
        "SELECT id, created_at, status, total FROM orders WHERE customer_id = %s ORDER BY created_at DESC LIMIT %s",
        (customer_id, limit)
    )
    
    # Return structured result
    return {
        "customer_id": customer_id,
        "order_count": len(orders),
        "orders": [
            {
                "id": order.id,
                "date": order.created_at.isoformat(),
                "status": order.status,
                "total": float(order.total)
            }
            for order in orders
        ]
    }

After execution, send the result back to the model along with the tool call ID, so it knows which query the result answers. The model then reads the actual data and formulates a response to the user. This loop—model requests, backend executes, model consumes results—repeats until the model has enough information to answer the original query or decides no further tools are needed.

Validate arguments and enforce permissions

The model is smart but not infallible. It will sometimes generate arguments that don't match your schema, request invalid resources, or attempt operations it shouldn't be allowed to perform. Your validation layer is non-negotiable in production.

Here's a validation pattern we use across multiple systems:

import jsonschema

def validate_and_execute_tool(tool_call, user_id, tool_schemas):
    tool_name = tool_call.function.name
    arguments = json.loads(tool_call.function.arguments)
    
    # Step 1: Verify tool exists and schema matches
    schema = next((t for t in tool_schemas if t["function"]["name"] == tool_name), None)
    if not schema:
        return {"error": f"Tool {tool_name} not found"}
    
    try:
        jsonschema.validate(arguments, schema["function"]["parameters"])
    except jsonschema.ValidationError as e:
        return {"error": f"Invalid arguments: {e.message}"}
    
    # Step 2: Check permissions
    permissions = get_user_permissions(user_id)
    if tool_name not in permissions.get("allowed_tools", []):
        return {"error": f"User not permitted to call {tool_name}", "status": 403}
    
    # Step 3: Rate limiting
    if exceeds_rate_limit(user_id, tool_name):
        return {"error": "Rate limit exceeded", "status": 429}
    
    # Step 4: Execute with error handling
    try:
        if tool_name == "get_customer_orders":
            return get_customer_orders(
                user_id=user_id,
                customer_id=arguments["customer_id"],
                limit=arguments.get("limit", 10)
            )
        # ... other tools
    except Exception as e:
        log_error(f"Tool execution failed: {tool_name}", e)
        return {"error": "Tool execution failed", "status": 500}

Validation runs before execution. Check that arguments conform to the schema, that the user has permission to call this tool, that they haven't exceeded rate limits, and that the data they're requesting actually belongs to them. Only then execute the underlying logic.

Common mistakes

Over-trusting the model's argument construction. Models hallucinate. A model might call a tool with a negative user ID or a customer ID that doesn't exist. Always validate before touching your database or external systems.

Exposing sensitive data in tool results. If a customer service agent calls a tool to fetch user data, you might accidentally include password hashes, API keys, or medical information. Return only what the model needs to answer the user's question. If the user asks about a customer they don't manage, your tool should refuse or return an error, not return partial data.

Forgetting to implement timeout and retry logic. Tool execution involves network calls and database queries. They fail. Add timeouts so a slow external API doesn't block the entire conversation, and implement exponential backoff for retries.

Defining tools too broadly. A tool named "execute_sql" or "run_command" is a security nightmare. Define granular tools with clear boundaries: get_customer_orders, not query_database.

Not logging tool invocations. When something goes wrong in production, you need an audit trail. Log which user requested what tool call with which arguments, what the result was, and what the model did with it. This is essential for debugging, compliance, and security investigations.

Key takeaways

• Function calling and tool use gives models the ability to request actions from your backend while keeping your code in control.
• Define tools explicitly with JSON schemas; the model reads these to understand what's available and what arguments each tool needs.
• The model generates a request specifying which tool to call and with what arguments; your code validates and executes—or refuses.
• Always validate arguments against the schema, check user permissions, enforce rate limits, and return only the data the model needs.
• Log every tool invocation for auditability, debugging, and compliance.
• Start with a small set of well-defined tools. Expand gradually as you understand the model's behavior in your domain.