Cloudflare Docs
Workers AI
Edit this page
Report an issue with this page
Log into the Cloudflare dashboard
Set theme to dark (⇧+D)

Function calling

Beta

Function calling enables people to take Large Language Models (LLMs) and use the model response to interact with external APIs. The developer usually defines a set of functions and the required input schema for each function, which we call tools. The model then intelligently understands when it needs to do a tool call, and it returns a JSON output which can be fed to another API.

In essence, you define a tool’s API schema, the LLM then generates a JSON object based on the schema, then you use the JSON object as an input to make an API call.

​​ What models support function calling?

There are open-source models which have been fine-tuned to do function calling. When browsing our model catalog, look for models with the function calling property beside it. For example, @hf/nousresearch/hermes-2-pro-mistral-7b is a fine-tuned variant of Mistral 7B that you can use for function calling.

​​ Basic function calling: input schema

With function calling, you define an array of tools with the name, description, and tool arguments. The example below shows how you would pass a tool called getWeather in an inference request to a model.

const response = await env.AI.run("@hf/nousresearch/hermes-2-pro-mistral-7b", {
messages: [{ role: "user", content: "what is the weather in london?" }],
tools: [
{
name: "getWeather",
description: "Return the weather for a latitude and longitude",
parameters: {
type: "object",
properties: {
latitude: {
type: "string",
description: "The latitude for the given location",
},
longitude: {
type: "string",
description: "The longitude for the given location",
},
},
required: ["latitude", "longitude"],
},
},
],
});
return new Response(JSON.stringify(response.tool_calls));

The LLM will then return a JSON object with the required arguments and the name of the tool that was called. You can then pass this JSON object to make an API call.

[{"arguments":{"latitude":"51.5074","longitude":"-0.1278"},"name":"getWeather"}]

For a working example on how to do function calling, take a look at our demo app.