In the last months MCP (Model Context Protocol) became one of the hot topics in the field of AI. While it is promoted for connecting tools/agents to LLM workflows there’s very little information about how things work on a technical basis.
On most places
- When does the LLM call an MCP connected service?
- How does it know which function to call (and with what parameters?)
So here’s my quest to answer those questions by writing a very basic MCP service to observe what’s going on behind the scenes.
Preparations
As a basic setup I assume a running instance of open-webui with mcpo proxy to contact mcp services.
A sample phonebook MCP service
For demonstration purposes we’ll use a very basic MCP service: a phonebook. And I’ll leave all extra complexity aside (no database, no nothing, just a static data array):
#!/usr/bin/env python3
from mcp.server.fastmcp import FastMCP
# This is the shared MCP server instance
mcp = FastMCP("phonebook")
phonebook = {
'Ashley Johnson': '555-11111111',
'Daniel Brown': '555-22222222',
'Emily Davis': '555-33333333',
'Jacob Thompson': '555-44444444',
'Olivia White': '555-55555555',
'Joshua Wilson': '555-66666666',
'Ethan Martinez': '555-77777777',
}
@mcp.tool()
async def get_phonenumber(name: str) -> str:
"""
Deliver the phone number for a certain person.
Args:
name: Name of the person (e.g., 'Axel Foley')
Returns:
A string with the person's phone number
"""
if name in phonebook.keys():
return(phonebook[name])
else:
return("No entry in phonebook")
# Entry point to run the server
if __name__ == "__main__":
mcp.run()You can run this service as simple as this:
linux # fastmcp run server.py --host 192.168.0.10 --port 8000 --transport streamable-http
INFO: Started server process [24444]
INFO: Waiting for application startup.
[11/08/25 16:25:18] INFO StreamableHTTP session manager started streamable_http_manager.py:110
INFO: Application startup complete.
INFO: Uvicorn running on http://192.168.0.10:8000 (Press CTRL+C to quit)
<...>Connecting open-webui to the MCP service
In open-webui’s admin interface go to Settings -> External tools and add a new tool server:
Either directly …
Type: MCP Streamable HTTP
URL: http://192.168.0.10:8000/mcpUpdate 08.11.2025 (open-webui 0.6.36): While the setup check succeeds, real queries fail to connect, so for now the workaround is to use mcpo instead:
…or using mcpo:
The entry in mpco’s config.json could then look like this (mind the changed IP address – this is the mcpo host’s IP!):
Type: OpenAPI
URL: http://192.168.0.5:8000/phonebookAnd configure mcpo’s /config/config.json file:
{
"mcpServers": {
"phonebook": {
"type": "streamable-http",
"url": "http://192.168.0.10:8000/mcp"
}
}
}Make sure to restart open-webui (or at least mcpo) after that: The current implementation connects to MCP services only during startup (no reconnect, no updates), so you may have to restart things in order to make MCP work!
Communication
But what’s happening behind the scenes when using MCP?
Who’s talking to whom (and about what?)
To find out what’s happening I was watching the network traffic between open-webui/mcpo, the MCP server and an ollama instance.
Step 1: Retrieve information about MCP tools (open-webui/mcpo)
open-webui (via mcpo) is querying supported tools from MCP server/service during startup/restart:
# mcpo -> MCP server (POST /mcp)
"{"method":"tools/list","jsonrpc":"2.0","id":1}"
# MCP server reply:
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"tools": [
{
"name": "get_phonenumber",
"description": "\n Deliver the phone number for a certain person.\n Args:\n name: Name of the person (e.g., 'Axel Foley')\n Returns:\n A string with the person's phone number\n ",
"inputSchema": {
"properties": {
"name": {
"title": "Name",
"type": "string"
}
},
"required": [
"name"
],
"title": "get_phonenumberArguments",
"type": "object"
},
"outputSchema": {
"properties": {
"result": {
"title": "Result",
"type": "string"
}
},
"required": [
"result"
],
"title": "get_phonenumberOutput",
"type": "object"
}
}
]
}
}So open-webui now knows about available tools (and how to call them). All the information about the MCP service are taken from the MCP service functions comment section.
Step 2: How does the LLM get to know about this?
When asking "What is the phone number of Joshua Wilson?" with our phonebook MCP enabled, the following data is transmitted to the LLM (shortened and formatted for better readability):
{"model": "gemma3:27b", "messages":
[
{"role": "system", "content": "Available Tools: [
{
"name": "tool_get_phonenumber_post",
"description": "Deliver the phone number for a certain person.
Args:
name: Name of the person (e.g., 'Axel Foley')
Returns:
A string with the person's phone number",
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"title": "Name",
"description": ""
}
},
"required": [ "name" ]
}
}]
Your task is to choose and return the correct tool(s) from the list of available tools based on the query. Follow these guidelines:
- Return only the JSON object, without any additional text or explanation.
- If no tools match the query, return an empty array:
{ "tool_calls": [] }
- If one or more tools match the query, construct a JSON response containing a "tool_calls" array with objects that include:
- "name": The tool's name.
- "parameters": A dictionary of required parameters and their corresponding values.
The format for the JSON response is strictly:
{"tool_calls": [
{"name": "toolName1", "parameters": {"key1": "value1"}},
{"name": "toolName2", "parameters": {"key2": "value2"}}
]}
},
{"role": "user",
"content": "Query: History:\nUSER: "" "What is the phone number of Joshua Wilson?"""
Query: What is the phone number of Joshua Wilson?"}
],
"stream": false}So the information about available tools that was collected before is now sent to the LLM by open-webui/mcpo.
Step 3: Let the LLM extract function an parameters
Now it’s the LLM’s part to decide if it needs to call a tool function, and if so which one and with what parameters:
{
"model": "gemma3:27b",
"created_at": "2025-11-08T15:53:10.989452276Z",
"message": {
"role": "assistant",
"content": "```json\n{\n \"tool_calls\": [\n {\n \"name\": \"tool_get_phonenumber_post\",\n \"parameters\": {\n \"name\": \"Joshua Wilson\"\n }\n }\n ]\n}\n```"
},
"done": true,
"done_reason": "stop",
"total_duration": 2035516062,
"load_duration": 316418425,
"prompt_eval_count": 349,
"prompt_eval_duration": 140556964,
"eval_count": 60,
"eval_duration": 1472558770
}Extracting the relevant part (s. below) tells us, that the LLM decided to order a call to tool_get_phonenumber_post(name="Joshua Wilson"):
json
{
"tool_calls": [
{
"name": "tool_get_phonenumber_post",
"parameters": {
"name": "Joshua Wilson"
}
}
]
}Step 4: Executing tool call by open-webui/mcpo
So let’s call this function:
{
"method":"tools/call",
"params": {
"name": "get_phonenumber",
"arguments":{
"name":"Joshua Wilson"
}
}
"jsonrpc":"2.0"
"id":2
}Response from our MCP server:
{
"jsonrpc":"2.0",
"id":2,
"result":{
"content":[
{
"type":"text",
"text":"555-66666666"
}
],
"structuredContent":{
"result":"555-66666666"
},
"isError":false
}
}Step 5: Send the tool information to the LLM
Now this data is sent back to the LLM (again the relevant part extracted and formatted):
<context>
<source id="1" name="server:3/tool_get_phonenumber_post">555-66666666</source>
</context>
<user_query>
What is the phone number of Joshua Wilson?
</user_query>
What is the phone number of Joshua Wilson?
Tool `server:3/tool_get_phonenumber_post` Output: 555-66666666Step 6: Get the LLMs final answer
The phone number associated with the request is 555-66666666 server:3/tool_g...umber_post.Side notes
You might encounter unexpected results while testing. Some LLMs seem to be quite aware of privacy issues and refuse to answer (even if your MCP service worked perfectly well):
Question:
What is the phone number of Joshua Wilson?
Answer: (Qwen3:32B)
I'm sorry, but I cannot assist with retrieving personal phone numbers or private contact information. Providing such details would violate privacy laws, ethical guidelines, and the principles of protecting individuals' personal data.