Using the Ollama REST API from Python in your company
Ollama exposes a local language model through a plain REST API, so a short Python script is enough to ask it questions. This tutorial covers the calls an administrator needs, from checking models to chat requests with history.

What you will have at the end and what you need first
At the end you have a small Python script that lists the models on your Ollama server, sends a chat request, reports the generation speed and keeps a conversation history. All calls go to a server in your own network, so prompts never reach an external provider. The calls follow the Ollama API reference, which also notes that the API docs are moving to docs.ollama.com/api.
You need a running Ollama server, which listens on port 11434 in the documentation's examples. You also need at least one pulled model, here llama3.2, and Python with the requests package. The documentation shows the endpoints with curl; the Python code below uses the same paths and JSON fields. Operating models locally is the subject of our page on local AI.
Step 1: check reachability and local models
The endpoint GET /api/tags lists the models that are available locally. The script below calls it with a timeout, stops on an HTTP error and prints each model name with its parameter size. If this call fails, the server cannot be reached under the address you set in BASE, and the later steps will fail as well.
import requests
BASE = "http://localhost:11434"
# List all models that are available locally
antwort = requests.get(f"{BASE}/api/tags", timeout=10)
antwort.raise_for_status()
for m in antwort.json()["models"]:
print(m["name"], m["details"]["parameter_size"])The documentation's sample response shows entries such as llama3.2:latest with a parameter size of 3.2B. Model names follow the model:tag format, and the tag defaults to latest if you leave it out.
Step 2: send a chat request
The endpoint POST /api/chat takes a model and a list of messages with the roles system, user, assistant or tool. Chat streams by default, so the payload sets stream to false to receive one JSON object. The options temperature 0 and seed 101 follow the documentation's example for reproducible outputs.
payload = {
"model": "llama3.2",
"messages": [
{"role": "system", "content": "Antworte knapp auf Deutsch."},
{"role": "user", "content": "Was ist ein Auftragsverarbeiter?"},
],
"stream": False,
"options": {"temperature": 0, "seed": 101},
}
# Send the chat request and print the answer
r = requests.post(f"{BASE}/api/chat", json=payload, timeout=120)
r.raise_for_status()
daten = r.json()
print(daten["message"]["content"])The answer sits in the message object of the response. The part of the documentation reproduced here does not cover the embeddings endpoint, so this tutorial leaves it out.
Step 3: evaluate speed and keep a history
The final response object carries statistics. All durations are returned in nanoseconds, and the documentation calculates tokens per second as eval_count divided by eval_duration, multiplied by 10 to the power of 9. The snippet below does that and then appends the model's reply to the original messages, which gives you the history for a follow-up question.
# eval_duration is given in nanoseconds
tps = daten["eval_count"] / daten["eval_duration"] * 10**9
print(f"{tps:.1f} Token pro Sekunde")
# Keep the history for a follow-up question
verlauf = payload["messages"] + [daten["message"]]For the next turn you add a new user message to verlauf and send the list as messages again. The documentation describes this approach in its chat example with history.
How do you check that everything runs locally and correctly?
Step 1 should print your models, and step 2 should return a German answer in the message field. With temperature 0 and a fixed seed, identical requests should give reproducible output, so a changed answer points to a changed model or option. A tokens per second value from step 3 confirms that the generation statistics are present.
The documentation mentions a few behaviours that look like errors. A model stays loaded for 5 minutes after a request by default, controlled by keep_alive, so the first call after a pause is slower. Setting keep_alive to 0 with an empty messages list unloads the model. Without stream set to false you receive a series of JSON objects, and reading them as one object fails. To confirm that traffic stays internal, check on the network side that the client talks only to your server address.
Where this fits in a company
A script like this suits internal tools, tests and small automations where the data should stay on your own hardware. Someone has to own the server: updating Ollama, deciding which models are pulled and restricting who can reach port 11434. The documentation shows no authentication in its examples, so access control belongs on the network side. Data protection questions, such as whether personal data may enter prompts, remain with your data protection officer and, for professional secrets under § 203 StGB, with the responsible partners. See also our overview of GDPR and AI at work.
souveraen.ai is an AI platform for company knowledge that runs either hosted in Germany or on-premise, and it offers an OpenAI-compatible developer API. Its on-premise option is called Private Spark and runs on an NVIDIA DGX Spark in your own network.
Sources
- Ollama API reference, Ollama


