Basic litellm setup

Basic config file

linux # cat litellm_config.yaml
model_list:
  - model_name: gemma3:27b
    litellm_params:
      model: ollama/gemma3:27b
      api_base: "http://127.0.0.1:11435"

general_settings: 
  master_key: sk-supermasterkey
  database_url: postgresql://litellm:mypassword@127.0.0.1:5432/litellm

Install and create postgres database

Install postgres, create litellm user and database:

linux # apt install postgresql
linux # sudo -u postgres createuser --pwprompt litellm
Enter password for new role: 
Enter it again:
linux # sudo -u postgres createdb --encoding=UTF8 --locale=C --template=template0 --owner=litellm litellm

Problems while setting up postgres

See bug report: In case you get some error on litellm startup mentioning “prisma generate” (path to schema.prisma may vary on your installation):

linux # pip install prisma
linux # python -m prisma generate --schema /opt/litellm/venv/lib/python3.12/site-packages/litellm/proxy/schema.prisma

Creating API keys

Once a database is configured and working, creating API keys is as easy as:

linux # MASTER_KEY=sk-supermasterkey
linux # curl -L -X POST 'http://127.0.0.1:80/key/generate' \
-H "Authorization: Bearer ${MASTER_KEY}" \
-H 'Content-Type: application/json' -d '{
  "user_id": "marcel"
}' | jq
{
  "key_alias": null,
  "duration": null,
  "models": [],
  "spend": 0.0,
  "max_budget": null,
  "user_id": "marcel",
  "team_id": null,
  "max_parallel_requests": null,
  "metadata": {},
  "tpm_limit": null,
  "rpm_limit": null,
  "budget_duration": null,
  "allowed_cache_controls": [],
  "config": {},
  "permissions": {},
  "model_max_budget": {},
  "model_rpm_limit": null,
  "model_tpm_limit": null,
  "guardrails": null,
  "prompts": null,
  "blocked": null,
  "aliases": {},
  "object_permission": null,
  "key": "sk-j-DhUashI7dasS3Cr3t",
  "budget_id": null,
  "tags": null,
  "enforced_params": null,
  "allowed_routes": [],
  "allowed_passthrough_routes": null,
  "allowed_vector_store_indexes": null,
  "rpm_limit_type": null,
  "tpm_limit_type": null,
  "key_name": "sk-...Cr3t",
  "expires": null,
  "token_id": "1234f18ef33d469bfec0f3c01c11379b9c0be6f85fa50b86fcd86e50dfaecdef",
  "organization_id": null,
  "litellm_budget_table": null,
  "token": "1234f18ef33d469bfec0f3c01c11379b9c0be6f85fa50b86fcd86e50dfaecdef",
  "created_by": "default_user_id",
  "updated_by": "default_user_id",
  "created_at": "2025-12-21T16:48:06.839000Z",
  "updated_at": "2025-12-21T16:48:06.839000Z"
}

In case you’re interested in load-balancing using litellm (and ollama) take a look here.