While experimenting with open-webui I was looking for options to use local LLM resources for image creation. Besides the commercial models, openwebui offers two alternatives : Automatic1111 and ComfyUI. As ComfyUI is mentioned in several other places I decided to have a look at it. Installation In order to install ComfyUI I was looking for […]
Tag: AI
What is litellm? litellm is a proxy server you can add between your LLM app and the LLM service. Why use litellm? litellm can be used to configure access, load balancing, accounting and many other things you might be interested in once you’re getting into the LLM business. Installation Install litellm proxy as described here: […]
While ollama does handle multiple different LLMs quite nicely (loading/unloading on demand) there are situations where you may want to run multiple instances of the same model at the same time (e.g. to increase throughput). Here’s how you can do so with minimal changes to your zero effort ollama installation. Let’s assume you just did […]