In the last months MCP (Model Context Protocol) became one of the hot topics in the field of AI. While it is promoted for connecting tools/agents to LLM workflows there’s very little information about how things work on a technical basis. On most places So here’s my quest to answer those questions by writing a […]
Category: Large Language Model (LLM)
Some time ago I tried to use my AMD iGPUs (not supported by AMDs ROCm) for LLMs. However I didn’t succeed. Now I read some benchmark for the newest AMD Strix Halo systems performing quite well in some LLM tasks using vulkan instead of ROCm. Building vulkan enabled llama.cpp (using docker) Building llama.cpp using docker […]
Motivation I’m currently playing around with some LLMs for different projects. One project aims to add textual descriptions to a large image gallery. Those descriptions can be used both for better image organization or as alternative texts when publishing them on websites. My basic setup contains ollama for running the LLMs and some python code […]
A LLM (Large Language Model) is the basis for most current AI tools. Fortunately there are lots of open source models that can be run on your own hardware (if it is powerful enough). However I asked myself more than once: what is powerful enough? Hardware requirements While experimenting with LLMs one of the first […]
For some tests with a large language model (LLM) I needed a test system with docker and a Nvidia card (for faster AI processing). Here’s what it takes to convert a basic Ubuntu 24.04.1 installation into a docker based LLM test machine: First let’s have a look at our hardware: Next: Install basic Nvidia drivers: […]