Some time ago I tried to use my AMD iGPUs (not supported by AMDs ROCm) for LLMs. However I didn’t succeed. Now I read some benchmark for the newest AMD Strix Halo systems performing quite well in some LLM tasks using vulkan instead of ROCm. Building vulkan enabled llama.cpp (using docker) Building llama.cpp using docker […]
Category: Artificial Intelligence
Motivation I’m currently playing around with some LLMs for different projects. One project aims to add textual descriptions to a large image gallery. Those descriptions can be used both for better image organization or as alternative texts when publishing them on websites. My basic setup contains ollama for running the LLMs and some python code […]
A LLM (Large Language Model) is the basis for most current AI tools. Fortunately there are lots of open source models that can be run on your own hardware (if it is powerful enough). However I asked myself more than once: what is powerful enough? Hardware requirements While experimenting with LLMs one of the first […]
For some tests with a large language model (LLM) I needed a test system with docker and a Nvidia card (for faster AI processing). Here’s what it takes to convert a basic Ubuntu 24.04.1 installation into a docker based LLM test machine: First let’s have a look at our hardware: Next: Install basic Nvidia drivers: […]