A LLM (Large Language Model) is the basis for most current AI tools. Fortunately there are lots of open source models that can be run on your own hardware (if it is powerful enough). However I asked myself more than once: what is powerful enough? Hardware requirements While experimenting with LLMs one of the first […]
Category: Large Language Model (LLM)
For some tests with a large language model (LLM) I needed a test system with docker and a Nvidia card (for faster AI processing). Here’s what it takes to convert a basic Ubuntu 24.04.1 installation into a docker based LLM test machine: First let’s have a look at our hardware: Next: Install basic Nvidia drivers: […]