The gap between open-weight and proprietary artificial intelligence models has narrowed significantly, with recent analysis indicating that open versions now trail the best closed models by an average of just four months. This convergence, highlighted in a May 29 report by Epoch AI, suggests that capable AI assistants are increasingly accessible on standard consumer hardware. As a result, users can now download and execute these models locally using free software, avoiding the privacy risks and monthly fees associated with cloud-based services.

Privacy and cost advantages of local execution

Running AI models locally offers distinct benefits regarding data privacy and financial cost. Cloud-based chatbots transmit every user prompt to external servers, a practice that has drawn legal scrutiny; Decrypt previously reported that a court ordered OpenAI to preserve ChatGPT logs, including deleted entries. In contrast, local models keep conversations entirely on the user's device, ensuring that no third party can access the data without permission. LM Studio’s documentation confirms that its application can operate completely offline, provided the necessary model files are initially downloaded. Additionally, while paid chatbot subscriptions typically cost around $20 per month, the software required to run local models is free, shifting the primary cost to the hardware the user likely already owns.

Hardware requirements and quantization

The ability to run these models depends on the available video memory (VRAM) on graphics cards or unified memory on Apple Silicon Macs. As a general guideline, users need at least 8GB of VRAM to run compressed, or quantized, versions of smaller models. Quantization reduces memory usage by storing weights with fewer bits, such as 8-bit integers instead of 32-bit floating-point numbers, though this comes at a slight cost to accuracy. For more demanding tasks, serious users are advised to utilize GPUs with 24GB or more of VRAM. Specific models have defined memory footprints: OpenAI’s gpt-oss-20b requires 16GB of memory, while the larger gpt-oss-120b runs within 80GB. Google’s Gemma 4 variants in Ollama’s library range from 4.6GB to 20GB, depending on the specific size.

Setting up Ollama and LM Studio

Two primary free tools facilitate local AI deployment: Ollama and LM Studio. LM Studio is designed for general users, offering a polished graphical interface that allows users to search for, download, and chat with models directly from the Hugging Face hub. Ollama, conversely, targets developers and experienced users who prefer command-line interfaces. Installation is straightforward, with simple scripts available for macOS, Linux, and Windows. Once installed, users can pull specific models, such as Gemma 4, and initiate a chat session. Ollama also exposes a local API, enabling other applications on the machine to interact with the model. It is important to note that models labeled with ':cloud' in Ollama run on remote servers, which negates the privacy benefits of local execution.