OfflineLLM: Private AI Chat

Acerca de esta app

OfflineLLM is a local large language model execution engine designed for Apple Silicon devices, including iPhone, iPad, Mac, and Vision Pro. By leveraging the full power of Metal 3, it aims to run faster than apps based on llama.cpp or MLC. The app emphasizes privacy by processing all data locally without sending information to the cloud, allowing usage in Airplane mode.

Key features include support for various GGUF models such as DeepSeek, Llama, Gemma, Phi, Mistral, and others. It offers both Beginner and Advanced modes, allowing novices to use pre-configured settings while experts can tune every parameter of the execution engine. The app supports multi-modal vision models, enabling users to send images to offline AI chatbots. Additionally, it integrates with system Shortcuts, Automations, and Widgets, and provides an OpenAI-compatible API for integration with other systems. Recent updates introduce MCP support to connect with external services like GitHub, Zapier, and Calendar.

Use cases range from drafting emails and proofreading text to summarizing articles, learning languages, and assisting with coding in multiple languages. Performance varies by device; advanced models require more VRAM and may struggle on older iPhones.