This topic covers the practical landscape for building consumer-facing AI devices and prototypes that run large-language-model (LLM) inference on-device or at the edge, and the market options for hardware and software stacks in 2025–2026. Interest in on-device inference is driven by demands for lower latency, offline capability, cost control, and stronger data privacy; developers now combine compact or quantized models, hardware NPUs, and local retrieval-augmented generation (RAG) to meet those needs. Key tool categories include MCP (Model Context Protocol) servers that expose local model and search capabilities to clients; local RAG/document search engines that index user files for private semantic search; multi-model orchestration layers that synthesize outputs from diverse models; and domain adapters that tie AI to specific consumer workflows (for example, music production). Representative implementations: FoundationModels runs Apple’s Foundation Models via an MCP server on macOS for local text generation; Minima provides a containerized on-prem RAG stack that can integrate with ChatGPT and MCP clients; Local RAG is a privacy-first MCP document indexer for offline semantic search; Multi-Model Advisor queries multiple Ollama models and synthesizes perspectives; Producer Pal embeds natural-language control into Ableton Live as a Max for Live device. Evaluators should weigh trade-offs: model size and quantization versus accuracy, hardware acceleration (NPUs, GPUs), memory and storage constraints, OS and container support, and integration with RAG pipelines and MCP-compatible clients. Together these tools illustrate a maturing, modular ecosystem where developers can mix local LLM inference, private retrieval, and orchestration to prototype consumer devices that emphasize responsiveness and data control rather than cloud dependency.