Learning to use local AI is exciting, overwhelming, and frustrating
A writer documents his ongoing journey of running local AI models on a high-end Mac Studio, finding it exciting, overwhelming, and sometimes frustrating.Summary
The author describes his hesitant first steps into regularly using AI, motivated by privacy concerns about sending personal data to cloud services. Rather than trusting OpenAI, Google, Microsoft, or Anthropic, he experiments with running powerful large language models locally on an M5 Ultra Mac Studio with 256GB of unified memory. He installs the open-source Hermes Agent desktop app and selects a 125-billion parameter Qwen model (~105GB). While setup is quick, he immediately faces the "blank text box" problem of not knowing what to do with the tool. He starts with simple cron-based tasks: a daily morning briefing that scans email, calendar, and weather (which initially failed because macOS was asleep), and reorganizing his 400+ game Steam library by genre using a Steam web API key. He also uses it for sensitive data analysis like financial records and an embargoed laptop spec comparison spreadsheet. His bigger ongoing project is automating laptop benchmark testing via vibe-coded Python scripts. Overall, he views local AI as just a tool, not a digital friend, and plans to remain cautious and vigilant as he continues learning its limits.
(Source：The Verge)