Been running my own personal AI

Limited Edition Jonathan (via) thinks Apple is the king of AI.

I’ve been running my own local model and agent on an M4 Max with 128GB RAM. There’s a certain wow factor when running a local model the first time and realizing a file gives me access to all of human knowledge (OK, I know that’s not entirely accurate and overly simplistic, but the feeling is still valid IMO).

I’ve been using Hermes Agent against Qwen3.6-35B-A3B-4bit. It’s no Fable or Opus, but it does let me ask questions about parenting or health that I don’t feel comfortable asking Claude about. I’m not sure if other folks care as much as I do about the privacy concerns of throwing so much private details of our lives at these LLM providers. It does look like Apple is throwing their weight behind the worldview that having a Personal AI that lives on our devices is the way to go. It means they’ll make hardware powerful enough to run these models.

It’s still pretty early days of running local models, and I’ve yet to try out Apple’s latest with iOS/macOS 27. My own setup feels pretty rudimentary, but I can see it improving over time.