Local Ai on Ivan: Thinking

Session Ownership

Fri, 31 Jul 2026 14:32:05 -0700

From The Session You Cannot Take With You:

A user should be able to close an account, keep a session, and hand it to another model. The new model may disagree, ask questions, or perform worse. It should not be staring at ciphertext where the old model saw the user’s history, evidence, plans, and delegated work.

Agreed, and relevant to folks running local models. I want to be able to swap between different models as needed, but it seems like session context is being hidden by model providers who don’t make that easy.

Coffee at Sightglass

Thu, 30 Jul 2026 12:42:20 -0700

Had a great chat with Josh Nguyen today. He’s working on RoverAI, doing super interesting work with local LLMs. I think more and more people and businesses will care about how much data we keep feeding model providers. I know I’m concerned about it, and I don’t think I’m alone. Beyond the privacy concern, having personal context available to models is a huge win too.

I don’t really want Claude to have all that data about me: how can I trust that they’re not using it to advertise to me or manipulate responses? Take them at their word?

I want to run my models locally where I control who gets to see my data, and it’s clear that the model is working for me rather than for a model provider.

Been running my own personal AI

Sun, 26 Jul 2026 22:01:55 -0700

Limited Edition Jonathan (via) thinks Apple is the king of AI.

I’ve been running my own local model and agent on an M4 Max with 128GB RAM. There’s a certain wow factor when running a local model the first time and realizing a file gives me access to all of human knowledge (OK, I know that’s not entirely accurate and overly simplistic, but the feeling is still valid IMO).

I’ve been using Hermes Agent against Qwen3.6-35B-A3B-4bit. It’s no Fable or Opus, but it does let me ask questions about parenting or health that I don’t feel comfortable asking Claude about. I’m not sure if other folks care as much as I do about the privacy concerns of throwing so much private details of our lives at these LLM providers. It does look like Apple is throwing their weight behind the worldview that having a Personal AI that lives on our devices is the way to go. It means they’ll make hardware powerful enough to run these models.

It’s still pretty early days of running local models, and I’ve yet to try out Apple’s latest with iOS/macOS 27. My own setup feels pretty rudimentary, but I can see it improving over time.