I Built a Personal AI Operating System on a Knowledge Graph. Here's the Architecture.

Every AI assistant you use today has amnesia.

You spend twenty minutes giving Claude or ChatGPT context about your week, your projects, your priorities. It gives you a genuinely good answer. Then you close the tab, and tomorrow you start from zero. The model is brilliant; the system around it is goldfish-brained.

Six months ago I decided to stop complaining about this and build the missing layer myself. The result is KAIROS — a personal AI operating system that runs my day, remembers everything I've told it, and gets smarter about me the longer I use it.

This post is the architecture, the design decisions, and the things that broke along the way. It's also the first in a regular series on what I'm building.

The core bet: memory belongs in a graph, not a vector store

Most "AI memory" implementations today are a vector database with retrieval bolted on. Embed everything, cosine-similarity your way to relevant chunks, stuff them into the prompt. It works — for single-hop factual lookups.

But my life isn't a pile of chunks. It's a network. A meeting connects to people, who connect to projects, which connect to decisions made three months ago, which connect to a commitment I made last Tuesday. When I ask "what should I prepare for tomorrow?", the answer requires traversing those relationships — multi-hop reasoning that flat vector search is fundamentally bad at.

So the core bet behind KAIROS: the memory layer is a knowledge graph.

Mine runs on Neo4j and currently holds roughly 24,000 nodes and 95,000 relationships — meetings, people, projects, decisions, documents, and the connective tissue between them, accumulated over months of daily use.

The research increasingly backs this up. Zep's temporal knowledge graph work (Graphiti, also Neo4j-based) shows graph memory beating vector baselines on long-horizon memory benchmarks, and Microsoft's GraphRAG paper demonstrates large comprehensiveness gains for "global" questions that require connecting information across a corpus. The honest caveat: for a simple one-shot lookup ("what's the WiFi password?"), plain vector search is still faster and simpler. Graphs earn their complexity on the multi-hop, temporal, relational queries — which, it turns out, is most of what a personal OS needs.

The stack

Six layers, deliberately boring:

1. Memory: Neo4j. The knowledge graph is the source of truth. Everything KAIROS learns — from meeting notes, calendar events, documents, conversations — gets extracted into entities and relationships. Cypher queries do the heavy lifting at retrieval time.

2. Integration: MCP everywhere. KAIROS talks to the outside world through Model Context Protocol servers — Outlook for mail and calendar, Slack for team conversations, Obsidian for my notes vault. Every source of daily context flows in through the same standard interface, so adding a new source is "plug in another MCP server," not "write another bespoke integration."

And here's the part I'm most pleased with: KAIROS exposes its own MCP server. The knowledge graph isn't locked inside one app — any MCP-capable client (Claude, Claude Code, other agents) can query my graph as a tool. My memory layer became infrastructure that everything else I use can tap. This one decision quietly turned KAIROS from an application into a platform.

3. Agents: Strands Agents SDK. The agent loop is built on Strands, AWS's open-source agent framework. I like it because it's model-driven rather than workflow-driven — you give the agent tools and let the model plan, instead of hand-coding a brittle DAG. KAIROS agents get the graph, the MCP integrations, and each other as tools.

4. Runtime: Amazon Bedrock AgentCore. The Strands agents don't run on a laptop or a hand-rolled EC2 setup — they're deployed on Bedrock AgentCore. This is the layer that took KAIROS from "script I run" to "system that runs." AgentCore gives me a managed, serverless runtime with session isolation, identity, and built-in observability, so I can see exactly what an agent did and why without building any of that plumbing myself. Deploying a new agent is a push, not a project — which matters a lot when you're iterating weekly.

5. Inference: Amazon Bedrock. Nova Pro handles the high-volume, latency-sensitive work (entity extraction, classification, routing). Claude handles the reasoning-heavy tasks (synthesis, prioritisation, anything where nuance matters). Routing by task type rather than betting on one model turned out to be one of the best cost/quality decisions in the whole system.

6. Surface: dashboards, not chat. The frontend is a lightweight Flask + React app, and it's deliberately not a chat box. It's a set of dashboards: how my week is tracking against what I planned, what's overdue, which projects are heating up or going quiet, and — front and center — what the prioritisation agents think I should be doing right now. Chat exists for ad-hoc questions, but the default interface is the system showing me its conclusions, not me interrogating it.

The redesign that mattered: calendar-first

Version one of this system (it had a different name — KAIROS is actually the second incarnation) tried to be everything: a chat interface, a dashboard, a knowledge explorer, a task manager. It was impressive in demos and useless in practice, because it had no center of gravity. I never knew where to look.

The redesign started with one question: what is the atomic unit of a knowledge worker's day? Answer: the calendar. Meetings are where commitments are made, decisions happen, and context accumulates.

So KAIROS v2 is calendar-first. The home screen is a "Today" view: your meetings, each one enriched from the graph — who's attending, what happened last time you met them, what's open on the related project, what you promised to deliver. The agent does the traversal; you get a briefing.

That single design decision — anchor the system to the calendar, let the graph serve it — took KAIROS from "cool demo" to "thing I actually open every morning."

The killer feature: agents that never stop prioritising

Here's the thing that justifies all of the above.

Running continuously on AgentCore is a set of prioritisation agents. Their job is a single question, asked over and over: given everything KAIROS knows, what is the most valuable thing Koushal should be working on right now?

To answer it, they traverse the graph and pull from every MCP source: what's in my inbox and who's waiting on me (Outlook), what conversations are escalating (Slack), what commitments I made in meetings, which project deadlines are converging, what I captured in my notes (Obsidian) but never acted on. Then they rank — not by recency or loudness, but by an attempt at actual value: deadline proximity, who's blocked on me, how a task connects to the goals I've told the system matter.

The output lands on the dashboard as a short, ranked answer: this is the most valuable task, here's why, here's what it's connected to. When the picture changes — a new email shifts a dependency, a meeting produces a new commitment — the ranking updates.

This flipped my relationship with my own to-do list. A task manager stores what you tell it. KAIROS argues with me about what matters — and because its reasoning is traceable through the graph, I can see why it thinks the unglamorous task I've been avoiding is the one that unblocks three other people.

It's not always right. But "system that proposes priorities with reasons attached" beats "list I sort by vibes" every single day.

What broke (and what I learned)

Entity resolution is the real boss fight. "Ahmed," "Ahmed K.," and "ahmed.k@..." are the same person, and if your extraction pipeline doesn't know that, your graph fragments into useless islands. I ended up with a resolution pass that uses the LLM itself to merge candidate duplicates, with a confidence threshold before auto-merging. This is where most of my debugging hours went, and where most graph-memory projects quietly die.

Extraction quality beats retrieval cleverness. I spent early weeks tuning retrieval. Wrong lever. Garbage entities in the graph poison everything downstream. Tightening the extraction prompts — forcing typed entities, explicit relationship schemas, and "if unsure, don't create the node" rules — improved answer quality more than any retrieval trick.

The graph needs a schema, even a loose one. "Let the LLM decide the ontology" sounds elegant and produces chaos. A small fixed vocabulary of node types (Person, Meeting, Project, Decision, Document) and relationship types, with an escape hatch for edge cases, keeps the graph queryable.

Infrastructure is not the hard part anymore. The agent runtime, model access, and hosting were the easy 20% — AgentCore in particular collapsed what used to be weeks of container-and-observability plumbing into an afternoon. The hard 80% was data modeling, extraction quality, and deciding what the system should do every morning. If you're procrastinating on an agent project because the infra feels daunting — it isn't. Start.

Why "operating system" and not "assistant"

An assistant answers when spoken to. An operating system schedules, remembers, and manages resources whether or not you're looking at it.

KAIROS ingests continuously, maintains its own memory, prioritises without being asked, and prepares my day before I open the app. It even offers services to other software through its MCP server, the way an OS offers system calls. The chat interface exists, but it's the smallest part. The point of the architecture is that intelligence without memory is a party trick — and memory without structure is a junk drawer. The graph is what turns a very good model into something that compounds.

What's next

KAIROS is a living project, and this blog is going to follow it. Coming up in this series: how the prioritisation agents actually score "value" (and where they get it wrong), the entity-extraction pipeline in detail, building KAIROS's MCP server, and the weekly agent-building discipline that feeds new capabilities into the system.

Innovation rarely fails. Simplification almost always does. KAIROS is my ongoing attempt to get the simplification right.

If you're building something similar — agent memory, personal knowledge graphs, or anything in the "AI that remembers" space — I'd genuinely like to compare notes. Find me on LinkedIn.