OpenNotch — Open-Source AI Assistant in the MacBook Notch

An open-source (MIT) macOS app that turns the MacBook notch into an AI assistant that actually does things: 40+ agent tools, 14 notch modules, hands-free voice, MCP connectors and an approval gate for anything risky. Native Swift with an in-process agent loop — no server, no account, no telemetry; chats, memory and keys stay on the Mac. Comes with Puff, a pokeable notch character, and a live three.js desktop companion.

• Role: Creator & Maintainer
• Timeline: 2026 - Present
• Team: Solo Developer · Open Source
• Technologies: Swift, SwiftUI, LLM Agents, MCP, Three.js, Open Source (MIT)
• Link: https://laxman824.github.io/opennotch/

Problem

The MacBook notch is dead screen space, and desktop AI assistants are either chat windows that can't act, or agents that act without asking — usually behind an account and a cloud backend that sees everything.

Solution

OpenNotch puts a native Swift agent in the notch. One hover or ⌥Space opens it; it plans, calls tools (files, shell, web, Mail drafts, Notes, calendar, reminders, contacts, weather, schedules) and any MCP server, streams its thinking and plan live, and pauses for approval before anything risky. It works with OpenAI, Anthropic, Gemini, Groq, OpenRouter, Ollama, LM Studio or Apple's on-device model, and keeps everything local.

Impact

• Open-source (MIT) — 40+ agent tools, 14 notch modules, any AI provider
• Zero backend: in-process agent, no account, no telemetry; keys in the macOS Keychain
• Every risky action (shell, file writes, calendar, mail drafts, MCP) gated behind an explicit OK
• Live-tested multi-tool agent turns with approval and decline paths; 80+ automated agent checks

Key features

• Agent loop with streaming thinking, plan cards, expandable tool calls and chat history (⌘Y)
• Approvals: glowing notch + chime when closed, spoken yes/no in hands-free mode, 5-minute timeout = deny
• MCP client (stdio) using the common mcpServers config, with per-tool autoApprove
• Hands-free voice with on-device speech recognition, barge-in and voice approvals
• Notch modules: clipboard history, drag-and-drop shelf, calendar & reminders, music, timers, captures and more
• Scheduled prompts ("every weekday at 9, brief me") and an opennotch CLI for piping text in
• Puff, a jelly-blob notch character with poke/pet reactions, and a three.js desktop companion (Ledge, Bee or Cat)

Tech stack

• App: Swift, SwiftUI, AppKit, AVFoundation, EventKit, Contacts
• Ai: OpenAI-compatible APIs, Anthropic Messages API, Apple FoundationModels, Model Context Protocol
• Companion: three.js, WKWebView
• Tooling: Swift Package Manager, GitHub Actions, DMG packaging

System architecture

Trust boundary — Your Mac — OpenNotch.app (one process, no server). No account, no telemetry. Chats, memory, schedules and traces stay in ~/Library/Application Support/OpenNotch and keys in the Keychain. The only outbound traffic goes to the AI you pick, plus Open-Meteo for weather and fetches you ask for.

01 · Input — Hover, ⌥Space, voice or CLI

• Notch UI — SwiftUI · NotchController. Collapsed ears, hover peek and the expanded panel: transcript, thinking rows, plan card, approval card and ⌘K palette. The panel stays mounted and is revealed by a mask, so open/close never re-lays out the chat.
• Hands-free voice — On-device speech · TTS · barge-in. Wake word follows the assistant's name. Speech recognition runs on device; audio objects live on their own serial queues, never the main thread. Approvals can be answered out loud — 'no' always wins.
• opennotch CLI — Distributed notifications. Ask, pipe text in (git diff | opennotch 'review this'), start timers or open modules from any terminal.
• MediaIntent fast path — Music commands skip the model. Narrow, tested patterns (play / pause / next…) go straight to Spotify or Music via AppleScript, off the main thread — no LLM call, no latency.

↓ Request → AgentCore

02 · Agent loop — AgentCore — in-process, streaming

• AgentCore — Turn loop · context window. Appends the user message, streams a provider turn (text, thinking, tool calls), runs the tools and loops. The context window starts at a user message, stays ≤ 300k chars and keeps images for the latest two image turns.
• Provider router — ChatProvider protocol. One event vocabulary across OpenAI-compatible APIs (OpenAI, OpenRouter, Groq, Gemini, Ollama, LM Studio), the Anthropic Messages API (raw-block replay, prompt caching, summarized thinking) and Apple's on-device FoundationModels.
• ToolRouter — Core set + keyword groups + more_tools. Sends core tools always, adds groups (mail, calendar, notes, browser, schedule…) by keyword or prior use, and exposes more_tools — saving thousands of prompt tokens per request.

↓ Stream → tool calls → results → next pass

03 · Guardrails — Nothing risky without an explicit OK

• Approval gate — Risky tools · 5-min timeout = deny. Shell, file writes, calendar/reminder creation, mail drafts, notes, schedules and MCP tools wait for approval. The waiter registers before the request is announced, so a synchronous answer can't be lost. Mail is drafted, never sent.
• Loop limits — 40 tools/turn · repeat bounce · JSON check. Identical calls are bounced after 3 repeats, tool-call JSON is validated (INVALID_JSON goes back to the model), and a pass that fails before streaming is retried once unless the error is permanent.
• ResultBudget — 40k chars · spill to file. Large tool results are truncated for the model with the full text spilled to a local file; tools report failure as failure (ok: false), so the model never claims a failed action worked.

↓ Tool call → validated → approved → run

04 · Tools — 26 core · 16 daily · any MCP server

• ToolKit — 26 core tools. Read/write/edit files (PDF, Word, images), search, find, run commands, fetch URLs, web search, open, clipboard, screenshot, system info, timer, keep-awake, memory and a plan tool.
• DailyTools — Mail · Notes · Calendar · Contacts · Weather. Active browser tab, Mail read/draft, Notes search/create, Contacts, EventKit calendar & reminders, Open-Meteo weather and scheduled prompts. Automation prompts yield the notch first so dialogs stay visible.
• MCP client — stdio JSON-RPC · mcpServers. Each configured server's tools join as mcp__server__tool; they require approval unless listed in autoApprove.

↓ Run off the main thread → result to the loop

05 · Local state — ~/Library/Application Support/OpenNotch

• SessionStore — sessions/chat_<id>.json. One file per chat with full-text history search (⌘Y). Empty chats are never saved.
• Memory & plans — memory.json · plan · schedule.json. Remembered preferences, the current plan and scheduled prompts — plain local JSON.
• Keychain — API keys. Provider keys live in the macOS Keychain, never in files or settings.
• Traces — traces/YYYY-MM-DD.jsonl. Per-turn traces power the local AI-usage heatmap; nothing is sent anywhere.

↓ Everything persisted on the Mac

06 · Presence — Personality without a server

• Puff — CharacterBrain + Canvas. Jelly-blob notch character driven by springs: poke → annoyed, 3 pokes → dizzy, stroking → love, a hop when a task finishes and peek-a-boo now and then.
• Desktop companion — WKWebView + three.js. Ledge, Bee or Cat walk on your windows, dance to music and point at the notch when an approval is waiting.
• Live activities — Ears · HUDs. Music wave, timer ring, approval glow, mic/camera indicator, battery and system-health pop-ups, in a fixed priority order.

↓ Agent events → UI, ears and characters

Outside your Mac — only what you choose

• Your AI provider — OpenAI · Anthropic · Gemini · Groq · OpenRouter. Only the conversation window and tool schemas are sent. Choose Ollama, LM Studio or Apple on-device instead and nothing leaves the Mac at all.
• MCP servers — GitHub, Slack, Notion, a browser…. Whatever servers you add to mcp.json; they may talk to their own services. Their tools still ask before acting unless you auto-approve them.
• Open-Meteo — Weather only. The weather tool's forecast lookup — no key, no account.

Control loops

• Tool loop — Provider streams tool calls → each runs off the main thread → results (and any images) are appended → next pass, until the model answers without tools or a limit is hit.
• Approval round-trip — A risky call emits an approval event → the open notch shows an Approve card; the closed notch glows yellow and chimes at 0/25/50 s; hands-free asks out loud → yes/no → the call runs or is declined.
• Scheduler — A 30-second timer runs due scheduled prompts while the assistant is idle; only future slots fire and anything more than 30 minutes stale is skipped.

Architecture highlights

• Provider abstraction streams a common event vocabulary (text, thinking, tool calls, usage) across OpenAI-compatible, Anthropic and on-device models
• Tool results are budgeted (40k chars, overflow spilled to a file) and repeated identical calls are bounced
• Audio and TCC-protected work never runs on the main thread; every permission prompt yields the notch first

What K Laxman learned

• A tool router (core set + keyword groups + more_tools) cuts thousands of prompt tokens per request and helps small models choose tools
• Approvals must register their waiter before they're announced — an answer can arrive synchronously
• UI performance on macOS is mostly about what you don't animate: no per-frame blurs or animated gradients over large areas

← All 28 projects by K Laxman

Explore more

• Home — overview, skills and a built-in AI assistant
• Experience — roles at Think360 AI (CAMS), CAMS Mutual Funds and IIT Delhi
• Projects — GenAI, LLM, RAG and full-stack builds
• Education — IIT Delhi, M.Tech & B.Tech Computer Science
• Learn — a working AI engineer's roadmap: LLMs, RAG, agents, evals, production
• GitHub Activity — open-source contributions
• Contact / Hire me

GitHub · LinkedIn · Email