Published on Sun Jul 12 2026 00:00:00 GMT+0000 (Coordinated Universal Time) by Jacob Cavazos

TL;DR

MemPalace is an open-source AI memory system with 57K+ GitHub stars, 96.6% R@5 on LongMemEval, and a local-first architecture inspired by the ancient method of loci. We’ve been running it as the memory layer for our agent infrastructure at Orkid Labs — 425,000 drawers, 7,800+ knowledge graph triples, hammered continuously by AI agents doing real execution work.

The retrieval engine is excellent. The operational layer for production deployments didn’t exist.

So we built it — and contributed it upstream as PR #1999.

The Problem

MemPalace’s --transport http mode speaks plain JSON over HTTP. It uses Python’s BaseHTTPRequestHandler with Connection: close on every response — no keep-alive, no SSE, no streaming. MCP clients that expect the streamable-HTTP transport protocol (POST/GET/DELETE /mcp with Mcp-Session-Id and text/event-stream) cannot connect directly.

But the transport mismatch is just the surface. The deeper problem is that MemPalace has no operational layer for production deployments:

Restart=on-failure in systemd catches crashes. It does not catch hangs — situations where the process is alive but not responding to JSON-RPC. This happens when the embedding model gets stuck loading, the storage backend deadlocks, or the HTTP handler thread is blocked.

What We Built

1. MCP Streamable-HTTP Proxy

A persistent forwarding layer that bridges MemPalace’s plain JSON HTTP to the MCP streamable-HTTP protocol. Built with aiohttp + httpx.

MCP Client (Claude/Devin/etc.)
    │
    │  streamable-HTTP (POST/GET/DELETE /mcp)
    │  Mcp-Session-Id, text/event-stream
    ▼
┌──────────────────────┐
│   MCP Proxy (:8766)  │
│  ┌────────────────┐  │
│  │ Circuit Breaker│  │
│  │ Retry w/ backoff│ │
│  │ Session Mgmt   │  │
│  │ /health        │  │
│  │ /metrics       │  │
│  └───────┬────────┘  │
│          │           │
│  pooled httpx client │
└──────────┼───────────┘
           │
           │  plain JSON HTTP (POST /mcp)
           │  Connection: close
           ▼
┌──────────────────────┐
│  MemPalace (:8765)   │
│  BaseHTTPRequestHandler│
│  ChromaDB / Qdrant   │
└──────────────────────┘

Connection pooling: Persistent httpx.AsyncClient with 20 max connections, 10 keepalive, 60s expiry. No more per-request TCP overhead.

Circuit breaker: 3 consecutive failures → circuit opens (requests fail fast with 503, no 120-second hang). 30 seconds later → half-open. Next request → probe; success closes the circuit, failure reopens it.

Retry with exponential backoff: Transient failures (5xx, timeouts, connect errors) get retried up to 2 times with 0.5s * attempt backoff. The circuit breaker tracks cumulative failures across retries.

Bearer token forwarding: Respects MEMPALACE_MCP_HTTP_TOKEN — if the MemPalace server has auth enabled, the proxy forwards it transparently.

2. Health Endpoint

GET /health doesn’t just check if a port is open. It sends an actual tools/list JSON-RPC call to the upstream and verifies the response contains tools. This distinguishes “listening on a port” from “actually working” — the exact gap that caused us to think the server was healthy when it was actually hung.

{
    "status": "ok",
    "upstream": "http://127.0.0.1:8765/mcp",
    "upstream_ok": true,
    "upstream_latency_ms": 33.0,
    "circuit_state": "closed",
    "circuit_failures": 0,
    "active_sessions": 0,
    "uptime_seconds": 386.8
}

3. Metrics Endpoint

GET /metrics exposes Prometheus-style counters for scraping or manual inspection:

mempalace_proxy_requests_total 42
mempalace_proxy_requests_success 40
mempalace_proxy_requests_failed 2
mempalace_proxy_mcp_errors 1
mempalace_proxy_connect_errors 3
mempalace_proxy_timeout_errors 0
mempalace_proxy_active_sessions 2
mempalace_proxy_uptime_seconds 3600.0
mempalace_proxy_circuit_state{state="closed"} 0

No telemetry. No phone-home. No data leaves the proxy→server path. This is consistent with MemPalace’s “local-first, zero external API” design principle.

4. Auto-Restart Watchdog

The watchdog (mempalace-watchdog.sh) complements systemd’s Restart=on-failure by catching hangs — the process is alive but not responding to JSON-RPC.

Every 30 seconds:

1. Check if the process is alive (pgrep)
2. Send a tools/list JSON-RPC call and verify a valid response
3. If either check fails, restart the server

Rate limiting: max 5 restarts per hour, 60s cooldown between restarts. Force-kill (SIGKILL) if SIGTERM doesn’t work. Waits up to 20s for the new server to become responsive.

5. Proactive Monitor

The monitor (mempalace-monitor.sh) runs every 5 minutes via cron and checks:

• Proxy /health endpoint responds with upstream_ok=true
• MCP tools/list returns the expected number of tools
• Upstream server is reachable via SSH (if remote)

After 3 consecutive failures: desktop notification (macOS osascript / Linux notify-send) + auto-remediation via launchd/systemd reload.

Verified Against a Live Deployment

We tested every component against our production MemPalace server — 425,000 drawers across 4 wings, 7,800+ knowledge graph triples, running on a Dell server with Qdrant backend.

Circuit Breaker Test

Pointed the proxy at a dead port and sent 4 requests:

Health endpoint correctly reported circuit_state: "open", circuit_failures: 3.

Watchdog Test

Killed the MemPalace process:

Total recovery: 4 seconds from kill to fully operational. The proxy automatically recovered — circuit breaker never even opened.

End-to-End Test

Design Decisions

Why a proxy, not a fork?

MemPalace’s core is excellent at what it does — verbatim storage, semantic search, the wing/room/drawer architecture. We didn’t want to change any of that. The proxy adds the operational layer without touching the core. If MemPalace adds native streamable-HTTP support in the future, you just remove the proxy — no migration needed.

Why no telemetry?

MemPalace’s design principle is “privacy by architecture — the system physically cannot send your data because it never leaves your machine.” The proxy honors this. No analytics, no phone-home, no external service dependencies. The /metrics endpoint is local-only Prometheus format — you scrape it if you want it, but it doesn’t push anywhere.

Why env vars, not config files?

Configuration via environment variables means the proxy works with any process manager (systemd, launchd, Docker, Kubernetes) without file path assumptions. The PR checklist requires “no hardcoded paths” — env vars are the cleanest way to achieve that.

The Contribution

PR #1999 adds 9 files, 1,513 lines to deploy/proxy/:

The PR follows all contributing guidelines: conventional commits, ruff check + ruff format pass, no hardcoded paths (verified by test class), tests included, no telemetry.

What This Means for Orkid Labs

The same hardening architecture drives our high-throughput routing engine. When institutional players land on orkidlabs.xyz, the engineering depth is verifiable — not marketing claims, but merged code in a 57K-star open source project.

Open source means you stand on the shoulders of people who built the foundation. Milla Jovovich and Ben Sigman built something that actually works for AI memory. We made it survive production.

PR: github.com/MemPalace/mempalace/pull/1999

MemPalace: github.com/MemPalace/mempalace

Orkid Labs: orkidlabs.xyz

Written by Jacob Cavazos