Redis vs Memcached

We evaluated Redis and Memcached as caching backends for the zetl index layer. The decision was driven by benchmark data and operational simplicity requirements.

(given redis-evaluated)
(given memcached-evaluated)
(given single-node-deployment)

^cache-decision-context

Benchmark Results

We ran read/write throughput tests at various concurrency levels against both backends using a representative workload of 10 000 graph index lookups.

Backend	Reads/sec (p50)	Reads/sec (p99)	Writes/sec	Memory overhead
Redis 7.2	185 000	210 000	92 000	~12 MB
Memcached	170 000	198 000	88 000	~8 MB

^benchmark-results

The p99 numbers show Redis edges out Memcached on read latency under load, primarily because Redis pipelines responses more aggressively over a single connection.

^benchmark-interpretation

Analysis

Given the benchmark data, Redis meets our throughput target of 150 000 reads/sec with comfortable headroom.

; Redis read throughput exceeds our threshold — grounded to the benchmark table
(meta redis-fast-enough (source "^benchmark-results"))

^analysis-block

Memcached also clears the threshold, but Redis offers additional features (pub/sub, persistence options, richer data structures) that future graph-query caching may exploit.

; Both backends acceptable for now; Redis preferred for future flexibility
(normally r-prefer-redis
  (and redis-fast-enough single-node-deployment)
  prefer-redis-over-memcached)

^redis-preference-rule

Operational considerations

Redis ships with redis-cli which simplifies debugging cache state during development.
Both are available in standard distro package managers.
Neither requires a separate service when running locally — zetl can be configured to skip caching entirely in read-only mode.

^operational-notes

Decision

We adopt Redis as the recommended caching backend. Memcached remains a supported alternative for environments where Redis is unavailable.

Principles

Read-only — zetl only reads Markdown and .spl files. It never writes to, renames, or deletes vault content.

Disposable cache — the .zetl/ directory contains only derived data (the Link Graph index and Reasoning Engine theory cache). Deleting it loses nothing; zetl index regenerates it.

No network — zetl makes no network calls. Everything runs locally.

No lock-in — your vault is plain Markdown with optional Spindle Lisp blocks. Removing zetl leaves your files untouched.

Why this matters

Users trust zetl with their knowledge base — years of accumulated notes. A tool that might corrupt, reformat, or accidentally delete files would be a non-starter. Read-only access removes that risk entirely.

The Cache is the only thing zetl writes, and it lives in a clearly-marked directory that can be gitignored.

Requirements

zetl must cold-start in under 200 ms on a vault of 10 000 Markdown files, complete incremental re-index in under 50 ms when fewer than 100 files have changed, and serve graph queries in under 10 ms at p99 on a fully-indexed vault of 50 000 nodes.

^perf-numbers

These figures were chosen to keep the tool imperceptibly fast during normal editing workflows, where the user expects results before they finish reading the command prompt.

^perf-rationale

How it works

The cache lives in .zetl/ at the vault root:

index.json — serialised link graph

theory.json — serialised reasoning theory and conclusions

Each entry records the file’s last-modified timestamp. When zetl starts, it compares mtimes and only re-scans changed files. Use --no-cache to force a full rebuild.

(given mtime-based-cache) (given incremental-rebuild)

Design tension

There is an unresolved design tension in how aggressively to cache reasoning results:

; Cache the theory for fast startup (normally r-cache-theory mtime-based-cache cache-reasoning-results) ; But recomputing ensures results are always fresh (normally r-recompute-theory incremental-rebuild (not cache-reasoning-results))

Both arguments have merit. Try zetl reason conflicts on this vault to see how zetl surfaces this kind of tension. A future Plugin System could let users configure the trade-off.

Redis vs Memcached

Benchmark Results

Analysis

Operational considerations

Decision

Backlinks