fedora-csb-system-manager

Email Indexing Alternatives to mu

Comparison of email indexing tools and their concurrency characteristics.

Quick Comparison Table

Tool Backend Persistent Process Concurrent Read Concurrent Write Lock Recovery Query Syntax
mu Xapian mu server (for mu4e) ⚠️ Limited ❌ No ⚠️ Manual kill Simple, intuitive
notmuch Xapian ❌ No ✅ Yes (with caveats) ❌ No ✅ Auto Tag-based
mairix Custom ❌ No ✅ Yes ❌ No ✅ –unlock flag Boolean search

Detailed Comparison

mu (Current Tool)

What you’re using now.

Strengths:

  • Fast, powerful query language
  • Excellent Emacs integration (mu4e)
  • Database can be rebuilt from messages
  • Smaller database (no positional indexing)
  • Everything stored in messages (easy sync)

Weaknesses:

  • mu server holds persistent write lock
  • Can’t run mu index while mu4e is open
  • Intermittent issue: server doesn’t always quit cleanly

Best For:

  • Primary email reading in mu4e
  • When you don’t need frequent background indexing
  • When email access is mostly through Emacs

Concurrency Model:

  • Read operations: ✅ Work while mu4e is running
  • Write operations: ❌ Blocked while mu4e is running
  • Workaround: Use emacsclient to trigger indexing

notmuch

A strong alternative with better concurrency.

Strengths:

  • No persistent server process
  • Better concurrent access (read during write)
  • Excellent CLI tools
  • Tag-based workflow (very flexible)
  • Good Emacs integration (notmuch.el)
  • Active development

Weaknesses:

  • Tags stored in database (not on messages)
  • Database must be synced across machines
  • Still has Xapian limitations (2-version concurrency)
  • Different query syntax (learning curve)

Best For:

  • When you need concurrent access (scripts + GUI)
  • Tag-based email organization
  • Multiple machines accessing same maildir

Concurrency Model:

  • Read operations: ✅ Work during indexing
  • Write operations: ❌ Only one at a time
  • Advantage: No persistent lock, better recovery

Migration Path:

# Install notmuch
nix-shell -p notmuch

# Initialize
notmuch setup  # Point to /home/vincent/desktop/mails

# Initial index
notmuch new

# Search (different syntax)
notmuch search from:alice@example.com
notmuch search tag:inbox and date:1w..

Emacs Integration:

  • Built-in notmuch.el (ships with Emacs)
  • Similar to mu4e but tag-focused
  • No server process, no lock issues

mairix

Simplest option, minimal locking issues.

Strengths:

  • No persistent process at all
  • Simple locking (lock file during operations)
  • Easy lock recovery (--unlock flag)
  • Fast incremental indexing
  • Very stable and mature

Weaknesses:

  • Less powerful query syntax
  • No Emacs integration (use with mutt, etc.)
  • Smaller community
  • Fewer features than mu/notmuch

Best For:

  • Simple search needs
  • Scripts and automation
  • When you want minimal complexity
  • Using with mutt or other MUAs

Concurrency Model:

  • Lock only during operation
  • No persistent locks
  • Easy recovery from crashes

Dovecot FTS (Server-Side)

For server-based email setups.

Options:

  • fts-flatcurve: Newest, becoming default
  • fts-xapian: Good for low memory
  • Apache Solr: Complex, being phased out

Best For:

  • Server-side email (IMAP servers)
  • When you control the email server
  • Not applicable for local Maildir

Not Recommended For:

  • Local Maildir setups (like yours)

Recommendation Matrix

Stick with mu if:

  • ✅ You primarily use mu4e in Emacs
  • ✅ You can trigger indexing from within Emacs
  • ✅ You rarely need background indexing
  • ✅ You like the current workflow

Mitigation: Use the Email skill’s smart-reindex.sh tool for scripts

Switch to notmuch if:

  • ✅ You need better concurrent access
  • ✅ You run many automated scripts
  • ✅ You want tag-based organization
  • ✅ The occasional read error during indexing is acceptable
  • ✅ You’re willing to learn new query syntax

Migration Effort: Medium (different query syntax, Emacs config changes)

Switch to mairix if:

  • ✅ You want simplest possible locking
  • ✅ You use email clients other than mu4e
  • ✅ You don’t need advanced features
  • ✅ You mostly use scripts/CLI

Migration Effort: High (no good Emacs integration)

Hybrid Approach

Use both mu and notmuch in parallel:

  1. Keep mu4e for interactive email reading
  2. Add notmuch for scripts/automation
  3. Maintain two separate indices

Pros:

  • Best of both worlds
  • No conflicts (separate databases)
  • Scripts always work

Cons:

  • 2x disk space for indices
  • Must maintain both
  • Slight complexity

Setup:

# mu uses: ~/.cache/mu/xapian
# notmuch uses: ~/desktop/mails/.notmuch

# Both can index the same Maildir
# No conflicts because separate databases

Current State of Your Email Skill

The Email skill has been updated to work well with mu’s locking:

✅ What works now:

  • mu find queries (read-only, always work)
  • mu view (read-only, always work)
  • mu extract (read-only, always work)
  • smart-reindex.sh (handles locks intelligently)
  • archive-emails.sh (updated with lock detection)

✅ Recommended pattern for new scripts:

#!/usr/bin/env bash
# Use mu find freely (read-only)
results=$(mu find "your query" --format=json)

# For reindexing, use the helper
~/.config/claude/skills/Email/tools/smart-reindex.sh

Further Reading