flake-update-20260201

Database Locking in mu/mu4e

Understanding and working with mu’s database locking behavior.

How mu Locking Works

The Xapian Backend

mu uses Xapian for its search database. Xapian’s locking model:

  • Multiple concurrent readers are supported
  • Only one writer can exist at a time
  • Attempting to open a second writer throws a DatabaseLockError

mu server Behavior

When mu4e is running in Emacs:

  1. mu4e starts mu server as a background process
  2. mu server opens the database in write mode and holds the lock
  3. The lock is held “just in case” for performance (avoids costly open/close cycles)
  4. The lock persists until mu server exits

What Works and What Doesn’t

While mu4e is running:

✅ WORKS - Read-only operations:

  • mu find <query> - Search emails
  • mu view <path> - View email content
  • mu extract <path> - Extract attachments
  • mu cfind <pattern> - Find contacts
  • Scripts that only search/read

❌ BLOCKED - Write operations:

  • mu index - Reindex maildir (fails with “already locked”)
  • mu add - Add messages
  • Any other process trying to write

Known Issues

mu server Doesn’t Exit Cleanly

Problem: Sometimes mu server doesn’t quit properly after exiting mu4e (Issue #2198)

Symptoms:

  • Error: “Unable to get write lock on ~/.cache/mu/xapian: already locked”
  • Can’t restart mu4e without manual intervention
  • Happens intermittently

Detection:

# Check if mu server is still running
pgrep -u $UID mu

# Check lock status directly
xapian-delve ~/.cache/mu/xapian | grep "currently open for writing"
# Shows: true (locked) or false (unlocked)

Quick Fix:

# Gracefully stop mu server
pkill -2 -u $UID mu  # SIGINT for graceful shutdown

# Wait for it to release the lock
sleep 1

# Verify it's gone
pgrep -u $UID mu || echo "mu server stopped"

Workarounds

Option 1: Use emacsclient for Indexing

Instead of running mu index externally, trigger it from within mu4e:

# Check if mu server is running
if pgrep -u $UID mu > /dev/null; then
    # Use emacsclient to trigger indexing
    emacsclient -e '(mu4e-update-index)'
else
    # Safe to run mu index directly
    mu index
fi

Option 2: Smart Lock Detection in Scripts

#!/usr/bin/env bash
# Robust indexing that works whether mu4e is running or not

reindex_mu() {
    if pgrep -u $UID mu > /dev/null 2>&1; then
        echo "mu4e is running, using emacsclient..."
        if command -v emacsclient >/dev/null 2>&1; then
            emacsclient -e '(mu4e-update-index)' 2>/dev/null || {
                echo "Warning: Could not trigger reindex via mu4e"
                return 1
            }
        else
            echo "Warning: emacsclient not available, skipping reindex"
            return 1
        fi
    else
        echo "Running mu index directly..."
        mu index
    fi
}

reindex_mu

Option 3: Read-Only Scripting

For scripts that only need to search/read emails:

  • Use mu find freely - it works even when mu4e is running
  • No need to check for locks
  • Script can run in parallel with mu4e

Best Practices for Scripts

1. Distinguish Read from Write Operations

# Read-only script - no lock checking needed
search_emails() {
    mu find "$@" --format=json
}

# Write operation - needs lock handling
archive_and_reindex() {
    # ... move/delete emails ...

    # Smart reindex
    if pgrep -u $UID mu > /dev/null; then
        echo "Note: mu4e is running, trigger reindex from Emacs with (mu4e-update-index)"
        return 0
    else
        mu index
    fi
}

2. Provide User Feedback

if ! mu index 2>/dev/null; then
    if pgrep -u $UID mu > /dev/null; then
        echo "Note: mu4e is running. Run (mu4e-update-index) in Emacs to refresh."
    else
        echo "Error: Failed to index. Check mu database."
        return 1
    fi
fi

3. Graceful Degradation

# Try to reindex, but don't fail the script if it can't
mu index 2>/dev/null || {
    echo "Warning: Could not reindex (mu4e may be running)"
    echo "Your changes will appear after next mu4e update"
}

Alternative: Consider notmuch

If concurrent access is critical for your workflow, consider notmuch as an alternative:

Advantages:

  • No persistent server process
  • Better concurrent read support
  • Automatic lock recovery on next run
  • Can search while indexing (with occasional errors)

Trade-offs:

  • Different query syntax
  • Tags stored in database (not on messages)
  • Must sync database across machines

See reference/Alternatives.md for a detailed comparison.

Troubleshooting

Error: “database @ ~/.cache/mu/xapian is write-locked”

Cause: mu server is running (or a stale lock exists)

Solution:

  1. Check if mu4e/mu server is running: pgrep mu
  2. If yes, close mu4e or use emacsclient for operations
  3. If no (stale lock), kill stale process: pkill -2 -u $UID mu

Error: “Unable to get write lock”

Same as above - indicates active or stale lock.

mu index works in terminal but not in scripts

Cause: Script may run while mu4e is open

Solution: Add lock detection to script (see workarounds above)

References