Nixpkgs Branch Consolidation Tool

Automatically consolidate multiple WIP/PR branches from your nixpkgs fork into a single consolidated branch using cherry-picking.

Overview

This tool helps manage multiple work-in-progress (WIP) branches in your nixpkgs fork by:

1. Fetching latest from upstream nixpkgs
2. Creating a fresh consolidated branch from the base (e.g., nixos-unstable)
3. Cherry-picking commits from each WIP branch in order
4. Force-pushing the result to your fork
5. Sending notifications on success/failure

Key Features:

• Uses git worktrees to avoid interfering with your main nixpkgs checkout
• Cherry-pick strategy (no merge commits)
• Handles conflicts gracefully with detailed error reporting
• Integrates with ntfy for notifications
• Dry-run mode for safe testing
• Comprehensive logging

Use Case

Perfect for nixpkgs contributors who:

• Maintain multiple WIP branches for different packages/features
• Want to test all changes together in a single branch
• Need an automated way to keep a consolidated branch up-to-date with upstream
• Want to reference one branch in their NixOS flake instead of managing overlays

Installation

Manual Usage

# Run directly from the tools directory
./tools/nixpkgs-consolidate/nixpkgs-consolidate.sh

Nix Package (Recommended)

# Build and install
nix build .#nixpkgs-consolidate
nix profile install .#nixpkgs-consolidate

# Run
nixpkgs-consolidate

Automated with NixOS Module

Add to your system configuration (e.g., systems/aomi/extra.nix):

{
  services.nixpkgs-consolidate = {
    enable = true;
    user = "vincent";
    schedule = "daily";  # or "Mon,Thu *-*-* 02:00:00"
    configFile = "/home/vincent/.config/nixpkgs-automation/branches.conf";
  };
}

Configuration

Automatic Base Branch Detection

NEW: The script automatically detects the nixpkgs revision from your home flake!

When running, the script will:

1. Read the configured base branch from branches.conf (e.g., nixos-unstable)
2. Check your home flake at $HOME_FLAKE_PATH/flake.lock
3. Extract the exact nixpkgs commit hash from the nixpkgs input
4. Use that commit hash as the base instead of the configured branch

Benefits:

• Consolidated branch is based on the exact same nixpkgs as your home configuration
• No manual hash updates needed - it follows your flake.lock
• More reliable testing of packages in your actual NixOS environment
• Falls back to configured base if detection fails

Example:

# branches.conf contains:
base: nixos-unstable

# But your flake.lock has nixpkgs pinned to:
31926ce9afb1c915fa4190b77ca9be389ccaf18e

# Script will use the commit hash from flake.lock as the base

Setup

1. Create configuration directory:

   mkdir -p ~/.config/nixpkgs-automation
   

2. Copy example configuration:

   cp tools/nixpkgs-consolidate/branches.conf.example \
      ~/.config/nixpkgs-automation/branches.conf
   

3. Edit configuration:

   vim ~/.config/nixpkgs-automation/branches.conf
   

Configuration File Format

~/.config/nixpkgs-automation/branches.conf:

# Base branch (required)
base: nixos-unstable

# WIP branches to consolidate (one per line)
wip-package-foo
wip-fix-bar
pr-12345-add-service

Rules:

• First line starting with base: defines the upstream base branch
• Other non-comment, non-empty lines are treated as WIP branch names
• Branches are cherry-picked in the order listed
• Comments start with #

Environment Variables

Override defaults with environment variables:

Usage

Basic Usage

# Run with defaults
nixpkgs-consolidate

# Or directly:
./tools/nixpkgs-consolidate/nixpkgs-consolidate.sh

Dry Run (Recommended First)

Test without actually pushing:

DRY_RUN=true nixpkgs-consolidate

This will:

• Parse configuration
• Fetch from remotes
• Create worktree
• Cherry-pick all commits
• Show what would be pushed
• NOT actually push to remote

Custom Branch Name

NIXPKGS_CONSOLIDATED_BRANCH=personal-main nixpkgs-consolidate

Different Base Branch

# Edit config file to change base:
echo "base: master" > ~/.config/nixpkgs-automation/branches.conf
# Then run:
nixpkgs-consolidate

Workflow

Initial Setup

1. Fork nixpkgs on GitHub (if you haven’t already)

2. Clone nixpkgs locally:

   git clone git@github.com:NixOS/nixpkgs.git ~/src/nixpkgs
   cd ~/src/nixpkgs
   git remote add upstream git@github.com:NixOS/nixpkgs.git
   # Assumes 'origin' points to your fork
   

3. Create WIP branches:

   git checkout -b wip-my-feature nixos-unstable
   # Make changes...
   git push origin wip-my-feature
   

4. Configure consolidation:

   mkdir -p ~/.config/nixpkgs-automation
   cat > ~/.config/nixpkgs-automation/branches.conf <<EOF
   base: nixos-unstable
   wip-my-feature
   wip-another-feature
   EOF
   

5. Run consolidation:

   DRY_RUN=true nixpkgs-consolidate  # Test first
   nixpkgs-consolidate                # Actually push
   

Regular Usage

1. Work on WIP branches as usual:

   git checkout wip-my-feature
   # Make changes...
   git commit -m "Update package foo"
   git push origin wip-my-feature
   

2. Run consolidation (manually or via systemd timer):

   nixpkgs-consolidate
   

3. Use consolidated branch in your flake:

   {
     inputs = {
       nixpkgs.url = "github:vdemeester/nixpkgs/wip-consolidated";
     };
   }
   

4. Clean up when PRs land:

   • Remove merged branches from branches.conf
   • Delete remote branches: git push origin --delete wip-merged-feature

Git Workflow Details

Worktree Usage

The script uses git worktrees to keep your main repository clean:

1. Main repository (~/src/nixpkgs):

   • Stays on whatever branch you’re working on
   • Not affected by consolidation process

2. Consolidation worktree (~/src/nixpkgs-consolidate-work):

   • Created automatically
   • Used for all cherry-picking operations
   • Removed automatically after completion

Cherry-Pick Strategy

For each WIP branch:

# Get commits unique to WIP branch
commits = $(git log upstream/nixos-unstable..origin/wip-feature)

# Cherry-pick each commit
for commit in commits; do
  git cherry-pick $commit
done

Why cherry-pick instead of merge?

• Clean linear history
• No merge commits
• Easy to see what changed
• Better for testing package updates

Conflict Handling

If a cherry-pick conflicts:

1. Script aborts immediately
2. Sends detailed notification with:
   • Which branch caused the conflict
   • Which commit failed
   • What was successfully completed
3. Cleans up worktree
4. Returns non-zero exit code

Resolution:

• Fix conflicts in the WIP branch
• Rebase WIP branch on latest upstream
• Run consolidation again

Notifications

Success Notification

Sent to ntfy.sbr.pm/git-builds:

✅ Nixpkgs Consolidation Success

Consolidated 3 branch(es) into wip-consolidated

Base: nixos-unstable
Branches:
wip-package-foo
wip-fix-bar
pr-12345-add-service

Total commits: 15

Pushed to origin/wip-consolidated

Failure Notification

❌ Nixpkgs Consolidation Failed

Cherry-pick conflict in branch: wip-fix-bar
Commit: abc123def456

Base: nixos-unstable
Completed: wip-package-foo
Failed at: wip-fix-bar

Logs: /var/log/nixpkgs-consolidate/20260108-103015.log

Logging

Logs are written to:

/var/log/nixpkgs-consolidate/YYYYMMDD-HHMMSS.log

Log format:

[2026-01-08 10:30:15] Starting nixpkgs branch consolidation
[2026-01-08 10:30:16] Reading configuration from ~/.config/nixpkgs-automation/branches.conf
[2026-01-08 10:30:17] Base branch: nixos-unstable
[2026-01-08 10:30:17] WIP branches (3): wip-foo wip-bar pr-12345
[2026-01-08 10:30:20] Fetching from upstream...
[2026-01-08 10:30:25] Creating worktree at ~/src/nixpkgs-consolidate-work
[2026-01-08 10:30:26] Processing WIP branch: wip-foo
[2026-01-08 10:30:27] Found 5 commit(s) to cherry-pick
[2026-01-08 10:30:28] Cherry-picking abc123: Update foo to 1.2.3
...

Troubleshooting

“Configuration file not found”

Problem: ~/.config/nixpkgs-automation/branches.conf doesn’t exist

Solution:

mkdir -p ~/.config/nixpkgs-automation
cp tools/nixpkgs-consolidate/branches.conf.example \
   ~/.config/nixpkgs-automation/branches.conf
vim ~/.config/nixpkgs-automation/branches.conf

“Remote ‘upstream’ not found”

Problem: nixpkgs repository doesn’t have upstream remote configured

Solution:

cd ~/src/nixpkgs
git remote add upstream git@github.com:NixOS/nixpkgs.git
git fetch upstream

“Branch not found on remote origin”

Problem: WIP branch listed in config doesn’t exist in your fork

Solution:

• Check branch exists: git ls-remote origin | grep wip-feature
• Push branch to fork: git push origin wip-feature
• Or remove from config if no longer needed

Cherry-pick conflicts

Problem: Commits conflict when cherry-picked

Solution:

1. Check the logs to see which branch caused the conflict
2. Rebase that branch on latest upstream:

   git checkout wip-problematic-branch
   git fetch upstream
   git rebase upstream/nixos-unstable
   git push --force-with-lease origin wip-problematic-branch
   

3. Run consolidation again

Force push rejected

Problem: git push --force-with-lease fails

Cause: Someone else (or another process) updated the consolidated branch

Solution:

• If it was you from another machine: Re-run consolidation
• If unexpected: Check remote branch with git log origin/wip-consolidated

Integration with NixOS Flake

Once consolidation is running, reference the consolidated branch in your flake:

{
  inputs = {
    # Use your consolidated branch instead of official nixpkgs
    nixpkgs.url = "github:vdemeester/nixpkgs/wip-consolidated";

    # Or for testing with local changes:
    # nixpkgs.url = "path:/home/vincent/src/nixpkgs";
  };

  outputs = { nixpkgs, ... }: {
    # Your configuration...
  };
}

Benefits:

• Test all your WIP packages together
• Single flake input instead of multiple overlays
• Automatically updated as you run consolidation

Automation with Systemd

Manual Trigger

systemctl start nixpkgs-consolidate.service

Check Status

systemctl status nixpkgs-consolidate.service
journalctl -u nixpkgs-consolidate.service -f

List Timer Schedule

systemctl list-timers | grep nixpkgs

Modify Schedule

Edit your system configuration:

services.nixpkgs-consolidate = {
  enable = true;
  schedule = "Mon,Thu *-*-* 02:00:00";  # Monday and Thursday at 2 AM
};

Best Practices

1. Test with dry-run first:

   DRY_RUN=true nixpkgs-consolidate
   

2. Keep WIP branches rebased:

   git checkout wip-feature
   git fetch upstream
   git rebase upstream/nixos-unstable
   git push --force-with-lease origin wip-feature
   

3. Clean up merged branches:

   • Remove from branches.conf
   • Delete from fork: git push origin --delete wip-merged

4. Use descriptive branch names:

   • wip-update-python-312 ✅
   • wip-stuff ❌

5. Order matters in config:

   • Put independent changes first
   • Put dependent changes later

6. Monitor notifications:

   • Subscribe to ntfy topic git-builds
   • Fix conflicts promptly

Security

• Git Authentication: Uses existing SSH keys (~/.ssh/)
• ntfy Token: Stored in agenix (/run/agenix/ntfy-token)
• Logs: Written to /var/log with restricted permissions
• Systemd: Service runs with security hardening (ProtectSystem, ProtectHome, etc.)

See Also

• Nixpkgs Contributing Guide
• Git Worktrees Documentation
• ntfy Documentation

License

MIT