main

Code Analysis Tool Comparison

Overview

This document compares different code analysis tools available for the home repository, helping you choose the right tool for each task.

Tools Comparison Matrix

Tool Type Speed Languages Strength Weakness
ast-grep AST pattern ⚡⚡⚡ Very Fast 20+ Structural search, refactoring No deep semantics
ripgrep Text search ⚡⚡⚡⚡ Fastest All Raw speed, regex False positives
semgrep Semantic ⚡ Slow 30+ Deep analysis, security Slow on large repos
statix Semantic ⚡⚡ Fast Nix Nix-specific checks Nix only
shellcheck Semantic ⚡⚡ Fast Bash/sh Deep shell analysis Shell only
golangci-lint Semantic ⚡⚡ Medium Go Deep Go analysis Go only
deadnix AST ⚡⚡⚡ Fast Nix Dead code detection Nix only

Detailed Comparison

ast-grep vs. ripgrep

When to use ripgrep:

# Simple text search - super fast
rg "password" --type nix

# Finding literal strings
rg "TODO: fix this"

# Counting occurrences
rg -c "import" --type go

When to use ast-grep:

# Structural patterns
ast-grep -p 'mkOption { $$$ }' -l nix

# Code refactoring (interactive)
ast-grep -p 'oldFunc($ARGS)' --rewrite 'newFunc($ARGS)' --interactive

# Avoiding false positives from comments/strings
ast-grep -p 'password = $VAL' -l nix  # Won't match "# password = ..."

Performance:

Searching 417 files in home repository:
- ripgrep: ~0.005s (2.5x faster)
- ast-grep: ~0.022s (still very fast)

Verdict: Use ripgrep for quick searches, ast-grep when you need accuracy or refactoring.

ast-grep vs. semgrep

When to use semgrep:

# Deep security analysis
semgrep --config=p/security-audit

# Complex data flow analysis
# Find SQL injection vulnerabilities
semgrep --config "r/python.lang.security.audit.sql-injection"

# Taint analysis (user input → dangerous function)

When to use ast-grep:

# Fast structural patterns
ast-grep -p 'curl $$$URL | sh' -l bash

# Custom repo-specific rules
ast-grep scan  # Uses .ast-grep/rules/

# Interactive refactoring
ast-grep -p 'v1beta1' --rewrite 'v1' --interactive

Performance:

Full scan of home repository:
- ast-grep: ~0.02s
- semgrep: ~30s (1500x slower)

Verdict: Use ast-grep for fast feedback and refactoring. Use semgrep for deep security audits (monthly/quarterly).

ast-grep vs. Language-Specific Tools

Nix: ast-grep vs. statix

statix (Nix linter):

statix check .

Checks:

  • Empty let-in blocks
  • Unused function arguments
  • Deprecated syntax (with keyword)
  • Semantic anti-patterns

ast-grep (Structural patterns):

ast-grep scan  # .ast-grep/rules/nix-*.yml

Checks:

  • Custom repository conventions
  • Specific patterns (mkHost usage, etc.)
  • Style preferences

Verdict: Use both. statix for Nix semantics, ast-grep for custom patterns.

.PHONY: lint-nix
lint-nix:
	statix check .
	ast-grep scan --rule .ast-grep/rules/nix-*.yml

Bash: ast-grep vs. shellcheck

shellcheck (Bash linter):

shellcheck script.sh

Checks:

  • Undefined variables
  • Quoting issues
  • Portability problems
  • Command usage errors
  • Many semantic issues

ast-grep (Structural patterns):

ast-grep scan --rule .ast-grep/rules/bash-*.yml

Checks:

  • Custom conventions (log function usage, etc.)
  • Specific dangerous patterns
  • Repository-specific requirements

Verdict: Use both. shellcheck is essential for Bash, ast-grep for custom rules.

# Combined Bash linting
shellcheck script.sh
ast-grep -l bash script.sh

Go: ast-grep vs. golangci-lint

golangci-lint (Go meta-linter):

golangci-lint run ./...

Checks:

  • Type errors
  • Unused code
  • Inefficiencies
  • Common bugs
  • Security issues (via gosec)
  • 100+ linters

ast-grep (Structural patterns):

ast-grep scan --rule .ast-grep/rules/go-*.yml

Checks:

  • Custom error wrapping patterns
  • Specific Tekton/K8s conventions
  • Repository-specific requirements

Verdict: golangci-lint is essential for Go. Add ast-grep only for project-specific patterns.

ast-grep vs. deadnix

deadnix (Find dead Nix code):

deadnix

Finds:

  • Unused function arguments
  • Unused let bindings
  • Dead code paths

ast-grep (General patterns):

ast-grep scan

Finds:

  • Active code patterns (both used and unused)
  • Style issues
  • Conventions

Verdict: Use deadnix to clean up code. Use ast-grep for linting active code.

Decision Matrix

Use ripgrep when:

  • Quick text search
  • Finding literal strings
  • Exploring unknown codebase
  • Counting occurrences
  • Speed is critical (< 0.01s)

Use ast-grep when:

  • Structural code search
  • Cross-language patterns
  • Code refactoring (especially interactive)
  • Custom linting rules
  • Want accuracy over speed
  • Moderate speed acceptable (< 0.1s)

Use semgrep when:

  • Deep security analysis
  • Complex data flow tracking
  • Monthly/quarterly security audits
  • Need pre-built security rule sets
  • Speed not critical (30s+ acceptable)

Use language-specific tools when:

  • Deep semantic analysis needed
  • Type checking required
  • Official language tooling exists
  • Community rule sets available
  • IDE integration important

Real-World Examples

Example 1: Finding a Pattern

Task: Find all password configuration in Nix files.

Option 1: ripgrep (fastest, least accurate)

rg "password" --type nix

Result: Many false positives (comments, descriptions, etc.)

Option 2: ast-grep (fast, accurate)

ast-grep -p 'passwordFile = $PATH' -l nix

Result: Only actual password assignments

Option 3: statix (semantic, might not catch this specific pattern)

statix check .

Result: Catches Nix anti-patterns, but not this specific one

Best choice: ast-grep for this task.

Example 2: Security Audit

Task: Find potential security issues in repository.

Option 1: ast-grep (fast, basic patterns)

ast-grep scan --rule .ast-grep/rules/security-*.yml

Time: ~0.02s Finds: Basic patterns (curl|sh, hardcoded secrets)

Option 2: semgrep (slow, comprehensive)

semgrep --config=p/security-audit

Time: ~30s Finds: Complex patterns, data flow issues, OWASP Top 10

Best choice:

  • Daily: ast-grep (fast feedback)
  • Before release: semgrep (comprehensive)

Example 3: Code Refactoring

Task: Rename function across codebase.

Option 1: ripgrep + sed (dangerous)

rg "oldFunc" -l | xargs sed -i 's/oldFunc/newFunc/g'

Risk: Matches in comments, strings, similar names

Option 2: ast-grep (safe, interactive)

ast-grep -p 'oldFunc($$$ARGS)' \
  --rewrite 'newFunc($$$ARGS)' \
  --interactive

Benefit: Review each change, only matches function calls

Option 3: Language-specific IDE refactoring (safest, requires IDE)

  • VSCode: F2 (rename symbol)
  • Requires LSP, type information

Best choice: ast-grep for cross-file refactoring with review.

Performance Benchmarks

Home Repository (417 files: Nix, Bash, Go, Python, YAML)

Tool Operation Time Files/sec
ripgrep Text search “password” 0.005s 83,400
ast-grep Scan all rules 0.022s 18,950
ast-grep Single rule 0.010s 41,700
semgrep Security audit 30s 14
statix Nix check 2s 175
shellcheck All .sh files 1.5s 33

Large Codebase (hypothetical: 10,000 files)

Tool Estimated Time Practical?
ripgrep 0.05s ✅ Yes
ast-grep 0.5s ✅ Yes
semgrep 10min ⚠️ CI only
golangci-lint 2min ⚠️ CI only

Daily Development

# Quick search (for exploration)
rg "pattern"

# Structural search (for accuracy)
ast-grep -p 'pattern' -l lang

# Language-specific linting
make lint  # Includes multiple tools

Pre-Commit

# Fast linting with ast-grep
ast-grep scan

# Plus language-specific tools
statix check .
shellcheck changed_scripts.sh

CI/CD Pipeline

# Everything
make lint              # Includes ast-grep, statix, shellcheck, etc.
make test              # Unit tests
semgrep --config=auto  # Deep security scan (weekly)

Refactoring

# 1. Plan with ripgrep
rg "oldPattern" -l

# 2. Execute with ast-grep
ast-grep -p 'oldPattern' --rewrite 'newPattern' --interactive

# 3. Verify with tests
make test

Tool Combination Strategies

Strategy 1: Speed Tiers

# Tier 1: Instant feedback (< 0.1s)
alias search='rg'

# Tier 2: Fast analysis (< 1s)
alias lint-quick='ast-grep scan'

# Tier 3: Comprehensive (minutes)
alias lint-full='make lint && semgrep --config=auto'

Strategy 2: By Task

# Search: ripgrep → ast-grep → IDE
# Refactor: ast-grep → IDE refactoring tools
# Lint: ast-grep + language-specific → semgrep
# Security: ast-grep (quick) → semgrep (deep)

Strategy 3: By Language

# Nix
statix check .               # Semantic
ast-grep scan -l nix         # Custom patterns
deadnix                       # Dead code

# Bash
shellcheck *.sh              # Semantic
ast-grep scan -l bash        # Custom patterns

# Go
golangci-lint run ./...      # Comprehensive
ast-grep scan -l go          # Custom patterns

# Cross-language
ast-grep scan                # All languages
semgrep --config=auto        # Security (all)

Conclusion

Golden Rules

  1. Start with ripgrep for quick searches
  2. Use ast-grep for structural patterns and refactoring
  3. Keep language-specific tools for deep analysis
  4. Add semgrep for security audits
  5. Combine tools for best results

Quick Reference

# Searching
rg "text"                    # Fast text search
ast-grep -p 'pattern' -l L   # Structural search

# Linting
ast-grep scan                # Fast custom rules
make lint                    # All linters

# Refactoring
ast-grep -p 'old' --rewrite 'new' --interactive

# Security
ast-grep scan --rule security-*.yml    # Quick
semgrep --config=p/security-audit      # Deep

Tool Selection Flowchart

Need to search code?
├─ Simple text? → ripgrep
└─ Structural pattern?
   ├─ One language + deep analysis? → Language tool
   └─ Cross-language or refactoring? → ast-grep

Need to lint?
├─ Fast feedback? → ast-grep
├─ Language-specific? → Use language tool
└─ Security audit? → semgrep

Need to refactor?
├─ Simple rename in IDE? → LSP/IDE
├─ Cross-file structural? → ast-grep
└─ Complex semantic? → Language refactoring tool

Summary: ast-grep fits perfectly between simple text search (ripgrep) and deep semantic analysis (semgrep, language-specific tools). Use it for fast structural search, custom linting, and interactive refactoring across multiple languages.