fedora-csb-system-manager

Build Workflow

Build container images with Docker/Podman, supporting multi-architecture builds and optimization techniques.

Quick Start

Detect runtime:

# Auto-detect (prefers Podman)
../tools/DetectRuntime.sh

# Specify runtime
../tools/DetectRuntime.sh --runtime docker
../tools/DetectRuntime.sh --runtime podman

Basic build:

docker build -t myapp:latest .
podman build -t myapp:latest .

Multi-architecture:

# Docker buildx
docker buildx build --platform linux/amd64,linux/arm64 -t myapp:latest --push .

# Podman manifest
podman build --platform linux/amd64 -t myapp:amd64 .
podman build --platform linux/arm64 -t myapp:arm64 .
podman manifest create myapp:latest
podman manifest add myapp:latest myapp:amd64
podman manifest add myapp:latest myapp:arm64

Optimize with multi-stage:

FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY . .
RUN go build -o myapp

FROM alpine:3.19
COPY --from=builder /app/myapp /
CMD ["/myapp"]

Runtime Detection

Detect Container Runtime

Auto-detect:

../tools/DetectRuntime.sh
# Prefers Podman, falls back to Docker

Specify runtime:

../tools/DetectRuntime.sh --runtime docker
../tools/DetectRuntime.sh --runtime podman

Check available builders:

# Docker buildx
docker buildx ls

# Podman buildah
buildah version

Process Integration

  1. Parse user prompt for explicit runtime mention
  2. Run DetectRuntime.sh with appropriate flags
  3. Identify build tools (buildx, buildah)
  4. Use detected runtime for all commands

Building Images

Basic Build Process

Docker:

# Basic build
docker build -t myapp:latest .

# With build arguments
docker build --build-arg VERSION=1.0 -t myapp:1.0 .

# With specific target
docker build --target production -t myapp:prod .

# Build without cache
docker build --no-cache -t myapp:latest .

# Build with custom context
docker build -t myapp:latest -f path/to/Dockerfile context/

Podman:

# Basic build
podman build -t myapp:latest .

# With build arguments
podman build --build-arg VERSION=1.0 -t myapp:1.0 .

# With specific target
podman build --target production -t myapp:prod .

# Build without cache
podman build --no-cache -t myapp:latest .

# Build with custom context
podman build -t myapp:latest -f path/to/Dockerfile context/

Build Arguments and Tags

Multiple tags:

docker build -t myapp:latest -t myapp:1.0 -t myregistry/myapp:1.0 .
podman build -t myapp:latest -t myapp:1.0 -t myregistry/myapp:1.0 .

Build arguments:

ARG VERSION=1.0
ARG BUILD_DATE
ENV APP_VERSION=${VERSION}

RUN echo "Building version ${VERSION} on ${BUILD_DATE}"

docker build --build-arg VERSION=2.0 --build-arg BUILD_DATE=$(date -u +%Y-%m-%d) .
podman build --build-arg VERSION=2.0 --build-arg BUILD_DATE=$(date -u +%Y-%m-%d) .

Dockerfile Best Practices

Base image selection:

# Use specific tags, never :latest
FROM alpine:3.19
FROM python:3.11-slim
FROM node:20-alpine

# For static binaries
FROM scratch
FROM gcr.io/distroless/static-debian12

Layer ordering (cache optimization):

# Good: dependencies cached separately
FROM node:20-alpine
WORKDIR /app
# Copy dependency files first (changes less frequently)
COPY package*.json ./
RUN npm ci
# Copy code last (changes most frequently)
COPY . .
RUN npm run build

Non-root user:

RUN addgroup -S appgroup && adduser -S appuser -G appgroup
USER appuser
WORKDIR /app
COPY --chown=appuser:appgroup . .

Minimal layers:

# Bad: creates multiple layers
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get clean

# Good: single layer with cleanup
RUN apt-get update && \
    apt-get install -y --no-install-recommends curl && \
    rm -rf /var/lib/apt/lists/*

Using .dockerignore

Create .dockerignore to exclude unnecessary files:

# Version control
.git
.gitignore

# Dependencies
node_modules
vendor

# Build artifacts
dist
build
*.o

# Documentation
*.md
docs/

# Tests
*_test.go
test/
__tests__

# CI/CD
.github
.gitlab-ci.yml

# Development
.env
.vscode
*.log

# OS files
.DS_Store

Multi-Architecture Builds

Docker with Buildx

Setup QEMU:

docker run --privileged --rm tonistiigi/binfmt --install all

Create builder:

docker buildx create --name multiarch --driver docker-container --use
docker buildx inspect --bootstrap

List available platforms:

docker buildx inspect --bootstrap | grep Platforms

Build multi-arch image:

# Build and push
docker buildx build \
  --platform linux/amd64,linux/arm64,linux/arm/v7 \
  -t myregistry/myapp:latest \
  --push \
  .

# Build single platform locally
docker buildx build \
  --platform linux/arm64 \
  -t myapp:arm64 \
  --load \
  .

Build with cache:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --cache-from type=registry,ref=myregistry/myapp:buildcache \
  --cache-to type=registry,ref=myregistry/myapp:buildcache,mode=max \
  -t myregistry/myapp:latest \
  --push \
  .

Manage builders:

# List builders
docker buildx ls

# Remove builder
docker buildx rm multiarch

# Use default builder
docker buildx use default

Podman with Buildah

Install QEMU:

# On host system
sudo dnf install qemu-user-static  # Fedora/RHEL
sudo apt install qemu-user-static  # Debian/Ubuntu

# Or with Podman
podman run --rm --privileged multiarch/qemu-user-static --reset -p yes

Build for specific platform:

podman build --platform linux/arm64 -t myapp:arm64 .

Build multi-arch with manifest:

# Build for each platform
podman build --platform linux/amd64 -t myapp:amd64 .
podman build --platform linux/arm64 -t myapp:arm64 .
podman build --platform linux/arm/v7 -t myapp:armv7 .

# Create manifest
podman manifest create myapp:latest

# Add images to manifest
podman manifest add myapp:latest myapp:amd64
podman manifest add myapp:latest myapp:arm64
podman manifest add myapp:latest myapp:armv7

# Push manifest
podman manifest push myapp:latest docker://myregistry/myapp:latest

Using buildah directly:

buildah bud --arch arm64 -t myapp:arm64 .
buildah bud --arch amd64 -t myapp:amd64 .

Platform Targeting

Common platforms:

Platform Description Use Case
linux/amd64 64-bit x86 Standard servers, desktops
linux/arm64 64-bit ARM Raspberry Pi 4, Apple Silicon, AWS Graviton
linux/arm/v7 32-bit ARM v7 Raspberry Pi 2/3
linux/arm/v6 32-bit ARM v6 Raspberry Pi Zero/1
linux/386 32-bit x86 Legacy systems

Platform-aware Dockerfile:

FROM --platform=$BUILDPLATFORM golang:1.21-alpine AS builder
ARG TARGETPLATFORM
ARG BUILDPLATFORM
ARG TARGETOS
ARG TARGETARCH

WORKDIR /app
COPY . .

# Build for target platform
RUN GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o app .

FROM --platform=$TARGETPLATFORM alpine:latest
COPY --from=builder /app/app /app
ENTRYPOINT ["/app"]

Conditional platform logic:

ARG TARGETARCH

RUN if [ "$TARGETARCH" = "arm64" ]; then \
      echo "Installing ARM64 packages"; \
    elif [ "$TARGETARCH" = "amd64" ]; then \
      echo "Installing AMD64 packages"; \
    fi

Verify Multi-Arch Images

Inspect manifest (Docker):

docker buildx imagetools inspect myregistry/myapp:latest
docker manifest inspect myregistry/myapp:latest

Inspect manifest (Podman):

podman manifest inspect myapp:latest
skopeo inspect docker://myregistry/myapp:latest

Test platform-specific images:

# Force ARM64 on AMD64 host
docker run --platform linux/arm64 myregistry/myapp:latest

# Verify architecture
docker run --rm myregistry/myapp:latest uname -m
# Output: x86_64, aarch64, armv7l, etc.

Image Optimization

Multi-Stage Builds

Basic pattern:

# Build stage
FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY go.* ./
RUN go mod download
COPY . .
RUN go build -o myapp

# Runtime stage
FROM alpine:3.19
WORKDIR /app
COPY --from=builder /app/myapp .
CMD ["/app/myapp"]
# Result: ~20 MB vs ~1 GB

Multiple build stages:

# Stage 1: Dependencies
FROM node:20-alpine AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production

# Stage 2: Build
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 3: Runtime
FROM node:20-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY --from=deps /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
CMD ["node", "dist/index.js"]

Base Image Selection

Size comparison:

# Large (1.1 GB)
FROM ubuntu:latest

# Medium (130 MB)
FROM ubuntu:22.04

# Small (77 MB)
FROM debian:bookworm-slim

# Smaller (7 MB)
FROM alpine:3.19

# Smallest (< 2 MB)
FROM scratch
FROM gcr.io/distroless/static-debian12

Language-specific minimal images:

# Python
FROM python:3.11-slim    # 50 MB vs 900 MB
FROM python:3.11-alpine  # 45 MB

# Node.js
FROM node:20-slim        # 240 MB vs 1 GB
FROM node:20-alpine      # 180 MB

# Go
FROM golang:1.21-alpine  # Build stage
FROM scratch             # Runtime (static binary)

Layer Optimization

Combine RUN commands:

# Bad: creates 4 layers
RUN apt-get update
RUN apt-get install -y curl
RUN apt-get install -y git
RUN apt-get clean

# Good: single layer
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        curl \
        git && \
    rm -rf /var/lib/apt/lists/*

Clean up in same layer:

# Bad: file remains in layer
RUN wget https://example.com/large-file.tar.gz
RUN tar -xzf large-file.tar.gz
RUN rm large-file.tar.gz

# Good: cleanup in same layer
RUN wget https://example.com/large-file.tar.gz && \
    tar -xzf large-file.tar.gz && \
    rm large-file.tar.gz

Size Reduction Techniques

Remove package manager cache:

# APT (Debian/Ubuntu)
RUN apt-get update && \
    apt-get install -y package && \
    rm -rf /var/lib/apt/lists/*

# APK (Alpine)
RUN apk add --no-cache package

# DNF (Fedora/RHEL)
RUN dnf install -y package && \
    dnf clean all

# npm
RUN npm ci --only=production && \
    npm cache clean --force

# pip
RUN pip install --no-cache-dir -r requirements.txt

Minimize installed packages:

# Bad: installs many unnecessary packages
RUN apt-get update && \
    apt-get install -y build-essential

# Good: minimal installation
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        gcc \
        make && \
    rm -rf /var/lib/apt/lists/*

Remove build dependencies:

RUN apk add --no-cache --virtual .build-deps \
        gcc \
        musl-dev \
        postgresql-dev && \
    pip install psycopg2 && \
    apk del .build-deps

Static Binaries

Go static binary:

FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o myapp .

FROM scratch
COPY --from=builder /app/myapp /myapp
ENTRYPOINT ["/myapp"]
# Result: ~10 MB (just the binary)

Rust static binary:

FROM rust:1.75-alpine AS builder
WORKDIR /app
COPY . .
RUN cargo build --release

FROM scratch
COPY --from=builder /app/target/release/myapp /myapp
ENTRYPOINT ["/myapp"]

Distroless Images

Language-specific distroless:

# Java
FROM gcr.io/distroless/java17-debian12
COPY target/app.jar /app.jar
CMD ["app.jar"]

# Python
FROM gcr.io/distroless/python3-debian12
COPY --from=builder /app /app
WORKDIR /app
CMD ["app.py"]

# Node.js
FROM gcr.io/distroless/nodejs20-debian12
COPY --from=builder /app /app
WORKDIR /app
CMD ["index.js"]

# Static binaries
FROM gcr.io/distroless/static-debian12
COPY --from=builder /app/myapp /
CMD ["/myapp"]

Build Cache Strategies

BuildKit cache mounts:

# syntax=docker/dockerfile:1
FROM golang:1.21-alpine
WORKDIR /app
COPY go.* ./
RUN --mount=type=cache,target=/go/pkg/mod \
    go mod download
COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    go build -o myapp

Remote cache:

docker buildx build \
  --cache-from type=registry,ref=myregistry/myapp:buildcache \
  --cache-to type=registry,ref=myregistry/myapp:buildcache,mode=max \
  -t myapp:latest .

Optimization Examples

Python Flask app (500 MB → 50 MB):

# Before
FROM python:3.11
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD ["python", "app.py"]

# After
FROM python:3.11-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN pip install --user --no-cache-dir -r requirements.txt

FROM python:3.11-slim
WORKDIR /app
COPY --from=builder /root/.local /root/.local
COPY app.py .
ENV PATH=/root/.local/bin:$PATH
CMD ["python", "app.py"]

Node.js React app (1.2 GB → 100 MB):

# Before
FROM node:20
WORKDIR /app
COPY . .
RUN npm install
RUN npm run build
CMD ["npm", "start"]

# After
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM nginx:alpine
COPY --from=builder /app/build /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Go microservice (800 MB → 10 MB):

# Before
FROM golang:1.21
WORKDIR /app
COPY . .
RUN go build -o myapp
CMD ["/app/myapp"]

# After
FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY go.* ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -ldflags="-w -s" -o myapp

FROM scratch
COPY --from=builder /app/myapp /
CMD ["/myapp"]

Analysis Tools

Check image size:

docker images myapp
podman images myapp

View image history:

docker history myapp:latest
docker history --no-trunc myapp:latest  # Full commands

podman history myapp:latest

Analyze with dive:

# Install dive
brew install dive

# Analyze image
dive myapp:latest

Find large layers:

docker history myapp:latest --format "{{.Size}}\t{{.CreatedBy}}" | sort -h

Best Practices

Dockerfile Quality Checklist

  • Use specific base image tags (not latest)
  • Minimize number of layers
  • Use multi-stage builds where appropriate
  • Run as non-root user
  • Have .dockerignore file
  • No secrets embedded
  • Proper layer ordering (least to most frequently changed)
  • Install only required packages
  • Clean up in same layer
  • Use minimal base images

Build Optimization Checklist

  • Leverage build cache effectively
  • Combine RUN commands where appropriate
  • Remove package manager cache
  • Use --no-install-recommends (apt)
  • Use --no-cache (apk)
  • Remove build dependencies after use
  • Exclude development files via .dockerignore
  • Order layers by change frequency

Security Best Practices

  • Use specific version tags
  • Update packages during build
  • Run as non-root user
  • Don’t include secrets in image
  • Scan for vulnerabilities
  • Use minimal base images (smaller attack surface)
  • Consider distroless for production

Multi-Arch Best Practices

  1. Use multi-stage builds for smaller final images
  2. Leverage build cache with remote cache
  3. Test on actual hardware when possible
  4. Use QEMU for cross-platform testing
  5. Build platform-specific optimizations with TARGETARCH
  6. Verify critical functionality on each platform

CI/CD Integration

GitHub Actions:

- name: Set up QEMU
  uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
  uses: docker/setup-buildx-action@v2

- name: Build and push
  uses: docker/build-push-action@v4
  with:
    platforms: linux/amd64,linux/arm64
    push: true
    tags: myregistry/myapp:latest

GitLab CI:

build-multiarch:
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker buildx create --use
    - docker buildx install
  script:
    - docker buildx build
      --platform linux/amd64,linux/arm64
      -t myregistry/myapp:latest
      --push .

Troubleshooting

Build Failures

Issue: Build fails with permission denied

Docker:

  • User not in docker group: sudo usermod -aG docker $USER
  • Daemon not running: sudo systemctl start docker

Podman:

  • Check rootless permissions: podman system migrate
  • Check user namespace: cat /etc/subuid /etc/subgid

Issue: Build cache not working

  • Check layer ordering (copy dependencies before code)
  • Verify .dockerignore excludes frequently changing files
  • Use --no-cache to force rebuild if needed

Issue: Large image size

  • Review Dockerfile layer ordering
  • Implement multi-stage builds
  • Use smaller base images
  • Clean up in same layer

Multi-Arch Issues

Issue: Build very slow for non-native platforms

  • Cause: QEMU emulation overhead
  • Solution: Use native builders or accept slower builds

Issue: Build fails with “exec format error”

  • Cause: QEMU not configured
  • Solution: docker run --privileged --rm tonistiigi/binfmt --install all

Issue: Cannot load multi-platform image locally

  • Cause: Local load only supports single platform
  • Solution: Use --platform to specify one, or push to registry

Issue: Different behavior on different platforms

  • Cause: Platform-specific dependencies
  • Solution: Review Dockerfile for cross-platform compatibility

Issue: Manifest push fails

  • Cause: Authentication or registry doesn’t support manifests
  • Solution: Verify registry supports manifest lists, check credentials

Optimization Issues

Issue: Removing files doesn’t reduce image size

  • Cause: Files removed in different layer
  • Solution: Clean up in same RUN command

Issue: Dependencies installed but not needed

  • Cause: Installing dev dependencies in production
  • Solution: Use npm ci --only=production, pip install --no-cache-dir

Issue: .dockerignore not working

  • Cause: File in wrong location or syntax error
  • Solution: Place in build context root, verify syntax

Output Examples

After successful build:

✓ Image built successfully
  Runtime: docker
  Image: myapp:latest
  Size: 125 MB
  Build time: 45s

Next steps:
  - Run the image: docker run myapp:latest
  - Scan for vulnerabilities: [trigger SecurityScan]
  - Push to registry: [trigger RegistryManage]

After multi-arch build:

✓ Multi-architecture image built successfully
  Runtime: docker buildx
  Image: myregistry/myapp:latest

  Platforms built:
    - linux/amd64 (152 MB)
    - linux/arm64 (148 MB)
    - linux/arm/v7 (145 MB)

  Total build time: 8m 32s

  Verify:
    docker buildx imagetools inspect myregistry/myapp:latest

  Test platforms:
    docker run --platform linux/arm64 myregistry/myapp:latest

After optimization:

✓ Image optimized successfully
  Image: myapp:latest

  Size reduction:
    Before: 850 MB
    After: 45 MB
    Savings: 805 MB (94.7%)

  Layers:
    Before: 15 layers
    After: 5 layers

  Optimizations applied:
    - Changed base image: python:3.11 → python:3.11-slim
    - Added multi-stage build
    - Combined RUN commands (4 → 1)
    - Removed package cache
    - Added .dockerignore (excluded 250 MB)

  Build time:
    Before: 3m 45s
    After: 1m 20s (faster caching)

Quick Reference

Basic builds:

docker build -t myapp:latest .
podman build -t myapp:latest .

Multi-architecture:

# Docker
docker buildx build --platform linux/amd64,linux/arm64 -t myapp:latest --push .

# Podman
podman build --platform linux/amd64 -t myapp:amd64 .
podman manifest create myapp:latest
podman manifest add myapp:latest myapp:amd64

Analyze image:

docker history myapp:latest
dive myapp:latest

Multi-stage pattern:

FROM golang:1.21-alpine AS builder
# ... build ...
FROM alpine:3.19
COPY --from=builder /app/binary /

Optimize layers:

RUN apk add --no-cache package && \
    # work && \
    cleanup