aptly/README.md

aptly-mirror.sh

A comprehensive bash script to create and maintain aptly mirrors for Debian and Ubuntu repositories. Automate the mirroring of package repositories, create time-based snapshots, and publish merged snapshot sets to serve to clients.

Features

• Multi-distribution support: Mirror Debian 11/12/13 and Ubuntu 20.04/22.04/24.04/26.04
• Flexible mirroring: Mirror main, updates, and security streams separately
• Snapshot management: Create dated snapshots of mirrors and merge them
• Automatic publishing: Publish snapshots as APT repositories for client consumption
• Retention policy: Keep only N recent snapshot sets, automatically cleanup old ones
• GPG verification: Verify repository signatures during download and signing
• Bandwidth control: Optional download speed limiting
• Syslog integration: All operations logged to syslog for monitoring

Getting Started

Prerequisites

• aptly (>= 1.4.0): Install via your system package manager or from aptly.info
• gpg: For GPG key management (usually pre-installed)
• Bash 4+
• Sufficient disk space for mirror storage (50GB+ recommended depending on architecture count)

Installation

1. Clone or download the script:

   git clone <repository> aptly-mirror
   cd aptly-mirror
   

2. Create configuration:

   cp aptly-mirror.conf.example aptly-mirror.conf
   

3. Edit configuration for your environment:

   nano aptly-mirror.conf
   

   Key settings:

   • ARCHITECTURES: CPU architectures to mirror (e.g., "amd64" or "amd64 arm64")
   • APTLY_ROOT: Where aptly stores its database and mirrors (default: ~/.aptly)
   • PUBLISH_ROOT: Where published repos are served (default: ~/.aptly/public)
   • KEEP_SNAPSHOTS: Number of old snapshot sets to retain (default: 2)

4. Make the script executable:

   chmod +x aptly-mirror.sh
   

5. Import GPG keys (recommended for production):

   ./aptly-mirror.sh import-keys
   

6. Create initial mirrors:

   ./aptly-mirror.sh create
   

   This may take 10-30 minutes depending on your network and disk speed.

7. Create initial snapshots and publish:

   ./aptly-mirror.sh publish
   

Verify Setup

List all mirrors, snapshots, and published repositories:

./aptly-mirror.sh list

Check syslog for operation logs:

journalctl -t aptly-mirror -f

Manual

Commands

create — Initialize all mirrors

Create mirrors for all configured Debian and Ubuntu releases. Mirrors are created empty and must be populated with the first update command.

./aptly-mirror.sh create

Output:

• Creates mirrors for each release:
  • debian-{codename}-main
  • debian-{codename}-updates
  • debian-{codename}-security
  • Similar pattern for Ubuntu

Time: 1-2 minutes

update — Fetch latest packages and republish

Updates all mirrors to the latest packages, creates timestamped snapshots, merges them, and publishes a fresh snapshot set. This is the main command to run regularly.

./aptly-mirror.sh update

Process:

1. Updates all existing mirrors (downloads new packages)
2. Creates dated snapshots (e.g., debian-bullseye-main-20260408123045)
3. Merges main + updates + security snapshots for each release
4. Publishes or switches the published repository to the merged snapshot

Time: 5-30 minutes (depends on network, disk speed, and download limit)

Output example:

Repositories are published under /var/aptly/public/

Client sources.list entries:
  deb http://<server>/debian bullseye main contrib non-free
  deb http://<server>/debian bookworm main contrib non-free non-free-firmware
  deb http://<server>/ubuntu focal main restricted universe multiverse

publish — Snapshot and publish current state

Creates fresh snapshots of current mirrors without updating them. Useful when you want to take a snapshot without fetching new packages.

./aptly-mirror.sh publish

Use cases:

• Create a point-in-time snapshot after verifying upstream stability
• Re-publish without re-fetching packages

cleanup — Remove old snapshots and optimize database

Removes old snapshot sets beyond the KEEP_SNAPSHOTS retention limit and runs aptly's database cleanup.

./aptly-mirror.sh cleanup

Safety:

• Only removes snapshots that are not currently published
• Never removes the active published snapshot
• Safe to run after each update

list — Show all mirrors, snapshots, and published repos

Display the current state of all mirrors, snapshots, and published repositories.

./aptly-mirror.sh list

import-keys — Import GPG signing keys

Imports GPG keys for verifying Debian and Ubuntu repository signatures. Requires network access to GPG keyserver.

./aptly-mirror.sh import-keys

Notes:

• Only needs to run once during setup
• Requires ~/.gnupg/ to exist
• Keyserver can be customized via GPG_KEYSERVER in config

Options

-c <path> — Specify config file

./aptly-mirror.sh -c /etc/aptly-mirror.conf update

Config file search order (if -c not specified):

1. $APTLY_MIRROR_CONF environment variable
2. ./aptly-mirror.conf (same directory as script)
3. /etc/aptly-mirror.conf

-h, --help — Show usage

./aptly-mirror.sh --help

Configuration

All settings are in aptly-mirror.conf. See the included example for detailed comments.

Key settings:

Setting	Default	Description
ARCHITECTURES	amd64	Architectures to mirror (space-separated)
KEEP_SNAPSHOTS	2	Number of old snapshot sets to retain
DOWNLOAD_LIMIT	0	Speed limit in KiB/s (0 = unlimited)
IGNORE_SIGNATURES	0	Skip GPG verification (not recommended)
SKIP_SIGNING	0	Skip GPG signing when publishing
APTLY_ROOT	~/.aptly	Aptly database and mirrors location
PUBLISH_ROOT	~/.aptly/public	Published repositories root

Logging

All output is logged via syslog with tag aptly-mirror. View logs with:

# Follow in real-time
journalctl -t aptly-mirror -f

# View recent entries
journalctl -t aptly-mirror -n 50

# Search for errors
journalctl -t aptly-mirror | grep ERROR

Older systemd-free systems can check /var/log/syslog:

grep aptly-mirror /var/log/syslog

Sample Crontab

Run weekly updates every Sunday at 2 AM:

# Edit crontab
crontab -e

# Add this line:
0 2 * * 0 /home/dak/Code/aptly/aptly-mirror.sh update

Full recommended setup with logging and error notifications:

# Weekly update and cleanup
0 2 * * 0 /home/dak/Code/aptly/aptly-mirror.sh update && /home/dak/Code/aptly/aptly-mirror.sh cleanup

# Check syslog for errors (runs at 3 AM, 1 hour after update)
0 3 * * 0 journalctl -t aptly-mirror -n 100 | grep -i error && echo "aptly-mirror errors detected" | mail -s "aptly-mirror Alert" admin@example.com

Multiple updates per week:

# Every 12 hours (twice daily)
0 2,14 * * * /home/dak/Code/aptly/aptly-mirror.sh update

# Cleanup on Sundays
0 4 * * 0 /home/dak/Code/aptly/aptly-mirror.sh cleanup

For systemd systems (alternative to cron):

Create /etc/systemd/system/aptly-mirror.timer:

[Unit]
Description=Weekly aptly mirror update
After=network-online.target
Wants=network-online.target

[Timer]
OnCalendar=Sun *-*-* 02:00:00
Persistent=true

[Install]
WantedBy=timers.target

Create /etc/systemd/system/aptly-mirror.service:

[Unit]
Description=Update aptly mirrors
After=network-online.target

[Service]
Type=oneshot
ExecStart=/home/dak/Code/aptly/aptly-mirror.sh update
ExecStartPost=/home/dak/Code/aptly/aptly-mirror.sh cleanup
User=aptly
StandardOutput=journal
StandardError=journal

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable aptly-mirror.timer
sudo systemctl start aptly-mirror.timer

Troubleshooting

"Mirror already exists"

This is normal on repeated runs. The script skips existing mirrors.

GPG signature verification failures

Either:

• Import keys: ./aptly-mirror.sh import-keys
• Or skip verification (not recommended): set IGNORE_SIGNATURES=1 in config

Disk space exhausted

Monitor with:

du -sh ~/.aptly/

Reduce KEEP_SNAPSHOTS in config, or clean up old mirrors manually via aptly mirror drop.

Network timeout during update

Set a DOWNLOAD_LIMIT to avoid overwhelming the network, or run during off-peak hours.

Can't write to log

Ensure log directory exists and is writable. Check that LOG_DIR in config is accessible.

AI Attribution

AIA EAI Hin R Claude Code v1.0

This work was entirely AI-generated. AI was prompted for its contributions, or AI assistance was enabled. AI-generated content was reviewed and approved. The following model(s) or application(s) were used: Claude Code.

License

MIT