daniel/aptly
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

init

Browse Source
This commit is contained in:
Daniel Akulenok
2026-04-08 14:19:47 +02:00
commit ca436981e1
3 changed files with 943 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

333
README.md Normal file
View File

@@ -0,0 +1,333 @@
# aptly-mirror.sh
A comprehensive bash script to create and maintain aptly mirrors for Debian and Ubuntu repositories. Automate the mirroring of package repositories, create time-based snapshots, and publish merged snapshot sets to serve to clients.
## Features
- **Multi-distribution support**: Mirror Debian 11/12/13 and Ubuntu 20.04/22.04/24.04/26.04
- **Flexible mirroring**: Mirror main, updates, and security streams separately
- **Snapshot management**: Create dated snapshots of mirrors and merge them
- **Automatic publishing**: Publish snapshots as APT repositories for client consumption
- **Retention policy**: Keep only N recent snapshot sets, automatically cleanup old ones
- **GPG verification**: Verify repository signatures during download and signing
- **Bandwidth control**: Optional download speed limiting
- **Syslog integration**: All operations logged to syslog for monitoring
## Getting Started
### Prerequisites
- `aptly` (>= 1.4.0): Install via your system package manager or from [aptly.info](https://www.aptly.info/)
- `gpg`: For GPG key management (usually pre-installed)
- Bash 4+
- Sufficient disk space for mirror storage (50GB+ recommended depending on architecture count)
### Installation
1. **Clone or download the script**:
```bash
git clone <repository> aptly-mirror
cd aptly-mirror
```
2. **Create configuration**:
```bash
cp aptly-mirror.conf.example aptly-mirror.conf
```
3. **Edit configuration** for your environment:
```bash
nano aptly-mirror.conf
```
Key settings:
- `ARCHITECTURES`: CPU architectures to mirror (e.g., `"amd64"` or `"amd64 arm64"`)
- `APTLY_ROOT`: Where aptly stores its database and mirrors (default: `~/.aptly`)
- `PUBLISH_ROOT`: Where published repos are served (default: `~/.aptly/public`)
- `KEEP_SNAPSHOTS`: Number of old snapshot sets to retain (default: `2`)
4. **Make the script executable**:
```bash
chmod +x aptly-mirror.sh
```
5. **Import GPG keys** (recommended for production):
```bash
./aptly-mirror.sh import-keys
```
6. **Create initial mirrors**:
```bash
./aptly-mirror.sh create
```
This may take 10-30 minutes depending on your network and disk speed.
7. **Create initial snapshots and publish**:
```bash
./aptly-mirror.sh publish
```
### Verify Setup
List all mirrors, snapshots, and published repositories:
```bash
./aptly-mirror.sh list
```
Check syslog for operation logs:
```bash
journalctl -t aptly-mirror -f
```
## Manual
### Commands
#### `create` — Initialize all mirrors
Create mirrors for all configured Debian and Ubuntu releases. Mirrors are created empty and must be populated with the first `update` command.
```bash
./aptly-mirror.sh create
```
**Output**:
- Creates mirrors for each release:
- `debian-{codename}-main`
- `debian-{codename}-updates`
- `debian-{codename}-security`
- Similar pattern for Ubuntu
**Time**: 1-2 minutes
#### `update` — Fetch latest packages and republish
Updates all mirrors to the latest packages, creates timestamped snapshots, merges them, and publishes a fresh snapshot set. This is the main command to run regularly.
```bash
./aptly-mirror.sh update
```
**Process**:
1. Updates all existing mirrors (downloads new packages)
2. Creates dated snapshots (e.g., `debian-bullseye-main-20260408123045`)
3. Merges main + updates + security snapshots for each release
4. Publishes or switches the published repository to the merged snapshot
**Time**: 5-30 minutes (depends on network, disk speed, and download limit)
**Output example**:
```
Repositories are published under /var/aptly/public/
Client sources.list entries:
deb http://<server>/debian bullseye main contrib non-free
deb http://<server>/debian bookworm main contrib non-free non-free-firmware
deb http://<server>/ubuntu focal main restricted universe multiverse
```
#### `publish` — Snapshot and publish current state
Creates fresh snapshots of current mirrors without updating them. Useful when you want to take a snapshot without fetching new packages.
```bash
./aptly-mirror.sh publish
```
**Use cases**:
- Create a point-in-time snapshot after verifying upstream stability
- Re-publish without re-fetching packages
#### `cleanup` — Remove old snapshots and optimize database
Removes old snapshot sets beyond the `KEEP_SNAPSHOTS` retention limit and runs aptly's database cleanup.
```bash
./aptly-mirror.sh cleanup
```
**Safety**:
- Only removes snapshots that are not currently published
- Never removes the active published snapshot
- Safe to run after each update
#### `list` — Show all mirrors, snapshots, and published repos
Display the current state of all mirrors, snapshots, and published repositories.
```bash
./aptly-mirror.sh list
```
#### `import-keys` — Import GPG signing keys
Imports GPG keys for verifying Debian and Ubuntu repository signatures. Requires network access to GPG keyserver.
```bash
./aptly-mirror.sh import-keys
```
**Notes**:
- Only needs to run once during setup
- Requires `~/.gnupg/` to exist
- Keyserver can be customized via `GPG_KEYSERVER` in config
### Options
#### `-c <path>` — Specify config file
```bash
./aptly-mirror.sh -c /etc/aptly-mirror.conf update
```
Config file search order (if `-c` not specified):
1. `$APTLY_MIRROR_CONF` environment variable
2. `./aptly-mirror.conf` (same directory as script)
3. `/etc/aptly-mirror.conf`
#### `-h`, `--help` — Show usage
```bash
./aptly-mirror.sh --help
```
### Configuration
All settings are in `aptly-mirror.conf`. See the included example for detailed comments.
**Key settings**:
| Setting | Default | Description |
|---------|---------|-------------|
| `ARCHITECTURES` | `amd64` | Architectures to mirror (space-separated) |
| `KEEP_SNAPSHOTS` | `2` | Number of old snapshot sets to retain |
| `DOWNLOAD_LIMIT` | `0` | Speed limit in KiB/s (0 = unlimited) |
| `IGNORE_SIGNATURES` | `0` | Skip GPG verification (not recommended) |
| `SKIP_SIGNING` | `0` | Skip GPG signing when publishing |
| `APTLY_ROOT` | `~/.aptly` | Aptly database and mirrors location |
| `PUBLISH_ROOT` | `~/.aptly/public` | Published repositories root |
### Logging
All output is logged via syslog with tag `aptly-mirror`. View logs with:
```bash
# Follow in real-time
journalctl -t aptly-mirror -f
# View recent entries
journalctl -t aptly-mirror -n 50
# Search for errors
journalctl -t aptly-mirror | grep ERROR
```
Older systemd-free systems can check `/var/log/syslog`:
```bash
grep aptly-mirror /var/log/syslog
```
## Sample Crontab
Run weekly updates every Sunday at 2 AM:
```bash
# Edit crontab
crontab -e
# Add this line:
0 2 * * 0 /home/dak/Code/aptly/aptly-mirror.sh update
```
**Full recommended setup** with logging and error notifications:
```bash
# Weekly update and cleanup
0 2 * * 0 /home/dak/Code/aptly/aptly-mirror.sh update && /home/dak/Code/aptly/aptly-mirror.sh cleanup
# Check syslog for errors (runs at 3 AM, 1 hour after update)
0 3 * * 0 journalctl -t aptly-mirror -n 100 | grep -i error && echo "aptly-mirror errors detected" | mail -s "aptly-mirror Alert" admin@example.com
```
**Multiple updates per week**:
```bash
# Every 12 hours (twice daily)
0 2,14 * * * /home/dak/Code/aptly/aptly-mirror.sh update
# Cleanup on Sundays
0 4 * * 0 /home/dak/Code/aptly/aptly-mirror.sh cleanup
```
**For systemd systems** (alternative to cron):
Create `/etc/systemd/system/aptly-mirror.timer`:
```ini
[Unit]
Description=Weekly aptly mirror update
After=network-online.target
Wants=network-online.target
[Timer]
OnCalendar=Sun *-*-* 02:00:00
Persistent=true
[Install]
WantedBy=timers.target
```
Create `/etc/systemd/system/aptly-mirror.service`:
```ini
[Unit]
Description=Update aptly mirrors
After=network-online.target
[Service]
Type=oneshot
ExecStart=/home/dak/Code/aptly/aptly-mirror.sh update
ExecStartPost=/home/dak/Code/aptly/aptly-mirror.sh cleanup
User=aptly
StandardOutput=journal
StandardError=journal
```
Enable and start:
```bash
sudo systemctl daemon-reload
sudo systemctl enable aptly-mirror.timer
sudo systemctl start aptly-mirror.timer
```
## Troubleshooting
### "Mirror already exists"
This is normal on repeated runs. The script skips existing mirrors.
### GPG signature verification failures
Either:
- Import keys: `./aptly-mirror.sh import-keys`
- Or skip verification (not recommended): set `IGNORE_SIGNATURES=1` in config
### Disk space exhausted
Monitor with:
```bash
du -sh ~/.aptly/
```
Reduce `KEEP_SNAPSHOTS` in config, or clean up old mirrors manually via `aptly mirror drop`.
### Network timeout during update
Set a `DOWNLOAD_LIMIT` to avoid overwhelming the network, or run during off-peak hours.
### Can't write to log
Ensure log directory exists and is writable. Check that `LOG_DIR` in config is accessible.
## AI Attribution
AIA EAI Hin R Claude Code v1.0
This work was entirely AI-generated. AI was prompted for its contributions, or AI assistance was enabled. AI-generated content was reviewed and approved. The following model(s) or application(s) were used: Claude Code.
## License
MIT

105
aptly-mirror.conf Normal file
View File

@@ -0,0 +1,105 @@
# ---------------------------------------------------------------------------
# aptly-mirror.conf — Configuration for aptly-mirror.sh
# ---------------------------------------------------------------------------
#
# This file is sourced by aptly-mirror.sh. It uses plain bash syntax.
# To override the default location, set APTLY_MIRROR_CONF or pass -c <path>.
#
# ---------------------------------------------------------------------------
# General settings
# ---------------------------------------------------------------------------
# Architectures to mirror (space-separated for multiple, e.g. "amd64 arm64")
ARCHITECTURES="amd64"
# GPG keyring for repository signature verification (empty = default trustedkeys.gpg)
GPG_KEYRING=""
# Set to 1 to skip GPG signature verification (not recommended for production)
IGNORE_SIGNATURES=0
# Number of old snapshot sets to retain during cleanup (0 = keep only current)
KEEP_SNAPSHOTS=2
# Download speed limit in KiB/s (0 = unlimited)
DOWNLOAD_LIMIT=0
# Set to 1 to skip GPG signing when publishing (useful if no GPG key configured)
SKIP_SIGNING=0
# ---------------------------------------------------------------------------
# Storage paths
# ---------------------------------------------------------------------------
# Root directory for aptly data (mirrors, snapshots, db)
# Leave empty to use aptly's default (~/.aptly)
APTLY_ROOT=""
# Directory where published repositories are served from
# Leave empty to use aptly's default (~/.aptly/public)
PUBLISH_ROOT=""
# ---------------------------------------------------------------------------
# Debian mirrors
# ---------------------------------------------------------------------------
DEBIAN_MIRROR="http://deb.debian.org/debian"
DEBIAN_SECURITY_MIRROR="http://security.debian.org/debian-security"
# Debian releases: codename -> version number
declare -A DEBIAN_RELEASES=(
[bullseye]="11"
[bookworm]="12"
[trixie]="13"
)
# Components for Debian 11 and earlier (before non-free-firmware split)
DEBIAN_COMPONENTS_LEGACY="main contrib non-free"
# Components for Debian 12+ (non-free-firmware is a separate component)
DEBIAN_COMPONENTS_MODERN="main contrib non-free non-free-firmware"
# Security mirror components use an "updates/" prefix (applies to all Debian releases)
DEBIAN_SECURITY_COMPONENTS_LEGACY="updates/main updates/contrib updates/non-free"
DEBIAN_SECURITY_COMPONENTS_MODERN="updates/main updates/contrib updates/non-free updates/non-free-firmware"
# ---------------------------------------------------------------------------
# Ubuntu mirrors
# ---------------------------------------------------------------------------
UBUNTU_MIRROR="http://archive.ubuntu.com/ubuntu"
UBUNTU_SECURITY_MIRROR="http://security.ubuntu.com/ubuntu"
# Ubuntu releases: codename -> version number
declare -A UBUNTU_RELEASES=(
[focal]="20.04"
[jammy]="22.04"
[noble]="24.04"
[resolute]="26.04"
)
UBUNTU_COMPONENTS="main restricted universe multiverse"
# ---------------------------------------------------------------------------
# GPG keys
# ---------------------------------------------------------------------------
# Debian archive signing key fingerprints
DEBIAN_GPG_KEYS=(
"1F89983E0081FDE018F3CC9673A4F27B8DD47936" # Debian Archive (11/bullseye)
"A7236886F3CCCAAD148A27F80E98404D386FA1D9" # Debian Archive (12/bookworm)
"B7C5D7829D669F88788D4C1BEFCC6169C4AE0E3C" # Debian Security Archive
"4D64FEC119C2029067D6E791F8D2585B8783D481" # Debian Stable Release Key
"A48449044AAD5C5DEEDE9B19B7B0C1CF28EA3E34" # Debian Archive (13/trixie)
"89C87ACEA5DD6B8E6A7068808E9F831205B4BA95" # Debian Security Archive (13/trixie)
)
# Ubuntu archive signing key fingerprints
UBUNTU_GPG_KEYS=(
"3B4FE6ACC0B21F32" # Ubuntu Archive
"871920D1991BC93C" # Ubuntu Archive (older)
)
# GPG keyserver to fetch keys from
GPG_KEYSERVER="hkps://keyserver.ubuntu.com"

505
aptly-mirror.sh Executable file
View File

@@ -0,0 +1,505 @@
#!/usr/bin/env bash
#
# aptly-mirror.sh — Create and maintain aptly mirrors for Debian and Ubuntu repositories.
#
# Mirrors:
# Debian: 11 (bullseye), 12 (bookworm), 13 (trixie)
# Ubuntu: 20.04 (focal), 22.04 (jammy), 24.04 (noble), 26.04 (roaring)
#
# Usage:
# ./aptly-mirror.sh create — Create all mirrors (first-time setup)
# ./aptly-mirror.sh update — Update all mirrors, snapshot, merge, and publish
# ./aptly-mirror.sh publish — Snapshot current state and publish/switch
# ./aptly-mirror.sh cleanup — Drop old snapshots and run db cleanup
# ./aptly-mirror.sh list — List all mirrors, snapshots, and published repos
#
set -euo pipefail
# ---------------------------------------------------------------------------
# Load configuration
# ---------------------------------------------------------------------------
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Config file search order:
# 1. -c <path> command-line flag
# 2. APTLY_MIRROR_CONF environment variable
# 3. ./aptly-mirror.conf (next to the script)
# 4. /etc/aptly-mirror.conf
find_config() {
if [[ -n "${APTLY_MIRROR_CONF:-}" && -r "$APTLY_MIRROR_CONF" ]]; then
echo "$APTLY_MIRROR_CONF"
elif [[ -r "${SCRIPT_DIR}/aptly-mirror.conf" ]]; then
echo "${SCRIPT_DIR}/aptly-mirror.conf"
elif [[ -r "/etc/aptly-mirror.conf" ]]; then
echo "/etc/aptly-mirror.conf"
else
echo ""
fi
}
CONFIG_FILE=""
# Parse flags before subcommand
while [[ "${1:-}" == -* ]]; do
case "$1" in
-c) CONFIG_FILE="$2"; shift 2 ;;
-h|--help) CONFIG_FILE="__help__"; break ;;
*) break ;;
esac
done
if [[ -z "$CONFIG_FILE" ]]; then
CONFIG_FILE=$(find_config)
fi
if [[ "$CONFIG_FILE" == "__help__" ]]; then
# Defer to main() which will show usage
:
elif [[ -z "$CONFIG_FILE" ]]; then
echo "Error: No config file found." >&2
echo "Searched: \$APTLY_MIRROR_CONF, ${SCRIPT_DIR}/aptly-mirror.conf, /etc/aptly-mirror.conf" >&2
echo "Create one from aptly-mirror.conf.example or pass -c <path>." >&2
exit 1
else
# shellcheck source=aptly-mirror.conf
source "$CONFIG_FILE"
fi
DATE=$(date +%Y%m%d%H%M%S)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
log() {
local msg="$*"
echo "$msg"
logger -t aptly-mirror "$msg"
}
die() {
log "FATAL: $*"
exit 1
}
run_aptly() {
log " -> aptly $*"
if ! aptly "$@" 2>&1; then
log " !! aptly $1 failed (exit $?)"
return 1
fi
}
mirror_exists() {
aptly mirror show "$1" &>/dev/null
}
snapshot_exists() {
aptly snapshot show "$1" &>/dev/null
}
published_exists() {
# $1 = distribution, $2 = prefix (default ".")
local dist="$1" prefix="${2:-.}"
aptly publish list -raw 2>/dev/null | grep -qE "^${prefix} ${dist}$"
}
common_mirror_flags() {
local flags="-architectures=${ARCHITECTURES}"
if [[ -n "$GPG_KEYRING" ]]; then
flags+=" -keyring=${GPG_KEYRING}"
fi
if (( IGNORE_SIGNATURES )); then
flags+=" -ignore-signatures"
fi
echo "$flags"
}
common_update_flags() {
local flags=""
if (( DOWNLOAD_LIMIT > 0 )); then
flags+=" -download-limit=${DOWNLOAD_LIMIT}"
fi
if (( IGNORE_SIGNATURES )); then
flags+=" -ignore-signatures"
fi
echo "$flags"
}
publish_flags() {
local flags=""
if (( SKIP_SIGNING )); then
flags+=" -skip-signing"
fi
echo "$flags"
}
# ---------------------------------------------------------------------------
# get_debian_components <codename>
# ---------------------------------------------------------------------------
get_debian_components() {
local codename="$1"
case "$codename" in
bullseye) echo "$DEBIAN_COMPONENTS_LEGACY" ;;
*) echo "$DEBIAN_COMPONENTS_MODERN" ;;
esac
}
get_debian_security_components() {
local codename="$1"
case "$codename" in
bullseye) echo "$DEBIAN_SECURITY_COMPONENTS_LEGACY" ;;
*) echo "$DEBIAN_SECURITY_COMPONENTS_MODERN" ;;
esac
}
# ---------------------------------------------------------------------------
# CREATE — set up all mirrors
# ---------------------------------------------------------------------------
create_debian_mirrors() {
local codename version components security_components
for codename in "${!DEBIAN_RELEASES[@]}"; do
version="${DEBIAN_RELEASES[$codename]}"
components=$(get_debian_components "$codename")
security_components=$(get_debian_security_components "$codename")
log "Creating Debian ${version} (${codename}) mirrors..."
local name="debian-${codename}-main"
if ! mirror_exists "$name"; then
run_aptly mirror create $(common_mirror_flags) \
"$name" "$DEBIAN_MIRROR" "$codename" $components
else
log " Mirror $name already exists, skipping."
fi
name="debian-${codename}-updates"
if ! mirror_exists "$name"; then
run_aptly mirror create $(common_mirror_flags) \
"$name" "$DEBIAN_MIRROR" "${codename}-updates" $components
else
log " Mirror $name already exists, skipping."
fi
name="debian-${codename}-security"
if ! mirror_exists "$name"; then
run_aptly mirror create $(common_mirror_flags) \
"$name" "$DEBIAN_SECURITY_MIRROR" "${codename}-security" $security_components
else
log " Mirror $name already exists, skipping."
fi
done
}
create_ubuntu_mirrors() {
local codename version
for codename in "${!UBUNTU_RELEASES[@]}"; do
version="${UBUNTU_RELEASES[$codename]}"
log "Creating Ubuntu ${version} (${codename}) mirrors..."
local name="ubuntu-${codename}-main"
if ! mirror_exists "$name"; then
run_aptly mirror create $(common_mirror_flags) \
"$name" "$UBUNTU_MIRROR" "$codename" $UBUNTU_COMPONENTS
else
log " Mirror $name already exists, skipping."
fi
name="ubuntu-${codename}-updates"
if ! mirror_exists "$name"; then
run_aptly mirror create $(common_mirror_flags) \
"$name" "$UBUNTU_MIRROR" "${codename}-updates" $UBUNTU_COMPONENTS
else
log " Mirror $name already exists, skipping."
fi
name="ubuntu-${codename}-security"
if ! mirror_exists "$name"; then
run_aptly mirror create $(common_mirror_flags) \
"$name" "$UBUNTU_SECURITY_MIRROR" "${codename}-security" $UBUNTU_COMPONENTS
else
log " Mirror $name already exists, skipping."
fi
done
}
do_create() {
log "=== Creating all mirrors ==="
create_debian_mirrors
create_ubuntu_mirrors
log "=== Mirror creation complete ==="
}
# ---------------------------------------------------------------------------
# UPDATE — fetch latest package metadata and files
# ---------------------------------------------------------------------------
update_all_mirrors() {
local mirror_name
log "=== Updating all mirrors ==="
while IFS= read -r mirror_name; do
[[ -z "$mirror_name" ]] && continue
log "Updating mirror: ${mirror_name}"
run_aptly mirror update $(common_update_flags) "$mirror_name" || \
log " !! Warning: failed to update ${mirror_name}, continuing..."
done < <(aptly mirror list -raw 2>/dev/null)
log "=== Mirror updates complete ==="
}
# ---------------------------------------------------------------------------
# SNAPSHOT & PUBLISH — create dated snapshots, merge, and publish/switch
# ---------------------------------------------------------------------------
snapshot_and_publish_debian() {
local codename version snap_main snap_updates snap_security snap_merged
local pub_flags
pub_flags=$(publish_flags)
for codename in "${!DEBIAN_RELEASES[@]}"; do
version="${DEBIAN_RELEASES[$codename]}"
log "Snapshotting Debian ${version} (${codename})..."
snap_main="debian-${codename}-main-${DATE}"
snap_updates="debian-${codename}-updates-${DATE}"
snap_security="debian-${codename}-security-${DATE}"
snap_merged="debian-${codename}-merged-${DATE}"
run_aptly snapshot create "$snap_main" from mirror "debian-${codename}-main"
run_aptly snapshot create "$snap_updates" from mirror "debian-${codename}-updates"
run_aptly snapshot create "$snap_security" from mirror "debian-${codename}-security"
log "Merging snapshots for Debian ${codename}..."
run_aptly snapshot merge -latest \
"$snap_merged" "$snap_main" "$snap_updates" "$snap_security"
local prefix="debian"
if published_exists "$codename" "$prefix"; then
log "Switching published Debian ${codename} to ${snap_merged}..."
run_aptly publish switch $pub_flags "$codename" "$prefix" "$snap_merged"
else
log "Publishing Debian ${codename} for the first time..."
run_aptly publish snapshot $pub_flags \
-distribution="$codename" -architectures="$ARCHITECTURES" \
"$snap_merged" "$prefix"
fi
done
}
snapshot_and_publish_ubuntu() {
local codename version snap_main snap_updates snap_security snap_merged
local pub_flags
pub_flags=$(publish_flags)
for codename in "${!UBUNTU_RELEASES[@]}"; do
version="${UBUNTU_RELEASES[$codename]}"
log "Snapshotting Ubuntu ${version} (${codename})..."
snap_main="ubuntu-${codename}-main-${DATE}"
snap_updates="ubuntu-${codename}-updates-${DATE}"
snap_security="ubuntu-${codename}-security-${DATE}"
snap_merged="ubuntu-${codename}-merged-${DATE}"
run_aptly snapshot create "$snap_main" from mirror "ubuntu-${codename}-main"
run_aptly snapshot create "$snap_updates" from mirror "ubuntu-${codename}-updates"
run_aptly snapshot create "$snap_security" from mirror "ubuntu-${codename}-security"
log "Merging snapshots for Ubuntu ${codename}..."
run_aptly snapshot merge -latest \
"$snap_merged" "$snap_main" "$snap_updates" "$snap_security"
local prefix="ubuntu"
if published_exists "$codename" "$prefix"; then
log "Switching published Ubuntu ${codename} to ${snap_merged}..."
run_aptly publish switch $pub_flags "$codename" "$prefix" "$snap_merged"
else
log "Publishing Ubuntu ${codename} for the first time..."
run_aptly publish snapshot $pub_flags \
-distribution="$codename" -architectures="$ARCHITECTURES" \
"$snap_merged" "$prefix"
fi
done
}
do_publish() {
log "=== Snapshot and publish ==="
snapshot_and_publish_debian
snapshot_and_publish_ubuntu
log "=== Publish complete ==="
log ""
log "Repositories are published under ${PUBLISH_ROOT:-~/.aptly/public}/"
log ""
log "Client sources.list entries:"
for codename in "${!DEBIAN_RELEASES[@]}"; do
log " deb http://<server>/debian ${codename} $(get_debian_components "$codename")"
done
for codename in "${!UBUNTU_RELEASES[@]}"; do
log " deb http://<server>/ubuntu ${codename} ${UBUNTU_COMPONENTS}"
done
}
# ---------------------------------------------------------------------------
# UPDATE (full) — update mirrors then snapshot & publish
# ---------------------------------------------------------------------------
do_update() {
update_all_mirrors
do_publish
}
# ---------------------------------------------------------------------------
# CLEANUP — drop old snapshots, run db cleanup
# ---------------------------------------------------------------------------
do_cleanup() {
log "=== Cleanup ==="
# Collect all snapshot names grouped by prefix (everything before the timestamp)
declare -A snap_groups
while IFS= read -r snap; do
[[ -z "$snap" ]] && continue
# Snapshot names follow: <prefix>-YYYYMMDDHHMMSS
# Strip the 14-digit timestamp suffix to get the group key
local prefix="${snap%-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]}"
if [[ "$prefix" == "$snap" ]]; then
# No timestamp suffix — skip (manually created snapshot)
continue
fi
snap_groups["$prefix"]+="${snap}"$'\n'
done < <(aptly snapshot list -raw -sort=name 2>/dev/null)
local dropped=0
for prefix in "${!snap_groups[@]}"; do
# Sort snapshots by name (timestamp order) and keep the newest KEEP_SNAPSHOTS
local snaps
snaps=$(echo -n "${snap_groups[$prefix]}" | sort)
local count
count=$(echo "$snaps" | wc -l)
local to_drop=$(( count - KEEP_SNAPSHOTS ))
if (( to_drop > 0 )); then
echo "$snaps" | head -n "$to_drop" | while IFS= read -r old_snap; do
[[ -z "$old_snap" ]] && continue
log "Dropping snapshot: ${old_snap}"
run_aptly snapshot drop "$old_snap" || \
log " !! Could not drop ${old_snap} (may be published or referenced)"
(( dropped++ )) || true
done
fi
done
log "Running database cleanup..."
run_aptly db cleanup
log "=== Cleanup complete ==="
}
# ---------------------------------------------------------------------------
# LIST — show current state
# ---------------------------------------------------------------------------
do_list() {
echo "==============================="
echo " Mirrors"
echo "==============================="
aptly mirror list 2>/dev/null || true
echo ""
echo "==============================="
echo " Snapshots"
echo "==============================="
aptly snapshot list -sort=time 2>/dev/null || true
echo ""
echo "==============================="
echo " Published Repositories"
echo "==============================="
aptly publish list 2>/dev/null || true
}
# ---------------------------------------------------------------------------
# GPG key import helper
# ---------------------------------------------------------------------------
do_import_keys() {
log "=== Importing repository signing keys ==="
local keyserver="${GPG_KEYSERVER:-hkps://keyserver.ubuntu.com}"
log "Importing Debian GPG keys..."
for key in "${DEBIAN_GPG_KEYS[@]}"; do
gpg --no-default-keyring --keyring trustedkeys.gpg \
--keyserver "$keyserver" --recv-keys "$key" 2>&1 || \
log " !! Warning: could not import key ${key}"
done
log "Importing Ubuntu GPG keys..."
for key in "${UBUNTU_GPG_KEYS[@]}"; do
gpg --no-default-keyring --keyring trustedkeys.gpg \
--keyserver "$keyserver" --recv-keys "$key" 2>&1 || \
log " !! Warning: could not import key ${key}"
done
log "=== Key import complete ==="
}
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
usage() {
cat <<USAGE
Usage: aptly-mirror.sh [-c <config>] <command>
Commands:
create Create all mirrors (first-time setup)
update Update mirrors, snapshot, merge, and publish
publish Snapshot current mirror state and publish/switch
cleanup Drop old snapshots and run aptly db cleanup
list List all mirrors, snapshots, and published repos
import-keys Import GPG keys for Debian and Ubuntu repositories
Options:
-c <path> Path to config file (default: auto-detected)
Configuration:
The config file is searched in this order:
1. -c <path> flag
2. \$APTLY_MIRROR_CONF environment variable
3. ${SCRIPT_DIR}/aptly-mirror.conf
4. /etc/aptly-mirror.conf
USAGE
}
main() {
if (( $# < 1 )); then
usage
exit 1
fi
local cmd="$1"; shift
case "$cmd" in
create) do_create ;;
update) do_update ;;
publish) do_publish ;;
cleanup) do_cleanup ;;
list) do_list ;;
import-keys) do_import_keys ;;
-h|--help|help)
usage
exit 0
;;
*)
echo "Unknown command: $cmd" >&2
usage
exit 1
;;
esac
}
main "$@"