Overview/xpkg — package builder

xpkg - Architecture

How the project is structured, how a package is produced, and the formats it reads and writes.

Related documents in this folder:

• 01 - Overview and status
• 02 - Usage
• 04 - Future integration

Cargo workspace layout

xpkg is a Cargo workspace (edition 2021) with two crates:

The workspace root Cargo.toml centralises shared dependencies and metadata (version 0.1.0, edition 2021, GPL-3.0-or-later, org xlnux). Notable third-party dependencies: clap (CLI), serde/serde_json/toml, thiserror/anyhow (errors), tracing (logging), ureq (HTTP), sha2 (checksums), flate2/tar/xz2/bzip2/zstd/zip (archives), sequoia-openpgp (signing), tempfile (tests).

Modules in xpkg-core

The build pipeline

xpkg build orchestrates, in order:

1. Parse and validate the recipe (XBUILD or PKGBUILD).
2. Apply CLI overrides for builddir/outdir.
3. Set up isolated build directories and environment.
4. Run the build phases: prepare then build then check (optional) then package, executing each recipe phase as shell scripts.
5. Strip ELF binaries (if strip_binaries = true).
6. Create the .xp archive (tar.zst by default).
7. Sign the package (if --sign or sign = true in config).

Rootless packaging

The package() phase writes into a fakeroot context so files are recorded with uid=0/gid=0 without real root privileges. xpkg uses a 3-layer fallback: unshare --user (kernel namespaces, Linux >= 3.8) when available, otherwise the fakeroot tool, otherwise direct execution with tar header rewriting.

Environment

The builder sets PKGDIR, SRCDIR, BUILDDIR, MAKEFLAGS, CFLAGS, CXXFLAGS and LDFLAGS for the phase scripts. The package() phase must install everything into $PKGDIR (never /).

The .xp package format

.xp is an ALPM-compatible compressed tar archive (tar.zst by default). At the archive root it carries the metadata files generated by the metadata module:

package-1.0-1-x86_64.xp (tar.zst)
+-- .PKGINFO       package identity, version, dependencies, sizes
+-- .BUILDINFO     build environment record (packager, builddate, toolchain)
+-- .MTREE          file integrity manifest (hashes, permissions, ownership, symlinks)
+-- .INSTALL        optional pre/post install/upgrade/remove hook scripts
+-- usr/            installed file tree
+-- ...

Optional signing produces an OpenPGP detached signature file next to the archive (package-...-x86_64.xp.sig).

Repository database format

A repository is a set of .xp packages plus a database index that xpm can query: an ALPM-compatible compressed tar archive (.db.tar.zst by default; .db.tar.gz and .db.tar.xz are auto-detected). Inside, one directory per package holds desc (package metadata, %FILENAME%, %NAME%, %VERSION%, %DESC%, sizes, checksum, ...) and depends (dependency information), in an ALPM-compatible key-value format. The repo module reads/writes these databases and can generate a static repository layout for HTTP hosting.

The XBUILD recipe format

XBUILD is the native TOML recipe format (file XBUILD, TOML v1.0, UTF-8), structured into four top-level sections. See the full XBUILD Specification.

Example:

[package]
name = "hello"
version = "2.12"
release = 1
description = "GNU Hello - the friendly greeter"
url = "https://www.gnu.org/software/hello/"
license = ["GPL-3.0-or-later"]
arch = ["x86_64"]

[dependencies]
depends = ["glibc"]
makedepends = ["gcc", "make"]

[source]
urls = ["https://ftp.gnu.org/gnu/hello/hello-2.12.tar.gz"]
sha256sums = ["cf04af86dc085268c5f4470fbae49b18afbc221b78096aab842d934a76bad0ab"]

[build]
build = """
cd hello-2.12
./configure --prefix=/usr
make
"""
package = """
cd hello-2.12
make DESTDIR=$PKGDIR install
"""

Validation rules applied by the parser: name must follow the naming rules (lowercase ASCII start, lowercase letters/digits/hyphens/underscores, max 128); version non-empty; release >= 1; arch in x86_64, aarch64, i686, armv7h, any; source URL schemes in http, https, ftp, file; checksum arrays must match the urls length. Errors are collected and reported together, not one by one.

PKGBUILD compatibility

xpkg build --pkgbuild parses legacy Arch Linux PKGBUILD bash scripts and extracts variables (pkgname, pkgver, pkgrel, depends arrays, source, sha256sums) and functions (prepare, build, check, package) for migration from the Arch ecosystem.

Source handling

Declared sources are downloaded (HTTP/HTTPS/FTP/file via ureq, Git repos via the system git), verified with SHA-256 and/or SHA-512 (each entry index-matched to urls, SKIP bypasses), extracted by extension (tar.gz/tgz, tar.xz/txz, tar.bz2/tbz2, tar.zst/tzst, zip; other files kept as-is), and cached under $XDG_CACHE_HOME/xpkg/sources/ (keyed by a truncated SHA-256 of the URL) to avoid re-downloads. See Source Management.

Configuration model

Configuration lives in ~/.config/xpkg/xpkg.conf (TOML) with [options], [environment] and [lint] sections, loaded at startup and used to build an XpkgConfig. Subcommands clone the loaded config and apply CLI overrides before acting. See etc/xpkg.conf.example for all options.

Self-hosting recipe

The repository carries its own build recipe at packaging/xpkg/XBUILD: it copies the repo tree into the source dir, runs cargo build -p xpkg --release --locked, and installs the binary, license, README and etc/xpkg.conf.example into the package - an example of a [source]-free local recipe.

Edit this page on GitHub