commit 5a794864d1eca8ee99e871be66cf0f4de41bac6d
parent 4df238a6568e6da97df0eb9c03877c3fc50dbfe8
Author: Florian Dold <florian@dold.me>
Date: Wed, 26 Nov 2025 23:33:50 +0100
update DD43
Diffstat:
1 file changed, 29 insertions(+), 35 deletions(-)
diff --git a/design-documents/043-managing-prebuilt-artifacts.rst b/design-documents/043-managing-prebuilt-artifacts.rst
@@ -50,47 +50,44 @@ prebuilt files are not managed alongside the source code. Instead, we use
(orphan-)branches) in a (possibly separate) repository to manage them. This
allows us to re-use Git authentication and Git commit signing.
-Due to a historical accident, prebuilt files are currently stored in the
-``prebuilt`` branch of the ``wallet-core.git`` repository. This might change
-in the future.
-
-To enable forwards-compatibility, we are implementing the following approach:
-The ``taler.net`` HTTP server redirects requests in the form of
-``https://artifacts.taler.net/$component/$version/$file`` to
-``https://git.taler.net/$repo.git/tree/$version/$file?h=$branch``, where
-``$component`` determines ``$repo`` and ``$branch``.
-
-We are also introducing the new rule that a prebuilt artifact must be a single
-archive file (preferable ``.tar.gz.``).
-
Repositories that produce artifacts *MUST* have phony Makefile helper targets
to (a) create the prebuilt artifact (b) check out the prebuilt branch as a Git worktree, (c)
move the artifact to the right place in the prebuilt branch and commit/push.
-Repositories that depend on prebuilt components must download them
-in their ``./bootstrap`` script. After downloading, the bootstrap
-script *SHOULD* verify a checksum of the downloaded artifact.
+Prebuilt files are consumed via git submodules. The following rules apply for consumers:
+
+* The ``./bootstrap`` script of a component must initialize the submodule.
+* The ``./bootstrap`` script *SHOULD* fall back gracefully when the
+ submodule commit of the prebuilt submodule is not available anymore.
+* The consuming module *MUST* rely on a particular commit of the prebuilt
+ branch and the commit *MUST* be tagged to enable sparse checkouts of the submodule.
+* Consumers *SHOULD* use sparse checkouts for prebuilt submodules.
+
+Layout of the prebuilt branch and prebuilt tags:
+
+* The prebult files for a component are under ``{component_name}/{version}/``
+* The ``version`` *MUST* correspond both to the current version number of the
+ component and a tag of the source repo of ``{component_name}``.
+* The prebuilt branch *MUST* have a tag of the form ``prebuilt-{serial}`` for
+ every version that may be consumed by another component.
+* The prebuilt branch *MAY* have additional tags of the form
+ ``{component_name}/{version}`` to indicate that a particular version of a
+ particular component was added as a prebuilt version.
-FIXME: Link to an example bootstrap file, once the approach from this document
-is fully implemented.
Alternatives
============
-* The prebuilt branch could be integrated into source repositories that
- depend on it via a git submodule. This has the following disadvantages:
-
- 1. Git submodules are difficult to handle for some developers. We've had issues
- in the past where the prebuilt submodule wasn't integrated properly and would
- lead to non-reproducible builds
- 2. Relying purely on the Git commit hash of the prebuilt branch
- makes it very difficult to know which version of the prebuilt
- artifact is being used. It also makes it very difficult
- to reliable find older versions of the artifact.
- 3. Relying on the commit hash instead of a version and artifact hash
- couples the artifact to the Git commit. That means that
- we can never compact the commit history of the prebuilt
- branch of move artifacts around.
+* Only maintain one version per module.
+
+ * We were doing this (as of 2025-10-26) and it doesn't work very well,
+ because the same package (say ``exchange.git``) might depend
+ on different versions of prebuilt packages (say ``aml-spa`` and ``kyc-spa``).
+ Thus the prebuilt branch would need a commit with exactly the right combination
+ of versions, which is hard to do correctly.
+
+ * This is okay for prebuilt branches that only contain one
+ component, such as taler-docs.git.
* Set up a full-blown artifact management solution like JFrog Artifactory.
That's costly and requires a lot of admin work.
@@ -98,9 +95,6 @@ Alternatives
* Never rely on prebuilt files. That slows down the build process
and in some cases requires exotic dependencies.
-Drawbacks
-=========
-
Discussion / Q&A
================
