1 files changed, 107 insertions, 0 deletions
diff --git a/design-documents/043-managing-prebuilt-artifacts.rst b/design-documents/043-managing-prebuilt-artifacts.rst
new file mode 100644
index 00000000..3d0debc8
--- /dev/null
+++ b/design-documents/043-managing-prebuilt-artifacts.rst
@@ -0,0 +1,107 @@
+DD 43: Managing Prebuilt Artifacts and Source-Level Dependencies
+################################################################
+
+Summary
+=======
+
+This design document defines how the GNU Taler project manages prebuilt
+artifacts and other source-level dependencies in repositories
+that are part of project.
+
+Motivation
+==========
+
+Some repositories have source-level dependencies on the build results of other
+repositories.  While it is possible to build these dependencies from scratch,
+we often want to avoid that extra step in order to make building the software
+easier and faster.
+
+Examples are:
+
+* man-pages built via Sphinx, produced in ``docs.git`` and consumed in ``exchange.git``.
+* SPAs used by the exchange, merchant libeufin.
+* (formerly): The ``taler-wallet-core-embedded.js`` file used
+  by the Android repo.
+
+Another type of source-level dependency is on other **source** files.
+Examples for this are:
+
+* The ``build-common.git`` repository that contains common build-time helpers
+* The ``gana.git`` repository 
+
+Requirements
+============
+
+* We are a free software project, users must always be able to
+  easily re-build the prebuilt artifacts for themselves.
+* We want to reduce reliance on third party infrastructure as much
+  as possible.
+* We want to keep our own infrastructure as slim as possible.
+* Builds must be reproducible and remain reproducible in the future.
+* Offline builds should be supported as well as possible.
+* Ideally, prebuilt artifacts can easily be cached / archived by third parties.
+
+Proposed Solution
+=================
+
+Instead of using a full-blown artifact management solution from the start, we
+make use of Git.  Since Git is not very good at managing binary artifacts,
+prebuilt files are not managed alongside the source code. Instead, we use
+(orphan-)branches) in a (possibly separate) repository to manage them.  This
+allows us to re-use Git authentication and Git commit signing.
+
+Due to a historical accident, prebuilt files are currently stored in the
+``prebuilt`` branch of the ``wallet-core.git`` repository.  This might change
+in the future.
+
+To enable forwards-compatibility, we are implementing the following approach:
+The ``taler.net`` HTTP server redirects requests in the form of
+``https://artifacts.taler.net/$component/$version/$file`` to
+``https://git.taler.net/$repo.git/tree/$version/$file?h=$branch``, where
+``$component`` determines ``$repo`` and ``$branch``.
+
+We are also introducing the new rule that a prebuilt artifact must be a single
+archive file (preferable ``.tar.gz.``).
+
+Repositories that produce artifacts *MUST* have phony Makefile helper targets
+to (a) create the prebuilt artifact (b) check out the prebuilt branch as a Git worktree, (c)
+move the artifact to the right place in the prebuilt branch and commit/push.
+
+Repositories that depend on prebuilt components must download them
+in their ``./bootstrap`` script.  After downloading, the bootstrap
+script *SHOULD* verify a checksum of the downloaded artifact.
+
+FIXME: Link to an example bootstrap file, once the approach from this document
+is fully implemented.
+
+Alternatives
+============
+
+* The prebuilt branch could be integrated into source repositories that
+  depend on it via a git submodule.  This has the following disadvantages:
+
+  1. Git submodules are difficult to handle for some developers. We've had issues
+     in the past where the prebuilt submodule wasn't integrated properly and would
+     lead to non-reproducible builds
+  2. Relying purely on the Git commit hash of the prebuilt branch
+     makes it very difficult to know which version of the prebuilt
+     artifact is being used.  It also makes it very difficult
+     to reliable find older versions of the artifact.
+  3. Relying on the commit hash instead of a version and artifact hash
+     couples the artifact to the Git commit.  That means that
+     we can never compact the commit history of the prebuilt
+     branch of move artifacts around.
+
+* Set up a full-blown artifact management solution like JFrog Artifactory.
+  That's costly and requires a lot of admin work.
+
+* Never rely on prebuilt files.  That slows down the build process
+  and in some cases requires exotic dependencies.
+
+Drawbacks
+=========
+
+Discussion / Q&A
+================
+
+(This should be filled in with results from discussions on mailing lists / personal communication.)