1 files changed, 130 insertions, 0 deletions
diff --git a/design-documents/044-ci-system.rst b/design-documents/044-ci-system.rst
new file mode 100644
index 00000000..e38632c0
--- /dev/null
+++ b/design-documents/044-ci-system.rst
@@ -0,0 +1,130 @@
+DD 44: CI System
+################
+
+Summary
+=======
+
+This documents describes Taler's CI system based on Buildbot.
+
+This document uses `RFC 2119 <https://tools.ietf.org/html/rfc2119>`_
+keywords throughout.
+
+Motivation
+==========
+
+With the current CI system there are an array of issues:
+
+- Central place for all the jobs.
+- The central config is poorly organized.
+- We should prefer to keep as much CI logic in respective project source repos
+  as possible.
+- Jobs should be split up further to allow for more granular control and
+  insight.
+- Job triggers are unclear.
+- The build environments are mutable.
+- Non-trivial and error-prone to keep track of environment state.
+- Hard to get an overview of what repo is causing a failure, at a quick glance.
+- Bad for development workflow on a single project when you are getting
+  false-negatives all the time.
+
+Proposed Solution
+=================
+
+General
+-------
+
+Jobs shall be executed inside of containers.
+
+One build pipeline (aka. "builder") per repo.
+
+Build steps are generated from directory structure within a given repo.
+
+Example directory structure:
+
+::
+
+  contrib
+  └── ci
+      ├── ci.sh
+      ├── Containerfile
+      └── jobs
+          ├── 0-codespell
+          │   ├── config.ini
+          │   ├── dictionary.txt
+          │   └── job.sh
+          ├── 1-build
+          │   ├── build.sh
+          │   └── job.sh
+          └── 2-docs
+              ├── docs.sh
+              └── job.sh
+
+Job directories **MUST** follow this pattern:
+``<repo_root>/contrib/ci/jobs/<n-job_name>/``
+
+``n`` is an integer used for ordering the build steps.
+
+Job directories **MUST** contain a script named ``job.sh`` which **MAY**
+execute other scripts.
+
+Config files may optionally be created, and MUST be named ``config.ini`` and
+placed in the job directory.
+
+Available config options:
+
+::
+
+  [build]
+  HALT_ON_FAILURE = True|False
+  WARN_ON_FAILURE = True|False
+  CONTAINER_BUILD = True|False
+  CONTAINER_NAME = <string>
+  CONTAINER_ARCH = <string>
+
+
+Unless *all* jobs specify a "CONTAINER_NAME" in their custom config a
+container file **MUST** be present at ``<repo_root>/contrib/ci/Containerfile``.
+The container file will be built and used to run all of a repo's jobs
+by default.
+
+All projects SHOULD have a ``build`` step and a ``test`` step, at a minimum.
+
+Running CI Locally
+------------------
+
+Running the CI scripts locally can be useful for development and testing.
+
+Included in each CI directory is a script which simplifies running jobs
+in the same way the CI Worker does, in containers, using ``podman``.
+
+::
+
+  # Usage:
+  ./contrib/ci/ci.sh <job-name>
+
+  # For example, if the CI jobs tree looks like this:
+  ./contrib/ci/jobs
+    ├── 0-codespell/
+    ├── 1-build/
+    ├── 2-test/
+    ├── 3-docs/
+    ├── 4-deb-package/
+    └── 5-deploy-package/
+
+  # Then you can run job '0-codespell' as follows:
+  ./contrib/ci/ci.sh 0-codespell
+
+  # If you are using podman and have "qemu-user-binfmt" installed
+  # then you may attempt to run any job under an alternative CPU
+  # architecture by providing a second argument.
+  # For example:
+  ./contrib/ci/ci.sh 0-codespell arm64
+
+
+Additional Builders
+-------------------
+
+To run some tests there is a need for many or most project's sourcecode to be
+available in the same environment. This will be a separate builder/pipeline
+from the per-repo builders. Triggers for this builder are yet to be
+determined.