Making a control board firmware release ======================================= This section describes how to make a new firmware release using the different subprojects. Repository layout and files --------------------------- Firmware is build using the **controlboard_fw** project. This project contains 4 submodules: * gateware - points to *spidr4_controlboard_gateware.git*, the FPGA gateware for SPIDR4 * software - points to *controlboard_linux.git*, Linux OS + applications * buildtools - Points to *buildtools.git*, contains a few simple scripts for versioning and template rendering, * updatetools - Points to *updatetools.git*, to contains tools for making SD card images and such. In addition it contains the continuous integration script for gitlab: ``.gitlab-ci.yml``, containing a script for building the firmware and generating the SD card images. It is triggered when a tag is committed. Updating the control-board application -------------------------------------- If the control board main application has changed, this change must be included in Petalinux. Tagging the control board application ````````````````````````````````````` For Petalinux to checkout the correct version of the control-board application, we must tag the the application it self. In ``controlboard_app`` make sure you are on the correct commit, and tag the version, then push the tags to the remote repository: .. code:: bash controlboard_app$ git tag v1.0.0b3 controlboard_app$ git push --tags See 'versioning' (*todo*) section on the versioning conventions used. Updating the Petalinux yocto recipe ``````````````````````````````````` In controlboard_linux, update the recipe filename to reflect the new tag, and then commit and push it: .. code:: bash controlboard_linux$ cd ctrl2019_1/project-spec/meta-user/recipes-apps/control controlboard_linux$ git mv control_1.0.0b1.bb control_1.0.0b2.bb controlboard_linux$ git commit -m "Bumped control app to v1.0.0b2" controlboard_linux$ git push Where ``control_1.0.0b1.bb`` and ``control_1.0.0b2.bb`` must be replaced by the old and new tag respectively. Note that it is important that the part after the underscore (``_``) and before the ``.bb`` match the version tag, excluding the ``v``-prefix. Now we're ready to update the firmware. Procedure for updating the the submodules ------------------------------------------ Before a new release can be made, first the submodules must be updated, in order to include the changes you want to. Updating the gateware ````````````````````` To update the gateware to the latest master, go to the gateware directory and pull it from its origin: .. code:: bash controlboard_fw$ cd gateware controlboard_fw/gateware$ git pull origin master Updating the software ````````````````````` For the software, it is the same: .. code:: bash controlboard_fw$ cd software controlboard_fw/software$ git pull origin master You should see the rename of the recipe in the logging output of the pull Tagging and initiating the build -------------------------------- Now that we've updated all the subparts of the firmware project we can commit it. First add the submodule directories in the root of the ``controlboard_fw`` project. .. code:: bash controlboard_fw$ git add software gateware controlboard_fw$ git commit -m "Updated control app to 1.0.0b2, latest gatware" controlboard_fw$ git tag v1.0.0b2 Note that I use the same tag for the firmware as I do for the control board application, but this is not mandatory. All that is left to do is to initate the CI script by pushing the tag: .. code:: bash controlboard_fw$ git push controlboard_fw$ git push --tags Go to the pipelines overview at the SPIDR4 controlboard_fw project to see the pipeline in progress: https://gitlab.nikhef.nl/spidr4/controlboard_fw/-/pipelines You should see something like the following: .. image:: ../_static/gitlab_ci.png :alt: gitlab-ci pipeline in progress Step-by-step gitlab-ci script guide ----------------------------------- In this last section we'll walk through the gitlab-ci script in the ``controlboard_fw`` repository, explaining each stage. The gitlab-ci script defines a 'pipeline' for building the firmware. A pipeline consists of some declarations and one or more job sections. Each job contains a script with building instructions. Default variables ````````````````` The first section of the ``.gitlab-ci.yml`` script starts with the variable declarations. .. code:: yaml variables: GIT_DEPTH: 0 parallel: 2 GIT_SUBMODULE_STRATEGY: recursive PUBL_HOST: amia.nikhef.nl PUBL_USER: spidr4 PUBL_ROOT: /var/www/spidr4/html/releases/firmware PACKAGE_PREFIX: spidr4-firmware Important variables are: * ``PUBL_HOST`` is the host system to which the firmware release must be copied. * ``PUBL_USER`` is the user on the host system to use for this. * ``PUBL_ROOT`` provides the directory in which the firmware releases must be placed, on the publication host. * ``PACKAGE_PREFIX`` is the prefix used for packaging, pre-pended as the package name. For the remaining variables, please see the gitlab-ci script reference guide: https://docs.gitlab.com/ee/ci/yaml/gitlab_ci_yaml.html Stages `````` The next section declares the stages: .. code:: yaml stages: - synth - build - image - deploy Each 'job' is bound to a stage, which provides an ordering. For example, jobs bound to ``build`` will be executed *after* after jobs bound to ``synth``. Generic properties for each job ``````````````````````````````` The next section defines a common set of properties named 'generic' for jobs. .. code:: yaml .generic: &generic only: - tags - web artifacts: expire_in: 3 months In this case the triggering (only when triggered by web, or when a tag is created). This common set of properties can be referenced by actual jobs. Also artifacts from the build will be stored by gitlab-ci for 3 months. Note that this is not related to the actual storage on ``spidr4.nikhef.nl``, just how long gitlab-ci stores it together with the job logs. It can be seen as an 'abstract super class' for jobs, which can be inherited by any job. FPGA synthesis `````````````` The following section is the actual FPGA synthesis step: .. code:: yaml synth: <<: *generic stage: synth timeout: 1 hours 30 minutes tags: - piava - shell script: - cd gateware/scripts/ - ./ci.sh artifacts: paths: - gateware/synt_output This actually executes the gateware ci-script, which generates the FPGA image and the hardware description file. It inherits the properties of 'generic'. It is bound to the ``synth`` step, and has a timeout of 1 and ha half hours. The tags 'piava' and 'shell' dictate it must use a runner (https://docs.gitlab.com/runner/) which has these tags. The actual script simply executes ``ci.sh``, which does the synthesis and related. The artifacts, e.g. the FPGA image and hardware description file are stored in ``gateware/synth_output``, and will be persisted during the build, and downloadable as an item of the job. Software build `````````````` The next section describes how to build the software. .. code:: yaml build: <<: *generic stage: build timeout: 1 hours 30 minutes script: # Source build environment - source /localstore/et/vincentb/plx2019.1/settings.sh # Get version (tag/hash) - VERSION=`python3 buildtools/version.py` - VERSION=${VERSION/\//-} - echo $VERSION >version.env # Inject into petalinux 'version' (/etc/petalinux/version) - sed -i "s/non-ci-build/${VERSION/\//\\/}/g" software/ctrl2019_1/project-spec/configs/config - cd software/ctrl2019_1 # Import hardware description - petalinux-config --get-hw-description=../../gateware/synt_output --silentconfig # Build Linux image - petalinux-build # Package boot image - petalinux-package --boot --force --format BIN --u-boot --fpga - cd .. # Make release - ./makerelease.py ctrl2019_1 ${PACKAGE_PREFIX}-${VERSION} tags: - piava - shell artifacts: paths: - software/ctrl2019_1/images/linux/BOOT.BIN - software/ctrl2019_1/images/linux/image.ub - software/ctrl2019_1/images/linux/rootfs.tar.gz - software/${PACKAGE_PREFIX}-*.zip - version.env The script section first line sources the Petalinux environment present on host *piava*. The tags also indicate that the runner must be executed on Piava. This is important! In the future it would be better to have a Petalinux docker container. The next section creates a version based on the git-tag. If there is no git-tag the branch name will be used. It also inserts this version into the Petalinux configuration prior to build, such that it is inside the rootfs (/etc/petalinux/version). The remaining part of the script builds and packages the Linux environment. Various artifacts are stored in the build, and the ``version.env`` file is used to store the version, which is used in the next jobs. SD card creation ```````````````` The SD card creation is the only part which uses **docker**. This is required because SD card creation requires mounting of a file (image) and creation of loop back devices. Something which you normally do not have the permissions for. .. code:: yaml sdcard: <<: *generic stage: image # ci script using docker image to create SD card image (CentOS latest) image: name: centos:latest entrypoint: ["/bin/bash","-l","-c"] tags: - docker - privileged script: # Get version from previous - VERSION=`cat version.env` # Install / clone required tools - yum install -y unzip zip git dosfstools e4fsprogs # Check if loop device exists, otherwise create - "[ ! -e /dev/loop0 ] && mknod /dev/loop0 b 7 0" # Run make sd image for docker - updatetools/sdimage/make_sd_img_docker_zip.sh software/${PACKAGE_PREFIX}-${VERSION}.zip artifacts: paths: - software/${PACKAGE_PREFIX}-*.img.zip The ``image`` section defines the docker container to use. In this case the latest *CentOS*. This container is pulled for the central docker repository and requires no configuration. The entry point must be modified for it to play nice with docker-ci. See (https://docs.docker.com/) for more information on docker. For docker to work, we must use a runner which support docker, but also runs in privileged mode in order to be allowed to create loop-back devices. A docker runner on Piava was configured with these rights, and uses the tags ``docker`` and ``privileged``. The script retrieves the version stored as an artifact, and installs dependencies required for SD card creation not yet present in the docker container. It creates a loop device, required for SD card creation and then starts the shell-script to generate the actual SD card image. Finally it stores ``software/${PACKAGE_PREFIX}-*.img.zip`` as an artifact. The wildcard * must be used as the ``version`` variable is not present in the gitlab-ci context, but as there is just one file to match, it is not a problem. Deployment to spidr4.nikhef.nl `````````````````````````````` The final stage publishes the SD card image to the SPIDR4 download website. .. code:: yaml deploy to site: <<: *generic stage: deploy tags: - piava - shell script: # Get version from previous - VERSION=`cat version.env` - mkdir ${PACKAGE_PREFIX}-${VERSION} - cp software/${PACKAGE_PREFIX}-*.zip ${PACKAGE_PREFIX}-${VERSION}/. - cp README.image.md ${PACKAGE_PREFIX}-${VERSION}/README.md - tar -czvf ${PACKAGE_PREFIX}-${VERSION}.tar.gz ${PACKAGE_PREFIX}-${VERSION}/. - scp ${PACKAGE_PREFIX}-${VERSION}.tar.gz ${PUBL_USER}@${PUBL_HOST}:${PUBL_ROOT}/. The script creates a directory, copies the SD card ZIP file into this directory, adds a *readme* file, compresses the whole thing, and finally uses secure shell copy (*scp*) to copy this to the server. Because the ID of the gitlab-runner user on host *piava* has been copied to the publication server no password is required.