diff --git a/source/mixer.rst b/source/mixer.rst index 0f2bee5d..fc59b8e7 100644 --- a/source/mixer.rst +++ b/source/mixer.rst @@ -31,78 +31,88 @@ Mixing #. **Configure builder.conf**. Copy the template conf file:: - # cp /usr/share/defaults/bundle-chroot-builder/builder.conf /etc/bundle-chroot-builder/ + # cp /usr/share/defaults/bundle-chroot-builder/builder.conf /home/clr/mix/ - The file ``builder.conf`` will be read automatically from ``/etc/bundle-chroot-builder``, + The file ``builder.conf`` will be read automatically from the current workspace directory, but all of the scripts accept a ``-c/--config`` option to specify where - the file is if you want to store it elsewhere. If you wish to use one in your current workspace, - you can copy the template and supply it to the scripts when you run them. The :file:``.yum-mix.conf`` - file will be auto-generated for you. + the file is if you want to store it elsewhere. To use one in your current workspace, + copy the template to /home/clr/mix. + The :file:``.yum-mix.conf`` file will be auto-generated for you. Note there are different sections to the builder.conf. The ``[Builder]`` section provides the mixer tools with required configuration options, defining where generated bundles and update metadata should get published. The ``[swupd]`` section is used by swupd-server to create an update with the newly mixed content. - Edit the template configuration file according to your needs:: + Edit the template configuration file according to your needs, for example like so:: # vim /etc/bundle-chroot-builder/builder.conf: [Builder] - SERVER_STATE_DIR = /var/lib/update - BUNDLE_DIR = /home/clr/mix/mix-bundles - YUM_CONF = /home/clr/mix/.yum-mix.conf + SERVER_STATE_DIR=/home/clr/mix/update + BUNDLE_DIR=/home/clr/mix/bundles + YUM_CONF=/home/clr/mix/.yum-mix.conf + CERT=/home/clr/mix/ClearLinuxRoot.pem + CLEAR_VERSION=VER + MIX_VERSION=VER [swupd] BUNDLE=os-core-update - CONTENTURL= ### URL where swupd content is - # published. Optional if not hosting updates - VERSIONURL= ### Should be the same as CONTENTURL. - # Optional if not hosting updates - FORMAT=1 ### Can be any number. - # See 'OS Epoch' discussion for details + CONTENTURL= + VERSIONURL= + FORMAT=1 - You may include a ``CERT=/path/to/cert`` line which tells the chroot builder to insert the certificate - specified for the mix in ``/os-core-update/usr/share/clear/update-ca/``. This would be the certificate used - by the software update client to verify the Manifest.MoM signature. + + You may change the ``CERT=/path/to/cert`` line which tells the chroot builder to insert the certificate + specified for the mix in ``/os-core-update/usr/share/clear/update-ca/``. This is the certificate used by the software update client to verify the Manifest.MoM signature. For now, it is HIGHLY recommended that you do not modify this line, as the certificate swupd expects needs a very specific configuration to sign and verify properly. The certificate will be automatically generated for you, and the Manifest.MoM will be signed automatically as well, providing security for the update content you create. + + The CLEAR_VERSION is the upstream Clear Linux* OS for Intel® Architecture version the mix will be based off of, and all required content (RPMs) not provided by the mixer will be downloaded from that release. + The MIX_VERSION is the version you want your mix to be. + + For this example, set CLEAR_VERSION=11230 and MIX_VERSION=10. You can of course choose any numbers you like for the MIX_VERSION, but it is recommended to use the current latest version of upstream for the CLEAR_VERSION. The CLEAR_VERSION can be updated as new upstream versions are released if needed. + + The CONTENTURL and VERSIONURL may be an IP address, or a domain name, which hosts the /home/clr/mix/update/www directory. A client running the mix will look to that URL to figure out if there is a new version available, and where to download update content from. + + To learn more about the FORMAT option, please refer to the bottom of this document "Format Version" and https://github.com/clearlinux/swupd-server/wiki/Format-Bumps for more information. For now leave the FORMAT value alone and do not increment it. #. **Generate the starting point for your Mix**. In your workspace, run:: - # mixer-init-mix.sh -c /etc/bundle-chroot-builder/builder.conf - Optionally, the ``-b/--clear-version`` option can be passed to tell the script which - Clear version to build the initial mix against, and the ``-m/--mix-version`` can be used - to specify what the starting mix version should be. This may be needed if the build - you are currently on is newer than the latest clr-bundles release tag. + # sudo mixer-init-mix.sh *If you wish to just build a mix that includes all Clear bundles with no modifications, run*:: - # mixer-init-mix.sh -c /etc/bundle-chroots-builder/builder.conf --all-bundles + # sudo mixer-init-mix.sh --all-bundles And skip to ``Creating an image``. All the required content will be automatically built, and this mix will be identical to the version of Clear it is being composed from. #. **Create/locate RPMs for mix.**. (Steps 4-6 are necessary only if you - want to add your own RPMs to the Mix. If you are simply working with Clear - only bundles, then skip to Step 7.) + want to add your own RPMs to the Mix. If you are working only with Clear + bundles, then skip to Step 7.) If you are creating RPMs from scratch, you may use ``autospec``, ``mock``, ``rpmbuild``, etc. to build them. If they are not built on Clear, make sure your configuration and toolchain builds them correctly for Clear. -#. **Import RPMs into workspace**. The easiest way to do this is to create a +#. **Import RPMs into workspace**. The way to do this is to create an ``rpms`` directory in your workspace (for example ``/home/clr/mix/rpms``), and to copy the RPMs you want into that directory. The mixer script will look here for RPMs in order to build a local RPM repo for yum to use. #. **Create a local RPM repo**. Create an empty directory in your workspace - named ``local`` and run:: + named ``local`` and add the paths in your builder.conf:: - # mixer-add-rpms.sh --rpmdir rpms --repodir local + RPMDIR=/home/clr/mix/rpms + REPODIR=/home/clr/mix/local + + These variables are automatically read; you simply need to run:: + + # sudo mixer-add-rpms.sh After the script exits, you should see your RPMs and a repodata directory in ``/home/clr/mix/local``. If the RPMs are not all in the local directory, check to make sure that they are indeed valid RPM files and not corrupt. -#. **Update bundle definitions**. The mixer uses a local clone of the +#. **Update/Add bundle definitions**. The mixer uses a local clone of the ``clr-bundles`` repo to define bundles for the mix. To define your bundles: @@ -112,6 +122,7 @@ Mixing $ git add . $ git commit -s -m 'Update bundles for mix #' + You can easily copy bundles over from the ``clr-bundles/bundles`` directory in the case that you want to simply use existing bundle sets. Note that ``mix-bundles`` should not have any folders inside of it, only bundle definitions. @@ -121,12 +132,10 @@ Mixing Why do this? With Git history, mixes are easy to revert to or refer to in the future if something were to go wrong with a new mix. If you're just testing this out, or if you really do not want to mess with Git, - you can ignore committing for now. The next feature will be to - implement an interactive way to modify/add/delete bundles, so much of - this work can be abstracted out so Git work will be more automated. + you can ignore committing for now. To add your own bundle, create a bundle definition file in ``mix-bundles/`` - and refer to :file:`os-core-update` for formatting, but be sure that + and refer to :file:`mix-bundles/os-core-update` for formatting, but be sure that the name does not conflict with another bundle. Add your package name(s) in that bundle definition file to tell it what package(s) must be installed as part of that bundle. @@ -134,38 +143,44 @@ Mixing #. **Build the bundle chroots** To build all of the ``chroots`` that are based on the bundles you defined, in your workspace run:: - # mixer-build-chroots.sh + # sudo mixer-build-chroots.sh If you have many bundles defined for your mix, this step may take some time. #. **Create update**. In the workspace, run:: - # mixer-create-update.sh + # sudo mixer-create-update.sh When the script completes, you'll find your mix update content under ``/var/lib/update/www/VER``, in this example, it will be located in ``/var/lib/update/www/``, where is the mix version you defined, or 10 by default. + All content to make a fully usable mix will be created by this step, but note that only zero packs are automatically generated. To create optional delta packs, run the pack-maker as follows:: + + # sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update + + The pack-maker will generate all delta packs for changed bundles from PAST_VERSION to MIX_VERSION. If your STATE_DIR is in a different location be sure to specify where with the -S option. + #. **Initialize next Mix version info**. To update the versions and prep for your - next mix, in the workspace run:: - - # mixer-init-versions.sh -m 20 - - This takes the Clear version from your image (or override it with - ``-c/--clear-version`` to use another Clear build's content), and sets - "20" for the mix version. From this point you can iterate through starting again at step - 4 and doing modifications as needed. + next mix:: + Update the MIX_VERSION in your builder.conf to the next version number you want to build. From this point you can iterate through, starting again at step 4 and doing modifications as needed, i.e. + - Add/Remove/Modify Bundles + - sudo mixer-build-chroots.sh + - sudo mixer-create-update.sh + - (Optionally) sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update + Next mix created. #. **Update Bundles (Optional)**. Update ``clr-bundles``. In the workspace, run:: - # mixer-update-bundles.sh + # sudo mixer-update-bundles.sh This step is optional because the script is already called by mixer-init-mix.sh, and only needs to be called again when you want to update the upstream clr-bundles - folder in your workspace. + folder in your workspace. It also does not need to be called unless you are updating + the CLEAR_VERSION number as well to match the newest upstream release. **Creating an image** To create a bootable image from your update content, you will need the configuration file for @@ -179,16 +194,16 @@ image this would be:: "Bundles": ["os-core", "os-core-update", "kernel-native"] And lastly, set the "Version:" to say which mix version content the image should be built from, -i.e 10 for your first build. To build the image, run:: +i.e. 10 for your first build. To build the image, run:: - # ister.py -t release-image-config.json -V file:///home/clr/mix/update/www/ -C file:///home/clr/mix/update/www/ + # sudo ister.py -t release-image-config.json -V file:///home/clr/mix/update/www/ -C file:///home/clr/mix/update/www/ -f 1 The output from this should be an image that is bootable as a VM or installable to baremetal. *Note* that you may need to pass in -f/--format if the format you are building is different than the format of Clear Linux OS you are currently building on. Format version can be found via:: # cat /usr/share/defaults/swupd/format -OS Epoch or Format version +Format Version -------------------------- The "format" used in ``builder.conf`` might be more precisely referred to as an