.. _mixer:

mixer
#####

The |CL-ATTR| team uses **mixer** to generate official update content and 
releases. The update content generated by mixer is then consumed by swupd on 
a downstream client. The same mixer tool is available to those who wish to create customized update content and releases.

.. contents::
   :local:
   :depth: 1

Description
***********

mixer uses the following sources as inputs to generate update content:

* Upstream |CL| bundles with their corresponding RPM packages
* Locally-defined bundles with their corresponding local RPM packages
* Locally-defined bundles with upstream RPM packages
* Locally-defined bundles with non-RPM content

Using the mixer tool, you select which content from these sources that
becomes part of your update. Your selection of sources produces a unique 
combination of functionality for your custom update content, known as 
a **mix**.

The update content that mixer generates consists of various pieces of OS
content, update metadata, as well as a complete image. The OS content
includes all files in an update, as well as zero- and delta-packs for
improved update performance. The update metadata, stored as manifests, 
describes all of the bundle information for the update. Update content 
produced by mixer is then published to a web server and consumed by clients 
via :command:`swupd`. Refer to :ref:`swupd <swupd-guide>` for additional 
information regarding updates and update content.

How it works
************

Learn the mixer tool set up and workflow.

.. contents::
   :local:
   :depth: 1

Prerequisites
=============

* :command:`mixer` bundle

  Add the mixer tool by installing the :command:`mixer` bundle. Refer to
  :ref:`swupd-guide` for more information on installing bundles.

* If you're working behind a corporate proxy, configure proxy settings using
  the :ref:`General proxy settings for many applications <proxy>` steps.

* Location to host the update content and images

  In order for :command:`swupd` to make use of your mix, the update content for your mix must be hosted on a web server. Your mix will be configured with an update location URL, which :command:`swupd` will use to pull down updates.

  Refer to `Set up a nginx web server for mixer`_ for an simple example of
  setting up an update location.

Mix setup
==========

Follow these steps to create and initialize the mixer workspace. Complete
the setup before you create a mix.

#. Create workspace.

   The mixer tool uses a simple workspace to contain all input and output in a
   basic directory structure. The workspace is simply an empty folder from
   which you execute the mixer commands. Each mix uses its own separate
   workspace.

#. Initialize the workspace and mix.

   Before you create a mix, you must explicitly initialize the mixer workspace.
   During initialization, the mixer workspace is configured and the base for
   your mix is defined. By default, your mix is based on the latest
   upstream version and starts with the minimum set of bundles. Your first custom
   mix version number starts at 10. Alternatively, you can select other
   versions or bundle sets from which to start.

   Initialization creates the directory structure within the workspace and adds
   the :file:`builder.conf` file, which is used to configure the mixer tool.

   View the `mixer.init man page`_ for more information on mixer
   initialization.

   View the list of suitable `releases`_ from which to mix.

#. Edit builder.conf.

   :file:`builder.conf` tells the mixer tool how to configure the mix. For
   example, it allows you to configure where mixer output is located and where swupd update content will be located.

   At minimum, set the URL of your update server so your custom OS knows where to get update content.

   Refer to the `builder.conf`_ section for more information.

Create a mix
============

A mix is created with the following steps:

#. Add custom RPMs and set up local repo (optional).

   If you are adding custom RPMs to your mix, you must add the RPMs to
   your mix workspace and set up a corresponding local repository.

   Go to the :ref:`autospec<autospec>` guide to learn to build RPMs from
   scratch. If the RPMs are not built on |CL|, make sure your
   configuration and toolchain builds them correctly for |CL|. Otherwise there
   is no guarantee they will be compatible.

   Refer to the :ref:`autospec` guide for more information on using autospec to
   build RPMs.

#. Update and build bundles.

   Add, edit, or remove bundles that will be part of your content and build
   them. mixer automatically updates the :file:`mixbundles` file when you
   update the bundles in your mix.

   View the `mixer.bundle man page`_ for more information on configuring bundles
   in a mix.

   View the `mixer.build man page`_ for more information on building bundles.

   View the `Bundles`_ section for more information on how mixer manages
   bundles.

#. Create the update content.

   mixer creates update content with this step. Zero-packs are created
   automatically, and delta-packs can be optionally created at the same time
   (for all builds after version 0).

   A zero-pack is the full set of content needed to go from mix version 0
   (nothing) to the mix version for which you just built content.

   A delta-pack provides the content *delta* between a `PAST_VERSION` to a
   `MIX_VERSION` that allows the transition from one mix version to another.

   View :ref:`swupd-guide`  for more information on update content.

#. Create image.

   mixer creates a bootable image from your updated content using
   the `clr-installer`_ tool. In this step you can specify which bundles you want
   *preinstalled* in the image. Users can later install other bundles available
   in your mix.

#. Make update available.

   Deploy update content and images to your update server.

   View the `Example 5: Deploy updates to target`_ for a simple deployment
   scenario.

Maintain or modify mix
======================

Update or modify your content to a new version by following the steps to
create a mix. Increment the mix version number for the next mix.

Examples
********

The following examples are designed to work together and in order. The examples
use:

* A stock installation of |CL|.
* A web server that comes with |CL| to host the content updates.
* A simple VM that updates against the locally produced content created in
  Example 2.

Complete all `Prerequisites`_ before using these examples.

Example 1: Mix set up
=====================

This example shows the basic steps for the first-time setup of
mixer for a new mix.

#. Create a directory to use as a workspace for mixer:

   .. code-block:: bash

      mkdir ~/mixer

#. In your mixer workspace, generate an initial mix based on the latest
   upstream |CL| version, with minimum bundles. In the initialization
   output, be aware that your initial mix version is set to 10 and that the
   minimum bundles have been added.

   .. code-block:: bash

      cd ~/mixer
      mixer init

   .. note::

      If you want to add all upstream bundles in your mix,
      initialize your mix as shown below.

   .. code-block:: bash

      mixer init --all-upstream

#. Look up your IP address:

   .. code-block:: bash

      networkctl status

#. Copy the IP “Address”, from above, for the next step.

   .. note::

      In this example, we put `mixer` and `nginx` on the same system. In a production environment, they would likely reside on different systems.

#. Edit :file:`builder.conf`.  Paste the IP address from the previous step
   as the value after \http:// for CONTENTURL and VERSIONURL. For example:

   .. code-block:: console

      CONTENTURL="http://192.168.25.52"
      VERSIONURL="http://192.168.25.52"

#. `Set up a nginx web server for mixer`_.


Example 2: Create a simple mix
==============================

This example shows how to create a simple custom mix using upstream content.
We'll create an image for a QEMU virtual machine that we can use later to
test our mix.

We can use the default bundles that were added during initialization, but
these include the :command:`native-kernel` bundle that is intended to be
used on a bare metal system instead of a VM. So we will modify the default
bundle set to get a smaller kernel image, which will also be faster to load.

.. note:: 
   
   The only bundles available to :command:`swupd` for a given release are 
   those that were added to the mix during build time. A mix doesn’t 
   automatically inherit all upstream bundles.

#. Ensure that you have run `mixer init`, shown in Example 1.

#. Update bundles in mix:

   .. code-block:: bash

      mixer bundle remove kernel-native
      mixer bundle add kernel-kvm

   .. note::
      The mixer bundle commands operate on the bundle description files but not on the bundle contents. To remove bundle contents and their tracking completely, follow `Example 6: Remove a bundle from client system`_, Advanced.

#. In this case, we will add the `editors` bundle from upstream, but we will
   remove the :command:`joe` editor.

   .. code-block:: bash

      mixer bundle add editors
      mixer bundle edit editors

#. Use an editor and manually remove `joe` from the bundle definition.

   .. code-block:: bash

      $EDITOR ./local-bundles/editors

#. List the bundles in the mix again to confirm removal of :command:`joe`.

   .. code-block:: bash

      mixer bundle list  --tree

#. Build bundles:

   .. code-block:: bash

      sudo mixer build bundles

#. First, browse to web server from Example 1. The web page appears yet
   has no update content. Build the update content:

   .. code-block:: bash

      sudo mixer build update

   After that is completed, on your web server, you can see the update
   content for mix version 10.

Example 3: Create an update for your mix
========================================

Next, let’s create a new version of the mix. We’ll add a new bundle.

#. Create a new version of your mix, for the live image to
   update to. Increment your mix version by 10:

   .. code-block:: bash

      mixer versions update

#. Add the upstream :command:`curl` bundle to version 20 of the mix:

   .. code-block:: bash

      mixer bundle add curl

#. Build your next mix version that incorporates the new bundle.

   .. code-block:: bash

      sudo mixer build bundles
      sudo mixer build update

#. Optionally, you can build delta-packs, which help reduce client update
   time:

   .. code-block:: bash

      sudo mixer build delta-packs --from 10 --to 20

Refresh your web server to see the update content for mix version 20.

You can also look in ~/mixer/update/www/<mix version> to see the update
content in your workspace.

Example 4: Build an image
=========================

This example shows how to build a bootable image containing the
:command:`kernel-kvm`, :command:`os-core`, and the :command:`os-core-update`
bundles from `Example 2: Create a simple mix`_. Complete that example before starting this one.  

Underneath, mixer uses `clr-installer`_ to generate the image.

#. Change directory into your mix.

#. Configure image.

   Create a YAML configuration file to specify aspects of your image
   such as image name, target media, bundles, etc.  See `Installer YAML Syntax`_ 
   for more information on clr-installer configuration YAML syntax.

   For this example, we will download a sample YAML and modify it.

   .. code-block:: bash

      curl -O https://raw.githubusercontent.com/clearlinux/clr-installer/master/scripts/kvm.yaml

   Make the following revisions to :file:`kvm.yaml`:

   * Reduce overall image size and root partition size by 5GB.
   * Remove these bundles from the image: ``editors``, ``network-basic``, 
     ``openssh-server``, ``sysadmin-basic``
   * Add ``version: 10`` to tell :command:`mixer` to generate an image based 
     on mix version 10

   .. note::

      When creating an image, it is not necessary to include all of the bundles
      that are in your entire mix.  Once you have a working image, you can use 
      :command:`swupd` to add them as needed.  

   Your :file:`kvm.yaml` should look like below:

   .. code-block:: console
      :linenos:
      :emphasize-lines: 11,26,29-33,44
      
      #clear-linux-config

      # switch between aliases if you want to install to an actuall block device
      # i.e /dev/sda
      block-devices: [
         {name: "bdevice", file: "kvm.img"}
      ]

      targetMedia:
      - name: ${bdevice}
        size: "3.54G"
        type: disk
        children:
        - name: ${bdevice}1
          fstype: vfat
          mountpoint: /boot
          size: "512M"
          type: part
        - name: ${bdevice}2
          fstype: swap
          size: "32M"
          type: part
        - name: ${bdevice}3
          fstype: ext4
          mountpoint: /
          size: "3G"
          type: part

      bundles: [
          bootloader,
          os-core,
          os-core-update,
        ]

      autoUpdate: false
      postArchive: false
      postReboot: false
      telemetry: false

      keyboard: us
      language: en_US.UTF-8
      kernel: kernel-kvm

      version: 10

#. Build the image.

   .. code-block:: bash

      sudo mixer build image --template $PWD/kvm.yaml

   The output from this step will be :file:`kvm.img`, which is a live
   image.

Example 5: Deploy updates to target
===================================

The image created in Example 4 is directly bootable in QEMU. In this example,
we'll boot the image and verify it. Then we'll update the image from
mix version 10 to mix version 20.

#. Set up the QEMU environment.

   Install the :command:`kvm-host` bundle to your |CL|:

   .. code-block:: bash

      sudo swupd bundle-add kvm-host

#. Get the virtual EFI firmware, download the image launch script, and make
   it executable:

   .. code-block:: bash

      curl -O https://download.clearlinux.org/image/OVMF.fd
      curl -O https://download.clearlinux.org/image/start_qemu.sh
      chmod +x start_qemu.sh

#. Start your VM image (created in Example 4):

   .. code-block:: bash

      sudo ./start_qemu.sh kvm.img

#. Log in as root and set a password.

#. By default, the :command:`swupd` client is designed to communicate with an
   HTTPS server. For development purposes, the swupd client can talk to
   an HTTP server if you add the flag ``allow-insecure-http``.

   To avoid adding a flag each time when invoking :command:`swupd`, enter:

   .. code-block:: bash

      mkdir -p /etc/swupd
      cat > /etc/swupd/config << EOF
      [GLOBAL]
      allow_insecure_http=true
      EOF

#. Try out your mix.

   a. Show the version and update URLs

      .. code-block:: bash

         swupd info

   #. List the bundles installed in your mix:

      .. code-block:: bash

         swupd bundle-list

   #. List available bundles on your update server.

      .. code-block:: bash

         swupd bundle-list -a

   #. Now we will add the :command:`editors` bundle that we modified.

      .. code-block:: bash

         swupd bundle-add editors

   #. Try to start the :command:`joe` editor.

      .. code-block:: bash

         joe

      It should not work because we removed it from the original
      :command:`editors` bundle.

   #. Next we will update from version 10 to 20 to capture the 
      newly-available bundles.

      .. code-block:: bash

         swupd check-update
         swupd update
         swupd bundle-list -a

   #. Now your mix should be at version 20 and :command:`curl` is available. 
      Try using :command:`curl`. This will fail because it is not yet installed.

      .. code-block:: console

         curl: command not found
         To install curl use: swupd bundle-add curl

   #. Add the new bundle from your update server to your VM. Retry :command:`curl`.
      It works!

      .. code-block:: bash

         swupd bundle-add curl
         curl -O https://download.clearlinux.org/image/start_qemu.sh

#. Shutdown your VM:

   .. code-block:: bash

      poweroff

Example 6: Remove a bundle from client system
=============================================

Removing a bundle in a future release requires more steps than deleting the
bundle description file, as shown in Example 2. After a bundle is built in
the mix, you must assure all of the files that are part of the bundle are
removed from the client where that bundle is installed. To do this, create a
version of this bundle in which all of its content is marked for deletion.

In the following example, we show how to remove the contents of the `editors`
bundle that we added to our mix in Example 2.

#. First update your mix version. This will set the mix to the next version.

   .. code-block:: bash

      mixer versions update

   .. note::
      Run this command every time that you want to build a new version.

#. Navigate to local-bundles:

   .. code-block:: bash

      cd local-bundles

#. Open the `editors` bundle with an editor and delete
   **all lines** that follow after the `[MAINTAINERS]` line.

#. Afterward, it should look like this:

   .. code-block:: console

      # [TITLE]: editors
      # [DESCRIPTION]: Run popular terminal text editors.
      # [STATUS]: Active
      # [CAPABILITIES]:
      # [TAGS]: Tools and Utilities, Editor
      # [MAINTAINER]: Developer <developer@intel.com>

#. Save and exit.

#. Next, run a build to capture recently edited bundles and update your mix.

   .. code-block:: bash

     sudo mixer build all

   .. note::
      :command:`mixer build all` runs both :command:`mixer build bundles` and :command:`mixer build update` in one step.

At this point the new mix, version 30, is complete. All the content of the
editors bundles is marked as deleted. If any clients of this mix upgraded to
mix build version 30, the content of the editors bundle would be removed.
Note that the bundle still exists and is being tracked by :command:`swupd`,
but it contains no files.

Example 7: Execute a format bump
================================

As a maintainer of your mix, you must execute a format bump if you wish to:

* Track upstream’s format bump on your downstream derivative
* Delete any custom bundles that were added

Follow the appropriate use case below depending on your needs.

Basic
-----

If you maintain your own downstream derivative and you want to track
upstream, you need to do a format bump when one occurs on upstream. This
method helps you track the latest changes on upstream; however, it does not
change any local content that was added or deleted. For example, if you
deprecated bundles, this method will **not remove the bundle tracking**.
Refer to `Advanced`_ for help on managing your local mix and removing bundle
tracking.

In this example, we show a mix version that was initialized to upstream
version 29740 (format 27). You need to update your mix to upstream version
30700 (format 28). To do so, you will go through a format bump.

#. Change to your mix location and verify the current version of the mix and
   its format.

   .. code-block:: bash

      mixer versions

#. Update to upstream version, which has a newer format.

   .. code-block:: bash

      mixer versions update --upstream-version 30700

   The output will look like this:

   .. code-block:: console

      Old mix:      10
      Old upstream: 29740 (format: 27)

      New mix:      20
      New upstream: 30700 (format: 28)
      [...]

   Read the output carefully:

   * The Old mix shows the current version (10) of  your mix.

   * The Old upstream shows the version and format (27) on which it’s based.

   * The New mix shows the new version (20) of your mix.

   * The New upstream shows the version and format (28) on which it’s based.

#. Given that the format in the output differs, you need to run a
   format bump:

   .. code-block:: bash

       sudo mixer build upstream-format --new-format 28

   .. note::

      You specify the :command:`--new-format` to indicate the format (28) to which you transition.

#. Your mix is now synchronized with the new format (28); however, you must
   still advance to the desired or latest version.

   .. code-block:: bash

      mixer versions update --upstream-version 30700

Advanced
--------

To properly remove a bundle from being tracked by :command:`swupd`,
do a manual format bump. This process can also be used to perform
customizations during the update, such as:

* Adjustment in the command parameters

* Change the content of the chroot

Follow the `afb.sh reference script`_ to learn how to do a manual format bump. The `afb.sh reference script`_ shows an example of how to:

* Create a mix

* Add a bundle

* Deprecate a bundle

* Do a format bump to remove the deprecated bundle

References
**********

Reference the `mixer man page`_ for details regarding mixer commands and options.

.. contents::
   :local:
   :depth: 1

.. rst-class:: content-collapse

builder.conf
============

mixer initialization creates a :file:`builder.conf` that stores the basic
configuration for the mixer tool. The items of primary interest are CONTENTURL
and VERSIONURL, which will be used by systems updating against your custom
content.

.. code-block:: console

   #builder.conf

   #VERSION 1.2

   [Builder]
      CERT = "/home/clr/mix/Swupd_Root.pem"
      SERVER_STATE_DIR = "/home/clr/mix/update"
      VERSIONS_PATH = "/home/clr/mix"
      YUM_CONF = "/home/clr/mix/.yum-mix.conf"

   [Swupd]
      BUNDLE = "os-core-update"
      CONTENTURL = "<URL where the content will be hosted>"
      VERSIONURL = "<URL where the version of the mix will be hosted>"
      COMPRESSION = ["external-xz"]
      UPSTREAM_BUNDLES_URL = "https://github.com/clearlinux/clr-bundles/archive/"

   [Server]
      DEBUG_INFO_BANNED = "true"
      DEBUG_INFO_LIB = "/usr/lib/debug"
      DEBUG_INFO_SRC = "/usr/src/debug"

   [Mixer]
      LOCAL_BUNDLE_DIR = "/home/clr/mix/local-bundles"
      LOCAL_REPO_DIR = "/home/clr/mix/local-yum"
      LOCAL_RPM_DIR = "/home/clr/mix/local-rpms"
      OS_RELEASE_PATH = ""

Additional explanation of variables in :file:`builder.conf` is provided in 
Table 1.

.. list-table:: **Table 1**: Variables in builder.conf   
   :widths: 50, 50
   :header-rows: 1

   * - **Variable**
     - **Description**

   * - `CERT` 
     - Sets the path where mixer stores the certificate file used to sign
       content for verification. mixer automatically generates the 
       certificate if you do not provide the path to an existing one, and 
       signs the :file:`Manifest.MoM` file to provide security for the 
       updated content you create. 

       chroot-builder uses the certificate file to sign the root :file:`
       Manifest.MoM` file to provide security for content verification. 
       swupd uses this certificate to verify the :file:`Manifest.MoM` file's 
       signature. 

       For now, we strongly recommend that you do not modify this variable, 
       as swupd expects a certificate with a very specific configuration to sign and verify properly.  

   * - `CONTENTURL` and `VERSIONURL`
     - Set these variables to the IP address of the web server hosting the
       update content. 

       VERSIONURL is the IP address where the swupd client 
       looks to determine if a new version is available. 

       CONTENTURL is the location from which swupd pulls content updates. If 
       the web server is on the same machine as the SERVER_STATE_DIR 
       directory, you can create a symlink to the directory in your web 
       server's document root to easily host the content. 

       These URLs are embedded in the images created by mixer.

   * - `LOCAL_BUNDLE_DIR` 
     - Sets the path where mixer stores the local bundle definition files. 
       The bundle definition files include any new, original bundles you 
       create, along with any edited versions of upstream bundles.  

   * - `SERVER_STATE_DIR`
     - Sets the path to which mixer outputs content. By default, mixer
       automatically sets the path.    

   * - `VERSIONS_PATH`
     - Sets the path for the mix version and upstream version's two state
       files: :file:`mixversion` and :file:`upstreamversion`. mixer creates 
       both files for you when you set up the workspace.
    
   * - `YUM_CONF`  
     - Sets the path where mixer automatically generates the  
       :file:`.yum-mix.conf` file. The yum configuration file points the 
       chroot-builder to where the RPMs are stored.

+-------------------------------+----------------------------------------------------------+
| **Variable**                  | **Explanation**                                          |
+-------------------------------+----------------------------------------------------------+
| `CERT`                        | Sets the path where mixer stores the certificate file    |
|                               | used to sign content for verification. mixer             |
|                               | automatically generates the certificate if you do not    |
|                               | provide the path to an existing one, and signs the       |
|                               | :file:`Manifest.MoM` file to provide security for the    |
|                               | updated content you create.                              |
|                               |                                                          |
|                               | chroot-builder uses the certificate file to sign         |
|                               | the root :file:`Manifest.MoM` file to provide            |
|                               | security for content verification.                       |
|                               |                                                          |
|                               | swupd uses this certificate to verify the                |
|                               | :file:`Manifest.MoM` file's signature.                   |
|                               |                                                          |
|                               | For now, we strongly recommend that you do not modify    |
|                               | this variable, as swupd expects a certificate with a     |
|                               | very specific configuration to sign and verify           |
|                               | properly.                                                |
+-------------------------------+----------------------------------------------------------+
| `CONTENTURL` and `VERSIONURL` | Set these variables to the IP address of the web server  |
|                               | hosting the update content.                              |
|                               |                                                          |
|                               | VERSIONURL is the IP address where the swupd client      |
|                               | looks to determine if a new version is available.        |
|                               |                                                          |
|                               | CONTENTURL is the location from which swupd pulls        |
|                               | content updates.                                         |
|                               |                                                          |
|                               | If the web server is on the same machine as the          |
|                               | SERVER_STATE_DIR directory, you can create a symlink to  |
|                               | the directory in your web server's document root to      |
|                               | easily host the content.                                 |
|                               |                                                          |
|                               | These URLs are embedded in the images created by mixer.  |
+-------------------------------+----------------------------------------------------------+
| `LOCAL_BUNDLE_DIR`            | Sets the path where mixer stores the local bundle        |
|                               | definition files. The bundle definition files include    |
|                               | any new, original bundles you create, along with any     |
|                               | edited versions of upstream bundles.                     |
+-------------------------------+----------------------------------------------------------+
| `SERVER_STATE_DIR`            | Sets the path to which mixer outputs content. By         |
|                               | default, mixer automatically sets the path.              |
+-------------------------------+----------------------------------------------------------+
| `VERSIONS_PATH`               | Sets the path for the mix version and upstream version's |
|                               | two state files: :file:`mixversion` and                  |
|                               | :file:`upstreamversion`. mixer creates both files for    |
|                               | you when you set up the workspace.                       |
+-------------------------------+----------------------------------------------------------+
| `YUM_CONF`                    | Sets the path where mixer automatically generates the    |
|                               | :file:`.yum-mix.conf` file.                              |
|                               |                                                          |
|                               | The yum configuration file points the chroot-builder to  |
|                               | where the RPMs are stored.                               |
+-------------------------------+----------------------------------------------------------+
| **Table 1**: *Variables in builder.conf*                                                 |
+-------------------------------+----------------------------------------------------------+

Format version
--------------

Compatible versions of an OS are tracked with an OS *compatibility epoch*.
Versions of an OS within an epoch are fully compatible and can update to any
other version within that epoch. The compatibility epoch is set as the
`Format` variable in the :file:`mixer.state` file. Variables in the
:file:`mixer.state` are used by mixer between executions and should not be
manually changed.

Format bump
-----------

Mixer needs to produce content that is consumable by swupd. For swupd to
consume the content, it needs a consistent protocol that describes the
requirements of the Manifest.

If the `Format` increments to a new epoch (a "format bump"),  the underlying
`swupd` protocol has changed such that updating from one build version in an
old format to a new build version in a new format is **only** allowed if one
performs a corresponding format bump.

Format bumps are “checkpoints” (see Figure 1). The first release (20) is
built on the previous format with a `swupd` that is capable of interpreting
the next format. The second release (30) has the same content, but it’s
built in the new format.

Suppose you have build version 10, but you need the tools in build version
40. Whereas version 10 belongs to Format 27, version 40 belongs to Format
28. The swupd client needs to follow formats sequentially. First, you must
update to version 20, which effectively enables a format bump to version 30.
Doing a format bump bridges the gap so your mix can progress to build
version 40.

.. figure:: ../../_figures/mixer/format-bump.png
   :alt: Format bump

   Figure 1: Format bump

.. note::
   if you update to build 20 and then check which format of the distro is
   used, the new build version will show 30, and the new format will show 28.

.. rst-class:: content-collapse

Bundles
=======

mixer stores information about the bundles included in a mix in a flat file
called :file:`mixbundles`, which is located in the path set by the 
VERSIONS_PATH variable in :file:`builder.conf`. :file:`mixbundles` is 
automatically created when the mix is initiated. mixer will refresh the file 
each time you change the bundles in the mix.

Bundles belong in one of two categories: upstream or local. Upstream
bundles are those provided by |CL|. Local bundles are either modified upstream bundles or new local bundles.

Upstream bundles
----------------

mixer automatically downloads and caches upstream bundle definition files.
These definition files are stored in the upstream-bundles directory in the
workspace. Do not modify the files in this directory. This directory is
simply a mirror for mixer to use. mixer will automatically delete the
contents of this directory before repopulating it on-the-fly if a new
version must be downloaded.

The mixer tool automatically caches the bundles for the |CL| version
configured in the :file:`upstreamversion` file. :command:`mixer` also 
cleans up old versions once they are no longer needed.

Local bundles
-------------

Local bundles are bundles that you create, or are edited versions of upstream
bundles. Local bundle definition files are stored in the local-bundles
directory in the workspace. The LOCAL_BUNDLE_DIR variable sets the path of this directory in the :file:`builder.conf` file.

*mixer always checks for local bundles first and the upstream bundles
second.* So bundles in the local-bundles directory will always take
precedence over any upstream bundles that have the same name. This
precedence enables you to copy upstream bundles locally, and edit into a
local variation.


Bundle definition files
-----------------------

A ``bundle definition`` file consists of a header, followed by a list 
of packages and directives. The header holds important meta-data, like 
the TITLE, DESCRIPTION, and STATUS. Other meta-data include TAGS, which 
define a bundle's function in the ecosystem, and MAINTAINER, which gives 
contact information. 

Following the header are the directives, shown in Table 2. 

.. list-table:: **Table 2**: Bundle directives
   :widths: 50,50
   :header-rows: 1 

   * - **Directive**
     - **Description**

   * - ``include(<required-bundle-name>)``
     - Add <required-bundle-name> with this bundle

   * - ``also-add(<optional-bundle-name>)``
     - Add <optional-bundle-name> unless the option ``--skip-optional`` is used with ``swupd bundle-add``.

   * - ``content(<full/path/to/non-packaged/content>)``
     - Add the non-packaged content to the bundle. Refer to :ref:`swupd-3rd-party` for usage of this directive. 

Following is `cluster-tools`, an upstream bundle definition file. The
directives are highlighted, and the rest are packages.

.. code-block:: bash
   :emphasize-lines: 8-12

   [TITLE]: cluster-tools
   [DESCRIPTION]: Utilities to manage computer clusters
   [STATUS]: Active
   [CAPABILITIES]: HPC
   [TAGS]: Tools and Utilities
   [MAINTAINER]: Juro Bystricky <juro.bystricky@intel.com>

   include(curl)
   include(libglib)
   include(libX11client)
   also-add(openmpi)
   also-add(modules)
   munge
   pmix
   pdsh
   slurm

Bundle configuration
--------------------

mixer provides commands to configure the bundles for a mix, such as to add a
bundle to a mix, to create a new bundle for a mix, or to remove a bundle from a mix. View the `mixer.bundle man page`_ for a full list of commands and more information on configuring bundles in a mix.

Editing an existing local bundle is as simple as opening the bundle definition
file in your favorite editor, making the desired edits, and saving your changes.

.. note::

   Removing bundles from a mix: By default, removing a bundle will only
   remove the bundle from the mix. The local bundle definition file will
   still remain. To completely remove a bundle, including its local bundle definition file, use the :command:`--local` flag.

   If you remove the bundle definition file for a local, edited version of an
   upstream bundle in a mix, the mix reverts to reference the original upstream version of the bundle.

.. rst-class:: content-collapse

.. _set-up-nginx-web-server-start:

Set up a nginx web server for mixer
===================================

A web server is needed to host your update content. In this example, 
the nginx web server is used.

#. Install the :command:`nginx` bundle.

   .. code-block:: bash

      sudo swupd bundle-add nginx

#. Create a symbolic link to the mixer update content directory. 

   .. code-block:: bash

      sudo mkdir -p /var/www

      sudo ln -sf $HOME/mixer/update/www /var/www/mixer

#. Set up nginx configuration files.

   .. code-block:: bash

      sudo mkdir -p  /etc/nginx/conf.d

      sudo cp -f /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf

#. Grant ``$USER`` permission to run the web server.

   .. code-block:: bash

      sudo tee -a /etc/nginx/nginx.conf << EOF
      user $USER;
      EOF

#. Configure the mixer update server. 

   .. code-block:: bash

      sudo tee -a /etc/nginx/conf.d/mixer-server.conf << EOF
      server {
        server_name localhost;
        location / {
          root /var/www/mixer;
          autoindex on;
        }
      }
      EOF

#. Restart the daemon, enable nginx on boot, and start the service.

   .. code-block:: bash

      sudo systemctl daemon-reload

      sudo systemctl enable nginx --now

#. Verify the web server is running at \http://<IP-address-of-web-server>.
   If there's no mix content yet, the expected response from nginx will be 
   a ``404 Not Found``.

.. _set-up-nginx-web-server-end:

Related topics
**************

* :ref:`autospec`
* :ref:`bundles-guide`
* :ref:`swupd-guide`
* :ref:`swupd-3rd-party`

.. _mixer man page: https://github.com/clearlinux/mixer-tools/blob/master/docs/mixer.1.rst
.. _mixer.init man page: https://github.com/clearlinux/mixer-tools/blob/master/docs/mixer.init.1.rst
.. _mixer.bundle man page: https://github.com/clearlinux/mixer-tools/blob/master/docs/mixer.bundle.1.rst
.. _mixer.build man page: https://github.com/clearlinux/mixer-tools/blob/master/docs/mixer.build.1.rst
.. _releases: https://github.com/clearlinux/clr-bundles/releases
.. _afb.sh reference script: https://github.com/clearlinux/mixer-tools/blob/master/afb.sh
.. _clr-installer: https://github.com/clearlinux/clr-installer
.. _Installer YAML Syntax:
   https://github.com/clearlinux/clr-installer/blob/master/scripts/InstallerYAMLSyntax.md

