mirror of
https://github.com/clearlinux/clear-linux-documentation.git
synced 2026-11-22 03:22:45 +00:00
Merge pull request #400 from krisdale/kd-autospec-apply-guide-template
Apply guide template to autospec guide + update examples
This commit is contained in:
@@ -1,182 +1,167 @@
|
||||
.. _autospec:
|
||||
|
||||
Build RPMs with autospec
|
||||
########################
|
||||
autospec
|
||||
########
|
||||
|
||||
This guide shows you how to create RPMs with :ref:`autospec <autospec-about>`,
|
||||
a tool that assists in automated creation and maintenance of RPM packaging
|
||||
on |CL-ATTR|.
|
||||
**autospec** is a tool to assist in the automated creation and maintenance of
|
||||
RPM packaging in |CL-ATTR|. Where a standard RPM build process using
|
||||
:command:`rpmbuild` requires a tarball and :file:`.spec` file to start, autospec
|
||||
requires only a tarball and package name to start.
|
||||
|
||||
See our :ref:`autospec concept page <autospec-about>` for a detailed explaination
|
||||
of how ``autospec`` works on |CL|. For a general understanding of how RPMs work,
|
||||
we recommend visiting the `rpm website`_ or the `RPM Packaging Guide`_ .
|
||||
.. contents::
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
Description
|
||||
***********
|
||||
|
||||
The autospec tool attempts to infer the requirements of the :file:`.spec` file
|
||||
by analyzing the source code and :file:`Makefile` information. It will
|
||||
continuously run updated builds based on new information discovered from build
|
||||
failures until it has a complete and valid :file:`.spec` file. If needed, you
|
||||
can influence the behavior of autospec and customize the build by providing
|
||||
optional `control files`_ to the autospec tool.
|
||||
|
||||
autospec uses mock as a sandbox to run the builds. Visit the `mock wiki`_ for
|
||||
additional information on using mock.
|
||||
|
||||
For a general understanding of how RPMs work, visit the `rpm website`_ or the
|
||||
`RPM Packaging Guide`_ .
|
||||
|
||||
How it works
|
||||
************
|
||||
|
||||
Learn the autospec tool set up and process.
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
Prerequisites
|
||||
*************
|
||||
=============
|
||||
|
||||
This guide requires that you:
|
||||
The setup for building source in |CL| must be completed before using the
|
||||
autospec tool.
|
||||
|
||||
* Have installed |CL| on a host machine or virtual environment. For detailed
|
||||
instructions on installing |CL|, visit the :ref:`get-started` section.
|
||||
Refer to `Setup environment to build source`_ for instructions on completing
|
||||
setup.
|
||||
|
||||
* :ref:`install-tooling`
|
||||
Create an RPM
|
||||
=============
|
||||
|
||||
.. _install-tooling:
|
||||
The basic autospec process is described in the following steps:
|
||||
|
||||
Install the |CL| development tooling framework
|
||||
==============================================
|
||||
#. The :command:`make autospec` command generates a :file:`.spec` file based on
|
||||
analysis of code and existing control files.
|
||||
|
||||
.. _install-tooling-after-header:
|
||||
Any control files should be located in the same directory as the resulting
|
||||
:file:`.spec` file.
|
||||
|
||||
#. Install the `os-clr-on-clr` developer bundle on your host system.
|
||||
View the `autospec README`_ for more information on `control files`_.
|
||||
|
||||
.. code-block:: bash
|
||||
#. autospec creates a build root with mock config.
|
||||
|
||||
sudo swupd bundle-add os-clr-on-clr
|
||||
#. autospec attempts to build an RPM from the generated :file:`.spec`.
|
||||
|
||||
#. Download the :file:`user-setup.sh` script.
|
||||
#. autospec detects any missed declarations in the :file:`.spec`.
|
||||
|
||||
.. code-block:: bash
|
||||
#. If build errors occur, autospec will scan the build log to try and detect
|
||||
the root cause.
|
||||
|
||||
curl -O https://raw.githubusercontent.com/clearlinux/common/master/user-setup.sh
|
||||
#. If autospec detects the root cause and knows how to continue, it will restart
|
||||
the build automatically at step 1 with updated build instructions.
|
||||
|
||||
#. Make :file:`user-setup.sh` executable.
|
||||
#. Otherwise, autospec will stop the build for user inspection to resolve the
|
||||
errors. Respond to the build process output by fixing source code issues
|
||||
and/or editing control files to resolve issues, which may include
|
||||
dependencies or exclusions. See `autospec README`_ for more information on
|
||||
control files.
|
||||
|
||||
.. code-block:: bash
|
||||
The user resumes the process at step 1 after errors are resolved.
|
||||
|
||||
chmod +x user-setup.sh
|
||||
If a binary dependency doesn't exist in |CL|, you will need to build it
|
||||
before running autospec again.
|
||||
|
||||
#. Run the script as an unprivileged user.
|
||||
Following these steps, autospec continues to rebuild the package, based on
|
||||
new information discovered from build failures, until it has a valid
|
||||
:file:`.spec`. If no build errors occur, RPM packages are successfully built.
|
||||
|
||||
.. code-block:: bash
|
||||
Examples
|
||||
********
|
||||
|
||||
./user-setup.sh
|
||||
Complete `Setup environment to build source`_ before using these examples.
|
||||
|
||||
#. After the script completes, log out and log in again to complete
|
||||
the setup process.
|
||||
.. contents::
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
The `user-setup script`_ creates a folder called :file:`clearlinux`, which
|
||||
contains the :file:`Makefile`, :file:`packages`, and :file:`projects`
|
||||
subfolders.
|
||||
Example 1: Build RPM with existing spec file
|
||||
============================================
|
||||
|
||||
The :file:`projects` folder contains the main tools, `autospec`
|
||||
and `common`, used for making packages in |CL|.
|
||||
This example shows how to build a RPM from a pre-packaged upstream package, with
|
||||
an existing spec file. The example uses the ``dmidecode`` package.
|
||||
|
||||
.. _install-tooling-end:
|
||||
|
||||
Create a RPM with autospec
|
||||
**************************
|
||||
|
||||
Choose one of the following options to build RPMs and manage source
|
||||
code:
|
||||
|
||||
* :ref:`build-a-new-rpm` and spec file using ``make autospecnew``.
|
||||
|
||||
* :ref:`build-source-code-with-existing-spec-file` using ``make build``, without changing the
|
||||
spec file.
|
||||
|
||||
* :ref:`generate-a-new-spec-file` using ``make autospec``, based on changes in the control files.
|
||||
|
||||
.. _build-a-new-rpm:
|
||||
|
||||
Option 1: Build a new RPM
|
||||
=========================
|
||||
|
||||
Use this method to build a new RPM with no spec file. In this example,
|
||||
we build a new helloclear RPM.
|
||||
|
||||
#. Navigate to the autospec workspace.
|
||||
#. Navigate to the autospec workspace and clone the ``dmidecode`` package:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/clearlinux
|
||||
|
||||
#. Enter the command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
make autospecnew URL="https://github.com/clearlinux/helloclear/archive/helloclear-v1.0.tar.gz"
|
||||
NAME="helloclear"
|
||||
make clone_dmidecode
|
||||
|
||||
.. note::
|
||||
|
||||
For a local tarball, use this type of *URL*: \file://<absolute-path-to-tarball>
|
||||
|
||||
#. If build failures or dependency issues occur, continue below.
|
||||
Otherwise, skip directly to `Next steps`_.
|
||||
|
||||
#. Navigate to the specific package.
|
||||
You can clone all package repos at once using:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/clearlinux/packages/[package-name]
|
||||
make [-j NUM] clone-packages
|
||||
|
||||
#. Respond to the build process output by editing control files to resolve
|
||||
issues, which may include dependencies or exclusions.
|
||||
See `autospec readme`_
|
||||
The optional NUM is the number of threads to use.
|
||||
|
||||
#. Run this command:
|
||||
For a list of available packages, view the
|
||||
:file:`~/clearlinux/projects/common/packages` file.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
make autospec
|
||||
|
||||
Repeat the last two steps above until all errors are resolved and you
|
||||
complete a successful build.
|
||||
|
||||
**Congratulations!**
|
||||
|
||||
You've successfully created a RPM.
|
||||
|
||||
Skip to `Next steps`_.
|
||||
|
||||
.. _build-source-code-with-existing-spec-file:
|
||||
|
||||
Option 2: Build source code with an existing spec file
|
||||
======================================================
|
||||
|
||||
Use this method if you only want to build the RPM using the spec file. This
|
||||
method assumes that a spec file already exists. In this example, we run a
|
||||
``make build`` on the ``dmidecode`` package.
|
||||
|
||||
#. Navigate to the ``dmidecode`` package in clearlinux:
|
||||
#. Navigate to the local copy of the ``dmidecode`` package and build it:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/clearlinux/packages/dmidecode/
|
||||
|
||||
#. To download the tarball and build, run the command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
make build
|
||||
|
||||
**Congratulations!**
|
||||
#. The resulting RPMs are in :file:`./rpms`. Build logs and additional RPMs are
|
||||
in :file:`./results`.
|
||||
|
||||
You've successfully created a RPM.
|
||||
Example 2: Build a new RPM
|
||||
==========================
|
||||
|
||||
Skip to `Next steps`_.
|
||||
This example shows how to build a new RPM with no spec file. The example will
|
||||
create a simple helloclear RPM.
|
||||
|
||||
.. _generate-a-new-spec-file:
|
||||
|
||||
Option 3: Generate a new spec file with a pre-defined package
|
||||
=============================================================
|
||||
|
||||
Use this method to modify an existing package. In this example, you will
|
||||
modify an existing |CL| package called ``dmidecode`` to create a custom
|
||||
RPM. You will make a simple change to this package, change the revision to
|
||||
a new number that is higher than the |CL| OS version, and rebuild the package.
|
||||
|
||||
#. Navigate to clearlinux:
|
||||
#. Navigate to the autospec workspace and build the helloclear RPM. The
|
||||
:file:`Makefile` provides a :command:`make autospecnew` that can
|
||||
automatically generate an RPM package using the autospec tool. You must pass
|
||||
the URL to the source tarball and the NAME of the RPM you wish to create:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/clearlinux
|
||||
make autospecnew URL="https://github.com/clearlinux/helloclear/archive/helloclear-v1.0.tar.gz" NAME="helloclear"
|
||||
|
||||
#. Copy the ``dmidecode`` package.
|
||||
The resulting RPMs are in :file:`./packages/helloclear/rpms`. Builde logs and
|
||||
additional RPMs are in :file:`./packages/helloclear/results`.
|
||||
|
||||
Example 3: Generate a new spec file with a pre-defined package
|
||||
==============================================================
|
||||
|
||||
This example shows how to modify an existing package to create a custom RPM. In
|
||||
this example you will make a simple change to the ``dmidecode`` package and
|
||||
rebuild the package.
|
||||
|
||||
#. Navigate to the autospec workspace and clone the ``dmidecode`` package:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/clearlinux
|
||||
make clone_dmidecode
|
||||
|
||||
#. Navigate into the *dmidecode* directory:
|
||||
@@ -185,9 +170,9 @@ a new number that is higher than the |CL| OS version, and rebuild the package.
|
||||
|
||||
cd packages/dmidecode
|
||||
|
||||
#. With an editor, open the :file:`excludes` file and add these lines:
|
||||
#. Open the :file:`excludes` file with an editor and add these lines:
|
||||
|
||||
.. code-block:: bash
|
||||
.. code-block:: console
|
||||
|
||||
/usr/bin/biosdecode
|
||||
/usr/bin/ownership
|
||||
@@ -201,49 +186,211 @@ a new number that is higher than the |CL| OS version, and rebuild the package.
|
||||
These files aren't needed by dmidecode, so we can remove them without
|
||||
any issues.
|
||||
|
||||
#. Save the file and exit.
|
||||
|
||||
#. At :file:`~/clearlinux/packages/dmidecode`, build the modified
|
||||
``dmidecode`` package:
|
||||
#. In the :file:`dmidecode` directory, build the modified ``dmidecode`` package:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
make autospec
|
||||
|
||||
When the process completes, you will see new RPM packages in the
|
||||
:file:`results/` folder.
|
||||
#. The resulting RPMs are in :file:`./rpms`. Logs are in :file:`./results`.
|
||||
|
||||
#. To view the new RPM packages, enter:
|
||||
Example 4: Provide control files to autospec
|
||||
============================================
|
||||
|
||||
This example shows how to modify control files to correct build failures that
|
||||
autospec is unable to resolve. In this example you will add a missing license
|
||||
and dependencies in order for autospec to complete a successful build.
|
||||
|
||||
#. Navigate to the autospec workspace:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
ls /clearlinux/packages/dmidecode/results/
|
||||
cd ~/clearlinux
|
||||
|
||||
**Congratulations!**
|
||||
#. If you have not already, clone all upstream package repos:
|
||||
|
||||
You've successfully created a RPM.
|
||||
.. code-block:: bash
|
||||
|
||||
Next steps
|
||||
make [-j NUM] clone-packages
|
||||
|
||||
The optional NUM is the number of threads to use.
|
||||
|
||||
.. note::
|
||||
|
||||
In a later step of this example, we will search the cloned package repos
|
||||
for a missing dependency.
|
||||
|
||||
#. Build the opae-sdk RPM:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
make autospecnew URL="https://github.com/OPAE/opae-sdk/archive/0.13.0.tar.gz" NAME="opae-sdk"
|
||||
|
||||
This will give an error for a missing license file:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
[FATAL] Cannot find any license or opae-sdk.license file!
|
||||
|
||||
#. Navigate to the package with build failures:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd packages/opae-sdk
|
||||
|
||||
#. Add a license:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
echo "Intel Corporation" > opae-sdk.license
|
||||
|
||||
#. Run autospec again:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
make autospec
|
||||
|
||||
This will result in a generic error:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
[FATAL] Build failed, aborting
|
||||
|
||||
#. Open the build log to view the error details:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cat ./results/build.log
|
||||
|
||||
In the build log, you will find details for the specific failures. In this
|
||||
instance, there are missing dependencies:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files:
|
||||
CJSON_LIBRARY
|
||||
linked by target "opae-c++-utils" in directory /builddir/build/BUILD/opae-sdk-0.13.0/tools/c++utilslib
|
||||
json-c_LIBRARIES
|
||||
linked by target "opae-c" in directory /builddir/build/BUILD/opae-sdk-0.13.0/libopae
|
||||
libuuid_LIBRARIES
|
||||
linked by target "opae-c" in directory /builddir/build/BUILD/opae-sdk-0.13.0/libopae
|
||||
|
||||
#. Search the spec files of upstream |CL| packages to see if the json-c library
|
||||
is availabe. In this case, it does exist and we'll add the json-c 'dev'
|
||||
package into the buildreq_add:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
grep 'json-c\.so$' ~/clearlinux/packages/*/*.spec
|
||||
echo "json-c-dev" >> buildreq_add
|
||||
|
||||
.. note::
|
||||
|
||||
This search step works only if the user cloned all of the upstream package
|
||||
repos. In this example, upstream package repos were cloned in a previous
|
||||
step.
|
||||
|
||||
#. Search the spec files of upstream |CL| packages to see if the libuuid library
|
||||
is available. In this case, it exists in the util-linux package, so we'll add
|
||||
util-linux-dev package into the buildreq_add:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
grep 'libuuid\.so$' ~/clearlinux/packages/*/*.spec
|
||||
echo "util-linux-dev" >> buildreq_add
|
||||
|
||||
#. Run autospec again and find the successfully-generated RPMs in the rpms
|
||||
directory:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
make autospec
|
||||
|
||||
.. note::
|
||||
|
||||
If you need a dependency that does not exist in the |CL| repo, you must first
|
||||
build it manually (see `Example 2: Build a new RPM`_), then add the repo so
|
||||
that autospec knows the package exists. For example:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/clearlinux/packages/<package-name>
|
||||
make repoadd
|
||||
make repostatus
|
||||
|
||||
You only need to add the dependency to the :file:`buildreq_add` control file
|
||||
if autospec is not able to automatically find the correct dependency on its
|
||||
own.
|
||||
|
||||
References
|
||||
**********
|
||||
|
||||
Now you can create a custom bundle with your new RPM and use it with |CL|:
|
||||
Reference the `autospec README`_ for details regarding autospec commands and
|
||||
options.
|
||||
|
||||
* Use the :ref:`Mixer tool <mixer>` to add a new bundle to your derivative of |CL|.
|
||||
* Use the :ref:`Mixin tool <mixin>` to customize your upstream |CL| installation with a new bundle.
|
||||
Setup environment to build source
|
||||
=================================
|
||||
|
||||
.. _install-tooling-after-header:
|
||||
|
||||
Setup of the workspace and tooling used for building source in |CL| is mostly
|
||||
automated for you with a setup script. It uses tools from the
|
||||
:command:`os-clr-on-clr` bundle.
|
||||
|
||||
The setup script creates a workspace in the :file:`clearlinux` folder, with the
|
||||
subfolders :file:`Makefile`, :file:`packages`, and :file:`projects`. The
|
||||
:file:`projects` folder contains the main tools used for making packages in
|
||||
|CL|: `autospec` and `common`.
|
||||
|
||||
Follow these steps to setup the workspace and tooling for building source:
|
||||
|
||||
#. Install the :command:`os-clr-on-clr` bundle:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd bundle-add os-clr-on-clr
|
||||
|
||||
#. Download the :file:`user-setup.sh` script:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
curl -O https://raw.githubusercontent.com/clearlinux/common/master/user-setup.sh
|
||||
|
||||
#. Make :file:`user-setup.sh` executable:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
chmod +x user-setup.sh
|
||||
|
||||
#. Run the script as an unprivileged user:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
./user-setup.sh
|
||||
|
||||
#. After the script completes, log out and log in again to complete the setup
|
||||
process.
|
||||
|
||||
#. Set your Git user email and username for the repos on your system:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git config --global user.email "you@example.com"
|
||||
git config --global user.name "Your Name"
|
||||
|
||||
This global setting is used by |CL| tools that make use of Git.
|
||||
|
||||
.. _install-tooling-end:
|
||||
|
||||
Related topics
|
||||
**************
|
||||
|
||||
* :ref:`Mixer tool <mixer>`
|
||||
* :ref:`Mixin tool <mixin>`
|
||||
* :ref:`autospec <autospec-about>`
|
||||
* :ref:`Bundles <bundles-about>`
|
||||
|
||||
|
||||
.. _rpm website: http://rpm.org
|
||||
|
||||
.. _RPM Packaging Guide: https://rpm-packaging-guide.github.io/
|
||||
|
||||
.. _user-setup script: https://github.com/clearlinux/common/blob/master/user-setup.sh
|
||||
|
||||
.. _autospec readme: https://github.com/clearlinux/autospec
|
||||
.. _autospec README: https://github.com/clearlinux/autospec
|
||||
.. _control files: https://github.com/clearlinux/autospec#control-files
|
||||
.. _mock wiki: https://github.com/rpm-software-management/mock/wiki
|
||||
.. _rpm website: http://rpm.org
|
||||
.. _RPM Packaging Guide: https://rpm-packaging-guide.github.io/
|
||||
|
||||
Reference in New Issue
Block a user