mirror of
https://github.com/clearlinux/clear-linux-documentation.git
synced 2026-09-27 08:40:32 +00:00
Merge branch 'master' of github.com:clearlinux/clear-linux-documentation into rtd-theme
Signed-off-by: Kevin Putnam <kevin.putnam@intel.com>
This commit is contained in:
@@ -0,0 +1,189 @@
|
||||
.. _faq:
|
||||
|
||||
Frequently Asked Questions (FAQ)
|
||||
################################
|
||||
|
||||
Below is a list of commonly asked questions with answers sourced from the
|
||||
|CL-ATTR| team and `Clear Linux community forums`_.
|
||||
|
||||
.. contents:: :local:
|
||||
:depth: 2
|
||||
|
||||
|
||||
General
|
||||
*******
|
||||
|
||||
Why did you make another distro?
|
||||
================================
|
||||
|
||||
The |CL| team felt that performance was left on the table with Linux software.
|
||||
|CL| takes a holistic approach to improving performance across the stack. We
|
||||
also wanted to take more modern approaches with OS updates and tooling.
|
||||
|
||||
|
|
||||
|
||||
Can other distros copy |CL| improvements?
|
||||
=========================================
|
||||
|
||||
Yes, we absolutely love open source reuse and upstreaming improvements.
|
||||
|
||||
|
|
||||
|
||||
How often do you update?
|
||||
========================
|
||||
|
||||
The |CL| team puts out multiple releases a week, often releasing 2 or more
|
||||
times a day. This rolling release approach allows |CL| to remain agile to
|
||||
upstream changes and security patches.
|
||||
|
||||
|
|
||||
|
||||
Is telemetry required?
|
||||
======================
|
||||
|
||||
The telemetry solution provided by |CL| is entirely optional and customizable.
|
||||
It is disabled by default. If you do choose to enable telemetry, the data
|
||||
helps the |CL| team proactively identify and resolve bugs. See the
|
||||
:ref:`telemetry <telemetry-about>` page for more information.
|
||||
|
||||
|
|
||||
|
||||
What is the default firewall?
|
||||
=============================
|
||||
|
||||
|CL| packages :command:`iptables` as a bundle, however, there are no default
|
||||
firewall rules. All network traffic is allowed by default.
|
||||
|
||||
|
|
||||
|
||||
Where are the files that I usually see under /etc like fstab?
|
||||
=============================================================
|
||||
|
||||
|CL| has a stateless design that maintains a separation between system files
|
||||
and user files. Default values are stored under :file:`/usr/share/defaults/`.
|
||||
Files under :file:`/etc/` are not created unless you create one.
|
||||
|
||||
A blog post explaining how this is accomplished with :file:`/etc/fstab/`
|
||||
specifically is available here:
|
||||
https://clearlinux.org/news-blogs/where-etcfstab-clear-linux
|
||||
|
||||
|
|
||||
|
||||
Software packages
|
||||
*****************
|
||||
|
||||
How is software installed and updated?
|
||||
======================================
|
||||
|
||||
|CL| provides software in the form of :ref:`bundles <bundles-about>` and
|
||||
updates software with :ref:`swupd <swupd-about>`.
|
||||
|
||||
:ref:`FlatPak\* <flatpak>` is an application virtualization solution that allows
|
||||
more software to be available to |CL| users by augmenting the software |CL|
|
||||
packages natively with software available through FlatPak.
|
||||
|
||||
Our goal is to have software packaged natively and made available through
|
||||
bundles whenever possible.
|
||||
|
||||
|
|
||||
|
||||
Does |CL| use RPMs like other distros?
|
||||
======================================
|
||||
|
||||
|CL| provides software in the form of :ref:`bundles <bundles-about>`. The RPM
|
||||
format is used as an intermediary step for packaging and determining software
|
||||
dependencies at OS build time.
|
||||
|
||||
Individual RPMs can sometimes be manually installed on a |CL| system with the
|
||||
right tools, but that is not the intended use case.
|
||||
|
||||
|
|
||||
|
||||
Can I install a software package from another OS on |CL|?
|
||||
=========================================================
|
||||
|
||||
Software that is packaged in other formats for other Linux distributions is
|
||||
not guaranteed to work on |CL| and may be impacted by |CL| updates.
|
||||
|
||||
If the software you're seeking is open source, please submit a request to add
|
||||
it to |CL|. Submit requests on GitHub\* here:
|
||||
https://github.com/clearlinux/distribution/issues
|
||||
|
||||
|
|
||||
|
||||
Software availability
|
||||
*********************
|
||||
|
||||
What software is available on |CL|?
|
||||
===================================
|
||||
|
||||
Available software can be found in the `Software Store`_, through the GNOME\*
|
||||
Software application on the desktop, or by using :ref:`swupd search <bundle-commands>`.
|
||||
|
||||
|
|
||||
|
||||
Is Google\* Chrome\* available?
|
||||
===============================
|
||||
|
||||
The Google Chrome web browser is not distributed as a bundle in |CL| due to
|
||||
copyright and licensing complexities.
|
||||
|
||||
A discussion on manually installing and maintaining Google Chrome can be found
|
||||
on GitHub: https://github.com/clearlinux/distribution/issues/422
|
||||
|
||||
|
|
||||
|
||||
Is FFmpeg available?
|
||||
====================
|
||||
|
||||
`FFmpeg`_ is a multimedia software suite, which is commonly used for
|
||||
various media encoding/decoding, streaming, and playback.
|
||||
|
||||
|CL| does not distribute FFmpeg due to well-known licensing and legal
|
||||
complexities (See https://www.ffmpeg.org/legal.html and
|
||||
http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html).
|
||||
|
||||
Read more in the |CL| repository, including discussion of an alternative
|
||||
hardware-based solution:
|
||||
https://github.com/clearlinux/distribution/issues/429.
|
||||
|
||||
While |CL| cannot distribute FFmpeg, a manual solution to build and install
|
||||
FFmpeg under :file:`/usr/local` has been shared on the community forums:
|
||||
https://community.clearlinux.org/t/how-to-h264-etc-support-for-firefox-including-ffmpeg-install.
|
||||
|
||||
|
|
||||
|
||||
Is ZFS\* available?
|
||||
===================
|
||||
|
||||
ZFS is not available with |CL| because of copyright and licensing
|
||||
complexities. BTRFS is an alternative filesystem that is available in |CL|
|
||||
natively.
|
||||
|
||||
A user on GitHub notes that the ZFS kernel module can be compiled, built, and
|
||||
installed manually: https://github.com/clearlinux/distribution/issues/631
|
||||
|
||||
|
|
||||
|
||||
Can you add a driver that I need?
|
||||
=================================
|
||||
|
||||
If a kernel module is available as part of the Linux kernel source tree but
|
||||
not enabled in the |CL| kernels, in many cases the |CL| team will enable it
|
||||
upon request. Submit requests on GitHub here:
|
||||
https://github.com/clearlinux/distribution/issues
|
||||
|
||||
The |CL| team does not typically add out-of-tree kernel modules as a matter of
|
||||
practice because of the maintenance overhead. If the driver was unable to be
|
||||
merged upstream, there is a good chance we may be unable to merge it for
|
||||
similar reasons.
|
||||
|
||||
Kernel modules can be individually built and installed on |CL|. See the
|
||||
:ref:`kernel modules <kernel-modules>` page for more information.
|
||||
|
||||
|
|
||||
|
||||
|
||||
.. _`Clear Linux community forums`: https://community.clearlinux.org
|
||||
.. _`Software Store`: https://clearlinux.org/software
|
||||
.. _`FFmpeg`: https://ffmpeg.org/
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 94 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 88 KiB |
@@ -0,0 +1,145 @@
|
||||
.. _stateless:
|
||||
|
||||
Stateless
|
||||
#########
|
||||
|
||||
In most operating systems, files can become intermingled with user and system
|
||||
data and configurations.
|
||||
|
||||
.. figure:: figures/stateless-1.png
|
||||
:scale: 45%
|
||||
:align: center
|
||||
:alt: Stateless: User and system files mixed
|
||||
|
||||
Figure 1: Without stateless, user and system files become mixed on the filesystem over time.
|
||||
|
||||
|CL-ATTR| has a stateless design philosophy of which the goal is to provide an
|
||||
:abbr:`OS (operating system)` that functions without excessive user
|
||||
configuration or customization. Stateless in this context does *not* mean
|
||||
ephemeral or non-persistent.
|
||||
|
||||
.. contents:: :local:
|
||||
:depth: 2
|
||||
|
||||
|
||||
File-level separation
|
||||
*********************
|
||||
|
||||
To accomplish a stateless design the Linux Filesystem Hierarchy is separated
|
||||
between user-owned areas and |CL|-owned areas.
|
||||
|
||||
.. figure:: figures/stateless-2.png
|
||||
:scale: 45%
|
||||
:align: center
|
||||
:alt: Stateless: User and system files separation
|
||||
|
||||
Figure 2: With stateless, user and system files are separated on the filesystem.
|
||||
|
||||
System areas
|
||||
============
|
||||
File under the :file:`/usr` directory are managed by |CL| as system files.
|
||||
Files written under the :file:`/usr` directory by users can get removed
|
||||
through system updates with :ref:`swupd <swupd-about>`.This operating
|
||||
assumption allows |CL| to verify and maintain integrity of system files.
|
||||
|
||||
User areas
|
||||
==========
|
||||
Files under the :file:`/etc/`, :file:`/home`, and :file:`/var` directories are
|
||||
owned and managed by the user. A freshly installed |CL| system will only have
|
||||
a minimal set of files in the :file:`/etc/` directory and software installed
|
||||
by |CL| does not write to :file:`/etc`. This operating assumption allows |CL|
|
||||
users to clearly identify the configuration that makes their system unique.
|
||||
|
||||
|
||||
Software Configuration
|
||||
**********************
|
||||
|
||||
With stateless separation, default software configurations are read in order
|
||||
from predefined source code, |CL| provided defaults, and user-provided
|
||||
configuration.
|
||||
|
||||
Default configurations
|
||||
======================
|
||||
|
||||
Software in |CL| provides default configuration values so that it is
|
||||
immediately functional, whenever it is appropriate to do so.
|
||||
|
||||
|CL| distributed software packages may be directly modified to include default
|
||||
configuration values or default configuration files may be provided by |CL|
|
||||
under :file:`/usr/share/defaults`. These files can be referenced as templates
|
||||
for customization.
|
||||
|
||||
For example, the default configuration that Apache uses when installed can be
|
||||
found at :file:`/usr/share/defaults/httpd/httpd.conf` directory.
|
||||
|
||||
|
||||
Overriding configurations
|
||||
=========================
|
||||
|
||||
If a configuration needs to be changed, the appropriate file should be
|
||||
modified by the user under :file:`/etc/`. If the configuration file does not
|
||||
already exist, it can be created in the appropriate location.
|
||||
|
||||
User defined configuration files should contain the minimal set of desired
|
||||
changes and rely on default configuration for the rest.
|
||||
|
||||
For example, a customized Apache configuration can be used instead by:
|
||||
|
||||
#. Create the destination directory for the configuration:
|
||||
|
||||
.. code :: bash
|
||||
|
||||
sudo mkdir /etc/httpd
|
||||
|
||||
#. Copy the default configuration as a reference template:
|
||||
|
||||
.. code :: bash
|
||||
|
||||
sudo cp /usr/share/defaults/httpd/httpd.conf /etc/httpd/
|
||||
|
||||
#. Make any desired modifications to the configurations:
|
||||
|
||||
.. code :: bash
|
||||
|
||||
sudoedit /etc/httpd/httpd.conf
|
||||
|
||||
#. Reload the service or reboot the system to pickup any changes:
|
||||
|
||||
.. code :: bash
|
||||
|
||||
systemctl daemon-reload httpd && systemctl restart httpd
|
||||
|
||||
|
||||
This pattern can be used to modify the configurations of other programs too.
|
||||
The `stateless man page`_ has application-specific examples.
|
||||
|
||||
|
||||
System reset
|
||||
************
|
||||
|
||||
Once advantage of the stateless design is that the system defaults can be
|
||||
easily restored by simply deleting everything under :file:`/etc/` and
|
||||
:file:`/var`.
|
||||
|
||||
Running the commands below effectively performs a system reset as if it was
|
||||
just installed:
|
||||
|
||||
.. code::
|
||||
|
||||
sudo rm -rf /etc
|
||||
sudo rm -rf /var
|
||||
|
||||
In other Linux distributions, this can be a catastrophic action that renders a
|
||||
system unable to boot.
|
||||
|
||||
Additional information
|
||||
**********************
|
||||
|
||||
* `stateless man page`_
|
||||
|
||||
* `Where is /etc/fstab in Clear Linux? <https://clearlinux.org/news-blogs/where-etcfstab-clear-linux>`_
|
||||
|
||||
|
||||
.. _`stateless man page`: https://github.com/clearlinux/clr-man-pages/blob/master/stateless.7.rst
|
||||
|
||||
|
||||
@@ -0,0 +1,196 @@
|
||||
.. _cpu-performance:
|
||||
|
||||
CPU Power and Performance
|
||||
#########################
|
||||
|
||||
Modern x86 :abbr:`CPUs (central processing units)` employ a number of features
|
||||
and technologies to balance performance, energy, and thermal efficiencies.
|
||||
|
||||
By default, |CL| prioritizes maximum CPU performance with the philosophy that
|
||||
the faster the program finishes execution, the faster the CPU can return to a
|
||||
low energy idle state. It is important to understand and evaluate all of these
|
||||
technologies when troubleshooting or considering changing the defaults.
|
||||
|
||||
.. contents:: :local:
|
||||
:depth: 1
|
||||
|
||||
CPU power saving mechanisms
|
||||
===========================
|
||||
|
||||
C-states and P-states are both CPU power saving mechanisms that are entered
|
||||
under different operating conditions. The tradeoff is a slightly longer time
|
||||
to exit these states when the CPU is needed once again.
|
||||
|
||||
.. _c-states-section:
|
||||
|
||||
C-states (idle states)
|
||||
----------------------
|
||||
|
||||
C-states are hardware sleep states that are entered when it is determined that
|
||||
the CPU is idle and not executing instructions.
|
||||
|
||||
C-states aim to reduce power utilization by increasingly reducing clock
|
||||
frequency, voltages, and features in each state.
|
||||
|
||||
Although C-states can typically be limited or disabled in a system's UEFI or
|
||||
BIOS configuration, these settings are overridden when the `intel_idle
|
||||
driver`_ is in use.
|
||||
|
||||
To view the current cpuidle driver run this command in a terminal:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
cat /sys/devices/system/cpu/cpuidle/current_driver
|
||||
|
||||
For troubleshooting, C-states can be limited with a kernel command line boot
|
||||
parameter by adding :command:`processor.max_cstate=N intel_idle.max_cstate=N`
|
||||
or completely disabled with :command:`idle=poll`.
|
||||
|
||||
.. note::
|
||||
|
||||
* :command:`processor.max_cstate=0` is changed to :command:`processor.max_cstate=1` by the kernel to be a valid value.
|
||||
|
||||
* :command:`intel_idle.max_cstate=0` disables the Intel Idle driver, not set
|
||||
it to C-state 0.
|
||||
|
||||
.. _p-states-section:
|
||||
|
||||
P-states (performance states)
|
||||
-----------------------------
|
||||
|
||||
P-states, also known as *SpeedStep* on Intel processors or *Cool'n'Quiet* on
|
||||
AMD processors, are states entered while the CPU is active and executing
|
||||
instructions.
|
||||
|
||||
P-states aim to reduce power utilization by adjusting CPU clock frequency and
|
||||
voltages based on CPU demand.
|
||||
|
||||
P-states can typically be limited or disabled in a system's firmware (UEFI/BIOS).
|
||||
|
||||
Turbo boost
|
||||
~~~~~~~~~~~
|
||||
|
||||
`Turbo Boost technology`_, found on some modern Intel CPUs, allows core(s) on
|
||||
a processor to temporarily operate at a higher than rated CPU clock frequency
|
||||
to accommodate demanding workloads if the CPU is under defined power and
|
||||
thermal thresholds.
|
||||
|
||||
Turbo boost is an extension of P-states. As such, changing or limiting
|
||||
C-states or P-states impact the ability of a process to enter Turbo boost.
|
||||
|
||||
Turbo boost can be disabled in a system's UEFI or BIOS. Turbo boost can also
|
||||
be disabled within |CL| with the command:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
echo 1 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo
|
||||
|
||||
Linux CPU clock frequency scaling
|
||||
=================================
|
||||
|
||||
The CPUFreq subsystem in Linux allows the OS to control :ref:`C-states
|
||||
<c-states-section>` and :ref:`P-states <P-states-section>`
|
||||
via CPU drivers and governors that provide algorithms that define how and when
|
||||
to enter these states.
|
||||
|
||||
Scaling driver
|
||||
--------------
|
||||
|
||||
Linux uses the `Intel P-state driver`_, *intel_pstate*, for modern Intel
|
||||
processors from the Sandy Bridge generation or newer. Other processors may
|
||||
default to the *acpi-cpufreq* driver which reads values from the systems UEFI
|
||||
or BIOS.
|
||||
|
||||
To view the current CPU frequency scaling driver run this command in a terminal:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver
|
||||
|
||||
Scaling governor
|
||||
----------------
|
||||
|
||||
|CL| sets the CPU governor to *performance* which calls for the CPU to operate
|
||||
at maximum clock frequency. In other words, P-state P0. While this may sound
|
||||
wasteful at first, it is important to remember that power utilization does not
|
||||
increase significantly simply because of a locked clock frequency without a
|
||||
workload.
|
||||
|
||||
To view the current CPU frequency scaling governor run this command in a terminal:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
|
||||
|
||||
To change the CPU frequency scaling governor:
|
||||
|
||||
#. Disable |CL| enforcement of certain power and performance settings:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
sudo systemctl mask clr-power.timer
|
||||
|
||||
#. Change the governor. In the example below, the governor is set to
|
||||
*performance*:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
|
||||
|
||||
The list of all governors can be found in the `Linux kernel documentation on
|
||||
CPUFreq Governors`_.
|
||||
|
||||
.. note::
|
||||
|
||||
The intel_pstate driver only supports *performance* and *powersave* governors.
|
||||
|
||||
Thermal management
|
||||
==================
|
||||
|
||||
`thermald`_ is a Linux thermal management daemon used to prevent the
|
||||
overheating of platforms. When temperature thresholds are exceeded, thermald
|
||||
forces a C-state by inserting CPU sleep cycles and adjusts available cooling
|
||||
methods. This can be especially desirable for laptops.
|
||||
|
||||
By default, thermald is disabled in |CL| and starts automatically if battery
|
||||
power is detected. thermald can be manually enabled using the systemd service
|
||||
by running the command:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
sudo systemctl enable thermald
|
||||
sudo systemctl start thermald
|
||||
|
||||
For more information, see the thermald man page:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
man thermald
|
||||
|
||||
`ThermalMonitor`_ is a GUI application that can visually graph and log
|
||||
temperatures from thermald. To use ThermalMonitor, add the desktop-apps-extras
|
||||
bundle and add your user account to the power group:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
sudo swupd bundle-add desktop-apps-extras
|
||||
sudo usermod -a -G power <USER>
|
||||
ThermalMonitor
|
||||
|
||||
.. note::
|
||||
|
||||
After adding a new group you must log out and log back in for the new group
|
||||
to take affect.
|
||||
|
||||
|
||||
.. _`Intel P-state driver`: https://www.kernel.org/doc/Documentation/cpu-freq/intel-pstate.txt
|
||||
|
||||
.. _`Linux kernel documentation on CPUFreq Governors`: https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt
|
||||
|
||||
.. _thermald: https://01.org/linux-thermal-daemon
|
||||
|
||||
.. _`intel_idle driver`: https://github.com/torvalds/linux/blob/master/drivers/idle/intel_idle.c
|
||||
|
||||
.. _`ThermalMonitor`: https://github.com/intel/thermal_daemon/tree/master/tools/thermal_monitor
|
||||
|
||||
.. _`Turbo Boost technology`: https://www.intel.com/content/www/us/en/architecture-and-technology/turbo-boost/turbo-boost-technology.html
|
||||
@@ -0,0 +1,89 @@
|
||||
.. _debug:
|
||||
|
||||
Debug System
|
||||
############
|
||||
|
||||
|CL-ATTR| introduces a novel approach to system software debugging using
|
||||
*clr-debug-info*. On the client side, the |CL| debug system obtains any
|
||||
necessary debug information on-the-fly over a network during a debugging
|
||||
session. On the server side, the system curates and compresses debug
|
||||
information into small pieces for efficient downloading.
|
||||
|
||||
For developers, this avoids the interruption during debugging that usually
|
||||
happens when debug information is missing. This can be especially useful on
|
||||
systems where storage is limited.
|
||||
|
||||
|
||||
.. contents:: :local:
|
||||
:depth: 2
|
||||
|
||||
|
||||
Background
|
||||
----------
|
||||
|
||||
Software that is compiled and packaged for general usage in an operating
|
||||
system typically only contains components that are used to execute the
|
||||
program, like binaries and libraries. Extra developer data like the the
|
||||
actual source code and symbol information are separated and excluded for
|
||||
efficiency.
|
||||
|
||||
The debug information helps relate binary code to human readable source code
|
||||
lines and variables. Most of the time this auxiliary information is not needed
|
||||
however without it, debugging a program results in limited visibility.
|
||||
|
||||
|
||||
Usage
|
||||
-----
|
||||
|
||||
The clr-debug-info system is integrated into |CL| and seamlessly engages once
|
||||
installed.
|
||||
|
||||
#. Install the *dev-utils* bundle.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
sudo swupd bundle-add dev-utils
|
||||
|
||||
.. note::
|
||||
|
||||
The *telemetrics* and *performance-tools* bundles also include
|
||||
clr-debug-info.
|
||||
|
||||
|
||||
#. Start a debugging session against a program using a debugger, like GDB.
|
||||
For example, to debug *gnome-control-center*:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
gdb /usr/bin/gnome-control-center
|
||||
|
||||
As the program is stepped through and debug information is needed, the
|
||||
clr_debug_daemon will obtain it in the background.
|
||||
|
||||
|
||||
Implementation
|
||||
--------------
|
||||
|
||||
The implementation of the |CL| debug system is open source and available on
|
||||
GitHub at: https://github.com/clearlinux/clr-debug-info/
|
||||
|
||||
.. figure:: figures/debug-diagram.png
|
||||
:scale: 75%
|
||||
:alt: Debug system communication flow
|
||||
|
||||
Figure 1: The communication flow of the |CL| debug system
|
||||
|
||||
The |CL| debug system implements a :abbr:`FUSE (filesystem in userspace)`
|
||||
filesystem mounted at :file:`/usr/lib/debug` and :file:`/usr/src/debug`. The
|
||||
FUSE filesystem starts automatically. Its status can be verified with
|
||||
:command:`systemctl status clr_debug_fuse.service`.
|
||||
|
||||
The *clr_debug_daemon* is responsible for fetching the appropriate package
|
||||
debug content from the server and making it available for any debugging
|
||||
programs needed it. It is socket activated whenever a request to the local
|
||||
FUSE filesystem occurs. Its status can be verified with :command:`systemctl
|
||||
status clr_debug_daemon.service`.
|
||||
|
||||
|
||||
|CL| hosts debuginfo content packaged for consumption by |CL| debug clients at
|
||||
https://download.clearlinux.org/debuginfo/
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 38 KiB |
@@ -0,0 +1,115 @@
|
||||
.. _tutorial-proxy:
|
||||
|
||||
Setting up proxy
|
||||
################
|
||||
|
||||
This tutorial shows you how to configure your system for use behind an
|
||||
outbound proxy to access the Internet.
|
||||
|
||||
|CL| :ref:`tooling` applications already benefit from the :ref:`autoproxy`
|
||||
feature.
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
Prerequisites
|
||||
*************
|
||||
|
||||
This tutorial assumes you have installed |CL| on your host system.
|
||||
For detailed instructions on installing |CL| on a bare metal system, visit
|
||||
the :ref:`bare metal installation guide <bare-metal-install-desktop>`.
|
||||
|
||||
Shells and programs in a desktop session
|
||||
****************************************
|
||||
|
||||
Terminal
|
||||
========
|
||||
|
||||
Add the following to your ~/.bashrc:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export http_proxy=http://your.http-proxy.url:port
|
||||
export https_proxy=http://your.https-proxy.url:port
|
||||
export ftp_proxy=http://your.ftp-proxy.url:port
|
||||
export socks_proxy=http://your.socks-proxy.url:port
|
||||
export no_proxy=".your-company-domain.com,localhost"
|
||||
export HTTP_PROXY=$http_proxy
|
||||
export HTTPS_PROXY=$https_proxy
|
||||
export FTP_PROXY=$ftp_proxy
|
||||
export SOCKS_PROXY=$socks_proxy
|
||||
export NO_PROXY=$no_proxy
|
||||
|
||||
wget
|
||||
****
|
||||
|
||||
Run this command to enable downloading from websites from the terminal:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
echo >> ~/.wgetrc <<EOF
|
||||
http_proxy = your.http-proxy.url:port
|
||||
https_proxy = your.https-proxy.url:port
|
||||
ftp_proxy = your.http-proxy.url:port
|
||||
no_proxy = your-company-domain.com, localhost
|
||||
EOF
|
||||
|
||||
System service (Docker)
|
||||
***********************
|
||||
|
||||
For Docker (and other services that use systemd), you can follow these steps to configure and check proxy settings:
|
||||
|
||||
.. note::
|
||||
|
||||
Be sure to use :command:`sudo`, as you will need elevated permissions.
|
||||
|
||||
#. Create :file:`/etc/systemd/system/docker.service.d` directory to host
|
||||
configuration information for the Docker service.
|
||||
|
||||
#. Create :file:`/etc/systemd/system/docker.service.d/http-proxy.conf` and add:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
[Service]
|
||||
Environment="HTTP_PROXY=http://your.http-proxy.url:port/"
|
||||
Environment="HTTPS_PROXY=http://your.https-proxy.url:port/"
|
||||
|
||||
#. Load the changes and restart the service:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart docker
|
||||
|
||||
#. Verify that changes have loaded:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
systemctl show --property=Environment docker
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
Environment=HTTP_PROXY=http://your.http-proxy.url:port/ HTTPS_PROXY=http://your.https-proxy.url:port/
|
||||
|
||||
.. note::
|
||||
|
||||
This process enables the ability to successfully run ``docker pull``.
|
||||
Containers themselves must be configured independently.
|
||||
|
||||
git over ssh
|
||||
************
|
||||
|
||||
Add the following to your :file:`~/.ssh/config`:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
host github.com
|
||||
port 22
|
||||
user git
|
||||
ProxyCommand connect-proxy -S your.ssh-proxy.url:port %h %p
|
||||
|
||||
.. note::
|
||||
|
||||
Though netcat is included with Clear Linux, it is not the BSD version,
|
||||
which is the one usually used to enable git over ssh.
|
||||
@@ -78,10 +78,5 @@ own records, then you must set up your own telemetry backend server.
|
||||
Next steps
|
||||
**********
|
||||
|
||||
To put this concept into practice, see the following resources:
|
||||
To put this concept into practice, refer to :ref:`telem-guide`.
|
||||
|
||||
* :ref:`telem-guide`
|
||||
* `Telemetry feature description`_
|
||||
|
||||
.. _`Telemetry feature description`:
|
||||
https://clearlinux.org/features/telemetry
|
||||
|
||||
@@ -1,31 +1,49 @@
|
||||
.. _gce:
|
||||
|
||||
Launch |CL-ATTR| Compute Engine on Google\* Cloud Platform
|
||||
Launch |CL-ATTR| Compute Engine on Google Cloud Platform\*
|
||||
##########################################################
|
||||
|
||||
This tutorial walks you through the steps to create a virtual machine
|
||||
instance of |CL-ATTR| on `Google Cloud Platform`_ (GCP).
|
||||
instance of |CL-ATTR| on `Google Cloud Platform`_ (:abbr:`GCP (Google Cloud Platform)`).
|
||||
|
||||
#. Sign in your Google account on the
|
||||
.. contents:: :local:
|
||||
:depth: 1
|
||||
|
||||
Prerequisites
|
||||
*************
|
||||
|
||||
* Set up a Google account and a GCP billing account.
|
||||
|
||||
* Generate and install a user SSH key in the Linux PCs that will connect to
|
||||
the VMs in GCP.
|
||||
|
||||
|
||||
Process
|
||||
*******
|
||||
|
||||
#. Sign in to your Google\* account on the
|
||||
`Google Cloud Console <https://console.cloud.google.com/>`_:
|
||||
|
||||
.. figure:: figures/gce/00-sign-in.png
|
||||
:scale: 50 %
|
||||
:alt: Sign in Google services
|
||||
:alt: Sign in to Google services
|
||||
|
||||
Figure 1: Google sign in screen
|
||||
|
||||
#. Google Cloud Platform uses **Projects** to manage resources.
|
||||
Select or create a new project for hosting the |CL| VM.
|
||||
|
||||
.. note::
|
||||
|
||||
Reference the
|
||||
Refer to the
|
||||
`Quickstart Using a Linux VM <https://cloud.google.com/compute/docs/quickstart-linux>`_
|
||||
guide to get familiar with the process of creating VM instances on GCP.
|
||||
guide to learn about the process of creating VM instances on GCP.
|
||||
|
||||
#. Navigate to the latest |CL|
|
||||
`release folder <https://download.clearlinux.org/releases/current/clear/>`_
|
||||
to view currently released ``gce`` image, and download the image archive
|
||||
*clear-<release number>-gce.tar.gz*.
|
||||
to view the currently released :abbr:`GCE (Google Compute Engine\*)`
|
||||
image, and download the :file:`clear-<release number>-gce.tar.gz`
|
||||
image archive.
|
||||
|
||||
You don't need to uncompress the image archive, the intact file will
|
||||
be uploaded to the Google Cloud Storage later.
|
||||
@@ -42,26 +60,32 @@ instance of |CL-ATTR| on `Google Cloud Platform`_ (GCP).
|
||||
:scale: 50 %
|
||||
:alt: Browse Google Cloud Storage
|
||||
|
||||
Figure 2: Browse Google Cloud Storage
|
||||
|
||||
.. note::
|
||||
You may need to create a billing account and link to this project
|
||||
before you create a bucket.
|
||||
|
||||
.. figure:: figures/gce/02-storage-browser.png
|
||||
:scale: 50 %
|
||||
:alt: Cloud Storage Broswer tool
|
||||
:alt: Cloud Storage Browser tool
|
||||
|
||||
Figure 3: Cloud Storage Browser tool
|
||||
|
||||
* Click the ``CREATE BUCKET`` button to enter the bucket creation tool.
|
||||
The bucket name must be unique as buckets in the Cloud Storage share
|
||||
The bucket name must be unique because buckets in the Cloud Storage share
|
||||
a single global namespace.
|
||||
|
||||
You can leave the rest of options as defaults, and select the
|
||||
Leave the remaining options set to the defaults, and select the
|
||||
``Create`` button at the bottom to create a *Bucket*.
|
||||
|
||||
.. figure:: figures/gce/03-create-bucket.png
|
||||
:scale: 50 %
|
||||
:alt: Set an unique bucket name
|
||||
:alt: Set a unique bucket name
|
||||
|
||||
#. Once the bucket is created, press the ``Upload files`` button
|
||||
Figure 4: Set bucket name
|
||||
|
||||
#. Once the bucket is created, click the ``Upload files`` button
|
||||
on the Bucket details page to upload the |CL| GCE image archive
|
||||
to the named bucket:
|
||||
|
||||
@@ -69,24 +93,32 @@ instance of |CL-ATTR| on `Google Cloud Platform`_ (GCP).
|
||||
:scale: 50 %
|
||||
:alt: Cloud Storage bucket is available for storing objects
|
||||
|
||||
Figure 5: Cloud Storage bucket
|
||||
|
||||
.. figure:: figures/gce/10-image-upload.png
|
||||
:scale: 50 %
|
||||
:alt: Uploading the image source archive file
|
||||
|
||||
Figure 6: Uploading the image source archive file
|
||||
|
||||
.. figure:: figures/gce/11-bucket-uploaded.png
|
||||
:scale: 50 %
|
||||
:alt: Image archive imported complete
|
||||
|
||||
Figure 7: Importing complete
|
||||
|
||||
#. Browse the Compute Engine Image library page:
|
||||
|
||||
* Click the *Navigation menu* icon on the upper left screen menu.
|
||||
|
||||
* Hover mouse over the *Compute Engine* menu and select *Images*.
|
||||
* Hover your mouse over the *Compute Engine* menu and select *Images*.
|
||||
|
||||
.. figure:: figures/gce/20-gce-image.png
|
||||
:scale: 50 %
|
||||
:alt: Go to Google Compute Engine Image library
|
||||
|
||||
Figure 8: Image library
|
||||
|
||||
#. On the Compute Engine Image library page, click the ``[+] CREATE IMAGE``
|
||||
menu item to create a custom image:
|
||||
|
||||
@@ -94,18 +126,22 @@ instance of |CL-ATTR| on `Google Cloud Platform`_ (GCP).
|
||||
:scale: 50 %
|
||||
:alt: Create a Google Compute Engine image
|
||||
|
||||
Figure 9: Create image
|
||||
|
||||
#. In the VM image creation page, change the image source type to
|
||||
*Cloud Storage file*.
|
||||
|
||||
#. Under :guilabel:`Cloud Storage file`, select :guilabel:`Browse`.
|
||||
|
||||
#. Locate the *clear-<release number>-gce.tar.gz*,
|
||||
#. Locate the :file:`clear-<release number>-gce.tar.gz` file,
|
||||
and click :guilabel:`Select`.
|
||||
|
||||
.. figure:: figures/gce/21-create-image.png
|
||||
:scale: 50 %
|
||||
:alt: Create the image using the imported image archive object
|
||||
|
||||
Figure 10: Create image using imported object
|
||||
|
||||
Accept all default options, and click the ``Create`` button
|
||||
at the bottom to import the Clear Linux GCE image to the image library.
|
||||
|
||||
@@ -113,18 +149,22 @@ instance of |CL-ATTR| on `Google Cloud Platform`_ (GCP).
|
||||
:scale: 50 %
|
||||
:alt: Clear Linux Compute Engine image is created
|
||||
|
||||
Figure 11: Image is created
|
||||
|
||||
#. After the |CL| image is imported, you can launch a VM instance running
|
||||
|CL|:
|
||||
|
||||
* Click the *Navigation menu* icon on the upper left screen menu.
|
||||
|
||||
* Hover mouse over the *Compute Engine* menu group and select
|
||||
* Hover your mouse over the *Compute Engine* menu group and select
|
||||
the *VM instances* item.
|
||||
|
||||
.. figure:: figures/gce/30-vm-instances.png
|
||||
:scale: 50 %
|
||||
:alt: Go to VM instances catalog
|
||||
|
||||
Figure 12: VM instances catalog
|
||||
|
||||
#. If no VM instance was created in this project, you will be prompted to
|
||||
create one.
|
||||
|
||||
@@ -135,25 +175,33 @@ instance of |CL-ATTR| on `Google Cloud Platform`_ (GCP).
|
||||
:scale: 50 %
|
||||
:alt: Prompt for VM creation
|
||||
|
||||
Figure 13: VM creation
|
||||
|
||||
.. figure:: figures/gce/30-vm-catalog.png
|
||||
:scale: 50 %
|
||||
:alt: List of VM instances
|
||||
|
||||
* In :guilabel:`Region`, decide on a region based on the
|
||||
`Best practices for Compute Engine regions selection`_ .
|
||||
Figure 14: VM instances list
|
||||
|
||||
* Under *Boot disk*, click the ``Change``.
|
||||
* In :guilabel:`Region`, choose a region based on the
|
||||
`Best practices for Compute Engine regions selection`_.
|
||||
|
||||
* Under *Boot disk*, click the ``Change`` button.
|
||||
|
||||
.. figure:: figures/gce/30-create-vm.png
|
||||
:scale: 50 %
|
||||
:alt: Use custom image while creating Clear Linux VM instance
|
||||
|
||||
Figure 15: Use custom image
|
||||
|
||||
* Select the *Custom images* tab for using Clear Linux OS GCE image.
|
||||
|
||||
.. figure:: figures/gce/31-select-boot-disk.png
|
||||
:scale: 50 %
|
||||
:alt: Select Clear Linux boot disk to create a VM instance
|
||||
|
||||
Figure 16: Select Clear Linux boot disk to create a VM instance
|
||||
|
||||
* Scroll down to the bottom of the VM instance creation page,
|
||||
expand the *Management, security, disks, networking, sole tenancy* group.
|
||||
|
||||
@@ -161,12 +209,14 @@ instance of |CL-ATTR| on `Google Cloud Platform`_ (GCP).
|
||||
:scale: 50 %
|
||||
:alt: Clear Linux requires setting up SSH keys
|
||||
|
||||
Figure 17: Set up SSH keys
|
||||
|
||||
.. note::
|
||||
|CL| does not allow SSH login with a root account by default.
|
||||
As a result, you need to configure the VM instance with your
|
||||
SSH public key, so that you are able to remote access to it.
|
||||
As a result, you must configure the VM instance with your
|
||||
SSH public key, so that you are able to access it remotely.
|
||||
|
||||
Refer also to :ref:`security` for more details.
|
||||
Refer to :ref:`security` for more details.
|
||||
|
||||
* Click the *Security* tab, copy and paste your SSH public key:
|
||||
|
||||
@@ -174,28 +224,43 @@ instance of |CL-ATTR| on `Google Cloud Platform`_ (GCP).
|
||||
:scale: 50 %
|
||||
:alt: Set SSH key for remote login
|
||||
|
||||
Figure 18: Set SSH key for remote login
|
||||
|
||||
.. warning::
|
||||
|
||||
The username is assigned from characters preceding ``@`` in the
|
||||
email address, included in the SSH key. The dot symbol "." is not
|
||||
allowed, as it is an invalid character while creating user accounts
|
||||
in |CL|.
|
||||
The username is assigned from characters preceding ``@`` in the email
|
||||
address, included in the SSH key. The dot symbol "." is not allowed,
|
||||
because it is an invalid character while creating user accounts in
|
||||
|CL|.
|
||||
|
||||
* Click the ``Create`` button to create the |CL| VM.
|
||||
|
||||
#. The Clear Linux VM instance is created and offered a public IP address:
|
||||
#. The Clear Linux VM instance is created and assigned a public IP address:
|
||||
|
||||
.. figure:: figures/gce/41-vm-created.png
|
||||
:scale: 50 %
|
||||
:alt: Clear Linux VM instance is created and started
|
||||
|
||||
#. You can now SSH login to the VM by using the IP address obtained in the
|
||||
Figure 19: Clear Linux VM instance is created and started
|
||||
|
||||
#. You can now SSH login to the VM using the IP address obtained in the
|
||||
previous step, and the username associated with the SSH public key:
|
||||
|
||||
.. figure:: figures/gce/42-ssh-vm.png
|
||||
:scale: 50 %
|
||||
:alt: SSH log in to the Clear Linux VM
|
||||
:alt: SSH login to the Clear Linux VM
|
||||
|
||||
Figure 20: SSH login to Clear Linux VM
|
||||
|
||||
Related topics
|
||||
**************
|
||||
|
||||
The following tutorials describe a similar process and may be useful references:
|
||||
|
||||
* :ref:`azure`
|
||||
* :ref:`aws-web`
|
||||
|
||||
|
||||
.. _Google Cloud Platform: https://cloud.google.com/
|
||||
|
||||
.. _Best practices for Compute Engine regions selection: https://cloud.google.com/solutions/best-practices-compute-engine-region-selection
|
||||
.. _Best practices for Compute Engine regions selection: https://cloud.google.com/solutions/best-practices-compute-engine-region-selection
|
||||
|
||||
@@ -61,7 +61,7 @@ Download and launch the virtual machine
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
curl -O https://cdn.download.clearlinux.org/image/$(curl https://cdn.download.clearlinux.org/image/latest-images | grep '[0-9]'-kvm)
|
||||
curl -O https://cdn.download.clearlinux.org/image/$(curl https://cdn.download.clearlinux.org/image/latest-images | grep '[0-9]'-kvm'\.')
|
||||
|
||||
#. Uncompress the downloaded image:
|
||||
|
||||
|
||||
@@ -129,8 +129,8 @@ Before you begin, you must:
|
||||
Obtain kernel module source
|
||||
===========================
|
||||
|
||||
A required :file:`dkms.conf` file inside of the kernel module's source code directory
|
||||
informs DKMS how the kernel module should be compiled.
|
||||
A required :file:`dkms.conf` file inside of the kernel module's source code
|
||||
directory informs DKMS how the kernel module should be compiled.
|
||||
|
||||
Kernel modules may come packaged as:
|
||||
|
||||
@@ -141,10 +141,10 @@ Kernel modules may come packaged as:
|
||||
- Precompiled module binaries only (without source code)
|
||||
|
||||
Of the package types listed above, only precompiled kernel module binaries
|
||||
will not work, because |CL| requires kernel modules to be built against
|
||||
the same kernel source tree before they can be loaded. If you are only able to
|
||||
obtain source code without a :file:`dkms.conf` file, you must manually create a
|
||||
:file:`dkms.conf` file, described later in this document.
|
||||
will not work, because |CL| requires kernel modules to be built against the
|
||||
same kernel source tree before they can be loaded. If you are only able to
|
||||
obtain source code without a :file:`dkms.conf` file, you must manually create
|
||||
a :file:`dkms.conf` file, described later in this document.
|
||||
|
||||
#. Download the kernel module's source code.
|
||||
|
||||
@@ -210,8 +210,8 @@ If the kernel module source does not contain a :file:`dkms.conf` file or the
|
||||
create the file.
|
||||
|
||||
Review the kernel module README documentation for guidance on what needs to be
|
||||
in the :file:`dkms.conf` file, including special variables that may be required to
|
||||
build successfully.
|
||||
in the :file:`dkms.conf` file, including special variables that may be
|
||||
required to build successfully.
|
||||
|
||||
Here are some additional resources that can be used for reference:
|
||||
|
||||
@@ -223,6 +223,10 @@ Here are some additional resources that can be used for reference:
|
||||
|
||||
* `Sample dkms.conf file`_ in the GitHub\* repository for the DKMS project.
|
||||
|
||||
.. note::
|
||||
|
||||
:command:`AUTOINSTALL=yes` must be set in the dkms.conf for the module to
|
||||
be automatically recompiled with |CL| updates.
|
||||
|
||||
The instructions below show a generic example:
|
||||
|
||||
@@ -240,6 +244,7 @@ The instructions below show a generic example:
|
||||
PACKAGE_NAME=custom_module
|
||||
PACKAGE_VERSION=1.0
|
||||
DEST_MODULE_LOCATION=/kernel/drivers/other
|
||||
AUTOINSTALL=yes
|
||||
|
||||
This example identifies a kernel module named *custom_module* with version
|
||||
*1.0*.
|
||||
|
||||
@@ -1,71 +1,175 @@
|
||||
.. _assign-static-ip:
|
||||
|
||||
Assign a static IP address to a network interface
|
||||
#################################################
|
||||
|
||||
Introduction
|
||||
************
|
||||
Assign a static IP address
|
||||
##########################
|
||||
|
||||
By default, your |CL-ATTR| system automatically gets an IP address from your
|
||||
network via DHCP. If you do not have a DHCP server on your network or simply
|
||||
want to use a static IP address, follow the steps in this guide.
|
||||
|
||||
Process
|
||||
*******
|
||||
.. contents::
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
#. Create this directory structure:
|
||||
Identify which program is managing the interface
|
||||
************************************************
|
||||
|
||||
.. code-block:: bash
|
||||
New installations of |CL| use NetworkManager as the default network interface
|
||||
manager for all network connections.
|
||||
|
||||
sudo mkdir -p /etc/systemd/network
|
||||
.. note::
|
||||
|
||||
#. Identify the interface to be assigned the static IP address:
|
||||
* The *cloud* |CL| images continue to use systemd-networkd to manage
|
||||
network connections.
|
||||
|
||||
.. code-block:: bash
|
||||
* In earlier |CL| versions, systemd-network was used to manage Ethernet
|
||||
interfaces and NetworkManager was used for wireless interfaces.
|
||||
|
||||
ip addr
|
||||
|
||||
The system returns the following:
|
||||
Before defining a configuration for assigning a static IP address, you should
|
||||
verify which program is managing the network interface.
|
||||
|
||||
.. code-block:: console
|
||||
#. Check the output of :command:`nmcli device` to see if NetworkManager is
|
||||
managing the device.
|
||||
|
||||
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
|
||||
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
|
||||
inet 127.0.0.1/8 scope host lo
|
||||
valid_lft forever preferred_lft forever
|
||||
inet6 ::1/128 scope host
|
||||
valid_lft forever preferred_lft forever
|
||||
.. code-block:: bash
|
||||
|
||||
2: wlp1s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
|
||||
link/ether 4a:98:8d:e5:43:15 brd ff:ff:ff:ff:ff:ff
|
||||
nmcli device status
|
||||
|
||||
3: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP group default qlen 1000
|
||||
link/ether f4:4d:30:68:96:20 brd ff:ff:ff:ff:ff:ff
|
||||
inet 10.0.1.2/24 brd 10.54.74.255 scope global dynamic eno1
|
||||
valid_lft 6766sec preferred_lft 6766sec
|
||||
inet6 fe80::f64d:30ff:fe68:9620/64 scope link
|
||||
valid_lft forever preferred_lft forever
|
||||
If the STATE column for the device shows *connected* or *disconnected*, the
|
||||
network configuration is being managed by NetworkManager and the instructions
|
||||
for :ref:`using NetworkManager <nm-static-ip>` should be used.
|
||||
|
||||
In this example, we will use the `eno1` interface.
|
||||
If the STATE column for the device shows *unmanaged*, check to see if the
|
||||
device is being managed by systemd-networkd
|
||||
|
||||
#. Create the :file:`70-static.network` file and add the following:
|
||||
|
||||
.. code-block:: bash
|
||||
#. Check the output of :command:`networkctl list` to see if
|
||||
systemd-networkd is managing the device.
|
||||
|
||||
sudo $EDITOR /etc/systemd/network/70-static.network
|
||||
.. code-block:: bash
|
||||
|
||||
[Match]
|
||||
Name=[interface name]
|
||||
[Network]
|
||||
Address=[IP address]/24
|
||||
DHCP=yes # to get DNS info, etc.
|
||||
networkctl list
|
||||
|
||||
Replace [interface name] and [IP address] with your specific settings.
|
||||
If the SETUP column for the device shows *configured*, the network
|
||||
configuration is being managed by systemd-networkd and the instructions for
|
||||
:ref:`using systemd-networkd <networkd-static-ip>` should be used.
|
||||
|
||||
#. Restart the networkd service:
|
||||
|
||||
.. code-block:: bash
|
||||
.. _nm-static-ip:
|
||||
|
||||
sudo systemctl restart systemd-networkd
|
||||
Using NetworkManager
|
||||
********************
|
||||
|
||||
Network connections managed by NetworkManager are stored as files with the
|
||||
:file:`.nmconnection` file extension in the
|
||||
:file:`/etc/NetworkManager/system-connections/` directory.
|
||||
|
||||
A few tools exists to aid to manipulate network connections managed by
|
||||
NetworkManager:
|
||||
|
||||
* nmcli - a command-line tool
|
||||
|
||||
* nmtui - a text user interface that provides a pseudo graphical menu in the
|
||||
terminal
|
||||
|
||||
* nm-connection-editor - a graphical user interface
|
||||
|
||||
The method below uses the command line tool *nmcli* to modify network
|
||||
connection.
|
||||
|
||||
|
||||
#. Identify the existing connection name.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
nmcli connection show
|
||||
|
||||
The output will look like this:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
NAME UUID TYPE DEVICE
|
||||
Wired connection 1 00000000-0000-0000-0000-000000000000 802-3-etherneten01
|
||||
|
||||
If a connection does not exist, it will need to be created with
|
||||
:command:`nmcli connection add`.
|
||||
|
||||
|
||||
#. Modify the connection to use a static IP address. Replace the variables in
|
||||
brackets with the appropriate values. *[CONNECTION_NAME]* should be
|
||||
replaced with the NAME from the command above.
|
||||
|
||||
.. code::
|
||||
|
||||
sudo nmcli connection modify "[CONNECTION_NAME]" \
|
||||
ipv4.method "manual" \
|
||||
ipv4.addresses "[IP_ADDRESS]/[CIDR_NETMASK]" \
|
||||
ipv4.gateway "[GATEWAY_IP_ADDRESS]" \
|
||||
ipv4.dns "[PRIMARY_DNS_IP],[SECONDARY_DNS_IP]"
|
||||
|
||||
|
||||
See the `nmcli developer page <https://developer.gnome.org/NetworkManager/stable/nmcli.html>`_ for more
|
||||
configuration options. For advanced configurations, the
|
||||
:file:`/etc/NetworkManager/system-connections/*.nmconnection`. can be edited
|
||||
directly.
|
||||
|
||||
#. Verify your static IP address details have been set
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
nmcli
|
||||
|
||||
|
||||
|
||||
.. _networkd-static-ip:
|
||||
|
||||
Using systemd-networkd
|
||||
**********************
|
||||
|
||||
Network connections managed by systemd-networkd are stored as files with the
|
||||
:file:`.network` file extension the :file:`/etc/systemd/network/` directory.
|
||||
|
||||
Files to manipulate network connections managed by systemd-networkd must be
|
||||
created manually.
|
||||
|
||||
#. Create the :file:`/etc/systemd/network` directory if it doesn't exist already:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo mkdir -p /etc/systemd/network
|
||||
|
||||
#. Create a :file:`.network` file and add the following content. Replace the
|
||||
variables in brackets with the appropriate values. *[INTERFACE_NAME]*
|
||||
should be replaced with LINK from the output of :command:`networkctl list`
|
||||
ran previously.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo $EDITOR /etc/systemd/network/70-static.network
|
||||
|
||||
[Match]
|
||||
Name=[INTERFACE_NAME]
|
||||
|
||||
[Network]
|
||||
Address=[IP_ADDRESS]/[CIDR_NETMASK]
|
||||
Gateway=[GATEWAY_IP_ADDRESS]
|
||||
DNS=[PRIMARY_DNS_IP]
|
||||
DNS=[SECONDARY_DNS_IP]
|
||||
|
||||
See the `systemd-network man page
|
||||
<https://www.freedesktop.org/software/systemd/man/systemd.network.html>`_
|
||||
for more configuration options.
|
||||
|
||||
#. Restart the systemd-networkd service:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo systemctl restart systemd-networkd
|
||||
|
||||
#. Verify your static IP address details have been set
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
networkctl status
|
||||
|
||||
**Congratulations!** You have successfully assigned a static IP address.
|
||||
|
||||
@@ -35,14 +35,11 @@ Table 2 lists the currently available images that are platform specific.
|
||||
* - Image Type
|
||||
- Description
|
||||
|
||||
* - installer.img
|
||||
- Preferred image of |CL| with interactive installer.
|
||||
* - live-desktop.img or live-desktop.iso
|
||||
- Image for booting to GNOME\* desktop to preview or install the OS.
|
||||
|
||||
* - installer.iso
|
||||
- ISO of |CL| with interactive installer. Only for special cases where ISO image format is required (not for use with a USB key)
|
||||
|
||||
* - live.img
|
||||
- Image for live booting into memory, without requiring installaton.
|
||||
* - live-server.img or live-server.iso
|
||||
- Image for booting to server command prompt to preview or install the OS.
|
||||
|
||||
.. list-table:: Table 2: Types of platform-specific |CL| images
|
||||
:widths: 15, 85
|
||||
@@ -51,20 +48,23 @@ Table 2 lists the currently available images that are platform specific.
|
||||
* - Image Type
|
||||
- Description
|
||||
|
||||
* - aws.img
|
||||
- Image suitable for use with Amazon\* AWS\*.
|
||||
|
||||
* - azure.vhd
|
||||
- Virtual Hard Disk for use on Microsoft\* Azure\* cloud platform
|
||||
- Virtual Hard Disk for use on Microsoft\* Azure\* cloud platform.
|
||||
|
||||
* - azure-docker.vhd
|
||||
- Virtual Hard Disk for use on Microsoft Azure cloud platform with Docker\* pre-installed
|
||||
- Virtual Hard Disk for use on Microsoft Azure cloud platform with Docker\* pre-installed.
|
||||
|
||||
* - azure-machine-learning.vhd
|
||||
- Virtual Hard Disk for use on Microsoft Azure cloud platform with the `machine-learning-basic` bundle installed
|
||||
- Virtual Hard Disk for use on Microsoft Azure cloud platform with the `machine-learning-basic` bundle installed.
|
||||
|
||||
* - cloud.img
|
||||
- Image for use by cloud deployments such as OpenStack\*
|
||||
* - cloudguest.img
|
||||
- Image with generic cloud guest virtual machine (VM) requirements installed.
|
||||
|
||||
* - containers.img
|
||||
- Image for use by Clear Containers runtime. Includes `optimized kernel`_ for Clear Containers.
|
||||
* - gce.tar
|
||||
- Image with the Google Compute Engine (GCE) specific kernel.
|
||||
|
||||
* - hyperv.vhdx
|
||||
- Virtual Hard Disk for use with Microsoft Hyper-V\* hypervisor. Includes `optimized kernel`_ for Hyper-V.
|
||||
@@ -73,9 +73,15 @@ Table 2 lists the currently available images that are platform specific.
|
||||
- Image for booting in a simple VM with start_qemu.sh. Includes
|
||||
`optimized kernel`_ for KVM.
|
||||
|
||||
* - kvm-legacy.img
|
||||
- Image for booting in a simple VM using legacy BIOS, if using start_qemu.sh make sure to remove -bios parameter.
|
||||
|
||||
* - pxe.tar
|
||||
- Image suitable for use with PXE server.
|
||||
|
||||
* - vmware.vmdk
|
||||
- Virtual Machine Disk for VMware\* platforms inclduing Player, Workstation, and ESXi.
|
||||
|
||||
.. _images: https://cdn.download.clearlinux.org/image
|
||||
.. _images: https://clearlinux.org/downloads
|
||||
.. _`optimized kernel`: https://clearlinux.org/documentation/clear-linux/reference/compatible-kernels
|
||||
|
||||
|
||||
@@ -20,22 +20,22 @@ manually configure the proxies.
|
||||
|
||||
Corporate and private networks can be very complex, needing to restrict and
|
||||
control network connections for security reasons. The typical side effects
|
||||
are limited or blocked connectivity and requiring manual configuration of
|
||||
proxies to perform the most mundane tasks such as cloning a repo or checking
|
||||
are limited or blocked connectivity, and require manual configuration of
|
||||
proxies to perform the most mundane tasks, such as cloning a repo or checking
|
||||
for updates. With |CL|, all of the work is done behind the scenes to
|
||||
effortlessly use your network and have connections “just work”.
|
||||
|
||||
This feature removes massive complications in network connectivity due to
|
||||
proxy issues. You can automate tasks like unit testing without worrying
|
||||
about the proxy not being set and you can remove unset proxies from the
|
||||
This feature removes severe complications with network connectivity due to
|
||||
proxy issues. You can automate tasks, such as unit testing, without worrying
|
||||
about the proxy not being set, and you can remove unset proxies from the
|
||||
equation when dealing with network unavailability across systems.
|
||||
|
||||
How it works
|
||||
************
|
||||
|
||||
We designed Autoproxy around tools provided by most Linux
|
||||
We designed Autoproxy around tools provided by most Linux\*
|
||||
distributions with a few minor additions and modifications. We leveraged the
|
||||
DHCP and network information provided from systemd and created a
|
||||
DHCP and network information obtained from systemd and created a
|
||||
PAC-discovery daemon. The daemon uses the information to resolve a URL for a
|
||||
PAC file. The daemon then passes the URL into PACrunner\*. PACrunner
|
||||
downloads the PAC file and uses the newly implemented Duktape\* engine to
|
||||
@@ -49,7 +49,7 @@ parse it.
|
||||
From that point on, any cURL\* or network requests query PACrunner for the
|
||||
correct proxy to use. We modified the cURL library to communicate with
|
||||
PACrunner over DBus. However, cURL will ignore PACrunner and run normally if
|
||||
no PAC file is loaded or if you set any proxies manually. Thus, your
|
||||
no PAC file is loaded or if you manually set any proxies. Thus, your
|
||||
environment settings are respected and no time is wasted trying to resolve a
|
||||
proxy. All these steps happen in the background with no user interaction.
|
||||
|
||||
|
||||
+29
-29
@@ -3,8 +3,8 @@
|
||||
autospec
|
||||
########
|
||||
|
||||
**autospec** is a tool to assist in the automated creation and maintenance of
|
||||
RPM packaging in |CL-ATTR|. Where a standard RPM build process using
|
||||
**autospec** is a tool used to assist with the automated creation and maintenance of
|
||||
RPM packaging in |CL-ATTR|. Where a standard :abbr:`RPM (RPM Package Manager)` build process using
|
||||
:command:`rpmbuild` requires a tarball and :file:`.spec` file to start, autospec
|
||||
requires only a tarball and package name to start.
|
||||
|
||||
@@ -16,17 +16,17 @@ Description
|
||||
***********
|
||||
|
||||
The autospec tool attempts to infer the requirements of the :file:`.spec` file
|
||||
by analyzing the source code and :file:`Makefile` information. It will
|
||||
continuously run updated builds based on new information discovered from build
|
||||
by analyzing the source code and :file:`Makefile` information. It
|
||||
continuously runs updated builds based on new information discovered from build
|
||||
failures until it has a complete and valid :file:`.spec` file. If needed, you
|
||||
can influence the behavior of autospec and customize the build by providing
|
||||
optional `control files`_ to the autospec tool.
|
||||
|
||||
autospec uses mock as a sandbox to run the builds. Visit the `mock wiki`_ for
|
||||
autospec uses **mock** as a sandbox to run the builds. Visit the `mock wiki`_ for
|
||||
additional information on using mock.
|
||||
|
||||
For a general understanding of how RPMs work, visit the `rpm website`_ or the
|
||||
`RPM Packaging Guide`_ .
|
||||
For a general understanding of how an RPM works, visit
|
||||
the `rpm website`_ or the `RPM Packaging Guide`_ .
|
||||
|
||||
How it works
|
||||
************
|
||||
@@ -44,7 +44,7 @@ The setup for building source in |CL| must be completed before using the
|
||||
autospec tool.
|
||||
|
||||
Refer to `Setup environment to build source`_ for instructions on completing
|
||||
setup.
|
||||
the setup.
|
||||
|
||||
Create an RPM
|
||||
=============
|
||||
@@ -52,7 +52,7 @@ Create an RPM
|
||||
The basic autospec process is described in the following steps:
|
||||
|
||||
#. The :command:`make autospec` command generates a :file:`.spec` file based on
|
||||
analysis of code and existing control files.
|
||||
the analysis of code and existing control files.
|
||||
|
||||
Any control files should be located in the same directory as the resulting
|
||||
:file:`.spec` file.
|
||||
@@ -65,13 +65,13 @@ The basic autospec process is described in the following steps:
|
||||
|
||||
#. autospec detects any missed declarations in the :file:`.spec`.
|
||||
|
||||
#. If build errors occur, autospec will scan the build log to try and detect
|
||||
#. If build errors occur, autospec scans the build log to try to detect
|
||||
the root cause.
|
||||
|
||||
#. If autospec detects the root cause and knows how to continue, it will restart
|
||||
#. If autospec detects the root cause and knows how to continue, it restarts
|
||||
the build automatically at step 1 with updated build instructions.
|
||||
|
||||
#. Otherwise, autospec will stop the build for user inspection to resolve the
|
||||
#. Otherwise, autospec stops the build for user inspection to resolve the
|
||||
errors. Respond to the build process output by fixing source code issues
|
||||
and/or editing control files to resolve issues, which may include
|
||||
dependencies or exclusions. See `autospec README`_ for more information on
|
||||
@@ -79,7 +79,7 @@ The basic autospec process is described in the following steps:
|
||||
|
||||
The user resumes the process at step 1 after errors are resolved.
|
||||
|
||||
If a binary dependency doesn't exist in |CL|, you will need to build it
|
||||
If a binary dependency doesn't exist in |CL|, you must build it
|
||||
before running autospec again.
|
||||
|
||||
Following these steps, autospec continues to rebuild the package, based on
|
||||
@@ -95,10 +95,10 @@ Complete `Setup environment to build source`_ before using these examples.
|
||||
:local:
|
||||
:depth: 1
|
||||
|
||||
Example 1: Build RPM with existing spec file
|
||||
============================================
|
||||
Example 1: Build RPM with an existing spec file
|
||||
===============================================
|
||||
|
||||
This example shows how to build a RPM from a pre-packaged upstream package, with
|
||||
This example shows how to build a RPM from a pre-packaged upstream package with
|
||||
an existing spec file. The example uses the ``dmidecode`` package.
|
||||
|
||||
#. Navigate to the autospec workspace and clone the ``dmidecode`` package:
|
||||
@@ -110,7 +110,7 @@ an existing spec file. The example uses the ``dmidecode`` package.
|
||||
|
||||
.. note::
|
||||
|
||||
You can clone all package repos at once using:
|
||||
You can clone all package repos at once using the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@@ -147,7 +147,7 @@ create a simple helloclear RPM.
|
||||
cd ~/clearlinux
|
||||
make autospecnew URL="https://github.com/clearlinux/helloclear/archive/helloclear-v1.0.tar.gz" NAME="helloclear"
|
||||
|
||||
The resulting RPMs are in :file:`./packages/helloclear/rpms`. Builde logs and
|
||||
The resulting RPMs are in :file:`./packages/helloclear/rpms`. Build logs and
|
||||
additional RPMs are in :file:`./packages/helloclear/results`.
|
||||
|
||||
Example 3: Generate a new spec file with a pre-defined package
|
||||
@@ -198,8 +198,8 @@ Example 4: Provide control files to autospec
|
||||
============================================
|
||||
|
||||
This example shows how to modify control files to correct build failures that
|
||||
autospec is unable to resolve. In this example you will add a missing license
|
||||
and dependencies in order for autospec to complete a successful build.
|
||||
autospec is unable to resolve. In this example, you will add a missing license
|
||||
and dependencies so autospec can complete a successful build.
|
||||
|
||||
#. Navigate to the autospec workspace:
|
||||
|
||||
@@ -226,7 +226,7 @@ and dependencies in order for autospec to complete a successful build.
|
||||
|
||||
make autospecnew URL="https://github.com/OPAE/opae-sdk/archive/0.13.0.tar.gz" NAME="opae-sdk"
|
||||
|
||||
This will give an error for a missing license file:
|
||||
This results in an error for a missing license file:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -238,7 +238,7 @@ and dependencies in order for autospec to complete a successful build.
|
||||
|
||||
cd packages/opae-sdk
|
||||
|
||||
#. Add one or more valid license identifier from the
|
||||
#. Add one or more valid license identifiers from the
|
||||
`SPDX License List <https://spdx.org/licenses/>`_.
|
||||
In the example below, two different licenses are appropriate based on the
|
||||
opae-sdk project licensing:
|
||||
@@ -253,7 +253,7 @@ and dependencies in order for autospec to complete a successful build.
|
||||
|
||||
make autospec
|
||||
|
||||
This will result in a generic error:
|
||||
This results in a generic error:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -265,7 +265,7 @@ and dependencies in order for autospec to complete a successful build.
|
||||
|
||||
cat ./results/build.log
|
||||
|
||||
In the build log, you will find details for the specific failures. In this
|
||||
The build log contains details for the specific failures. In this
|
||||
instance, there are missing dependencies:
|
||||
|
||||
.. code-block:: console
|
||||
@@ -279,7 +279,7 @@ and dependencies in order for autospec to complete a successful build.
|
||||
linked by target "opae-c" in directory /builddir/build/BUILD/opae-sdk-0.13.0/libopae
|
||||
|
||||
#. Search the spec files of upstream |CL| packages to see if the json-c library
|
||||
is availabe. In this case, it does exist and we'll add the json-c 'dev'
|
||||
is available. In this case, it does exist and we'll add the json-c 'dev'
|
||||
package into the buildreq_add:
|
||||
|
||||
.. code-block:: bash
|
||||
@@ -302,7 +302,7 @@ and dependencies in order for autospec to complete a successful build.
|
||||
grep 'libuuid\.so$' ~/clearlinux/packages/*/*.spec
|
||||
echo "util-linux-dev" >> buildreq_add
|
||||
|
||||
#. Run autospec again and find the successfully-generated RPMs in the rpms
|
||||
#. Run autospec again and find the successfully-generated RPMs in the :file:`rpms`
|
||||
directory:
|
||||
|
||||
.. code-block:: bash
|
||||
@@ -384,7 +384,7 @@ To test an autospec-created package inside a VM:
|
||||
The code that makes this possible can be viewed by searching for the
|
||||
*install:* target in the `Makefile.common file on GitHub`_.
|
||||
|
||||
#. Return back to the :file:`~/clearlinux` directory and start the |CL| VM:
|
||||
#. Return to the :file:`~/clearlinux` directory and start the |CL| VM:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@@ -411,7 +411,7 @@ Test directly on a development machine
|
||||
|
||||
The |CL| development tooling also includes a method to extract
|
||||
autospec-created RPMs locally onto a |CL| development system for testing.
|
||||
Extracting an RPM directly onto a system offers quicker testing, however
|
||||
Extracting an RPM directly onto a system offers quicker testing; however
|
||||
conflicts may occur and responsibility to remove the software after testing is
|
||||
up to the developer.
|
||||
|
||||
@@ -456,7 +456,7 @@ automated for you with a setup script. It uses tools from the
|
||||
The setup script creates a workspace in the :file:`clearlinux` folder, with the
|
||||
subfolders :file:`Makefile`, :file:`packages`, and :file:`projects`. The
|
||||
:file:`projects` folder contains the main tools used for making packages in
|
||||
|CL|: `autospec` and `common`.
|
||||
|CL| :file:`autospec` and :file:`common`.
|
||||
|
||||
Follow these steps to setup the workspace and tooling for building source:
|
||||
|
||||
|
||||
+11
-12
@@ -3,7 +3,7 @@
|
||||
ister.py image builder
|
||||
######################
|
||||
|
||||
The `ister.py tool`_ is a template based installer used by |CL-ATTR| to produce
|
||||
The `ister.py tool`_ is a template-based installer used by |CL-ATTR| to produce
|
||||
images for each release. The same ister tool is available for use in |CL| to
|
||||
create custom images based on an upstream image.
|
||||
|
||||
@@ -14,21 +14,21 @@ create custom images based on an upstream image.
|
||||
Description
|
||||
***********
|
||||
|
||||
|CL| is a rolling release and produces on average 10 releases per week using the
|
||||
|CL| is a rolling release and produces an average of 10 releases per week using the
|
||||
ister tool. With each release we produce multiple
|
||||
`image types for different environments`_ and use cases such as installers,
|
||||
Hyper-V, KVM, or VMWare.
|
||||
:ref:`image types for different environments <image-types>` and use cases such
|
||||
as installers, Hyper-V, KVM, or VMWare.
|
||||
|
||||
Each image has a JSON configuration file used by ister to generate the image.
|
||||
These JSON configuration files describe the image type, partitions, version,
|
||||
and which bundles will be preinstalled by default with the image. For each image
|
||||
type we produce, the corresponding JSON configuration file for the image is also
|
||||
Each image has a JSON configuration file that is used by ister to generate the
|
||||
image. These JSON configuration files describe the image type, partitions, version,
|
||||
and bundles that will be preinstalled by default with the image. For each image
|
||||
type we produce, the corresponding JSON configuration file for the image also is
|
||||
published.
|
||||
|
||||
The :ref:`mixer<mixer>` tool also uses ister to build images for your custom
|
||||
mix. Like upstream images, a JSON configuration file is defined for the image,
|
||||
which ister uses to generate the image. Refer to the :ref:`mixer<mixer>` guide
|
||||
for instruction on using ister to build an image for a custom mix.
|
||||
for instructions on using ister to build an image for a custom mix.
|
||||
|
||||
Examples
|
||||
********
|
||||
@@ -37,7 +37,7 @@ Recreate an upstream image
|
||||
==========================
|
||||
|
||||
The published configuration files for upstream images may be used to recreate an
|
||||
image, for example when you want to:
|
||||
image. Here are some examples:
|
||||
|
||||
* Use an older version of |CL| and the image is no longer available (only after
|
||||
March 2017).
|
||||
@@ -64,7 +64,7 @@ configuration file:
|
||||
:file:`/clear/config/image`. For example:
|
||||
``https://cdn.download.clearlinux.org/releases/15700/clear/config/image/``
|
||||
|
||||
#. Download “PostNonChroot” script (if applicable).
|
||||
#. Download the “PostNonChroot” script (if applicable).
|
||||
|
||||
The JSON configuration file for the image may have an accompanying
|
||||
“PostNonChroot” script that is executed at the end of the image creation
|
||||
@@ -88,7 +88,6 @@ Related topics
|
||||
* :ref:`bulk-provision`
|
||||
|
||||
.. _ister.py tool: https://github.com/bryteise/ister
|
||||
.. _image types for different environments: https://cdn.download.clearlinux.org/image/README-IMAGES.html
|
||||
.. _Configuration files for the current release: https://cdn.download.clearlinux.org/current/config/image/
|
||||
.. _Previous releases: https://cdn.download.clearlinux.org/releases/
|
||||
.. _Install a bundle: https://clearlinux.org/documentation/clear-linux/guides/maintenance/swupd-guide#adding-a-bundle
|
||||
+93
-102
@@ -8,77 +8,68 @@ software.
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
:depth: 2
|
||||
:depth: 1
|
||||
|
||||
Description
|
||||
***********
|
||||
|
||||
:command:`swupd` has two main functions:
|
||||
|
||||
#. It manages software replacing APT or YUM, installing bundles
|
||||
#. Manages software and replaces APT or YUM, by installing bundles
|
||||
rather than packages.
|
||||
#. It checks for system updates and installs them.
|
||||
#. Checks for system updates and installs them.
|
||||
|
||||
:ref:`Bundles <bundles>` are the smallest granularity component that is
|
||||
managed by |CL| and contain everything needed to deliver a software
|
||||
capability. Rather than downloading a cascade of package dependencies when
|
||||
installing a piece of software, a bundle comes with all of its dependencies.
|
||||
:command:`swupd` manages overlapping dependencies behind the scenes ensuring
|
||||
that all software is compatible across the system.
|
||||
:command:`swupd` manages overlapping dependencies behind the scenes, ensuring
|
||||
that all software is compatible across the system. It can be used to verify
|
||||
the OS, clean cached files, and fix issues.
|
||||
|
||||
:ref:`Bundles <bundles>` contain everything needed to deliver a software
|
||||
capability. They are the smallest granularity component that is
|
||||
managed by |CL|. A bundle comes with all of its dependencies, rather than
|
||||
downloading a cascade of package dependencies when installing a piece of
|
||||
software.
|
||||
|
||||
Versioning
|
||||
==========
|
||||
|
||||
In a traditional distribution, the process of describing current software
|
||||
versioning usually involves:
|
||||
Using package managers to keep track of software version compatibility or
|
||||
compare multiple systems on many Linux distributions can be cumbersome.
|
||||
|
||||
- Listing and keeping track of the current OS release (generally
|
||||
uninformative about any singular packages or functionality).
|
||||
With |CL| :command:`swupd`, versioning happens at the individual file-level.
|
||||
This means |CL| generates an entirely new OS version with any set of software
|
||||
changes to the system (including software downgrades or removals). This
|
||||
rolling release versioning model is similar to :command:`git` internal version
|
||||
tracking, where any of the individual file commits are tracked and move the
|
||||
pointer forward when changed.
|
||||
|
||||
- Keeping track of packages and repositories being used, and updating them
|
||||
individually.
|
||||
|
||||
- Listing and tracking every package available and installed on the
|
||||
system, none of which are directly tied to the current OS release.
|
||||
|
||||
This can be done effectively, but given the nearly endless combinations of
|
||||
packages and versions of packages a server may have, it quickly becomes
|
||||
non-trivial to define what "version" the system is and what software it
|
||||
is running without explicitly going through each system and inspecting
|
||||
every package.
|
||||
|
||||
With |CL|, we need track:
|
||||
|
||||
- One number
|
||||
|
||||
A number representing the **current** release of the OS is sufficient to
|
||||
describe the versions of all the software on the OS. Each build is
|
||||
composed of a specific set of bundles made from a particular version of
|
||||
packages. This matters on a daily basis to system administrators, who
|
||||
need to determine which of their systems do not have the latest security
|
||||
fixes, or which combinations of software have been tested. Every release
|
||||
of the same number is guaranteed to contain the same versions of software,
|
||||
so there's no ambiguity between two systems running the same version of |CL|.
|
||||
A number that represents the **current** release of the OS describes the
|
||||
versions of all the software on the OS. Each build is composed of a specific
|
||||
set of bundles made from a particular version of packages. This matters on a
|
||||
daily basis to system administrators, who need to determine which of their
|
||||
systems do not have the latest security fixes, or which combinations of
|
||||
software have been tested. Every release of the same number is guaranteed to
|
||||
contain the same versions of software, so there's no ambiguity between two
|
||||
systems running the same version of |CL|.
|
||||
|
||||
Updating
|
||||
========
|
||||
|
||||
|CL| enforces regular updating of the OS by default and will automatically
|
||||
check for updates against a version server. The content server provides the
|
||||
|CL| enforces regular updating of the OS by default and automatically
|
||||
checks for updates against a version server. The content server provides the
|
||||
file and metadata content for all versions and can be the same as the
|
||||
version server. The content url server provides metadata in the form of
|
||||
manifests. These Manifest files list and describe file contents, symlinks,
|
||||
*manifests*, which list and describe file contents, symlinks,
|
||||
and directories. Additionally, the actual content is
|
||||
provided to clients in the form of archive files.
|
||||
|
||||
Software updates with |CL| are also efficient. Unlike package-based
|
||||
distributions, :command:`swupd` only updates files that have changed rather
|
||||
distributions, :command:`swupd` only updates files that have changed, rather
|
||||
than entire packages. For example, it is quite common for an OS security
|
||||
patch to be as small as 15 KB. Using binary deltas, the |CL| is able to
|
||||
patch to be as small as 15 KB. Using binary deltas, |CL| is able to
|
||||
apply only what is needed.
|
||||
|
||||
To get a more detailed understanding of how to generate update content for
|
||||
|CL| see the :ref:`mixer <mixer>` tool.
|
||||
For details on how to generate update content for |CL|, see the
|
||||
:ref:`mixer <mixer>` tool.
|
||||
|
||||
How it works
|
||||
************
|
||||
@@ -93,42 +84,41 @@ Prerequisites
|
||||
Updates
|
||||
=======
|
||||
|
||||
|CL| updates are automatic by default but can be set to occur only on
|
||||
|CL| updates are automatic by default, but can be set to occur only on
|
||||
demand. :command:`swupd` makes sure that regular updates are simple and
|
||||
secure. It can also check the validity of currently installed files and
|
||||
software and correct any problems.
|
||||
software, and can correct any problems.
|
||||
|
||||
Manifests
|
||||
---------
|
||||
|
||||
The Clear Linux OS software update content consists of data and
|
||||
metadata. The data is the files that end up in the OS. The metadata
|
||||
contains relevant information to properly provision the data to the OS
|
||||
file system, as well as update the system and add or remove additional
|
||||
content to the OS.
|
||||
The |CL| software update content consists of data and metadata. The data is
|
||||
the files that end up in the OS. The metadata contains relevant information to
|
||||
properly provision the data to the OS file system, as well as update the
|
||||
system and add or remove additional content to the OS.
|
||||
|
||||
The Manifests are mostly long lists of hashes that describe content.
|
||||
The manifests are mostly long lists of hashes that describe content.
|
||||
Each bundle gets its own manifest file. There is a master manifest
|
||||
file that describes all manifests to tie it all together.
|
||||
|
||||
Fullfiles, packs, and delta packs
|
||||
---------------------------------
|
||||
|
||||
The data that an update provisions to a system can be obtained in
|
||||
three different ways. There are three different methods, and they
|
||||
exist to optimize the delivery of content and speed up updates.
|
||||
To speed up updates and optimize content delivery, update data provisioned to
|
||||
a system is obtained by one of the following methods:
|
||||
|
||||
Fullfiles are always generated for every file in every release. This
|
||||
allows any Clear Linux OS to obtain the exact copy of the content
|
||||
for each version directly. This would be used if the OS verification
|
||||
needed to replace a single file, for instance.
|
||||
* *Fullfiles* are always generated for every file in every release. This
|
||||
allows any |CL| to obtain the exact copy of the content
|
||||
for each version directly. This is used if the OS verification
|
||||
needs to replace a single file, for instance.
|
||||
|
||||
Packs are available for some releases and combine many files to speed
|
||||
up the creation of installation media and large updates. Delta packs
|
||||
are an optimized version of packs that only contain updates (binary
|
||||
diffs) and cannot be used without having the original file content.
|
||||
* *Packs* are available for some releases. They combine many files to speed
|
||||
up the creation of installation media and large updates.
|
||||
|
||||
Bundle Search
|
||||
* *Delta packs* are an optimized version of packs that only contain updates
|
||||
(binary diffs). They cannot be used without having the original file content.
|
||||
|
||||
Bundle search
|
||||
=============
|
||||
|
||||
:command:`swupd` search downloads manifest data and searches for
|
||||
@@ -139,21 +129,21 @@ Only the base bundle is returned. Bundles can contain other bundles via
|
||||
includes. For more details, see `Bundle Definition Files`_ and its
|
||||
subdirectory bundles.
|
||||
|
||||
Bundles that are already installed, will be marked [installed] in search
|
||||
Bundles that are already installed are marked **(installed)** in search
|
||||
results.
|
||||
|
||||
Optionally, you can review our `bundles`_ on GitHub.
|
||||
Optionally, you can review our `bundles`_ on GitHub\*.
|
||||
|
||||
Examples
|
||||
********
|
||||
|
||||
Example 1: Disable and Enable automatic updates
|
||||
Example 1: Disable and enable automatic updates
|
||||
===============================================
|
||||
|
||||
|CL| updates are automatic by default but can be set to occur only
|
||||
|CL| updates are automatic by default, but can be set to occur only
|
||||
on demand.
|
||||
|
||||
#. First verify your current auto-update setting.
|
||||
#. Verify your current auto-update setting.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@@ -202,7 +192,7 @@ Example 2: Find and install Kata\* Containers
|
||||
|
||||
Kata Containers is a popular container implementation. Unlike other
|
||||
container implementations, each Kata Container has its own
|
||||
kernel instance and runs on its own :abbr:`Virtual Machine (VM)` for
|
||||
kernel instance and runs on its own :abbr:`VM (Virtual Machine)` for
|
||||
improved security.
|
||||
|
||||
|CL| makes it very easy to install, since you only need to add
|
||||
@@ -211,34 +201,33 @@ number of dependencies. Also, check out our tutorial: :ref:`kata`.
|
||||
|
||||
#. Find the right bundle.
|
||||
|
||||
* To return all possible matches for the search string enter
|
||||
:command:`swupd search`, followed by 'kata':
|
||||
To return all possible matches for the search string, enter
|
||||
:command:`swupd search`, followed by 'kata':
|
||||
|
||||
.. code-block:: bash
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd search kata
|
||||
sudo swupd search kata
|
||||
|
||||
The output should be similar to:
|
||||
The output should be similar to:
|
||||
|
||||
.. code-block:: console
|
||||
.. code-block:: console
|
||||
|
||||
Bundle with the best search result:
|
||||
Bundle with the best search result:
|
||||
|
||||
containers-virt - Run container applications from Dockerhub in lightweight virtual machines
|
||||
containers-virt - Run container applications from Dockerhub in lightweight virtual machines
|
||||
|
||||
This bundle can be installed with:
|
||||
This bundle can be installed with:
|
||||
|
||||
swupd bundle-add containers-virt
|
||||
swupd bundle-add containers-virt
|
||||
|
||||
Alternative bundle options are
|
||||
Alternative bundle options are
|
||||
|
||||
cloud-native-basic - Contains ClearLinux native software for Cloud
|
||||
cloud-native-basic - Contains ClearLinux native software for Cloud
|
||||
|
||||
.. note::
|
||||
.. note::
|
||||
|
||||
If your search does not produce results with a specific
|
||||
term, shorten the search term. For example, use *kube* instead of
|
||||
*kubernetes*.
|
||||
If your search does not produce results with a specific term, shorten
|
||||
the search term. For example, use *kube* instead of *kubernetes*.
|
||||
|
||||
#. Add the bundle.
|
||||
|
||||
@@ -248,7 +237,7 @@ number of dependencies. Also, check out our tutorial: :ref:`kata`.
|
||||
|
||||
.. note::
|
||||
|
||||
To add multiple bundles simply add a space followed by the bundle name.
|
||||
To add multiple bundles, add a space followed by the bundle name.
|
||||
|
||||
The output of a successful installation should be similar to:
|
||||
|
||||
@@ -279,7 +268,7 @@ been added to, overwritten, removed, or modified (e.g., permissions).
|
||||
sudo swupd diagnose
|
||||
|
||||
All directories that are watched by :command:`swupd` are verified according
|
||||
to the manifest data and hash mismatches are flagged as follows:
|
||||
to the manifest data. Hash mismatches are flagged as follows:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -295,47 +284,49 @@ to the manifest data and hash mismatches are flagged as follows:
|
||||
423 files did not match
|
||||
Verify successful
|
||||
|
||||
In this case, python packages that were installed on top of the default
|
||||
In this case, Python\* packages that were installed on top of the default
|
||||
install were flagged as mismatched. :command:`swupd` can be directed to
|
||||
ignore or fix issues based on command line options.
|
||||
|
||||
:command:`swupd` can correct any issues it detects. Additional directives
|
||||
can be added including a white list of directories that will be ignored.
|
||||
can be added including a white list of directories to be ignored.
|
||||
|
||||
The following command will repair issues, remove unknown items, and
|
||||
ignore files or directories matching `/usr/lib/python`:
|
||||
The following command repairs issues, removes unknown items, and
|
||||
ignores files or directories matching :file:`/usr/lib/python`:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd repair --picky --picky-whitelist=/usr/lib/python
|
||||
|
||||
Quick Reference
|
||||
Quick reference
|
||||
***************
|
||||
|
||||
swupd info
|
||||
To see the currently installed version and update servers.
|
||||
Returns the currently installed version and update servers.
|
||||
|
||||
swupd update <version number>
|
||||
To update to a specific version or with no arguments to update to latest.
|
||||
Updates to a specific version or updates to latest version if no
|
||||
arguments are used.
|
||||
|
||||
swupd bundle-list [--all]
|
||||
To list installed bundles.
|
||||
Lists installed bundles.
|
||||
|
||||
swupd bundle-add [-b] <search term>
|
||||
To find a bundle that contains your search term.
|
||||
Finds a bundle that contains your search term.
|
||||
|
||||
swupd bundle-add <bundle name>
|
||||
To add a bundle.
|
||||
Adds a bundle.
|
||||
|
||||
swupd bundle-remove <bundle name>
|
||||
To remove a bundle.
|
||||
Removes a bundle.
|
||||
|
||||
swupd --help
|
||||
For additional :command:`swupd` commands.
|
||||
Lists additional :command:`swupd` commands.
|
||||
|
||||
man swupd
|
||||
To reference the :command:`swupd` man page, or see the
|
||||
`source documentation`_ available on github.
|
||||
Opens the :command:`swupd` man page.
|
||||
|
||||
Refer to :command:`swupd` `source documentation`_ on GitHub for more details.
|
||||
|
||||
Related topics
|
||||
**************
|
||||
|
||||
@@ -6,7 +6,6 @@ Deep Learning Reference Stack
|
||||
This tutorial describes how to run benchmarking workloads for TensorFlow\*,
|
||||
PyTorch\*, and Kubeflow in |CL-ATTR| using the Deep Learning Reference Stack.
|
||||
|
||||
|
||||
.. contents::
|
||||
:local:
|
||||
:depth: 1
|
||||
@@ -26,14 +25,14 @@ The Deep Learning Reference Stack is available in the following versions:
|
||||
for Deep Neural Networks (Intel® MKL-DNN) primitives and introduces support
|
||||
for Intel® AVX-512 Vector Neural Network Instructions (VNNI).
|
||||
* `Intel MKL-DNN`_, which includes the TensorFlow framework optimized using
|
||||
Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN) primitives.
|
||||
Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN)
|
||||
primitives.
|
||||
* `Eigen`_, which includes `TensorFlow`_ optimized for Intel® architecture.
|
||||
* `PyTorch with OpenBLAS`_, which includes PyTorch with OpenBlas.
|
||||
* `PyTorch with Intel MKL-DNN`_, which includes PyTorch optimized using Intel®
|
||||
Math Kernel Library (Intel® MKL) and Intel MKL-DNN.
|
||||
|
||||
|
||||
.. note::
|
||||
.. important::
|
||||
|
||||
To take advantage of the Intel® AVX-512 and VNNI functionality with the Deep
|
||||
Learning Reference Stack, you must use the following hardware:
|
||||
@@ -41,14 +40,16 @@ The Deep Learning Reference Stack is available in the following versions:
|
||||
* Intel® AVX-512 images require an Intel® Xeon® Scalable Platform
|
||||
* VNNI requires a 2nd generation Intel® Xeon® Scalable Platform
|
||||
|
||||
|
||||
Stack features
|
||||
==============
|
||||
|
||||
* Deep Learning Reference Stack `V3.0 release announcement`_.
|
||||
* Deep Learning Reference Stack v2.0 including current `PyTorch benchmark results`_.
|
||||
* Deep Learning Reference Stack v1.0 including current `TensorFlow benchmark results`_.
|
||||
* `Release notes on Github\*`_ for the latest release of Deep Learning Reference Stack.
|
||||
* Deep Learning Reference Stack v2.0 including current
|
||||
`PyTorch benchmark results`_.
|
||||
* Deep Learning Reference Stack v1.0 including current
|
||||
`TensorFlow benchmark results`_.
|
||||
* `Release notes on Github\*`_ for the latest release of Deep Learning Reference
|
||||
Stack.
|
||||
|
||||
.. note::
|
||||
|
||||
@@ -58,19 +59,21 @@ Stack features
|
||||
Prerequisites
|
||||
=============
|
||||
|
||||
* :ref:`Install <bare-metal-install-desktop>` |CL| on your host system.
|
||||
* :ref:`Install <bare-metal-install-desktop>` |CL| on your host system
|
||||
* :command:`containers-basic` bundle
|
||||
* :command:`cloud-native-basic` bundle
|
||||
|
||||
In |CL|, :command:`containers-basic` includes Docker\*, which is required for
|
||||
TensorFlow and PyTorch benchmarking. Use the :command:`swupd` utility to
|
||||
check if :command:`containers-basic` and :command:`cloud-native-basic` are present:
|
||||
check if :command:`containers-basic` and :command:`cloud-native-basic` are
|
||||
present:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd bundle-list
|
||||
|
||||
To install the :command:`containers-basic` or :command:`cloud-native-basic` bundles, enter:
|
||||
To install the :command:`containers-basic` or :command:`cloud-native-basic`
|
||||
bundles, enter:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@@ -91,7 +94,7 @@ Version compatibility
|
||||
|
||||
We validated these steps against the following software package versions:
|
||||
|
||||
* |CL| 26240 (Lower version not supported.)
|
||||
* |CL| 26240 (Lower version not supported)
|
||||
* Docker 18.06.1
|
||||
* Kubernetes 1.11.3
|
||||
* Go 1.11.12
|
||||
@@ -137,8 +140,8 @@ TensorFlow.
|
||||
You can replace the model with one of your choice supported by the
|
||||
TensorFlow benchmarks.
|
||||
|
||||
If you are using an FP32 based model, it can be converted to an int8 model using `Intel® quantization tools`_
|
||||
|
||||
If you are using an FP32 based model, it can be converted to an int8 model
|
||||
using `Intel® quantization tools`_.
|
||||
|
||||
PyTorch single and multi-node benchmarks
|
||||
****************************************
|
||||
@@ -182,6 +185,11 @@ Kubeflow multi-node benchmarks
|
||||
The benchmark workload runs in a Kubernetes cluster. The tutorial uses
|
||||
`Kubeflow`_ for the Machine Learning workload deployment on three nodes.
|
||||
|
||||
.. warning::
|
||||
|
||||
If you choose the Intel® MKL-DNN or Intel® MKL-DNN-VNNI image, your platform must support the Intel® AVX-512 instruction set. Otherwise, an *illegal instruction* error may appear, and you won’t be able to complete this tutorial.
|
||||
|
||||
|
||||
Kubernetes setup
|
||||
================
|
||||
|
||||
@@ -196,12 +204,48 @@ We used `flannel`_ as the network provider for these tests. If you
|
||||
prefer a different network layer, refer to the Kubernetes
|
||||
`networking documentation`_ for setup.
|
||||
|
||||
Kubectl
|
||||
=======
|
||||
|
||||
You can use kubectl to run commands against your Kubernetes cluster. Refer to
|
||||
the `kubectl overview`_ for details on syntax and operations. Once you have a
|
||||
working cluster on Kubernetes, use the following YAML script to start a pod with
|
||||
a simple shell script, and keep the pod open.
|
||||
|
||||
#. Copy this example.yaml script to your system:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: example-pod
|
||||
labels:
|
||||
app: ex-pod
|
||||
spec:
|
||||
containers:
|
||||
- name: ex-pod-container
|
||||
image: clearlinux/stacks-dlrs-mkl:latest
|
||||
command: ['/bin/bash', '-c', '--']
|
||||
args: [ "while true; do sleep 30; done" ]
|
||||
|
||||
#. Execute the script with kubectl:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
kubectl apply –f <path-to-yaml-file>/example.yaml
|
||||
|
||||
This script opens a single pod. More robust solutions would create a deployment
|
||||
or inject a python script or larger shell script into the container.
|
||||
|
||||
Images
|
||||
======
|
||||
|
||||
You must add `launcher.py` to the Docker image to include the Deep
|
||||
You must add `launcher.py`_ to the Docker image to include the Deep
|
||||
Learning Reference Stack and put the benchmarks repo in the correct
|
||||
location. From the Docker image, run the following:
|
||||
location. Note that this tutorial uses Kubeflow v0.4.0, and cannot guarantee results if you use a different version.
|
||||
|
||||
From the Docker image, run the following:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@@ -210,7 +254,7 @@ location. From the Docker image, run the following:
|
||||
cp launcher.py /opt
|
||||
chmod u+x /opt/*
|
||||
|
||||
Your entry point becomes: :file:`/opt/launcher.py`
|
||||
Your entry point becomes: :file:`/opt/launcher.py`.
|
||||
|
||||
This builds an image that can be consumed directly by TFJob from Kubeflow.
|
||||
|
||||
@@ -220,9 +264,9 @@ ksonnet\*
|
||||
Kubeflow uses ksonnet\* to manage deployments, so you must install it
|
||||
before setting up Kubeflow.
|
||||
|
||||
ksonnet was added to the :command:`cloud-native-basic` bundle in |CL| version 27550. If
|
||||
you are using an older |CL| version (not recommended), you must manually
|
||||
install ksonnet as described below.
|
||||
ksonnet was added to the :command:`cloud-native-basic` bundle in |CL| version
|
||||
27550. If you are using an older |CL| version (not recommended), you must
|
||||
manually install ksonnet as described below.
|
||||
|
||||
On |CL|, follow these steps:
|
||||
|
||||
@@ -294,7 +338,8 @@ Run a TFJob
|
||||
|
||||
Replace <docker_name> with the image name you specified in previous steps.
|
||||
|
||||
#. Generate Kubernetes manifests for the workloads and apply them using these commands:
|
||||
#. Generate Kubernetes manifests for the workloads and apply them using these
|
||||
commands:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@@ -372,7 +417,8 @@ A new, blank notebook is displayed, with a cell ready for input.
|
||||
:alt: New blank notebook
|
||||
|
||||
|
||||
To verify that PyTorch is working, copy the following snippet into the blank cell, and run the cell.
|
||||
To verify that PyTorch is working, copy the following snippet into the blank
|
||||
cell, and run the cell.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -405,7 +451,6 @@ Related topics
|
||||
* :ref:`kubernetes` tutorial
|
||||
* `Jupyter Notebook`_
|
||||
|
||||
|
||||
.. _TensorFlow: https://www.tensorflow.org/
|
||||
|
||||
.. _Kubeflow: https://www.kubeflow.org/
|
||||
@@ -446,6 +491,10 @@ Related topics
|
||||
|
||||
.. _Jupyter Notebook: https://jupyter.org/
|
||||
|
||||
.. _kubectl overview: https://kubernetes.io/docs/reference/kubectl/overview/
|
||||
|
||||
.. _launcher.py: https://github.com/clearlinux/dockerfiles/tree/master/stacks/dlrs/kubeflow
|
||||
|
||||
.. _Release notes on Github\*: https://github.com/clearlinux/dockerfiles/blob/master/stacks/dlrs/releasenote.md
|
||||
|
||||
.. _Intel® quantization tools: https://github.com/IntelAI/tools/blob/master/tensorflow_quantization/README.md#quantization-tools
|
||||
|
||||
@@ -28,3 +28,4 @@ Explore our tutorials to discover what you can do with |CL|!
|
||||
nvidia
|
||||
dars
|
||||
redis
|
||||
tutorial-proxy
|
||||
|
||||
Reference in New Issue
Block a user