commit 90b41ced3a1f08e80131689d5e3822aab18fce50 Author: alexjch Date: Mon Nov 4 18:56:31 2024 +0000 latest html output diff --git a/.buildinfo b/.buildinfo new file mode 100644 index 00000000..6f085372 --- /dev/null +++ b/.buildinfo @@ -0,0 +1,4 @@ +# Sphinx build info version 1 +# This file records the configuration used when building these files. When it is not found, a full rebuild will be done. +config: f8974e043de468370d3f73767671d8de +tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/.nojekyll b/.nojekyll new file mode 100644 index 00000000..e69de29b diff --git a/FAQ/index.html b/FAQ/index.html new file mode 100644 index 00000000..cb8058fe --- /dev/null +++ b/FAQ/index.html @@ -0,0 +1,405 @@ + + + + + + + + + FAQ — Documentation for Clear Linux* project + + + + + + + + + + + + + + + + + + + + + + +
+
+
+
+ +
+

FAQ

+

Below is a list of commonly asked questions with answers sourced from the +Clear Linux* OS team and Clear Linux community forums.

+ +
+

General

+
+

What is Clear Linux OS?

+

Clear Linux OS is an open source, rolling release Linux distribution optimized for +performance and security. See the about page +for more information.

+
+

+
+
+
+

Why another Linux distribution?

+

The Clear Linux OS team felt that performance was left on the table with Linux software. +Clear Linux OS takes a holistic approach to improve performance across the stack. We +also wanted to take more modern approaches with OS updates and tooling.

+
+

+
+
+
+

Is it a derivative of another Linux distribution?

+

No. Clear Linux OS is a new Linux distribution. It is not a fork and does not have a +parent Linux distribution.

+
+

+
+
+
+

Can others copy improvements from Clear Linux OS?

+

Yes, we absolutely love open source reuse and upstreaming improvements.

+
+

+
+
+
+

How often does it update?

+

The Clear Linux OS team puts out multiple releases a week, often releasing two or more +times a day. This rolling release approach allows Clear Linux OS to remain agile to +upstream changes and security patches.

+
+

+
+
+
+

Is telemetry required?

+

The telemetry solution provided by Clear Linux OS is entirely optional and customizable. +It is disabled by default. If you do choose to enable telemetry, the data +helps the Clear Linux OS team proactively identify and resolve bugs. See the +telemetry guide for more information.

+
+

+
+
+
+

What is the default firewall?

+

Clear Linux OS packages iptables and firewalld as optional +bundles, however, there are no default firewall rules. All network traffic is +allowed by default.

+
+

+
+
+
+

Where are the files that I usually see under /etc like fstab?

+

Clear Linux OS has a stateless design that maintains a separation between system files +and user files. Default values are stored under /usr/share/defaults/. +Clear Linux OS starts with a mostly empty /etc directory to store user-defined +configurations. See the stateless page for more information.

+

See this blog post for an +example explaining how this is accomplished with /etc/fstab/ +specifically.

+
+

+
+
+
+

Does it use the Intel Compiler (icc)?

+

No. Clear Linux OS uses open source compilers: gcc and clang. Clear Linux OS +does not compile any packages with icc.

+

For a more detailed explanation, see this discussion on the community forum.

+
+

+
+
+
+
+

Software

+
+

How is software installed and updated?

+

Clear Linux OS provides software in the form of bundles and +updates software with swupd.

+

Flatpak* is an application virtualization solution +that allows more software to be available to Clear Linux OS users by augmenting the +software Clear Linux OS packages natively with software available through Flatpak.

+

Our goal is to have software packaged natively and made available through +bundles whenever possible.

+
+

+
+
+
+

Does it use RPMs or DEBs packages like other distros?

+

No. Clear Linux OS provides software to systems in the form of Bundles. +Under the hood, Clear Linux OS developers use the RPM format as an intermediary step for +packaging and determining software dependencies at OS build time.

+

Individual RPMs and DEBs can sometimes be manually extracted and installed on +a Clear Linux OS system with the right tools, but that is not the intended use case.

+
+

+
+
+
+

Why does it have a different approach to software management?

+

The Clear Linux OS team wants software installation and updates to be as efficient +and error free as possible. Clear Linux OS packages software differently and uses a +novel updater to solve some of the classic problems with how the software +packages are on Linux.

+

For a more detailed explanation, see this discussion on our community forum.

+
+

+
+
+
+

Can I install a software package from another OS on Clear Linux OS?

+

Software that is packaged in other formats for other Linux distributions is +not guaranteed to work on Clear Linux OS and may be impacted by Clear Linux OS updates.

+

If the software you’re seeking is open source, please submit a request to add +it to Clear Linux OS. Submit requests on GitHub* here: +https://github.com/clearlinux/distribution/issues

+
+

+
+
+
+
+

Software availability

+
+

What software is available on Clear Linux OS?

+

Available software can be found in the Software Store, through the GNOME* +Software application on the desktop, or by using swupd search.

+
+

+
+
+
+

Is Google* Chrome* available?

+

The Google Chrome web browser is not distributed as a bundle in Clear Linux OS due to +copyright and licensing complexities.

+

A discussion on manually installing and maintaining Google Chrome can be found on +GitHub.

+
+

+
+
+
+

Is Microsoft* Visual Studio Code* available?

+

Yes. Find the CLI command for installing VS Code and other Flatpak apps in +the software store. Installing Flatpak apps is also covered in our +tutorial.

+

The Clear Linux OS team is working on a natively packaged version of Visual Studio Code +for future release.

+

Join a community forum discussion about manually installing and maintaining +Visual Studio Code.

+
+

+
+
+
+

Is FFmpeg available?

+

FFmpeg is a multimedia software suite, which is commonly used for +various media encoding/decoding, streaming, and playback.

+

Clear Linux OS does not distribute FFmpeg due to well-known licensing and legal +complexities (See https://www.ffmpeg.org/legal.html and +http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html).

+

While Clear Linux OS cannot distribute FFmpeg, solutions for manually building and +installing FFmpeg have been shared by users on GitHub and the community +forums.

+
+

+
+
+
+

Is ZFS* available?

+

ZFS is not available with Clear Linux OS because of copyright and licensing +complexities. BTRFS is an alternative filesystem that is available in Clear Linux OS +natively.

+

A community contributed tutorial has been shared on how to manually +install ZFS.

+
+

+
+
+
+

Can you add a driver that I need?

+

If a kernel module is available as part of the Linux kernel source tree but +not enabled in the Clear Linux OS kernels, in many cases the Clear Linux OS team will enable it +upon request. Submit requests on GitHub here: +https://github.com/clearlinux/distribution/issues

+

The Clear Linux OS team does not typically add out-of-tree kernel modules as a matter of +practice because of the maintenance overhead. If the driver was unable to be +merged upstream, there is a good chance we may be unable to merge it for +similar reasons.

+

Kernel modules can be individually built and installed on Clear Linux OS. See the +kernel modules page for more information.

+
+

+
+
+
+
+ + +
+
+
+
+ +
+
+ + + + \ No newline at end of file diff --git a/_errors/403.html b/_errors/403.html new file mode 100644 index 00000000..bca14094 --- /dev/null +++ b/_errors/403.html @@ -0,0 +1,79 @@ + + + + + Page Not Found + + + + + + + + +

403 Forbidden

+

Sorry, looks like you don't have permission to access this page.

+

Please go back to the documentation home page or try searching:

+ +
+
+ + + +
+
+ + diff --git a/_errors/404.html b/_errors/404.html new file mode 100644 index 00000000..32046bfb --- /dev/null +++ b/_errors/404.html @@ -0,0 +1,79 @@ + + + + + Page Not Found + + + + + + + + +

404 Not Found

+

Sorry, we didn't find the page you're looking for.

+

Please go back to the documentation home page or try searching:

+ +
+
+ + + +
+
+ + diff --git a/_images/00-sign-in.png b/_images/00-sign-in.png new file mode 100644 index 00000000..4b9840ee Binary files /dev/null and b/_images/00-sign-in.png differ diff --git a/_images/01-cloud-storage.png b/_images/01-cloud-storage.png new file mode 100644 index 00000000..0888723d Binary files /dev/null and b/_images/01-cloud-storage.png differ diff --git a/_images/01-digitalocean.png b/_images/01-digitalocean.png new file mode 100644 index 00000000..399c81fe Binary files /dev/null and b/_images/01-digitalocean.png differ diff --git a/_images/01-increase-virtual-disk-size.png b/_images/01-increase-virtual-disk-size.png new file mode 100644 index 00000000..63a23b1c Binary files /dev/null and b/_images/01-increase-virtual-disk-size.png differ diff --git a/_images/02-digitalocean.png b/_images/02-digitalocean.png new file mode 100644 index 00000000..2c93fd83 Binary files /dev/null and b/_images/02-digitalocean.png differ diff --git a/_images/02-increase-virtual-disk-size.png b/_images/02-increase-virtual-disk-size.png new file mode 100644 index 00000000..207888c5 Binary files /dev/null and b/_images/02-increase-virtual-disk-size.png differ diff --git a/_images/02-storage-browser.png b/_images/02-storage-browser.png new file mode 100644 index 00000000..72db76fd Binary files /dev/null and b/_images/02-storage-browser.png differ diff --git a/_images/03-create-bucket.png b/_images/03-create-bucket.png new file mode 100644 index 00000000..ad00ca00 Binary files /dev/null and b/_images/03-create-bucket.png differ diff --git a/_images/03-digitalocean.png b/_images/03-digitalocean.png new file mode 100644 index 00000000..313cc6f3 Binary files /dev/null and b/_images/03-digitalocean.png differ diff --git a/_images/04-bucket-created.png b/_images/04-bucket-created.png new file mode 100644 index 00000000..064719cb Binary files /dev/null and b/_images/04-bucket-created.png differ diff --git a/_images/04-digitalocean.png b/_images/04-digitalocean.png new file mode 100644 index 00000000..72f17284 Binary files /dev/null and b/_images/04-digitalocean.png differ diff --git a/_images/05-digitalocean.png b/_images/05-digitalocean.png new file mode 100644 index 00000000..81fe8b1b Binary files /dev/null and b/_images/05-digitalocean.png differ diff --git a/_images/06-digitalocean.png b/_images/06-digitalocean.png new file mode 100644 index 00000000..ed914674 Binary files /dev/null and b/_images/06-digitalocean.png differ diff --git a/_images/07-digitalocean.png b/_images/07-digitalocean.png new file mode 100644 index 00000000..15b694d6 Binary files /dev/null and b/_images/07-digitalocean.png differ diff --git a/_images/08-digitalocean.png b/_images/08-digitalocean.png new file mode 100644 index 00000000..6c336241 Binary files /dev/null and b/_images/08-digitalocean.png differ diff --git a/_images/09-digitalocean.png b/_images/09-digitalocean.png new file mode 100644 index 00000000..9ecd4231 Binary files /dev/null and b/_images/09-digitalocean.png differ diff --git a/_images/10-digitalocean.png b/_images/10-digitalocean.png new file mode 100644 index 00000000..a182643d Binary files /dev/null and b/_images/10-digitalocean.png differ diff --git a/_images/10-image-upload.png b/_images/10-image-upload.png new file mode 100644 index 00000000..babaf2d1 Binary files /dev/null and b/_images/10-image-upload.png differ diff --git a/_images/11-bucket-uploaded.png b/_images/11-bucket-uploaded.png new file mode 100644 index 00000000..ce368e44 Binary files /dev/null and b/_images/11-bucket-uploaded.png differ diff --git a/_images/11-digitalocean.png b/_images/11-digitalocean.png new file mode 100644 index 00000000..fb24087b Binary files /dev/null and b/_images/11-digitalocean.png differ diff --git a/_images/20-gce-image.png b/_images/20-gce-image.png new file mode 100644 index 00000000..7b8a2ccb Binary files /dev/null and b/_images/20-gce-image.png differ diff --git a/_images/20-image-library.png b/_images/20-image-library.png new file mode 100644 index 00000000..a70cceee Binary files /dev/null and b/_images/20-image-library.png differ diff --git a/_images/21-create-image.png b/_images/21-create-image.png new file mode 100644 index 00000000..4ef4d43b Binary files /dev/null and b/_images/21-create-image.png differ diff --git a/_images/22-image-list.png b/_images/22-image-list.png new file mode 100644 index 00000000..4347e462 Binary files /dev/null and b/_images/22-image-list.png differ diff --git a/_images/30-create-vm.png b/_images/30-create-vm.png new file mode 100644 index 00000000..49854b5f Binary files /dev/null and b/_images/30-create-vm.png differ diff --git a/_images/30-vm-catalog.png b/_images/30-vm-catalog.png new file mode 100644 index 00000000..06071cf6 Binary files /dev/null and b/_images/30-vm-catalog.png differ diff --git a/_images/30-vm-instances.png b/_images/30-vm-instances.png new file mode 100644 index 00000000..3c38bd8d Binary files /dev/null and b/_images/30-vm-instances.png differ diff --git a/_images/30-vm-none.png b/_images/30-vm-none.png new file mode 100644 index 00000000..3c118f49 Binary files /dev/null and b/_images/30-vm-none.png differ diff --git a/_images/31-select-boot-disk.png b/_images/31-select-boot-disk.png new file mode 100644 index 00000000..b0b44cdf Binary files /dev/null and b/_images/31-select-boot-disk.png differ diff --git a/_images/40-clear-vm-security.png b/_images/40-clear-vm-security.png new file mode 100644 index 00000000..8d45b4a4 Binary files /dev/null and b/_images/40-clear-vm-security.png differ diff --git a/_images/40-ssh-key.png b/_images/40-ssh-key.png new file mode 100644 index 00000000..e18a390c Binary files /dev/null and b/_images/40-ssh-key.png differ diff --git a/_images/41-vm-created.png b/_images/41-vm-created.png new file mode 100644 index 00000000..ec4f3a20 Binary files /dev/null and b/_images/41-vm-created.png differ diff --git a/_images/42-ssh-vm.png b/_images/42-ssh-vm.png new file mode 100644 index 00000000..92badb80 Binary files /dev/null and b/_images/42-ssh-vm.png differ diff --git a/_images/QingCloud-1.png b/_images/QingCloud-1.png new file mode 100644 index 00000000..3c3b3c1b Binary files /dev/null and b/_images/QingCloud-1.png differ diff --git a/_images/QingCloud-10.png b/_images/QingCloud-10.png new file mode 100644 index 00000000..79919743 Binary files /dev/null and b/_images/QingCloud-10.png differ diff --git a/_images/QingCloud-11.png b/_images/QingCloud-11.png new file mode 100644 index 00000000..a29370a3 Binary files /dev/null and b/_images/QingCloud-11.png differ diff --git a/_images/QingCloud-12.png b/_images/QingCloud-12.png new file mode 100644 index 00000000..d344d368 Binary files /dev/null and b/_images/QingCloud-12.png differ diff --git a/_images/QingCloud-13.png b/_images/QingCloud-13.png new file mode 100644 index 00000000..8c0a9f87 Binary files /dev/null and b/_images/QingCloud-13.png differ diff --git a/_images/QingCloud-14.png b/_images/QingCloud-14.png new file mode 100644 index 00000000..61caa0a7 Binary files /dev/null and b/_images/QingCloud-14.png differ diff --git a/_images/QingCloud-15.png b/_images/QingCloud-15.png new file mode 100644 index 00000000..0152fca4 Binary files /dev/null and b/_images/QingCloud-15.png differ diff --git a/_images/QingCloud-16.png b/_images/QingCloud-16.png new file mode 100644 index 00000000..48040924 Binary files /dev/null and b/_images/QingCloud-16.png differ diff --git a/_images/QingCloud-17.png b/_images/QingCloud-17.png new file mode 100644 index 00000000..99827193 Binary files /dev/null and b/_images/QingCloud-17.png differ diff --git a/_images/QingCloud-18.png b/_images/QingCloud-18.png new file mode 100644 index 00000000..e8c47b82 Binary files /dev/null and b/_images/QingCloud-18.png differ diff --git a/_images/QingCloud-19.png b/_images/QingCloud-19.png new file mode 100644 index 00000000..ba6c26b9 Binary files /dev/null and b/_images/QingCloud-19.png differ diff --git a/_images/QingCloud-2.png b/_images/QingCloud-2.png new file mode 100644 index 00000000..ed9d5701 Binary files /dev/null and b/_images/QingCloud-2.png differ diff --git a/_images/QingCloud-20.png b/_images/QingCloud-20.png new file mode 100644 index 00000000..c71dae6d Binary files /dev/null and b/_images/QingCloud-20.png differ diff --git a/_images/QingCloud-21.png b/_images/QingCloud-21.png new file mode 100644 index 00000000..990e7cff Binary files /dev/null and b/_images/QingCloud-21.png differ diff --git a/_images/QingCloud-3.png b/_images/QingCloud-3.png new file mode 100644 index 00000000..bd326a6c Binary files /dev/null and b/_images/QingCloud-3.png differ diff --git a/_images/QingCloud-4.png b/_images/QingCloud-4.png new file mode 100644 index 00000000..35f480ba Binary files /dev/null and b/_images/QingCloud-4.png differ diff --git a/_images/QingCloud-6.png b/_images/QingCloud-6.png new file mode 100644 index 00000000..a800fe1e Binary files /dev/null and b/_images/QingCloud-6.png differ diff --git a/_images/QingCloud-7.png b/_images/QingCloud-7.png new file mode 100644 index 00000000..b1f3d880 Binary files /dev/null and b/_images/QingCloud-7.png differ diff --git a/_images/QingCloud-8.png b/_images/QingCloud-8.png new file mode 100644 index 00000000..688ad825 Binary files /dev/null and b/_images/QingCloud-8.png differ diff --git a/_images/QingCloud-9.png b/_images/QingCloud-9.png new file mode 100644 index 00000000..f79ee9dd Binary files /dev/null and b/_images/QingCloud-9.png differ diff --git a/_images/architect-lifecycle-1.png b/_images/architect-lifecycle-1.png new file mode 100644 index 00000000..294d0f35 Binary files /dev/null and b/_images/architect-lifecycle-1.png differ diff --git a/_images/autoproxy_0.png b/_images/autoproxy_0.png new file mode 100644 index 00000000..7a02cd81 Binary files /dev/null and b/_images/autoproxy_0.png differ diff --git a/_images/aws-web-1.png b/_images/aws-web-1.png new file mode 100644 index 00000000..c4fad685 Binary files /dev/null and b/_images/aws-web-1.png differ diff --git a/_images/aws-web-10.png b/_images/aws-web-10.png new file mode 100644 index 00000000..b4fbeb13 Binary files /dev/null and b/_images/aws-web-10.png differ diff --git a/_images/aws-web-11.png b/_images/aws-web-11.png new file mode 100644 index 00000000..2a6dcab0 Binary files /dev/null and b/_images/aws-web-11.png differ diff --git a/_images/aws-web-12.png b/_images/aws-web-12.png new file mode 100644 index 00000000..1091ea72 Binary files /dev/null and b/_images/aws-web-12.png differ diff --git a/_images/aws-web-13.png b/_images/aws-web-13.png new file mode 100644 index 00000000..507c8c12 Binary files /dev/null and b/_images/aws-web-13.png differ diff --git a/_images/aws-web-14.png b/_images/aws-web-14.png new file mode 100644 index 00000000..36e10661 Binary files /dev/null and b/_images/aws-web-14.png differ diff --git a/_images/aws-web-2.png b/_images/aws-web-2.png new file mode 100644 index 00000000..8bb6ffa0 Binary files /dev/null and b/_images/aws-web-2.png differ diff --git a/_images/aws-web-3.png b/_images/aws-web-3.png new file mode 100644 index 00000000..7beb5ad8 Binary files /dev/null and b/_images/aws-web-3.png differ diff --git a/_images/aws-web-4.png b/_images/aws-web-4.png new file mode 100644 index 00000000..cd143b51 Binary files /dev/null and b/_images/aws-web-4.png differ diff --git a/_images/aws-web-5.png b/_images/aws-web-5.png new file mode 100644 index 00000000..9c70a903 Binary files /dev/null and b/_images/aws-web-5.png differ diff --git a/_images/aws-web-6.png b/_images/aws-web-6.png new file mode 100644 index 00000000..a5819f7a Binary files /dev/null and b/_images/aws-web-6.png differ diff --git a/_images/aws-web-7.png b/_images/aws-web-7.png new file mode 100644 index 00000000..dad60bd8 Binary files /dev/null and b/_images/aws-web-7.png differ diff --git a/_images/aws-web-8.png b/_images/aws-web-8.png new file mode 100644 index 00000000..9e927a43 Binary files /dev/null and b/_images/aws-web-8.png differ diff --git a/_images/aws-web-9.png b/_images/aws-web-9.png new file mode 100644 index 00000000..2c1f411c Binary files /dev/null and b/_images/aws-web-9.png differ diff --git a/_images/azure-1.png b/_images/azure-1.png new file mode 100644 index 00000000..25c1800f Binary files /dev/null and b/_images/azure-1.png differ diff --git a/_images/azure-2.png b/_images/azure-2.png new file mode 100644 index 00000000..8364393c Binary files /dev/null and b/_images/azure-2.png differ diff --git a/_images/azure-3.png b/_images/azure-3.png new file mode 100644 index 00000000..dfb1c11c Binary files /dev/null and b/_images/azure-3.png differ diff --git a/_images/balenaEtcher_Done.PNG b/_images/balenaEtcher_Done.PNG new file mode 100644 index 00000000..d2c884fc Binary files /dev/null and b/_images/balenaEtcher_Done.PNG differ diff --git a/_images/balenaEtcher_DriveSlect.PNG b/_images/balenaEtcher_DriveSlect.PNG new file mode 100644 index 00000000..1bb0f2cf Binary files /dev/null and b/_images/balenaEtcher_DriveSlect.PNG differ diff --git a/_images/balenaEtcher_Flashing.PNG b/_images/balenaEtcher_Flashing.PNG new file mode 100644 index 00000000..8cf9c5ce Binary files /dev/null and b/_images/balenaEtcher_Flashing.PNG differ diff --git a/_images/balenaEtcher_ImageSelect.PNG b/_images/balenaEtcher_ImageSelect.PNG new file mode 100644 index 00000000..c1836bfd Binary files /dev/null and b/_images/balenaEtcher_ImageSelect.PNG differ diff --git a/_images/balenaEtcher_ReadyToFlash.PNG b/_images/balenaEtcher_ReadyToFlash.PNG new file mode 100644 index 00000000..4662cbe5 Binary files /dev/null and b/_images/balenaEtcher_ReadyToFlash.PNG differ diff --git a/_images/balenaEtcher_Start.PNG b/_images/balenaEtcher_Start.PNG new file mode 100644 index 00000000..9b02db9a Binary files /dev/null and b/_images/balenaEtcher_Start.PNG differ diff --git a/_images/balenaEtcher_StartingToFlash.PNG b/_images/balenaEtcher_StartingToFlash.PNG new file mode 100644 index 00000000..c0b6f7ff Binary files /dev/null and b/_images/balenaEtcher_StartingToFlash.PNG differ diff --git a/_images/bare-metal-install-desktop-01.png b/_images/bare-metal-install-desktop-01.png new file mode 100644 index 00000000..57f998de Binary files /dev/null and b/_images/bare-metal-install-desktop-01.png differ diff --git a/_images/bare-metal-install-desktop-02.png b/_images/bare-metal-install-desktop-02.png new file mode 100644 index 00000000..bce97b57 Binary files /dev/null and b/_images/bare-metal-install-desktop-02.png differ diff --git a/_images/bare-metal-install-desktop-03.png b/_images/bare-metal-install-desktop-03.png new file mode 100644 index 00000000..a2650b35 Binary files /dev/null and b/_images/bare-metal-install-desktop-03.png differ diff --git a/_images/bare-metal-install-desktop-04.png b/_images/bare-metal-install-desktop-04.png new file mode 100644 index 00000000..7efc721f Binary files /dev/null and b/_images/bare-metal-install-desktop-04.png differ diff --git a/_images/bare-metal-install-desktop-05.png b/_images/bare-metal-install-desktop-05.png new file mode 100644 index 00000000..8a966ab7 Binary files /dev/null and b/_images/bare-metal-install-desktop-05.png differ diff --git a/_images/bare-metal-install-desktop-06.png b/_images/bare-metal-install-desktop-06.png new file mode 100644 index 00000000..13a3d5f2 Binary files /dev/null and b/_images/bare-metal-install-desktop-06.png differ diff --git a/_images/bare-metal-install-desktop-07.png b/_images/bare-metal-install-desktop-07.png new file mode 100644 index 00000000..059625e5 Binary files /dev/null and b/_images/bare-metal-install-desktop-07.png differ diff --git a/_images/bare-metal-install-desktop-08.png b/_images/bare-metal-install-desktop-08.png new file mode 100644 index 00000000..8497270d Binary files /dev/null and b/_images/bare-metal-install-desktop-08.png differ diff --git a/_images/bare-metal-install-desktop-09.png b/_images/bare-metal-install-desktop-09.png new file mode 100644 index 00000000..db05151c Binary files /dev/null and b/_images/bare-metal-install-desktop-09.png differ diff --git a/_images/bare-metal-install-desktop-10.png b/_images/bare-metal-install-desktop-10.png new file mode 100644 index 00000000..96dec049 Binary files /dev/null and b/_images/bare-metal-install-desktop-10.png differ diff --git a/_images/bare-metal-install-desktop-11.png b/_images/bare-metal-install-desktop-11.png new file mode 100644 index 00000000..a5973cd4 Binary files /dev/null and b/_images/bare-metal-install-desktop-11.png differ diff --git a/_images/bare-metal-install-desktop-12.png b/_images/bare-metal-install-desktop-12.png new file mode 100644 index 00000000..dbcfae3f Binary files /dev/null and b/_images/bare-metal-install-desktop-12.png differ diff --git a/_images/bare-metal-install-desktop-13.png b/_images/bare-metal-install-desktop-13.png new file mode 100644 index 00000000..1f64a847 Binary files /dev/null and b/_images/bare-metal-install-desktop-13.png differ diff --git a/_images/bare-metal-install-desktop-14.png b/_images/bare-metal-install-desktop-14.png new file mode 100644 index 00000000..1986af12 Binary files /dev/null and b/_images/bare-metal-install-desktop-14.png differ diff --git a/_images/bare-metal-install-desktop-15.png b/_images/bare-metal-install-desktop-15.png new file mode 100644 index 00000000..75ef9436 Binary files /dev/null and b/_images/bare-metal-install-desktop-15.png differ diff --git a/_images/bare-metal-install-desktop-16.png b/_images/bare-metal-install-desktop-16.png new file mode 100644 index 00000000..8fdfdcf0 Binary files /dev/null and b/_images/bare-metal-install-desktop-16.png differ diff --git a/_images/bare-metal-install-desktop-17.png b/_images/bare-metal-install-desktop-17.png new file mode 100644 index 00000000..98edfbb7 Binary files /dev/null and b/_images/bare-metal-install-desktop-17.png differ diff --git a/_images/bare-metal-install-desktop-18.png b/_images/bare-metal-install-desktop-18.png new file mode 100644 index 00000000..99cc6763 Binary files /dev/null and b/_images/bare-metal-install-desktop-18.png differ diff --git a/_images/bare-metal-install-desktop-19.png b/_images/bare-metal-install-desktop-19.png new file mode 100644 index 00000000..f47bd702 Binary files /dev/null and b/_images/bare-metal-install-desktop-19.png differ diff --git a/_images/bare-metal-install-desktop-20.png b/_images/bare-metal-install-desktop-20.png new file mode 100644 index 00000000..d81b6dc3 Binary files /dev/null and b/_images/bare-metal-install-desktop-20.png differ diff --git a/_images/bare-metal-install-desktop-21.png b/_images/bare-metal-install-desktop-21.png new file mode 100644 index 00000000..506bfa00 Binary files /dev/null and b/_images/bare-metal-install-desktop-21.png differ diff --git a/_images/bare-metal-install-desktop-22.png b/_images/bare-metal-install-desktop-22.png new file mode 100644 index 00000000..d51a0fa7 Binary files /dev/null and b/_images/bare-metal-install-desktop-22.png differ diff --git a/_images/bare-metal-install-desktop-23.png b/_images/bare-metal-install-desktop-23.png new file mode 100644 index 00000000..d6ac2d52 Binary files /dev/null and b/_images/bare-metal-install-desktop-23.png differ diff --git a/_images/bare-metal-install-desktop-24.png b/_images/bare-metal-install-desktop-24.png new file mode 100644 index 00000000..a82cb2a6 Binary files /dev/null and b/_images/bare-metal-install-desktop-24.png differ diff --git a/_images/bare-metal-install-server-01.png b/_images/bare-metal-install-server-01.png new file mode 100644 index 00000000..bb4e2058 Binary files /dev/null and b/_images/bare-metal-install-server-01.png differ diff --git a/_images/bare-metal-install-server-02.png b/_images/bare-metal-install-server-02.png new file mode 100644 index 00000000..688f85d5 Binary files /dev/null and b/_images/bare-metal-install-server-02.png differ diff --git a/_images/bare-metal-install-server-03.png b/_images/bare-metal-install-server-03.png new file mode 100644 index 00000000..b644a0a5 Binary files /dev/null and b/_images/bare-metal-install-server-03.png differ diff --git a/_images/bare-metal-install-server-04.png b/_images/bare-metal-install-server-04.png new file mode 100644 index 00000000..73bb3f90 Binary files /dev/null and b/_images/bare-metal-install-server-04.png differ diff --git a/_images/bare-metal-install-server-05.png b/_images/bare-metal-install-server-05.png new file mode 100644 index 00000000..9be288ee Binary files /dev/null and b/_images/bare-metal-install-server-05.png differ diff --git a/_images/bare-metal-install-server-06.png b/_images/bare-metal-install-server-06.png new file mode 100644 index 00000000..3e45ec66 Binary files /dev/null and b/_images/bare-metal-install-server-06.png differ diff --git a/_images/bare-metal-install-server-07.png b/_images/bare-metal-install-server-07.png new file mode 100644 index 00000000..2ebac81e Binary files /dev/null and b/_images/bare-metal-install-server-07.png differ diff --git a/_images/bare-metal-install-server-08.png b/_images/bare-metal-install-server-08.png new file mode 100644 index 00000000..63c9bb66 Binary files /dev/null and b/_images/bare-metal-install-server-08.png differ diff --git a/_images/bare-metal-install-server-09.png b/_images/bare-metal-install-server-09.png new file mode 100644 index 00000000..50f6c361 Binary files /dev/null and b/_images/bare-metal-install-server-09.png differ diff --git a/_images/bare-metal-install-server-10.png b/_images/bare-metal-install-server-10.png new file mode 100644 index 00000000..0af6f922 Binary files /dev/null and b/_images/bare-metal-install-server-10.png differ diff --git a/_images/bare-metal-install-server-11.png b/_images/bare-metal-install-server-11.png new file mode 100644 index 00000000..ae967bd7 Binary files /dev/null and b/_images/bare-metal-install-server-11.png differ diff --git a/_images/bare-metal-install-server-12.png b/_images/bare-metal-install-server-12.png new file mode 100644 index 00000000..5a2b87e9 Binary files /dev/null and b/_images/bare-metal-install-server-12.png differ diff --git a/_images/bare-metal-install-server-13.png b/_images/bare-metal-install-server-13.png new file mode 100644 index 00000000..ea195577 Binary files /dev/null and b/_images/bare-metal-install-server-13.png differ diff --git a/_images/bare-metal-install-server-14.png b/_images/bare-metal-install-server-14.png new file mode 100644 index 00000000..22b3b3b8 Binary files /dev/null and b/_images/bare-metal-install-server-14.png differ diff --git a/_images/bare-metal-install-server-15.png b/_images/bare-metal-install-server-15.png new file mode 100644 index 00000000..48061408 Binary files /dev/null and b/_images/bare-metal-install-server-15.png differ diff --git a/_images/bare-metal-install-server-16.png b/_images/bare-metal-install-server-16.png new file mode 100644 index 00000000..32a5ec49 Binary files /dev/null and b/_images/bare-metal-install-server-16.png differ diff --git a/_images/bare-metal-install-server-17.png b/_images/bare-metal-install-server-17.png new file mode 100644 index 00000000..b96aca8f Binary files /dev/null and b/_images/bare-metal-install-server-17.png differ diff --git a/_images/bare-metal-install-server-18.png b/_images/bare-metal-install-server-18.png new file mode 100644 index 00000000..1816d84c Binary files /dev/null and b/_images/bare-metal-install-server-18.png differ diff --git a/_images/bare-metal-install-server-19.png b/_images/bare-metal-install-server-19.png new file mode 100644 index 00000000..a1b1ce64 Binary files /dev/null and b/_images/bare-metal-install-server-19.png differ diff --git a/_images/bare-metal-install-server-20.png b/_images/bare-metal-install-server-20.png new file mode 100644 index 00000000..b3741d40 Binary files /dev/null and b/_images/bare-metal-install-server-20.png differ diff --git a/_images/bare-metal-install-server-21.png b/_images/bare-metal-install-server-21.png new file mode 100644 index 00000000..8045633f Binary files /dev/null and b/_images/bare-metal-install-server-21.png differ diff --git a/_images/bare-metal-install-server-22.png b/_images/bare-metal-install-server-22.png new file mode 100644 index 00000000..14f3492f Binary files /dev/null and b/_images/bare-metal-install-server-22.png differ diff --git a/_images/bare-metal-install-server-23.png b/_images/bare-metal-install-server-23.png new file mode 100644 index 00000000..315d7673 Binary files /dev/null and b/_images/bare-metal-install-server-23.png differ diff --git a/_images/bare-metal-install-server-24.png b/_images/bare-metal-install-server-24.png new file mode 100644 index 00000000..4a082720 Binary files /dev/null and b/_images/bare-metal-install-server-24.png differ diff --git a/_images/bare-metal-install-server-25.png b/_images/bare-metal-install-server-25.png new file mode 100644 index 00000000..79db68cb Binary files /dev/null and b/_images/bare-metal-install-server-25.png differ diff --git a/_images/bare-metal-install-server-26.png b/_images/bare-metal-install-server-26.png new file mode 100644 index 00000000..4aed6b7c Binary files /dev/null and b/_images/bare-metal-install-server-26.png differ diff --git a/_images/bare-metal-install-server-27.png b/_images/bare-metal-install-server-27.png new file mode 100644 index 00000000..11ed3660 Binary files /dev/null and b/_images/bare-metal-install-server-27.png differ diff --git a/_images/bare-metal-install-server-28.png b/_images/bare-metal-install-server-28.png new file mode 100644 index 00000000..7a4b9676 Binary files /dev/null and b/_images/bare-metal-install-server-28.png differ diff --git a/_images/bare-metal-install-server-29.png b/_images/bare-metal-install-server-29.png new file mode 100644 index 00000000..6a584f88 Binary files /dev/null and b/_images/bare-metal-install-server-29.png differ diff --git a/_images/bare-metal-install-server-30.png b/_images/bare-metal-install-server-30.png new file mode 100644 index 00000000..0979ba9d Binary files /dev/null and b/_images/bare-metal-install-server-30.png differ diff --git a/_images/bare-metal-install-server-31.png b/_images/bare-metal-install-server-31.png new file mode 100644 index 00000000..fc885d5f Binary files /dev/null and b/_images/bare-metal-install-server-31.png differ diff --git a/_images/bare-metal-install-server-32.png b/_images/bare-metal-install-server-32.png new file mode 100644 index 00000000..36073c23 Binary files /dev/null and b/_images/bare-metal-install-server-32.png differ diff --git a/_images/bare-metal-install-server-33.png b/_images/bare-metal-install-server-33.png new file mode 100644 index 00000000..7a046191 Binary files /dev/null and b/_images/bare-metal-install-server-33.png differ diff --git a/_images/bare-metal-install-server-34.png b/_images/bare-metal-install-server-34.png new file mode 100644 index 00000000..863561c3 Binary files /dev/null and b/_images/bare-metal-install-server-34.png differ diff --git a/_images/clear-lifecycle.png b/_images/clear-lifecycle.png new file mode 100644 index 00000000..a45449ea Binary files /dev/null and b/_images/clear-lifecycle.png differ diff --git a/_images/debug-diagram.png b/_images/debug-diagram.png new file mode 100644 index 00000000..d85a4045 Binary files /dev/null and b/_images/debug-diagram.png differ diff --git a/_images/download-verify-decompress-windows-fig-1.png b/_images/download-verify-decompress-windows-fig-1.png new file mode 100644 index 00000000..0b0ee7f6 Binary files /dev/null and b/_images/download-verify-decompress-windows-fig-1.png differ diff --git a/_images/dptf_bios.png b/_images/dptf_bios.png new file mode 100644 index 00000000..d99080d5 Binary files /dev/null and b/_images/dptf_bios.png differ diff --git a/_images/dual-boot-linux-01.png b/_images/dual-boot-linux-01.png new file mode 100644 index 00000000..a4a0d2ba Binary files /dev/null and b/_images/dual-boot-linux-01.png differ diff --git a/_images/dual-boot-linux-02.png b/_images/dual-boot-linux-02.png new file mode 100644 index 00000000..a4a6cc5a Binary files /dev/null and b/_images/dual-boot-linux-02.png differ diff --git a/_images/dual-boot-linux-03.png b/_images/dual-boot-linux-03.png new file mode 100644 index 00000000..7120b7d4 Binary files /dev/null and b/_images/dual-boot-linux-03.png differ diff --git a/_images/dual-boot-linux-04.png b/_images/dual-boot-linux-04.png new file mode 100644 index 00000000..6e297d2e Binary files /dev/null and b/_images/dual-boot-linux-04.png differ diff --git a/_images/dual-boot-linux-05.png b/_images/dual-boot-linux-05.png new file mode 100644 index 00000000..262306e1 Binary files /dev/null and b/_images/dual-boot-linux-05.png differ diff --git a/_images/dual-boot-linux-06.png b/_images/dual-boot-linux-06.png new file mode 100644 index 00000000..41e9fbe9 Binary files /dev/null and b/_images/dual-boot-linux-06.png differ diff --git a/_images/dual-boot-linux-07.png b/_images/dual-boot-linux-07.png new file mode 100644 index 00000000..6e541be8 Binary files /dev/null and b/_images/dual-boot-linux-07.png differ diff --git a/_images/dual-boot-linux-08.png b/_images/dual-boot-linux-08.png new file mode 100644 index 00000000..776feacf Binary files /dev/null and b/_images/dual-boot-linux-08.png differ diff --git a/_images/dual-boot-linux-09.png b/_images/dual-boot-linux-09.png new file mode 100644 index 00000000..f3e0bef3 Binary files /dev/null and b/_images/dual-boot-linux-09.png differ diff --git a/_images/dual-boot-linux-10.png b/_images/dual-boot-linux-10.png new file mode 100644 index 00000000..7a7e142c Binary files /dev/null and b/_images/dual-boot-linux-10.png differ diff --git a/_images/dual-boot-linux-11.png b/_images/dual-boot-linux-11.png new file mode 100644 index 00000000..040d0fe5 Binary files /dev/null and b/_images/dual-boot-linux-11.png differ diff --git a/_images/dual-boot-linux-12.png b/_images/dual-boot-linux-12.png new file mode 100644 index 00000000..2a26e29d Binary files /dev/null and b/_images/dual-boot-linux-12.png differ diff --git a/_images/dual-boot-win-01.png b/_images/dual-boot-win-01.png new file mode 100644 index 00000000..a1a99feb Binary files /dev/null and b/_images/dual-boot-win-01.png differ diff --git a/_images/dual-boot-win-02.png b/_images/dual-boot-win-02.png new file mode 100644 index 00000000..6110b47e Binary files /dev/null and b/_images/dual-boot-win-02.png differ diff --git a/_images/dual-boot-win-03.png b/_images/dual-boot-win-03.png new file mode 100644 index 00000000..23e9026c Binary files /dev/null and b/_images/dual-boot-win-03.png differ diff --git a/_images/dual-boot-win-04.png b/_images/dual-boot-win-04.png new file mode 100644 index 00000000..9eff8198 Binary files /dev/null and b/_images/dual-boot-win-04.png differ diff --git a/_images/dual-boot-win-06.png b/_images/dual-boot-win-06.png new file mode 100644 index 00000000..6d7e0a95 Binary files /dev/null and b/_images/dual-boot-win-06.png differ diff --git a/_images/dual-boot-win-07.png b/_images/dual-boot-win-07.png new file mode 100644 index 00000000..b0fba462 Binary files /dev/null and b/_images/dual-boot-win-07.png differ diff --git a/_images/dual-boot-win-08.png b/_images/dual-boot-win-08.png new file mode 100644 index 00000000..25c4fe75 Binary files /dev/null and b/_images/dual-boot-win-08.png differ diff --git a/_images/flatpak-01.png b/_images/flatpak-01.png new file mode 100644 index 00000000..8cc7b7cd Binary files /dev/null and b/_images/flatpak-01.png differ diff --git a/_images/flatpak-02.png b/_images/flatpak-02.png new file mode 100644 index 00000000..55f0f6f3 Binary files /dev/null and b/_images/flatpak-02.png differ diff --git a/_images/flatpak-03.png b/_images/flatpak-03.png new file mode 100644 index 00000000..ec4950c5 Binary files /dev/null and b/_images/flatpak-03.png differ diff --git a/_images/flatpak-04.png b/_images/flatpak-04.png new file mode 100644 index 00000000..af9137fa Binary files /dev/null and b/_images/flatpak-04.png differ diff --git a/_images/format-bump.png b/_images/format-bump.png new file mode 100644 index 00000000..d95e642d Binary files /dev/null and b/_images/format-bump.png differ diff --git a/_images/hpc-01.png b/_images/hpc-01.png new file mode 100644 index 00000000..91d94556 Binary files /dev/null and b/_images/hpc-01.png differ diff --git a/_images/hyper-v-01.png b/_images/hyper-v-01.png new file mode 100644 index 00000000..b1841260 Binary files /dev/null and b/_images/hyper-v-01.png differ diff --git a/_images/hyper-v-02.png b/_images/hyper-v-02.png new file mode 100644 index 00000000..6d6e0225 Binary files /dev/null and b/_images/hyper-v-02.png differ diff --git a/_images/hyper-v-03.png b/_images/hyper-v-03.png new file mode 100644 index 00000000..aa1cad9e Binary files /dev/null and b/_images/hyper-v-03.png differ diff --git a/_images/import-clr-aws-01.png b/_images/import-clr-aws-01.png new file mode 100644 index 00000000..45c8ac63 Binary files /dev/null and b/_images/import-clr-aws-01.png differ diff --git a/_images/import-clr-aws-02.png b/_images/import-clr-aws-02.png new file mode 100644 index 00000000..0f4d1c46 Binary files /dev/null and b/_images/import-clr-aws-02.png differ diff --git a/_images/import-clr-aws-03.png b/_images/import-clr-aws-03.png new file mode 100644 index 00000000..88104eb2 Binary files /dev/null and b/_images/import-clr-aws-03.png differ diff --git a/_images/import-clr-aws-04.png b/_images/import-clr-aws-04.png new file mode 100644 index 00000000..236cac4e Binary files /dev/null and b/_images/import-clr-aws-04.png differ diff --git a/_images/import-clr-aws-05.png b/_images/import-clr-aws-05.png new file mode 100644 index 00000000..73e3fa46 Binary files /dev/null and b/_images/import-clr-aws-05.png differ diff --git a/_images/import-clr-aws-06.png b/_images/import-clr-aws-06.png new file mode 100644 index 00000000..b92cf8d6 Binary files /dev/null and b/_images/import-clr-aws-06.png differ diff --git a/_images/import-clr-aws-07.png b/_images/import-clr-aws-07.png new file mode 100644 index 00000000..0460dd61 Binary files /dev/null and b/_images/import-clr-aws-07.png differ diff --git a/_images/import-clr-aws-08.png b/_images/import-clr-aws-08.png new file mode 100644 index 00000000..86312beb Binary files /dev/null and b/_images/import-clr-aws-08.png differ diff --git a/_images/import-clr-aws-09.png b/_images/import-clr-aws-09.png new file mode 100644 index 00000000..8b918243 Binary files /dev/null and b/_images/import-clr-aws-09.png differ diff --git a/_images/import-clr-aws-10.png b/_images/import-clr-aws-10.png new file mode 100644 index 00000000..7a8396e7 Binary files /dev/null and b/_images/import-clr-aws-10.png differ diff --git a/_images/import-clr-aws-11.png b/_images/import-clr-aws-11.png new file mode 100644 index 00000000..e51fb39b Binary files /dev/null and b/_images/import-clr-aws-11.png differ diff --git a/_images/import-clr-aws-12.png b/_images/import-clr-aws-12.png new file mode 100644 index 00000000..abed9621 Binary files /dev/null and b/_images/import-clr-aws-12.png differ diff --git a/_images/import-clr-aws-13.png b/_images/import-clr-aws-13.png new file mode 100644 index 00000000..e84cc103 Binary files /dev/null and b/_images/import-clr-aws-13.png differ diff --git a/_images/import-clr-aws-14.png b/_images/import-clr-aws-14.png new file mode 100644 index 00000000..ea759c73 Binary files /dev/null and b/_images/import-clr-aws-14.png differ diff --git a/_images/import-clr-aws-15.png b/_images/import-clr-aws-15.png new file mode 100644 index 00000000..6e91bb85 Binary files /dev/null and b/_images/import-clr-aws-15.png differ diff --git a/_images/import-clr-aws-16.png b/_images/import-clr-aws-16.png new file mode 100644 index 00000000..451baa2e Binary files /dev/null and b/_images/import-clr-aws-16.png differ diff --git a/_images/import-clr-aws-17.png b/_images/import-clr-aws-17.png new file mode 100644 index 00000000..3eb22a5a Binary files /dev/null and b/_images/import-clr-aws-17.png differ diff --git a/_images/import-clr-aws-18.png b/_images/import-clr-aws-18.png new file mode 100644 index 00000000..a8ddabf1 Binary files /dev/null and b/_images/import-clr-aws-18.png differ diff --git a/_images/import-clr-aws-19.png b/_images/import-clr-aws-19.png new file mode 100644 index 00000000..76c1427b Binary files /dev/null and b/_images/import-clr-aws-19.png differ diff --git a/_images/import-clr-aws-20.png b/_images/import-clr-aws-20.png new file mode 100644 index 00000000..9ff3e5ba Binary files /dev/null and b/_images/import-clr-aws-20.png differ diff --git a/_images/import-clr-aws-21.png b/_images/import-clr-aws-21.png new file mode 100644 index 00000000..6fd9f68a Binary files /dev/null and b/_images/import-clr-aws-21.png differ diff --git a/_images/import-clr-aws-22.png b/_images/import-clr-aws-22.png new file mode 100644 index 00000000..e9464e6f Binary files /dev/null and b/_images/import-clr-aws-22.png differ diff --git a/_images/import-clr-aws-23.png b/_images/import-clr-aws-23.png new file mode 100644 index 00000000..eaaac37b Binary files /dev/null and b/_images/import-clr-aws-23.png differ diff --git a/_images/import-clr-aws-24.png b/_images/import-clr-aws-24.png new file mode 100644 index 00000000..a9831dd5 Binary files /dev/null and b/_images/import-clr-aws-24.png differ diff --git a/_images/import-clr-aws-25.png b/_images/import-clr-aws-25.png new file mode 100644 index 00000000..2ab4a5a4 Binary files /dev/null and b/_images/import-clr-aws-25.png differ diff --git a/_images/import-clr-aws-26.png b/_images/import-clr-aws-26.png new file mode 100644 index 00000000..464a20bf Binary files /dev/null and b/_images/import-clr-aws-26.png differ diff --git a/_images/import-clr-aws-27.png b/_images/import-clr-aws-27.png new file mode 100644 index 00000000..e6230e39 Binary files /dev/null and b/_images/import-clr-aws-27.png differ diff --git a/_images/import-clr-aws-28.png b/_images/import-clr-aws-28.png new file mode 100644 index 00000000..e23e9b94 Binary files /dev/null and b/_images/import-clr-aws-28.png differ diff --git a/_images/ipxe-install-1.png b/_images/ipxe-install-1.png new file mode 100644 index 00000000..52bab4b1 Binary files /dev/null and b/_images/ipxe-install-1.png differ diff --git a/_images/ipxe-install-2.png b/_images/ipxe-install-2.png new file mode 100644 index 00000000..cfa3f517 Binary files /dev/null and b/_images/ipxe-install-2.png differ diff --git a/_images/machine-learning-1.png b/_images/machine-learning-1.png new file mode 100644 index 00000000..caddc867 Binary files /dev/null and b/_images/machine-learning-1.png differ diff --git a/_images/machine-learning-2.png b/_images/machine-learning-2.png new file mode 100644 index 00000000..70ee944c Binary files /dev/null and b/_images/machine-learning-2.png differ diff --git a/_images/machine-learning-3.png b/_images/machine-learning-3.png new file mode 100644 index 00000000..b54503a5 Binary files /dev/null and b/_images/machine-learning-3.png differ diff --git a/_images/machine-learning-4.png b/_images/machine-learning-4.png new file mode 100644 index 00000000..7cfdbe5d Binary files /dev/null and b/_images/machine-learning-4.png differ diff --git a/_images/machine-learning-5.png b/_images/machine-learning-5.png new file mode 100644 index 00000000..ae7e7d03 Binary files /dev/null and b/_images/machine-learning-5.png differ diff --git a/_images/machine-learning-6.png b/_images/machine-learning-6.png new file mode 100644 index 00000000..b074d787 Binary files /dev/null and b/_images/machine-learning-6.png differ diff --git a/_images/machine-learning-7.png b/_images/machine-learning-7.png new file mode 100644 index 00000000..0c6715fa Binary files /dev/null and b/_images/machine-learning-7.png differ diff --git a/_images/machine-learning-8.png b/_images/machine-learning-8.png new file mode 100644 index 00000000..add49999 Binary files /dev/null and b/_images/machine-learning-8.png differ diff --git a/_images/mirror-upstream-server-01.png b/_images/mirror-upstream-server-01.png new file mode 100644 index 00000000..30511b68 Binary files /dev/null and b/_images/mirror-upstream-server-01.png differ diff --git a/_images/mirror-upstream-server-02.png b/_images/mirror-upstream-server-02.png new file mode 100644 index 00000000..2846250d Binary files /dev/null and b/_images/mirror-upstream-server-02.png differ diff --git a/_images/nmtui_1.png b/_images/nmtui_1.png new file mode 100644 index 00000000..22848744 Binary files /dev/null and b/_images/nmtui_1.png differ diff --git a/_images/nmtui_2.png b/_images/nmtui_2.png new file mode 100644 index 00000000..8d925699 Binary files /dev/null and b/_images/nmtui_2.png differ diff --git a/_images/nmtui_3.png b/_images/nmtui_3.png new file mode 100644 index 00000000..94ac3f03 Binary files /dev/null and b/_images/nmtui_3.png differ diff --git a/_images/nmtui_4.png b/_images/nmtui_4.png new file mode 100644 index 00000000..0fb8677b Binary files /dev/null and b/_images/nmtui_4.png differ diff --git a/_images/nmtui_5.png b/_images/nmtui_5.png new file mode 100644 index 00000000..f27df24a Binary files /dev/null and b/_images/nmtui_5.png differ diff --git a/_images/nvidia-gnome-crash.png b/_images/nvidia-gnome-crash.png new file mode 100644 index 00000000..0e11eb95 Binary files /dev/null and b/_images/nvidia-gnome-crash.png differ diff --git a/_images/openfaas-function-output.png b/_images/openfaas-function-output.png new file mode 100644 index 00000000..74f940f4 Binary files /dev/null and b/_images/openfaas-function-output.png differ diff --git a/_images/openfaas-invoke-function.png b/_images/openfaas-invoke-function.png new file mode 100644 index 00000000..320d27f8 Binary files /dev/null and b/_images/openfaas-invoke-function.png differ diff --git a/_images/openfaas-login.png b/_images/openfaas-login.png new file mode 100644 index 00000000..ff905d9d Binary files /dev/null and b/_images/openfaas-login.png differ diff --git a/_images/parallels-01.png b/_images/parallels-01.png new file mode 100644 index 00000000..9ba5dcc1 Binary files /dev/null and b/_images/parallels-01.png differ diff --git a/_images/parallels-02.png b/_images/parallels-02.png new file mode 100644 index 00000000..7764b351 Binary files /dev/null and b/_images/parallels-02.png differ diff --git a/_images/parallels-03.png b/_images/parallels-03.png new file mode 100644 index 00000000..67bb0d7f Binary files /dev/null and b/_images/parallels-03.png differ diff --git a/_images/parallels-04.png b/_images/parallels-04.png new file mode 100644 index 00000000..672797d6 Binary files /dev/null and b/_images/parallels-04.png differ diff --git a/_images/parallels-05.png b/_images/parallels-05.png new file mode 100644 index 00000000..57bf13e6 Binary files /dev/null and b/_images/parallels-05.png differ diff --git a/_images/parallels-06.png b/_images/parallels-06.png new file mode 100644 index 00000000..32b7bf38 Binary files /dev/null and b/_images/parallels-06.png differ diff --git a/_images/parallels-07.png b/_images/parallels-07.png new file mode 100644 index 00000000..e110b66a Binary files /dev/null and b/_images/parallels-07.png differ diff --git a/_images/parallels-08.png b/_images/parallels-08.png new file mode 100644 index 00000000..668c72dd Binary files /dev/null and b/_images/parallels-08.png differ diff --git a/_images/parallels-09.png b/_images/parallels-09.png new file mode 100644 index 00000000..8a1a2f54 Binary files /dev/null and b/_images/parallels-09.png differ diff --git a/_images/pktgen_lw3fd.png b/_images/pktgen_lw3fd.png new file mode 100644 index 00000000..d17318b1 Binary files /dev/null and b/_images/pktgen_lw3fd.png differ diff --git a/_images/proxmox-01.png b/_images/proxmox-01.png new file mode 100644 index 00000000..0dd781b5 Binary files /dev/null and b/_images/proxmox-01.png differ diff --git a/_images/proxmox-02.png b/_images/proxmox-02.png new file mode 100644 index 00000000..feb4cb42 Binary files /dev/null and b/_images/proxmox-02.png differ diff --git a/_images/proxmox-03.png b/_images/proxmox-03.png new file mode 100644 index 00000000..829de890 Binary files /dev/null and b/_images/proxmox-03.png differ diff --git a/_images/proxmox-04.png b/_images/proxmox-04.png new file mode 100644 index 00000000..28be650f Binary files /dev/null and b/_images/proxmox-04.png differ diff --git a/_images/proxmox-05.png b/_images/proxmox-05.png new file mode 100644 index 00000000..fee1a2b6 Binary files /dev/null and b/_images/proxmox-05.png differ diff --git a/_images/proxmox-06.png b/_images/proxmox-06.png new file mode 100644 index 00000000..f64ae361 Binary files /dev/null and b/_images/proxmox-06.png differ diff --git a/_images/proxmox-07.png b/_images/proxmox-07.png new file mode 100644 index 00000000..43bb7fa6 Binary files /dev/null and b/_images/proxmox-07.png differ diff --git a/_images/proxmox-08.png b/_images/proxmox-08.png new file mode 100644 index 00000000..cce7f0ea Binary files /dev/null and b/_images/proxmox-08.png differ diff --git a/_images/proxmox-09.png b/_images/proxmox-09.png new file mode 100644 index 00000000..7dc1794c Binary files /dev/null and b/_images/proxmox-09.png differ diff --git a/_images/proxmox-10.png b/_images/proxmox-10.png new file mode 100644 index 00000000..960661b8 Binary files /dev/null and b/_images/proxmox-10.png differ diff --git a/_images/proxmox-11.png b/_images/proxmox-11.png new file mode 100644 index 00000000..501f4bfc Binary files /dev/null and b/_images/proxmox-11.png differ diff --git a/_images/proxmox-12.png b/_images/proxmox-12.png new file mode 100644 index 00000000..43675134 Binary files /dev/null and b/_images/proxmox-12.png differ diff --git a/_images/proxmox-13.png b/_images/proxmox-13.png new file mode 100644 index 00000000..9656b6cc Binary files /dev/null and b/_images/proxmox-13.png differ diff --git a/_images/pyshical_net.png b/_images/pyshical_net.png new file mode 100644 index 00000000..182ba6ce Binary files /dev/null and b/_images/pyshical_net.png differ diff --git a/_images/run-cell-button.png b/_images/run-cell-button.png new file mode 100644 index 00000000..b61de4ce Binary files /dev/null and b/_images/run-cell-button.png differ diff --git a/_images/smb-desktop-1.png b/_images/smb-desktop-1.png new file mode 100644 index 00000000..866a45ef Binary files /dev/null and b/_images/smb-desktop-1.png differ diff --git a/_images/smb-desktop-2.png b/_images/smb-desktop-2.png new file mode 100644 index 00000000..9ed4a61d Binary files /dev/null and b/_images/smb-desktop-2.png differ diff --git a/_images/smb-desktop-3.png b/_images/smb-desktop-3.png new file mode 100644 index 00000000..7688e484 Binary files /dev/null and b/_images/smb-desktop-3.png differ diff --git a/_images/smb-server-01.png b/_images/smb-server-01.png new file mode 100644 index 00000000..596e153d Binary files /dev/null and b/_images/smb-server-01.png differ diff --git a/_images/smb-server-02.png b/_images/smb-server-02.png new file mode 100644 index 00000000..7ffbe04f Binary files /dev/null and b/_images/smb-server-02.png differ diff --git a/_images/stateless-1.png b/_images/stateless-1.png new file mode 100644 index 00000000..550f37ce Binary files /dev/null and b/_images/stateless-1.png differ diff --git a/_images/stateless-2.png b/_images/stateless-2.png new file mode 100644 index 00000000..94e8af22 Binary files /dev/null and b/_images/stateless-2.png differ diff --git a/_images/telemetry-backend-1.png b/_images/telemetry-backend-1.png new file mode 100644 index 00000000..9cd8df75 Binary files /dev/null and b/_images/telemetry-backend-1.png differ diff --git a/_images/telemetry-e2e.png b/_images/telemetry-e2e.png new file mode 100644 index 00000000..75868291 Binary files /dev/null and b/_images/telemetry-e2e.png differ diff --git a/_images/tutorial-ratings-01.svg b/_images/tutorial-ratings-01.svg new file mode 100644 index 00000000..9458529a --- /dev/null +++ b/_images/tutorial-ratings-01.svg @@ -0,0 +1,3 @@ + + +

Are there ~ 8 or  > “complex” steps? 

“Complex steps”:

  • require more than one action 

  • require external reading/review

  • include explanation or context

  • give alternative(s) 

    (Must have one or more.)

Are there ~ 8 or  > “complex” steps?...

Experienced Linux user?

Experienced Linux user?

Advanced Linux user?

 Advanced Linux user?

Are there ~ 10 or > code/console examples, including those in figures?


Are there ~ 10 or > code/console examples, includi...

Are there ~ 10 or > code/console examples, including those in figures?


Are there ~ 10 or > code/console examples, includin...
NO
NO

Will impact of errors be 

inconsequential? 

Will impact of errors be...

Will impact of errors

cause system failure
(and difficult to recover) ?

Will impact of errors...

Will impact of errors

cause inconvenience 

(but system still works) ?

Will impact of errors...

Does troubleshooting require background knowledge?

Does troubleshooting require backgroun...
EASY
EASY
NO
NO
YES
YES
MODERATE
MODERATE
DIFFICULT
DIFFICULT
YES
YES
YES
YES
NO
NO
NO
NO
YES
YES
NO
NO
NO
NO
YES
YES
NO
NO
NO
NO
YES
YES
YES
YES
YES
YES
YES
YESViewer does not support full SVG 1.1 \ No newline at end of file diff --git a/_images/virt-manager-01.png b/_images/virt-manager-01.png new file mode 100644 index 00000000..bbc5335b Binary files /dev/null and b/_images/virt-manager-01.png differ diff --git a/_images/virt-manager-02.png b/_images/virt-manager-02.png new file mode 100644 index 00000000..75a5fe27 Binary files /dev/null and b/_images/virt-manager-02.png differ diff --git a/_images/virt-manager-03.png b/_images/virt-manager-03.png new file mode 100644 index 00000000..8993017b Binary files /dev/null and b/_images/virt-manager-03.png differ diff --git a/_images/virt-manager-04.png b/_images/virt-manager-04.png new file mode 100644 index 00000000..441703d5 Binary files /dev/null and b/_images/virt-manager-04.png differ diff --git a/_images/virt-manager-05.png b/_images/virt-manager-05.png new file mode 100644 index 00000000..89cf534e Binary files /dev/null and b/_images/virt-manager-05.png differ diff --git a/_images/virt-manager-06.png b/_images/virt-manager-06.png new file mode 100644 index 00000000..962f9efc Binary files /dev/null and b/_images/virt-manager-06.png differ diff --git a/_images/virt-manager-07.png b/_images/virt-manager-07.png new file mode 100644 index 00000000..80aba6d6 Binary files /dev/null and b/_images/virt-manager-07.png differ diff --git a/_images/virt-manager-08.png b/_images/virt-manager-08.png new file mode 100644 index 00000000..3a3bb04f Binary files /dev/null and b/_images/virt-manager-08.png differ diff --git a/_images/virt-manager-09.png b/_images/virt-manager-09.png new file mode 100644 index 00000000..384601b3 Binary files /dev/null and b/_images/virt-manager-09.png differ diff --git a/_images/virt-manager-10.png b/_images/virt-manager-10.png new file mode 100644 index 00000000..d83b4ac4 Binary files /dev/null and b/_images/virt-manager-10.png differ diff --git a/_images/virt-manager-11.png b/_images/virt-manager-11.png new file mode 100644 index 00000000..62cb96ba Binary files /dev/null and b/_images/virt-manager-11.png differ diff --git a/_images/virtualbox-cl-installer-01.png b/_images/virtualbox-cl-installer-01.png new file mode 100644 index 00000000..7722a886 Binary files /dev/null and b/_images/virtualbox-cl-installer-01.png differ diff --git a/_images/virtualbox-cl-installer-02.png b/_images/virtualbox-cl-installer-02.png new file mode 100644 index 00000000..365f8545 Binary files /dev/null and b/_images/virtualbox-cl-installer-02.png differ diff --git a/_images/virtualbox-cl-installer-03.png b/_images/virtualbox-cl-installer-03.png new file mode 100644 index 00000000..496f270a Binary files /dev/null and b/_images/virtualbox-cl-installer-03.png differ diff --git a/_images/virtualbox-cl-installer-04.png b/_images/virtualbox-cl-installer-04.png new file mode 100644 index 00000000..31662623 Binary files /dev/null and b/_images/virtualbox-cl-installer-04.png differ diff --git a/_images/virtualbox-cl-installer-05.png b/_images/virtualbox-cl-installer-05.png new file mode 100644 index 00000000..c250776d Binary files /dev/null and b/_images/virtualbox-cl-installer-05.png differ diff --git a/_images/virtualbox-cl-installer-06.png b/_images/virtualbox-cl-installer-06.png new file mode 100644 index 00000000..9926af81 Binary files /dev/null and b/_images/virtualbox-cl-installer-06.png differ diff --git a/_images/virtualbox-cl-installer-07.png b/_images/virtualbox-cl-installer-07.png new file mode 100644 index 00000000..8f2c2b34 Binary files /dev/null and b/_images/virtualbox-cl-installer-07.png differ diff --git a/_images/virtualbox-cl-installer-08.png b/_images/virtualbox-cl-installer-08.png new file mode 100644 index 00000000..7b23c454 Binary files /dev/null and b/_images/virtualbox-cl-installer-08.png differ diff --git a/_images/virtualbox-cl-installer-09.png b/_images/virtualbox-cl-installer-09.png new file mode 100644 index 00000000..f6feabfa Binary files /dev/null and b/_images/virtualbox-cl-installer-09.png differ diff --git a/_images/virtualbox-cl-installer-10.png b/_images/virtualbox-cl-installer-10.png new file mode 100644 index 00000000..0ffd96e8 Binary files /dev/null and b/_images/virtualbox-cl-installer-10.png differ diff --git a/_images/virtualbox-cl-installer-11.png b/_images/virtualbox-cl-installer-11.png new file mode 100644 index 00000000..b0ca6d35 Binary files /dev/null and b/_images/virtualbox-cl-installer-11.png differ diff --git a/_images/virtualbox-cl-installer-12.png b/_images/virtualbox-cl-installer-12.png new file mode 100644 index 00000000..ba6344de Binary files /dev/null and b/_images/virtualbox-cl-installer-12.png differ diff --git a/_images/vmw-player-01.png b/_images/vmw-player-01.png new file mode 100644 index 00000000..ece95983 Binary files /dev/null and b/_images/vmw-player-01.png differ diff --git a/_images/vmw-player-02.png b/_images/vmw-player-02.png new file mode 100644 index 00000000..3498a786 Binary files /dev/null and b/_images/vmw-player-02.png differ diff --git a/_images/vmw-player-03.png b/_images/vmw-player-03.png new file mode 100644 index 00000000..6d5cdcb4 Binary files /dev/null and b/_images/vmw-player-03.png differ diff --git a/_images/vmw-player-04.png b/_images/vmw-player-04.png new file mode 100644 index 00000000..719ee1b3 Binary files /dev/null and b/_images/vmw-player-04.png differ diff --git a/_images/vmw-player-05.png b/_images/vmw-player-05.png new file mode 100644 index 00000000..f9a867e2 Binary files /dev/null and b/_images/vmw-player-05.png differ diff --git a/_images/vmw-player-06.png b/_images/vmw-player-06.png new file mode 100644 index 00000000..b40b499c Binary files /dev/null and b/_images/vmw-player-06.png differ diff --git a/_images/vmw-player-07.png b/_images/vmw-player-07.png new file mode 100644 index 00000000..0c78ae36 Binary files /dev/null and b/_images/vmw-player-07.png differ diff --git a/_images/vmw-player-08.png b/_images/vmw-player-08.png new file mode 100644 index 00000000..f6842c11 Binary files /dev/null and b/_images/vmw-player-08.png differ diff --git a/_images/vmw-player-09.png b/_images/vmw-player-09.png new file mode 100644 index 00000000..9ef2d938 Binary files /dev/null and b/_images/vmw-player-09.png differ diff --git a/_images/vmw-player-10.png b/_images/vmw-player-10.png new file mode 100644 index 00000000..3b7f6b65 Binary files /dev/null and b/_images/vmw-player-10.png differ diff --git a/_images/vmw-player-11.png b/_images/vmw-player-11.png new file mode 100644 index 00000000..e4281ace Binary files /dev/null and b/_images/vmw-player-11.png differ diff --git a/_images/vmw-player-12.png b/_images/vmw-player-12.png new file mode 100644 index 00000000..08744dc5 Binary files /dev/null and b/_images/vmw-player-12.png differ diff --git a/_images/vmw-player-13.png b/_images/vmw-player-13.png new file mode 100644 index 00000000..f09ccedd Binary files /dev/null and b/_images/vmw-player-13.png differ diff --git a/_images/vmw-player-14.png b/_images/vmw-player-14.png new file mode 100644 index 00000000..6daf0f80 Binary files /dev/null and b/_images/vmw-player-14.png differ diff --git a/_images/vmw-player-15.png b/_images/vmw-player-15.png new file mode 100644 index 00000000..339e70ad Binary files /dev/null and b/_images/vmw-player-15.png differ diff --git a/_images/vmw-player-16.png b/_images/vmw-player-16.png new file mode 100644 index 00000000..6b983f01 Binary files /dev/null and b/_images/vmw-player-16.png differ diff --git a/_images/vmware-esxi-install-cl-1.png b/_images/vmware-esxi-install-cl-1.png new file mode 100644 index 00000000..828ca838 Binary files /dev/null and b/_images/vmware-esxi-install-cl-1.png differ diff --git a/_images/vmware-esxi-install-cl-10.png b/_images/vmware-esxi-install-cl-10.png new file mode 100644 index 00000000..85688105 Binary files /dev/null and b/_images/vmware-esxi-install-cl-10.png differ diff --git a/_images/vmware-esxi-install-cl-11.png b/_images/vmware-esxi-install-cl-11.png new file mode 100644 index 00000000..bb134ba2 Binary files /dev/null and b/_images/vmware-esxi-install-cl-11.png differ diff --git a/_images/vmware-esxi-install-cl-12.png b/_images/vmware-esxi-install-cl-12.png new file mode 100644 index 00000000..ba26d552 Binary files /dev/null and b/_images/vmware-esxi-install-cl-12.png differ diff --git a/_images/vmware-esxi-install-cl-13.png b/_images/vmware-esxi-install-cl-13.png new file mode 100644 index 00000000..3f11505d Binary files /dev/null and b/_images/vmware-esxi-install-cl-13.png differ diff --git a/_images/vmware-esxi-install-cl-14.png b/_images/vmware-esxi-install-cl-14.png new file mode 100644 index 00000000..59e07de1 Binary files /dev/null and b/_images/vmware-esxi-install-cl-14.png differ diff --git a/_images/vmware-esxi-install-cl-15.png b/_images/vmware-esxi-install-cl-15.png new file mode 100644 index 00000000..9d654081 Binary files /dev/null and b/_images/vmware-esxi-install-cl-15.png differ diff --git a/_images/vmware-esxi-install-cl-16.png b/_images/vmware-esxi-install-cl-16.png new file mode 100644 index 00000000..bb134ba2 Binary files /dev/null and b/_images/vmware-esxi-install-cl-16.png differ diff --git a/_images/vmware-esxi-install-cl-2.png b/_images/vmware-esxi-install-cl-2.png new file mode 100644 index 00000000..a3554e92 Binary files /dev/null and b/_images/vmware-esxi-install-cl-2.png differ diff --git a/_images/vmware-esxi-install-cl-3.png b/_images/vmware-esxi-install-cl-3.png new file mode 100644 index 00000000..aec0c8d0 Binary files /dev/null and b/_images/vmware-esxi-install-cl-3.png differ diff --git a/_images/vmware-esxi-install-cl-4.png b/_images/vmware-esxi-install-cl-4.png new file mode 100644 index 00000000..30cabb0e Binary files /dev/null and b/_images/vmware-esxi-install-cl-4.png differ diff --git a/_images/vmware-esxi-install-cl-5.png b/_images/vmware-esxi-install-cl-5.png new file mode 100644 index 00000000..f1e7cee1 Binary files /dev/null and b/_images/vmware-esxi-install-cl-5.png differ diff --git a/_images/vmware-esxi-install-cl-6.png b/_images/vmware-esxi-install-cl-6.png new file mode 100644 index 00000000..b0a58487 Binary files /dev/null and b/_images/vmware-esxi-install-cl-6.png differ diff --git a/_images/vmware-esxi-install-cl-7.png b/_images/vmware-esxi-install-cl-7.png new file mode 100644 index 00000000..4971a78e Binary files /dev/null and b/_images/vmware-esxi-install-cl-7.png differ diff --git a/_images/vmware-esxi-install-cl-8.png b/_images/vmware-esxi-install-cl-8.png new file mode 100644 index 00000000..b188da8a Binary files /dev/null and b/_images/vmware-esxi-install-cl-8.png differ diff --git a/_images/vmware-esxi-install-cl-9.png b/_images/vmware-esxi-install-cl-9.png new file mode 100644 index 00000000..24e72696 Binary files /dev/null and b/_images/vmware-esxi-install-cl-9.png differ diff --git a/_images/vnc-1.png b/_images/vnc-1.png new file mode 100644 index 00000000..bb496aaf Binary files /dev/null and b/_images/vnc-1.png differ diff --git a/_images/vnc-10.png b/_images/vnc-10.png new file mode 100644 index 00000000..be6751fc Binary files /dev/null and b/_images/vnc-10.png differ diff --git a/_images/vnc-2.png b/_images/vnc-2.png new file mode 100644 index 00000000..aa7561f7 Binary files /dev/null and b/_images/vnc-2.png differ diff --git a/_images/vnc-3.png b/_images/vnc-3.png new file mode 100644 index 00000000..29365e08 Binary files /dev/null and b/_images/vnc-3.png differ diff --git a/_images/vnc-4.png b/_images/vnc-4.png new file mode 100644 index 00000000..b3608289 Binary files /dev/null and b/_images/vnc-4.png differ diff --git a/_images/vnc-6.png b/_images/vnc-6.png new file mode 100644 index 00000000..5a808e34 Binary files /dev/null and b/_images/vnc-6.png differ diff --git a/_images/vnc-7.png b/_images/vnc-7.png new file mode 100644 index 00000000..a20e6c97 Binary files /dev/null and b/_images/vnc-7.png differ diff --git a/_images/vnc-8.png b/_images/vnc-8.png new file mode 100644 index 00000000..fa702cb4 Binary files /dev/null and b/_images/vnc-8.png differ diff --git a/_images/vnc-9.png b/_images/vnc-9.png new file mode 100644 index 00000000..a8cafba4 Binary files /dev/null and b/_images/vnc-9.png differ diff --git a/_images/web-server-install-1.png b/_images/web-server-install-1.png new file mode 100644 index 00000000..d1dd0e22 Binary files /dev/null and b/_images/web-server-install-1.png differ diff --git a/_images/web-server-install-2.png b/_images/web-server-install-2.png new file mode 100644 index 00000000..b0fd4d3f Binary files /dev/null and b/_images/web-server-install-2.png differ diff --git a/_images/web-server-install-3.png b/_images/web-server-install-3.png new file mode 100644 index 00000000..5fbdad01 Binary files /dev/null and b/_images/web-server-install-3.png differ diff --git a/_images/web-server-install-4.png b/_images/web-server-install-4.png new file mode 100644 index 00000000..2c553ce9 Binary files /dev/null and b/_images/web-server-install-4.png differ diff --git a/_images/web-server-install-5.png b/_images/web-server-install-5.png new file mode 100644 index 00000000..292dfbc6 Binary files /dev/null and b/_images/web-server-install-5.png differ diff --git a/_images/web-server-install-6.png b/_images/web-server-install-6.png new file mode 100644 index 00000000..0700a210 Binary files /dev/null and b/_images/web-server-install-6.png differ diff --git a/_images/web-server-install-7.png b/_images/web-server-install-7.png new file mode 100644 index 00000000..f3f9c327 Binary files /dev/null and b/_images/web-server-install-7.png differ diff --git a/_images/web-server-install-8.png b/_images/web-server-install-8.png new file mode 100644 index 00000000..3e631ec1 Binary files /dev/null and b/_images/web-server-install-8.png differ diff --git a/_images/wifi-1.1.png b/_images/wifi-1.1.png new file mode 100644 index 00000000..44ac556a Binary files /dev/null and b/_images/wifi-1.1.png differ diff --git a/_images/wifi-2.png b/_images/wifi-2.png new file mode 100644 index 00000000..9b1fc490 Binary files /dev/null and b/_images/wifi-2.png differ diff --git a/_images/wifi-3.png b/_images/wifi-3.png new file mode 100644 index 00000000..3a8b4cc4 Binary files /dev/null and b/_images/wifi-3.png differ diff --git a/_images/wifi-4.png b/_images/wifi-4.png new file mode 100644 index 00000000..4e69f736 Binary files /dev/null and b/_images/wifi-4.png differ diff --git a/_images/wifi-5.png b/_images/wifi-5.png new file mode 100644 index 00000000..41442245 Binary files /dev/null and b/_images/wifi-5.png differ diff --git a/_images/wp-install-1.png b/_images/wp-install-1.png new file mode 100644 index 00000000..3d1239fe Binary files /dev/null and b/_images/wp-install-1.png differ diff --git a/_images/wp-install-2.png b/_images/wp-install-2.png new file mode 100644 index 00000000..6adc942a Binary files /dev/null and b/_images/wp-install-2.png differ diff --git a/_images/wp-install-3.png b/_images/wp-install-3.png new file mode 100644 index 00000000..9e713210 Binary files /dev/null and b/_images/wp-install-3.png differ diff --git a/_images/wp-install-4.png b/_images/wp-install-4.png new file mode 100644 index 00000000..124f3b8e Binary files /dev/null and b/_images/wp-install-4.png differ diff --git a/_images/wp-install-5.png b/_images/wp-install-5.png new file mode 100644 index 00000000..6651b16f Binary files /dev/null and b/_images/wp-install-5.png differ diff --git a/_images/wp-install-6.png b/_images/wp-install-6.png new file mode 100644 index 00000000..6118eebb Binary files /dev/null and b/_images/wp-install-6.png differ diff --git a/_images/wp-install-7.png b/_images/wp-install-7.png new file mode 100644 index 00000000..554a0d36 Binary files /dev/null and b/_images/wp-install-7.png differ diff --git a/_images/wp-install-8.png b/_images/wp-install-8.png new file mode 100644 index 00000000..1e477c25 Binary files /dev/null and b/_images/wp-install-8.png differ diff --git a/_sources/FAQ/index.rst.txt b/_sources/FAQ/index.rst.txt new file mode 100644 index 00000000..267050b8 --- /dev/null +++ b/_sources/FAQ/index.rst.txt @@ -0,0 +1,255 @@ +.. _faq: + +FAQ +### + +Below is a list of commonly asked questions with answers sourced from the +|CL-ATTR| team and `Clear Linux community forums`_. + +.. contents:: :local: + :depth: 2 + +General +******* + +What is |CL|? +============= + +|CL| is an open source, rolling release Linux distribution optimized for +performance and security. See `the about page `_ +for more information. + +| + +Why another Linux distribution? +=============================== + +The |CL| team felt that performance was left on the table with Linux software. +|CL| takes a holistic approach to improve performance across the stack. We +also wanted to take more modern approaches with OS updates and tooling. + +| + +Is it a derivative of another Linux distribution? +================================================= + +No. |CL| is a new Linux distribution. It is not a fork and does not have a +parent Linux distribution. + +| + +Can others copy improvements from |CL|? +======================================= + +Yes, we absolutely love open source reuse and upstreaming improvements. + +| + +How often does it update? +========================= + +The |CL| team puts out multiple releases a week, often releasing two or more +times a day. This rolling release approach allows |CL| to remain agile to +upstream changes and security patches. + +| + +Is telemetry required? +====================== + +The telemetry solution provided by |CL| is entirely optional and customizable. +It is disabled by default. If you do choose to enable telemetry, the data +helps the |CL| team proactively identify and resolve bugs. See the +:ref:`telemetry ` guide for more information. + +| + +What is the default firewall? +============================= + +|CL| packages :command:`iptables` and :command:`firewalld` as optional +bundles, however, there are no default firewall rules. All network traffic is +allowed by default. + +| + +Where are the files that I usually see under /etc like fstab? +============================================================= + +|CL| has a stateless design that maintains a separation between system files +and user files. Default values are stored under :file:`/usr/share/defaults/`. +|CL| starts with a mostly empty :file:`/etc` directory to store user-defined +configurations. See the :ref:`stateless ` page for more information. + +See `this blog post +`_ for an +example explaining how this is accomplished with :file:`/etc/fstab/` +specifically. + + +| + +Does it use the Intel Compiler (icc)? +===================================== + +No. |CL| uses open source compilers: :command:`gcc` and :command:`clang`. |CL| +does not compile any packages with :command:`icc`. + +For a more detailed explanation, see `this discussion on the community forum +`_. + +| + +Software +******** + +How is software installed and updated? +====================================== + +|CL| provides software in the form of :ref:`bundles ` and +updates software with :ref:`swupd `. + +:ref:`Flatpak\* ` is an application virtualization solution +that allows more software to be available to |CL| users by augmenting the +software |CL| packages natively with software available through Flatpak. + +Our goal is to have software packaged natively and made available through +bundles whenever possible. + +| + +Does it use RPMs or DEBs packages like other distros? +===================================================== + +No. |CL| provides software to systems in the form of :ref:`bundles-guide`. +Under the hood, |CL| developers use the RPM format as an intermediary step for +packaging and determining software dependencies at OS build time. + +Individual RPMs and DEBs can sometimes be manually extracted and installed on +a |CL| system with the right tools, but that is not the intended use case. + +| + +Why does it have a different approach to software management? +============================================================= + +The |CL| team wants software *installation* and *updates* to be as efficient +and error free as possible. |CL| packages software differently and uses a +novel updater to solve some of the classic problems with how the software +packages are on Linux. + +For a more detailed explanation, see `this discussion on our community forum +`_. + + +| + +Can I install a software package from another OS on |CL|? +========================================================= + +Software that is packaged in other formats for other Linux distributions is +not guaranteed to work on |CL| and may be impacted by |CL| updates. + +If the software you're seeking is open source, please submit a request to add +it to |CL|. Submit requests on GitHub\* here: +https://github.com/clearlinux/distribution/issues + +| + +Software availability +********************* + +What software is available on |CL|? +=================================== + +Available software can be found in the `Software Store`_, through the GNOME\* +Software application on the desktop, or by using :ref:`swupd search `. + +| + +Is Google\* Chrome\* available? +=============================== + +The Google Chrome web browser is not distributed as a bundle in |CL| due to +copyright and licensing complexities. + +A `discussion on manually installing and maintaining Google Chrome +`_ can be found on +GitHub. + +| + +Is Microsoft\* Visual Studio Code\* available? +============================================== + +Yes. Find the CLI command for installing `VS Code`_ and other Flatpak apps in +the `software store`_. Installing Flatpak apps is also covered in our +:ref:`tutorial `. + +The |CL| team is working on a natively packaged version of Visual Studio Code +for future release. + +Join a community `forum discussion about manually installing and maintaining +Visual Studio Code +`_. + + +.. _VS Code: https://clearlinux.org/software?search_api_fulltext=vscode + +| + +.. _licensing_restrict: + +Is FFmpeg available? +==================== + +`FFmpeg`_ is a multimedia software suite, which is commonly used for +various media encoding/decoding, streaming, and playback. + +|CL| does not distribute FFmpeg due to well-known licensing and legal +complexities (See https://www.ffmpeg.org/legal.html and +http://blog.pkh.me/p/13-the-ffmpeg-libav-situation.html). + + +While |CL| cannot distribute FFmpeg, solutions for manually building and +installing FFmpeg have been shared by users `on GitHub +`_ and `the community +forums +`_. + +| + +Is ZFS\* available? +=================== + +ZFS is not available with |CL| because of copyright and licensing +complexities. BTRFS is an alternative filesystem that is available in |CL| +natively. + +A community contributed tutorial has been shared on how to :ref:`manually +install ZFS `. + +| + +Can you add a driver that I need? +================================= + +If a kernel module is available as part of the Linux kernel source tree but +not enabled in the |CL| kernels, in many cases the |CL| team will enable it +upon request. Submit requests on GitHub here: +https://github.com/clearlinux/distribution/issues + +The |CL| team does not typically add out-of-tree kernel modules as a matter of +practice because of the maintenance overhead. If the driver was unable to be +merged upstream, there is a good chance we may be unable to merge it for +similar reasons. + +Kernel modules can be individually built and installed on |CL|. See the +:ref:`kernel modules ` page for more information. + +| + + +.. _`Clear Linux community forums`: https://community.clearlinux.org +.. _`Software Store`: https://clearlinux.org/software +.. _`FFmpeg`: https://ffmpeg.org/ diff --git a/_sources/about.rst.txt b/_sources/about.rst.txt new file mode 100644 index 00000000..5e1c91a3 --- /dev/null +++ b/_sources/about.rst.txt @@ -0,0 +1,311 @@ +.. _about: + +About +##### + +|CL-ATTR| does things differently. Our software architecture provides a +unique and innovative platform for Linux* developers focused on +performance and security for compute, server, and the cloud. + +.. contents:: + :local: + :depth: 1 + +What is |CL|? +************* + +|CL| is an open source, rolling-release Linux distribution, optimized for +performance and security from the cloud to the Edge. Designed from the ground up, +|CL| provides an industry blueprint on how to incorporate Intel® architecture +features for a modern, modular Linux OS. |CL| is not based on any other Linux +distro. + +What |CL| isn't? +**************** + +|CL| is not intended to be a general-purpose Linux distribution, suitable +for novice end-users. While we ship common applications, our purpose isn’t +to make an OS for routine desktop tasks and provide immunity from all +security threats in all situations. Our unique focus means that what we consider *essential* use cases, *optional* use cases, or even *unsupported* use cases, differs from other Linux distros. See our :ref:`target audience ` below. + +Is |CL| completely Open Source? +******************************* + +|CL| aims to be completely open source. Our project `source code`_ and +`packages source code`_ are available on GitHub\*. When considering projects +for inclusion, we check that they are in active development and are well +maintained. We have a very strict requirement for not accepting proprietary +packages and non-open source components. For example, many Linux distros +may not be able to include certain media codecs due to +:ref:`licensing restrictions `, but manual installation and `third party alternatives`_ are available. + +.. _target-audience: + +Who is the target audience? +*************************** + +|CL| mainly targets professionals in IT, DevOps, Cloud/Container deployments, and :abbr:`AI (Artificial Intelligence)`. + +Rather than making a standard Linux distribution, the |CL| team decided to +build a unique Linux distro. Developing a distro in house allows us to experiment and iterate faster, which means we continually optimize performance and deliver security patches, :ref:`several times per week `. Yet our experiments are only valuable if our software architecture gives you the freedom to innovate, too. To improve manageability, |CL| employs a :ref:`stateless` design, separating user and system management. + +We leverage the pool of knowledge and skills at Intel to drive improvements to |CL|. + +Intel has worked with the Linux community and other distros for many years. +Understanding what it takes to integrate features in our own Linux distro +helps us collaborate with other distro owners and submit enhancements to +upstream. We demonstrate the value of our distro by offering users the same +tools we use. For example, :ref:`mixer`, a tool unique to |CL|, allows users +to build custom derivatives and act as their own :abbr:`OSV (Operating System +Vendor)`. + +For more details on |CL| features, visit our :ref:`cl-guides` guides. + +How does |CL| address security? +******************************* + +Several :ref:`security features ` are designed to work +out-of-the-box, yet they're not intended to be intrusive. We focus on +*essential* use cases and ignore *unwanted* or *unsupported* use cases. +For example, while |CL| does not enable antivirus by default, we provide a +bundle for it (``clamav``). We leave antivirus configuration to our users. +In addition, firewalls are less important if the OS doesn’t expose services +to the outside by default. In |CL|, we enforce this strategy by disabling +network services by default - e.g. ``mariadb`` listens on a UNIX socket; +``nginx`` won’t listen at all; and other services similarly are restricted +from being accessed over the network. This strategy alone makes firewall +software much less urgent--there simply isn’t anything that a firewall could +easily block. + +What’s the thinking around Server vs. Desktop? +********************************************** + +|CL| focuses on performance for server and cloud use-cases first because +many design decisions associated with them are applicable to other +use-cases, such as IoT and the desktop client. While our initial focus was +on the command line, we realized that many people valued the ease-of-use of +a desktop environment. Whereas in the past we tried to accommodate those +interested in a desktop version, we were forced to confront clear limits as +to how we could meet this need. |CL| minimizes the customizations and patches in support of the desktop and provides a generic GNOME implementation. Other window managers or desktops are available; however, testing in |CL| is focused on GNOME. + +What makes |CL| different? +************************** + +.. _release-cadence: + +Release Cadence +=============== + +|CL| updates are based on a rolling release that can occur daily, up to a few +times per week. Each release has a unique version number that identifies +every component in the OS from kernel, to driver, to tool, to GUI +application. Most components are included in entities called :ref:`bundles`. + +Updates +======= + +By default, |CL| automatically checks for updates, ensuring the latest +performance and security fixes are installed as soon as they are available. +|CL| stays in lockstep with upstream for current security upgrades and is +designed to rapidly deliver security mitigations to customers. +:ref:`swupd-guide` is designed to manage updates and bundles. + +Ease of Use +=========== + +|CL| makes it easier to manage a number of difficult problems. + +* :ref:`autoproxy` makes it possible for |CL| tools to operate in some proxy + environments without needing to be configured. + +* :ref:`stateless` means that configuration settings are easier to manage + and remain untouched when system software is updated. + +* :ref:`swupd-guide` simplifies managing software and maintaining + compatibility. + +Custom Derivatives +================== + +The same tools used to build the |CL| are available *in* the OS. These tools can be used to create a custom distribution that continues to benefit from upstream rolling releases. + +.. figure:: /_figures/about/clear-lifecycle.png + :scale: 75% + :align: center + :alt: Creating and managing a Clear Linux* OS version (or derivative) + + Figure 1: Creating and managing a Clear Linux\* OS version (or derivative) + +Create +====== + +To create a custom distribution you need to understand how to use the +:ref:`autospec` and :ref:`mixer` tools. Additional training materials are available in the `how-to-clear`_ GitHub project to help you get started with |CL| tools. + +Deploy +====== + +We also provide training on how to :ref:`deploy-at-scale`. + +Administrate +============ + +|CL| provides a :ref:`telem-guide` solution for collecting useful information +about a deployment, as well as :ref:`debug` capabilities. + +Why create new components rather than modifying existing projects? +****************************************************************** + +One question that's often asked: “Why did you develop your own solution +instead of using ?” (e.g. `swupd post`_). We do evaluate existing +projects for inclusion in |CL|, yet there are cases where our unique +architecture and components would require too much customization to use +off-the-shelf projects. In other situations, we may feel that using a new +language to develop the component would give us a performance advantage, +ease code development and maintenance, and grow the skills of our engineers +on new and upcoming programming languages. And yes, sometimes there are +personal biases for and against some projects by the architects and +engineers. We tend to move fast, and sometimes it’s easier to live with +suboptimal choices until we have the time or incentive to re-architect them +properly. + +Which Components are used in Clear Linux? +***************************************** + +.. list-table:: + :widths: 33,33,33 + :header-rows: 1 + + * - Component + - Enabled in OS/Bundle + - Optional + + * - OS Installer + - `Clear Linux installer`_ + - + + * - Bootloader + - `systemd-boot`_ (UEFI) / `syslinux`_ (Legacy) + - + + * - Boot Manager + - `Clear Linux Boot Manager`_ + - + + * - Configuration initialization and management + - *NA* + - `micro-config-drive`_ (minimal cloud-init), Ansible + + * - Software component installer, manager, updater + - `swupd`_ + - + + * - Software bundle generator - + - `mixer`_ and `Clear Linux Distro Factory`_ + - + + * - Software package builder + - `autospec`_ + - + + * - Software debugging + - *NA* + - `clr-debug-info`_ + + * - Unified TLS Trust Store Management + - `clrtrust`_ + - + + * - System and software telemetry + - *NA* + - `Telemetrics`_ (disabled by default) + + * - File system + - `EXT4`_ (default for rootfs), `VFAT`_, `EXT2 and EXT3`_, `F2FS`_ + - + + * - Disk encryption + - *NA* + - `LUKS`_ + + * - System /Service manager + - `systemd`_ + - + + * - Display manager + - `GNOME`_ + - ``KDE``, ``Xfce``, ``lightdm``, ``sddm`` (see `Clear Linux store`_) + + * - Display services (Desktop installed) + - `X.Org`_ + - `Wayland`_ compositor + + * - Network services + - `NetworkManager`_ by default, `systemd-networkd`_ See Note below. + - + + * - SSH Port scanning blocker + - `Tallow`_ + - + + * - Firewall + - *NA* + - iptables and `firewalld`_ + + * - Antivirus + - *NA* + - `ClamAV*`_ + + * - Web browser + - `Lynx`_ or `links`_ for text environments, `Firefox*`_ for GUI + - + + * - Additional Software + - `Supplied Bundles`_ + - Flatpak, 3rd-party software bundles + +.. note:: + + The |CL| OS images targeted for cloud deployments continue to use + ``systemd-networkd`` to manage network connections. In earlier |CL|, + ``systemd-networkd`` was used to manage Ethernet interfaces and NetworkManager was used for wireless interfaces. + + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _third party alternatives: https://community.clearlinux.org/t/about-the-3rd-party-sw-category/4072 +.. _how-to-clear: https://github.com/clearlinux/how-to-clear +.. _Clear Linux store: https://clearlinux.org/software +.. _source code: https://github.com/clearlinux +.. _swupd post: https://community.clearlinux.org/t/why-does-clearlinux-use-swupd-and-not-apt-deb-rpm/ +.. _swupd: https://github.com/clearlinux/swupd-client +.. _Clear Linux installer: https://github.com/clearlinux/clr-installer/ +.. _systemd-boot: https://www.freedesktop.org/software/systemd/man/systemd-boot.html +.. _syslinux: https://wiki.syslinux.org/wiki/index.php?title=The_Syslinux_Project +.. _Clear Linux Boot Manager: https://github.com/clearlinux/clr-boot-manager +.. _mixer: https://github.com/clearlinux/mixer-tools +.. _Clear Linux Distro Factory: https://github.com/clearlinux/clr-distro-factory +.. _autospec: https://github.com/clearlinux/common +.. _clr-debug-info: https://github.com/clearlinux/clr-debug-info +.. _clrtrust: https://github.com/clearlinux/clrtrust +.. _EXT4: https://ext4.wiki.kernel.org/index.php/Main_Page +.. _VFAT: https://www.kernel.org/doc/html/latest/filesystems/vfat.html +.. _EXT2 and EXT3: https://ext4.wiki.kernel.org/index.php/Main_Page +.. _F2FS: https://www.kernel.org/doc/Documentation/filesystems/f2fs.txt +.. _LUKS: https://gitlab.com/cryptsetup/cryptsetup/ +.. _systemd: https://www.freedesktop.org/wiki/Software/systemd/ +.. _GNOME: https://www.gnome.org/ +.. _X.Org: https://www.x.org/ +.. _Wayland: https://wayland.freedesktop.org/ +.. _NetworkManager: https://wiki.gnome.org/Projects/NetworkManager +.. _systemd-networkd: https://www.freedesktop.org/software/systemd/man/systemd.network.html +.. _Tallow: https://github.com/clearlinux/tallow +.. _firewalld: https://docs.01.org/clearlinux/latest/guides/network/firewall.html#firewalld +.. _ClamAV*: https://www.clamav.net/ +.. _Lynx: https://lynx.invisible-island.net/ +.. _links: http://links.twibright.com/ +.. _Firefox*: https://www.mozilla.org/en-US/firefox/ +.. _Supplied Bundles: https://clearlinux.org/software +.. _micro-config-drive: https://github.com/clearlinux/micro-config-drive +.. _Telemetrics: https://github.com/clearlinux/telemetrics-backend +.. _packages source code: https://github.com/clearlinux-pkgs/ diff --git a/_sources/collaboration/collaboration.rst.txt b/_sources/collaboration/collaboration.rst.txt new file mode 100644 index 00000000..bcdbbec5 --- /dev/null +++ b/_sources/collaboration/collaboration.rst.txt @@ -0,0 +1,74 @@ +.. _collaboration: + +Contribute +########## + +There are multiple ways to help improve our documentation: + +* `Contribute via GitHub`_: Submit pull requests in the GitHub\* documentation + repository. +* `Log an issue`_: Enter an issue in the documentation repository for + minor issues such as typos. +* `Make a suggestion`_: Send your documentation suggestion to the dev email inbox. +* Test documentation: Step through our guides and tutorials to verify the + instructions. `Log an issue`_ or `submit a pull request`_ with your findings. + +All contributions must follow our `code of conduct`_. + +Contribute via GitHub +********************* + +Our documentation is hosted in GitHub and we follow the standard `GitHub flow`_. +Here are the basic steps for contributing: + +#. Clone the `documentation repository`_. + +#. Create your own fork of the repository. + +#. Create a branch for your contribution. + +#. Add your commits. + +#. Open a pull request. + +#. Discuss, review, and update your contributions. + +#. Once the maintainer approves, your contribution is merged and published as + part of the documentation. + + +Contribution guidelines +*********************** + +The |CL| documentation is written using reStructuredText. Use our guidelines +and best practices to write consistent, readable documentation. If you're writing a tutorial, review our skill levels to better target a user group. + +.. toctree:: + :maxdepth: 1 + + Writing guide + Structure and formatting guide + +.. _references: + +References +********** + +We use the following references for grammar, style, and formatting: + +* `Microsoft Writing Style Guide`_ +* `Merriam-Webster Dictionary`_ +* The Chicago Manual of Style (15th edition), The University of Chicago Press +* Microsoft Press Computer Dictionary, Microsoft Press +* Read Me First!, Oracle Technical Publications + + +.. _`code of conduct`: https://clearlinux.org/community/code-of-conduct +.. _Make a suggestion: mailto:dev@clearlinux.discoursemail.com +.. _GitHub flow: https://guides.github.com/introduction/flow/ +.. _Log an issue: https://github.com/clearlinux/clear-linux-documentation/issues +.. _Contribute via GitHub: https://github.com/clearlinux/clear-linux-documentation +.. _submit a pull request: https://github.com/clearlinux/clear-linux-documentation +.. _documentation repository: https://github.com/clearlinux/clear-linux-documentation +.. _Microsoft Writing Style Guide: https://docs.microsoft.com/en-us/style-guide/welcome/ +.. _Merriam-Webster Dictionary: https://www.merriam-webster.com/ diff --git a/_sources/collaboration/structure-formatting.rst.txt b/_sources/collaboration/structure-formatting.rst.txt new file mode 100644 index 00000000..56f4c954 --- /dev/null +++ b/_sources/collaboration/structure-formatting.rst.txt @@ -0,0 +1,482 @@ +.. _structure-formatting: + +Structure and formatting +######################## + +Content should be organized to support scanning. Consistent organization, +formatting, and writing style helps readers quickly find what they need and to +understand the content more effectively. This document describes our +organization and formatting guidelines. + +Refer to :ref:`writing-guide` to learn how we keep our documents clear and +concise. + +.. contents:: :local: + :depth: 1 + +Markup +****** + +Our documentation is written in the reStructuredText markup language, using +Sphinx roles and directives. We use Sphinx to generate the final documentation. +You can read more about reStructuredText and Sphinx on their respective +websites: + +* `Sphinx documentation`_ +* `reStructuredText Primer`_ + +You can view the content directly in the .rst markup files, or generate the HTML +content by installing and building the documentation locally. To run the +documentation locally, follow the instructions found in the +`documentation repository`_ README. + +New pages +========= + +There are a few additional steps to consider when adding a new page to the +documentation. First, identify where your new page should be located within the +existing `Documentation organization`_. Second, make sure the new page is picked +up in the Sphinx build and easily linkable from other content. + +Each page must be included in a `Sphinx toctree`_ in order to be included in the +documentation content tree. Typically, pages are added to the section landing +page toctree. + +For example, the :ref:`collaboration` page toctree looks like: + +.. code-block:: rest + + .. toctree:: + :maxdepth: 1 + + writing-guide + structure-formatting + +Additionally, each page must include a uniquely named reST label directly before +the page title, to enable the `Sphinx ref role`_ for linking to a page. + +For example, this page "Structure and formatting" has the label +``.. _structure-formatting``: + +.. code-block:: rest + + .. _structure-formatting: + + Structure and formatting + ######################## + +This page can then be referenced from other pages in the documentation using the +`:ref:` role: + +.. code-block:: rest + + :ref:`structure-formatting` + +Documentation organization +************************** + +The documentation is organized into five general sections: + +#. **Concepts**: Introduction and overview of |CL| specific concepts or + features. +#. **Get started**: Information about getting started with |CL|. +#. **Guides**: Detailed information and instruction on using |CL| features. +#. **Tutorials**: Step-by-step instruction for using |CL| in specific use cases. +#. **Reference**: Supplementary and reference information for |CL|. + +Page structure +============== + +Each page in the documentation should follow the basic format of: + +* Overview: 1-2 sentences describing what this page shows and why it matters +* Prerequisites: Describe any pre-work necessary to the content (if appropriate) +* Content +* Next steps: List links to next steps (if appropriate) +* Related topics: List links to related content (if appropriate) + +Headings +======== + +Use headings to section and organize your content for better readability and +clarity. + +* All files must have a top level heading, which is the title for the page. +* Up to three additional levels of headings are allowed under the title heading. +* Each heading should be followed by at least one paragraph of content. Avoid + two or more consecutive headings. + +Refer to the :ref:`writing-guide` for tips on using headings to create +:ref:`scannable content `. + +To mark up headings in the .rst file: + +* Use hash-tags to underline the file's main title: + + .. code-block:: rest + + Main title + ########## + +* Use asterisks to underline the file's first level headings: + + .. code-block:: rest + + First level heading + ******************* + +* Use equal signs to underline the file's second level of headings: + + .. code-block:: rest + + Second level heading + ==================== + +* Use dashes to underline the file's third level of headings: + + .. code-block:: rest + + Third level heading + ------------------- + +In-page navigation +================== + +If a page has three or more sections, provide quick links to each section. Place +the quick links after the overview section. + +Use the standard `reST contents directive`_ with depth: 1 for quick links. + +Inline text formatting +********************** + +We use the `Microsoft Writing Style Guide`_ as our starting point for text +formatting. We apply the formatting using reST and Sphinx markup. + +Use our quick reference for the most commonly used inline text elements: + ++--------------------------------+---------------------------------------+-----------------------------+ +| **Element** | **Convention** | **reST/Sphinx** | ++--------------------------------+---------------------------------------+-----------------------------+ +| Acronyms | Define acronym when first used. After | Use the ``:abbr:`` role, in | +| | first use and definition, use the | the following format: | +| | acronym only. | | +| | | ``:abbr:`Acronym (Def)``` | ++--------------------------------+---------------------------------------+-----------------------------+ +| Bundle names | Bold | Use the ``:command:`` role. | ++--------------------------------+---------------------------------------+-----------------------------+ +| Callouts | | Use ``.. note::`` | ++--------------------------------+---------------------------------------+-----------------------------+ +| Code/command examples | Monospace, visually distinct | Use ``.. code-block::`` | +| | from rest of text. Use an | with the correct language | +| | indented call-out box. | setting. | ++--------------------------------+---------------------------------------+-----------------------------+ +| Commands | Bold | Use the ``:command:`` role. | ++--------------------------------+---------------------------------------+-----------------------------+ +| Command flags | Bold | Use the ``:command:`` role. | ++--------------------------------+---------------------------------------+-----------------------------+ +| Console output | Monospace, visual distinction | Use ``.. code-block::`` | +| | from rest of text. Use an | with console as the | +| | indented call-out box. | language setting. | ++--------------------------------+---------------------------------------+-----------------------------+ +| Emphasis | Italic | ``*strong*`` | ++--------------------------------+---------------------------------------+-----------------------------+ +| Environment variables | Use the case format of the | Use ``:envvar:`` | +| | environment variable. | | ++--------------------------------+---------------------------------------+-----------------------------+ +| Example commands with | Use angle brackets for swapping | | +| optional or replaceable | in the specific name, | | +| parts | e.g. . | | +| | | | +| | Use square brackets for optional | | +| | parts, | | +| | e.g. [--build]. | | ++--------------------------------+---------------------------------------+-----------------------------+ +| Example URLs (not linked) | Plain text | | ++--------------------------------+---------------------------------------+-----------------------------+ +| File extensions | Lowercase | | ++--------------------------------+---------------------------------------+-----------------------------+ +| File names, directories, paths | Title style capitalization | Use the ``:file:`` role. | ++--------------------------------+---------------------------------------+-----------------------------+ +| GUI labels | | Use ``:guilabel:`` | ++--------------------------------+---------------------------------------+-----------------------------+ +| Inline comments | | Use ``..`` | ++--------------------------------+---------------------------------------+-----------------------------+ +| Keystrokes | | Use ``:kbd:`` | ++--------------------------------+---------------------------------------+-----------------------------+ +| Local navigation | | ``.. contents:: :local:`` | +| | | with a depth of 1 | ++--------------------------------+---------------------------------------+-----------------------------+ +| Menu selection | | Use ``:menuselection:`` | ++--------------------------------+---------------------------------------+-----------------------------+ +| New terms | Italic for first use, normal for all | ``*term*`` | +| | subsequent uses. | | +| | | | +| | If it is used outside of the source | | +| | of definition, link the term. | | ++--------------------------------+---------------------------------------+-----------------------------+ +| Product name | Follow correct trademark and | | +| | attribution guidelines. | | ++--------------------------------+---------------------------------------+-----------------------------+ +| Tool names | Correctly capitalized, no quotes, | | +| | bold, or italics as the basic rule. | | +| | | | +| | If the tool name is the command, like | | +| | most Linux tools, treat it like a | | +| | command. | | +| | | | +| | If the tool name is lowercase and | | +| | used at the start of a sentence, use | | +| | bold. | | ++--------------------------------+---------------------------------------+-----------------------------+ + +White space and line length +=========================== + +Limit line length to 78 characters. The GitHub web interface forces this +limitation for readability. + +Remove trailing whitespace from your documents. + +Code blocks and examples +************************ + +When providing example code or commands use the `Sphinx code-block directive`_. +Select the appropriate syntax highlighting for the example command or code. + +For example, if showing console output, use console highlighting: + +.. code-block:: rest + + .. code-block:: console + +Sphinx provides other ways of `marking up example code`_ if needed. + +Lists and instructions +********************** + +Use a numbered list when the order or priority of the items is important, such +as step-by-step instructions. + +Use a bulleted list when the order of the items is not important. + +For both list types, keep all items in the list parallel. See +:ref:`parallelism`. + +Use standard `reST list markup`_. + +Numbered lists +============== + +Numbered lists are most frequently used for procedures. Use numbered lists to +show sequence for the items. Follow our guidelines for numbered lists: + +* Make sure the list is sequential and not just a collection of items. +* Introduce a numbered list with a sentence. End the setup text with a + colon. Example: "To configure the unit, perform the following steps:" +* Each item in the list should be parallel. +* Treat numbered list items as full sentences with correct ending + punctuation. +* You may interrupt numbered lists with other content, if relevant, + e.g. explanatory text, commands, or code. +* Second-level steps are acceptable; avoid third-level steps. +* Avoid single-step procedures; the minimum number of steps in a procedure + is two. +* Do not create numbered lists that emulate flowcharts. The reader should be + able to execute the list of steps from first to last without branching or + looping. +* Avoid over-using numbered lists, except in procedural documents such as + tutorials and step-by-step guides. + +Bulleted lists +============== + +Use bulleted lists to reduce wordiness and paragraph density, especially when +a sequence is not required. Here are some guidelines for bulleted lists: + +* Introduce a bulleted list with a sentence. End the setup text with a + colon. Example: "To repair the unit, you will need the following items:" +* Each item in the list should be parallel. +* Avoid interrupting bulleted lists with other paragraph styles. +* Second-level bullets are acceptable; avoid third-level bullets. + +Use the correct ending punctuation for sentence style bullet lists. For example: + +**Use this:** + +:: + + When setting the user code, remember: + + * Use a number that has a meaning for you. + * Change the code once a month. + * Do not disclose the user code to anyone, including the security company. + +**Not this:** + +:: + + When setting the user code remember: + + * make the user code easy to remember. Use a number that has a meaning for you + * change the code once a month + * do not disclose the user code to anyone else. This includes the security + company + +Instructions +============ + +When presenting instructions, such as in a tutorial, present them in a numbered +list according to these guidelines: + +* Each step (list item) should describe one action. + +* If the same steps are repeated, refer to the earlier steps rather than + repeating them. + +* When a step includes a command or code block as an example, put the command + or code block after the step that includes them. + +* Use supporting images where appropriate. If the series of steps is supported + by one figure, refer to the figure in the introductory text. + + For example: "See Figure 15 and do the following:" + + When a series of steps is supported by two or more figures, refer to the + specific figure in the relevant step and show the figure immediately after + the reference. **Do not write**: "See figures 15 through 22 and do the + following:" + +Notices +******* + +We use four special types of notices: notes, cautions, warnings, and dangers. +Here are some specific rules and tips regarding use of these notices: + +* Do not use a notice directly after a heading. Notices must follow a variant of + body text. +* Do not include more than one notice in a single notice block. +* Avoid back-to-back notices. +* If back-to-back notices are not avoidable, make sure each distinct notice in + the notice block is clearly defined. + +Use the standard `reST admonition directive`_. + +Notes, cautions, and warnings +============================= + +Use notes sparingly. Avoid having more than one note per section. If you exceed +this number consistently, consider rewriting the notes as main body text. + +Use cautions and warnings to alert readers of potential problems or pitfalls. +Use conditional phrases in cautions and warnings, such as "If you do X, then Y +will occur." + +These are examples of typical notices and the conditions for their usage: + +.. note:: + Notes are extra bits of information that supplement the main content. Notes + should be relatively short. + +.. caution:: + Cautions are low-level hazard messages that alert the user of possible + equipment, product, and software damage, including loss of data. + +.. warning:: + Warnings are mid-level hazards that are likely to cause product damage. + +Links +***** + +Use the standard `reST markup for links`_. + +To add a cross-reference to another documentation page, use the `:ref:` role: + +.. code-block:: rest + + :ref:`structure-formatting` + +To add an external link, we use named references that refer to a defined +link/label at the bottom of the page. + +For example, an external link is defined at the bottom of the page like this: + +.. code-block:: rest + + .. _wiki about dogs: https://en.wikipedia.org/wiki/Dog + +The defined link is then used in the content like this: + +.. code-block:: rest + + Check out the great `wiki about dogs`_. + +Images +****** + +Use images or figures to convey information that may be difficult to explain +using words alone. Well-planned graphics reduce the amount of text required to +explain a topic or example. + +Follow these guidelines when using graphics in support of your documentation: + +* Keep it simple. Use images that serve a specific purpose in your document, + and contain only the information the reader needs. + +* Avoid graphics that will need frequent updating. Don't include information in + a graphic that might change with each release, such as product versions. + +* Use either PNG or JPEG bitmap files for screenshots and SVG files for vector + graphics. + +* Place the image immediately after the text it helps clarify, or as close as + possible. + +* Use the `Sphinx figure directive`_ to insert images and figures into the + document. Include both alt text, a figure name, and caption. + + For example: + + .. code-block:: rest + + .. figure:: figures/topic-1.png + :alt: An image supporting the topic. + + Figure 1: This is the figure 1 caption. + +* Include at least one direct reference to an image from the main text, using + the figure number. For example: + + **Use this:** :: + + Figure 1 + + **Not this:** :: + + The figure above or below + +Images should follow these naming and location conventions: + +* Save the image files in a :file:`figures` folder at the same level as the file + that will reference the image. +* Name image files according to the following rules: + + * Use only lower case letters. + * Separate multiple words in filenames using dashes. + * Name images using the filename of the file they appear on and add a number + to indicate their place in the file. For example, the third figure added to + the :file:`welcome.rst` file must be named :file:`welcome-3.png`. + +.. _Sphinx documentation: http://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html +.. _reStructuredText Primer: http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html +.. _documentation repository: https://github.com/clearlinux/clear-linux-documentation +.. _Sphinx toctree: https://www.sphinx-doc.org/en/master/usage/quickstart.html?highlight=toctree#defining-document-structure +.. _Sphinx ref role: https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-ref +.. _reST contents directive: http://docutils.sourceforge.net/docs/ref/rst/directives.html#table-of-contents +.. _Microsoft Writing Style Guide: https://docs.microsoft.com/en-us/style-guide/welcome/ +.. _Sphinx code-block directive: http://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-code-block +.. _marking up example code: http://www.sphinx-doc.org/en/1.6/markup/code.html +.. _reST list markup: http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#lists-and-quote-like-blocks +.. _reST admonition directive: http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#directives +.. _reST markup for links: http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#hyperlinks +.. _Sphinx figure directive: http://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#directives diff --git a/_sources/collaboration/writing-guide.rst.txt b/_sources/collaboration/writing-guide.rst.txt new file mode 100644 index 00000000..40c6830f --- /dev/null +++ b/_sources/collaboration/writing-guide.rst.txt @@ -0,0 +1,410 @@ +.. _writing-guide: + +Writing guide +############# + +We want our documentation to be easy to read and understand. This document +describes guidelines for writing documentation that is clear, concise, +confident, and courteous. + +Refer to :ref:`structure-formatting` for details on organizing content and how +we use reStructuredText and Sphinx. + +.. contents:: :local: + :depth: 1 + +Use simple English +****************** + +Write using simple English: Be brief and communicate only the information that +is needed. Be friendly and informative. Emphasize clarity and avoid +unnecessary complicated or technical terms. Make the content accessible to +non-native speakers. + +Be brief +======== + +Use short sentences and paragraphs. Stick to the principle of one main +idea per sentence, plus one additional point if needed. Each paragraph +should address one main idea. Remember the basic structure of a paragraph: +Introduction, body, and conclusion. + +Be friendly +=========== + +We write for our peers and want to be familiar. Take a personal tone as if you +were speaking directly to the reader. Use "you" to address the reader and "we" +to refer to our view. Be professional, respectful, and cooperative. + +Assume your audience has the same level of technical understanding and expertise +as you did when you first started collaborating. Do not talk down to our +readers, but also do not assume they know everything about the subject. Offer +brief explanations or summaries of common knowledge if a significant portion of +readers might benefit. + +Use simple words +================ + +Use simple words to increase reader comprehension and reduce ambiguity. Follow +our tips for making good word choices: + +* **Avoid jargon**: Write for your audience, using everyday language where + possible, and technical terms where appropriate. Avoid clichés, idioms, and + metaphors. +* **Be consistent**: Use one term for each concept or action and use it + consistently. +* **Avoid "fancy" words and phrases**: If there is a simpler word or phrase, + use it. + + For example: + + =================== =================== + Use this Not this + =================== =================== + start, begin commence + so consequently + more than in excess of + if in the event of + before prior to + if you want should one wish + use utilize + example instance + =================== =================== + +Avoid overuse of product name +============================= + +Use product names only when necessary. Typically, you can rewrite sentences to +remove the product name with no change in meaning, which keeps the content +concise and scannable. + +Avoid using the product name in page titles and headings. + +.. _scannable-content: + +Make content scannable +********************** + +Organize your content to make it scannable for the reader, which helps them find +what they need quickly, and to understand the information more efficiently. + +* **Put the most important content first.** Make sure your introduction clearly + communicates what the reader can find on the page. Present the point of the + document first, and organize supporting information towards the end of the + page. +* **Write scannable headings.** Expect readers of documentation to skim and scan + the content, and to leave if they don't find what they need quickly. Good + headings add organization to your content and help the reader to find and + understand content more effectively. Follow our guidelines for writing + effective `Headings`_. +* **Write great link text.** Great link text tells the reader what they can + expect when they click on a link. It also helps make the page more scannable. + Follow our guidelines for writing `Link text`_. + +Headings +======== + +Use these guidelines to write effective headings: + +* **Be concise and descriptive.** Use only the words necessary to describe the + section. +* **Use sentence case.** Capitalize only the first word and proper nouns in a + heading. +* **Avoid punctuation.** Unless your heading is a question, don't use sentence + punctuation in headings. +* **Use parallel structure.** Headings at the same level should use the same + grammatical pattern. This provides structure to the document and helps users + find information more easily. See :ref:`parallelism`. +* **Use strong verbs.** Strong, active verbs get to the point. Avoid -ing verbs, + such as *Running*, *Testing*, etc. + +For example, two headings at the same level: + +**Use this:** :: + + Install software + + Configure software + +**Not this:** :: + + Installing the Software on the Platform + + Software Configuration. + +Link text +========= + +All links in content should follow these guidelines: + +* **Write descriptive link text**: Link text should describe where the link + goes, without having to read the surrounding text. +* **Keep link text concise**: Use only the words needed to accurately describe + the destination. +* **Use unique link text**: Each link on a page should be unique. If users see + the same link text twice on a page, they'll assume it goes to the same place. +* **Start link text with keywords**: Frontload the link text with the most + important words to help users scan the text. +* **Avoid generic text**: Don't use generic, uninformative link text such as + "click here" or "read more". + +For example: + +**Use this:** :: + + For more information about dogs, read the `dog wiki article`_. + +**Not this:** :: + + For more information about dogs, `click here`_. + +Use strong verbs +**************** + +Passive verbs make writing stuffy and formal. Use strong verbs to get to the +point and avoid unnecessary words and phrases. + +Use imperatives +=============== + +Commands, also called imperatives, are the fastest and most direct way of giving +someone instructions. For example: + +**Use this:** :: + + Send it to me. + +**Not this:** :: + + I would appreciate it if you would send it to me. + +Use present tense +================= + +Use simple present tense instead of future tense for most text. Search for the +words "will" or "shall" to find future tense instances. Future tense is +acceptable for conditional statements, such as in a caution or a warning. For +example: + +**Use this:** :: + + The system operates at a nominal temperature of 180 degrees Fahrenheit. + +**Not this:** :: + + The system will operate at a nominal temperature of 180 degrees Fahrenheit. + +Avoid nominalizations +===================== + +Avoid nominalizations, which are nouns formed from verbs. + +For example: + +===================== ===================== + Verb Nominalization +===================== ===================== + complete completion + provide provision + fail failure + install installation +===================== ===================== + +For example: + +**Use this:** :: + + We discussed the matter. + +**Not this:** :: + + We had a discussion about the matter. + +Or: + +**Use this:** :: + + IT has installed the software. + +**Not this:** :: + + IT has completed the installation of the software. + +Avoid words ending in -ing +========================== + +Avoid using words ending in -ing unless they are part of a technical name. For +example: + +**Use this:** :: + + There is no way to verify this. + +**Not this:** :: + + There is no way of verifying this. + +Use the active voice +==================== + +Use active voice whenever possible to show who or what is performing an +action. + +* Active voice follows standard English word order: SUBJECT–VERB–OBJECT + (where the OBJECT is optional). +* Passive voice reverses the order and weakens the verb: OBJECT–be VERB–by + SUBJECT (where the OBJECT is optional). + +For example: + +**Use this:** :: + + I made a mistake. + +**Not this:** :: + + A mistake was made. *(By whom?)* + +Or: + +**Use this:** :: + + We released version 2.0 in June. + +**Not this:** :: + + Version 2.0 was released in June. + +Avoid long noun phrases +*********************** + +Noun phrases (a noun and other words that describe or modify it) can be +difficult to understand. Try to limit the number of modifiers in a noun phrase +to two. For example: + +**Use this:** :: + + Integration policies for power management mechanisms. + +**Not this:** :: + + Power management mechanism integration policies. + +.. _parallelism: + +Parallelism +*********** + +Parallelism refers to the practice of using similar patterns of grammar, and +sometimes length, to coordinate words, phrases, and clauses. + +Use parallel construction in lists. The table below shows some unparallel +structures and how they can be made parallel with a little rewording. + ++----------------------------------+----------------------------------+ +| Parallel (do) | Unparallel (don't) | ++==================================+==================================+ +| 1. Mount the panel. | 1. Mount the panel. | +| 2. Install the battery. | 2. Battery installation. | +| 3. Wire the keypad. | 3. Wiring the keypad. | ++----------------------------------+----------------------------------+ +| I like practicing my accordion, | I like practicing my accordion, | +| reading sci-fi, and eating | reading sci-fi, and to eat | +| peanut butter and pickle | peanut butter and pickle | +| sandwiches. | sandwiches. | ++----------------------------------+----------------------------------+ +| For breakfast he likes coffee | For breakfast he likes coffee | +| and bacon. | and to fry bacon. | ++----------------------------------+----------------------------------+ +| Apples or bananas are a good | Apples or a banana are a good | +| snack. | snack. | ++----------------------------------+----------------------------------+ + +Grammar and punctuation +*********************** + +This section covers common grammatical topics relevant to our +documentation. For detailed explanations of correct grammar and punctuation, +use one of our :ref:`preferred references `. + +Capitalization +============== + +The capitalization style for all documentation is sentence case. Words should +only be capitalized when they are proper nouns or refer to trademarked product +names. + +.. note:: + Do not capitalize a word to indicate it is more important than other + words. Never change the case of variable, function or file names - always + keep the original case. + +Menu capitalization +------------------- + +When referring to software menu items by name, use the same capitalization as +seen in the actual menu. + +A few other tips when referring to menu items: + +* Reference the specific menu item using "Select :menuselection:`File --> New`." + +* Put the option to be selected last. "Select + :menuselection:`View --> Side Bar --> Hide Side Bar`" + +* Do not include more than 3 navigation steps in a menu selection. If + more than three steps are needed, divide the steps using + ``:guilabel:`` or ``:menuselection:``. + + For example: "Go to :guilabel:`File` and select + :menuselection:`Print --> Print Preview --> Set Up`." + +Software version capitalization +------------------------------- + +When listing software or hardware version numbers, the word “version” or letter +"v" are lowercase. The v is closed with the number (no period). + +For example: + +* Widget Pro version 5.0 +* Widget Master v2.1.12 + +Contractions +============ + +Avoid using contractions, such as it's, they're, and you're, because they may be +unclear to non-native English-speaking audiences. + +Quotation marks +=============== + +Follow these guidelines for quotation marks: + +* Restrict use of quotation marks to terms as terms. +* Do not use quotation marks for emphasis; use *italics* for emphasis. +* Avoid using single-quote marks. + +Commas and colons +================= + +This section addresses common use of commas, semicolons, and colons in our +documentation. Refer to one of our :ref:`preferred references ` +for further details. + +Use the serial comma +-------------------- + +When writing a series of items, use the serial comma before the final *and* and +*or* to avoid confusion and ambiguity. For example: + +**Use this:** :: + + Mom, Dad, and I are going to the game. + +**Not this:** :: + + Mom, Dad and I are going to the game. + +.. _click here: https://en.wikipedia.org/wiki/Dog +.. _dog wiki article: https://en.wikipedia.org/wiki/Dog diff --git a/_sources/disclaimers.rst.txt b/_sources/disclaimers.rst.txt new file mode 100644 index 00000000..6f2dfb6e --- /dev/null +++ b/_sources/disclaimers.rst.txt @@ -0,0 +1,12 @@ +:orphan: + +.. _disclaimers: + + +.. rubric:: Disclaimer + +.. important:: + This documentation is a work in progress and is being provided for + informative purposes only. Because it is a work in progress, there are + parts that are either missing or will be revised as code development + continues. diff --git a/_sources/documentation_license.rst.txt b/_sources/documentation_license.rst.txt new file mode 100644 index 00000000..758a7a24 --- /dev/null +++ b/_sources/documentation_license.rst.txt @@ -0,0 +1,12 @@ +:orphan: + +.. raw:: html + + Creative Commons License
This work is licensed under a Creative Commons + Attribution 4.0 International License + diff --git a/_sources/get-started/bare-metal-install-desktop.rst.txt b/_sources/get-started/bare-metal-install-desktop.rst.txt new file mode 100644 index 00000000..b92c0cd4 --- /dev/null +++ b/_sources/get-started/bare-metal-install-desktop.rst.txt @@ -0,0 +1,686 @@ +.. _bare-metal-install-desktop: + +Install |CL-ATTR| from the live desktop +####################################### + +This page explains how to boot the |CL-ATTR| live desktop image, from which +you can install |CL| or explore without modifying the host system. +Alternatively, use a :ref:`YAML configuration file ` +to install |CL|. + +.. contents:: + :local: + :depth: 1 + +System requirements +******************* + +Before installing |CL|, verify that the host system supports the +installation: + +* Requires 20 GB or more disk space +* :ref:`system-requirements` +* :ref:`compatibility-check` + +.. _preliminary-steps-install-desktop: + +Preliminary steps +***************** + +#. Visit our `Downloads`_ page. + +#. Download the file :file:`clear--live-desktop.iso`, + also called the |CL| Desktop. + + .. note:: + + is the latest |CL| auto-numbered release. + +#. Follow your OS instructions to + :ref:`create a bootable usb drive `. + +.. _install-on-target-start: + +Install from live image +*********************** + +After you download and burn the live desktop image on a USB drive, follow +these steps. + +#. Insert the USB drive into an available USB slot. + +#. Power on the system. + +#. Open the system BIOS setup menu by pressing the :kbd:`F2` key. + Your BIOS setup menu entry point may vary. + +#. In the setup menu, enable the UEFI boot and set the USB drive as the + first option in the device boot order. + +#. Save these settings, e.g. :kbd:`F10`, and exit. + +#. Reboot the target system. + +.. _preliminary-steps-install-desktop-end: + +Choose boot menu option +======================= + +#. Choose one of the options shown in Figure 1. + + a. Follow `Verify integrity of installer media (optional)`_. + + #. Select :guilabel:`Clear Linux OS` in the boot menu. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-01.png + :scale: 100% + :alt: Clear Linux OS in boot menu + + Figure 1: Clear Linux OS in boot menu + + .. note:: + + If no action is taken, the live image starts by default. + +.. _install-on-target-end: + +Verify integrity of installer media (optional) +============================================== + +Use :guilabel:`Verify ISO Integrity` to verify the checksum of +the image burned to the installer media. The checksum ensures that the ISO +is uncorrupted (see Figure 1). For every ISO generated, the +:guilabel:`clr-installer` implants checksums, which are verified during +early boot stage as part of :command:`initrd`. + +#. Select :guilabel:`Verify ISO Integrity`. The media will be validated. + +#. If the check passes, it will boot into the live image. Continue in + the next section. + +#. If the check fails, a failure message appears. + + * Restart the process at `Preliminary Steps`_. + +.. _install-clr-desktop-start: + +Launch the |CL| installer +========================= + +#. After the live desktop image boots, scroll over the vertical + :guilabel:`Activities` menu at left. + +#. Click the |CL| penguin icon to launch the installer, shown in Figure 2. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-02.png + :scale: 100% + :alt: Install Clear Linux OS icon + + Figure 2: |CL| installer icon + +#. After the installer is launched, it will appear as shown in Figure 3. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-03.png + :scale: 100% + :alt: |CL| Desktop Installer + + Figure 3: |CL| OS Desktop Installer + +#. In :guilabel:`Select Language`, select a language from the options, or + type your preferred language in the search bar. + +#. Select :guilabel:`Next`. + + +Network Proxy (optional) +------------------------ + +#. Configure :guilabel:`Network Proxy` settings. + +#. In the top right menu bar, select the :guilabel:`Power button`. + +#. Select :guilabel:`Wired Connected` and then :guilabel:`Wired Settings`. + + #. In :guilabel:`Network Proxy`, select the :guilabel:`Gear` icon to view + options. + + #. Select an option from `Automatic`, `Manual` or `Disabled`. + + #. Close :guilabel:`Network Proxy`. + +#. Close :guilabel:`Settings`. + +.. _incl-bare-metal-beta-start: + +Minimum installation requirements +********************************* + +To fulfill minimum installation requirements, complete the +`Required options`_. We also recommend completing `Advanced options`_. + +.. note:: + + * The :kbd:`Install` button is only highlighted **after** you complete + `Required options`_. + + * Check marks indicate a selection has been made. + + * The installer image contains the default bundles required for + installation. An Internet connection is only required if you install + additional bundles from `Advanced options`_. + +|CL| Desktop Installer +********************** + +The |CL| Desktop Installer Main Menu appears as shown in Figure 4. To meet +the minimum requirements, enter values in all submenus for the +:guilabel:`Required options`. After you complete them, your selections appear +below submenus and a check mark appears at right. + +.. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-04.png + :scale: 100% + :alt: Clear Linux OS Desktop Installer - Main Menu + + Figure 4: Clear Linux OS Desktop Installer - Main Menu + +Navigation +********** + +* Use the :kbd:`mouse` to navigate or select options. + +* Use :kbd:`Tab` key to navigate between :guilabel:`Required options` + and :guilabel:`Advanced options` + +* Use :kbd:`Up` or :kbd:`Down` arrow keys to navigate the submenus. + +* Select :kbd:`Confirm`, or :kbd:`Cancel` in submenus. + +Required options +**************** + +Select Time Zone +================ + +#. From the Main Menu, select :guilabel:`Select Time Zone`. `UTC` is selected + by default. + +#. In :guilabel:`Select Time Zone`, navigate to the desired time zone. + Or start typing the region and then the city. + (.e.g., :file:`America/Los_Angeles`). + +#. Select :guilabel:`Confirm`. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-05.png + :scale: 100% + :alt: Select System Timezone + + Figure 5: Select System Time Zone + +Select Keyboard +=============== + +#. From the Main Menu, select :guilabel:`Select Keyboard`. + +#. Navigate to your desired keyboard layout. We select "us" for the + United States. + +#. Select :guilabel:`Confirm`. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-06.png + :scale: 100% + :alt: Select Keyboard menu + + Figure 6: Select Keyboard menu + +Select Installation Media +========================= + +#. From the Main Menu, select :guilabel:`Select Installation Media`. + +#. Choose an installation method: `Safe Installation`_ or + `Destructive Installation`_. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-07.png + :scale: 100% + :alt: Select Installation Media + + Figure 7: Select Installation Media + +Safe Installation +----------------- + +Use this method to safely install |CL| on media with available space, or +alongside existing partitions, and accept the `Default partition schema`_. +If enough free space exists, safe installation is allowed. + +.. note:: + + |CL| allows installation alongside another OS. Typically, when you boot + your system, you can press an `F key` to view and select a bootable + device or partition during the BIOS POST stage. Some BIOSes present the + |CL| partition, and you can select and boot it. However, other + BIOSes may only show the primary partition, in which case you will not be + able boot |CL|. Be aware of this possible limitation. + +Destructive Installation +------------------------ + +Use this method to destroy the contents of the target device, install |CL| +on it, and accept the `Default partition schema`_. + +Disk encryption +--------------- + +For greater security, disk encryption is supported using LUKS. Encryption is +optional. + +#. To encrypt the root partition, select :guilabel:`Enable Encryption`, + as shown in Figure 8. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-08.png + :scale: 100% + :alt: Enable Encryption + + Figure 8: Enable Encryption + +#. When :guilabel:`Encryption Passphrase` appears, enter a passphrase. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-09.png + :scale: 100% + :alt: Encryption Passphrase + + Figure 9: Encryption Passphrase + + .. note:: + + Minimum length is 8 characters. Maximum length is 94 characters. + +#. Enter the same passphrase in the second field. + +#. Select :guilabel:`Confirm` in the dialogue box. + + .. note:: + + :guilabel:`Confirm` is only highlighted if passphrases match. + +#. Select :guilabel:`Confirm` in submenu. + +Advanced Installation +--------------------- + +Use this method to manually partition the target media using `gparted`. +Our example uses the `Default partition schema`_. The space you allocate for +``root``, or additional partitions, may vary. + +#. Select :guilabel:`Advanced Installation`. + +#. Select :guilabel:`Partition Media`, shown in Figure 11. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-10.png + :scale: 100% + :alt: Advanced Installation + + Figure 10: Advanced Installation + +boot partition +-------------- + +#. Select the available target media shown as `unallocated`. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-11.png + :scale: 100% + :alt: Advanced Disk Partitioning + + Figure 11: Advanced Disk Partitioning + +#. Choose :menuselection:`Device --> Create Partition Table`. + +#. In the `Warning` screen, under :guilabel:`Select new partition table type` + , select `gpt` from the pull-down menu. + +#. Select :guilabel:`Apply`. + +#. Select :menuselection:`Partition --> New`. + + .. note:: + + The `/boot` partition must be `VFAT(FAT32)`. + +#. In :guilabel:`Create new Partition`, complete the following fields to + match Figure 12. Don't change other default values. + + * :guilabel:`New size:` 150 + * :guilabel:`Partition name:` CLR_BOOT + * :guilabel:`File system:` fat32 + * :guilabel:`Label:` boot + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-12.png + :scale: 100% + :alt: boot partition + + Figure 12: boot partition + +#. Select :guilabel:`Add`. + +swap partition (optional) +------------------------- + +A swapfile is generated by default during installation. However, if you prefer to create a swap partition, follow the steps below. + +#. With :guilabel:`unallocated` highlighted, select from the menu + :menuselection:`Partition --> New`. + +#. In :guilabel:`Create new Partition`, complete the following fields to + match Figure 13. Don't change other default values. + + * :guilabel:`New size:` 256 + * :guilabel:`Partition name:` CLR_SWAP + * :guilabel:`File system:` linux-swap + * :guilabel:`Label:` swap + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-13.png + :scale: 100% + :alt: swap partition + + Figure 13: swap partition + +#. Select :guilabel:`Add`. + +root partition +-------------- + +#. With :guilabel:`unallocated` highlighted, select from the menu + :menuselection:`Partition --> New`. + +#. In :guilabel:`Create new Partition`, complete the following fields to + match Figure 14. Don't change other default values. + +#. In :guilabel:`New size`, enter the desired size, or leave as is + to accept the *default: remaining size*. + + * :guilabel:`New size:` + * :guilabel:`Partition name:` CLR_ROOT + * :guilabel:`File system:` ext[234], XFS, or f2fs + * :guilabel:`Label:` root + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-14.png + :scale: 100% + :alt: root partition + + Figure 14: root partition + +#. After all partitions are defined, verify your partition + configuration is similar to Figure 15. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-15.png + :scale: 100% + :alt: Final partition configuration + + Figure 15: Final partition configuration + +#. Select :menuselection:`Edit --> Apply All Operations`. + +#. A dialog box appears asking "Are you sure you want to apply the pending + operations?" + +#. Select :guilabel:`Apply`. + +#. When dialog :guilabel:`Applying pending operations` is complete, select + :guilabel:`Close`. + +#. Select :menuselection:`GParted --> Quit`. + +You are returned to installer. + +Manage User +=========== + +#. In Required Options, select :guilabel:`Manage User`. + +#. In :guilabel:`User Name`, enter a user name. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-16.png + :scale: 100% + :alt: Manage User + + Figure 16: Manage User + +#. In :guilabel:`Login`, create a login name. It must start with a letter + and can use numbers, hyphens, and underscores. Maximum length is 31 + characters. + +#. In :guilabel:`Password`, enter a password. Minimum length is + 8 characters. Maximum length is 255 characters. + +#. In :guilabel:`Confirm`, enter the same password. + + .. note:: + + :guilabel:`Administrator` rights are selected by default. + For security purposes, the default user must be assigned as an + Administrator. + +#. Select :kbd:`Confirm`. + + .. note:: + + Select :guilabel:`Cancel` to return to the Main Menu. + +Modify User +----------- + +#. In Manager User, select :guilabel:`Manage User`. + +#. Modify user details as desired. + +#. Select :guilabel:`Confirm` to save the changes you made. + + .. note:: + + Optional: Select :guilabel:`Cancel` to return to the Main Menu to + revert changes. + +Optional: Skip to `Finish installation`_. + +Telemetry +========= + +Choose whether to participate in `telemetry`. :ref:`telem-guide` is a |CL| +feature that reports failures and crashes to the |CL| development +team for improvements. + +#. From :guilabel:`Required Options`, select :guilabel:`Telemetry`. + +#. Select :kbd:`Yes`. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-17.png + :scale: 100% + :alt: Enable Telemetry + + Figure 17: Enable Telemetry + +#. If you don't wish to participate, select :kbd:`No`. + +Advanced options +**************** + +After you complete the `Required options`_, we recommend completing +:guilabel:`Advanced options`--though they're not required. Doing so +customizes your development environment, so you're ready to go immediately +after reboot. + +.. note:: + + You can always add more bundles later with :ref:`swupd-guide`. + +Select Additional Bundles +========================= + +This option is only available with a valid network connection. +Bundle selection is disabled if no network connection exists. + +#. On the Advanced menu, select :guilabel:`Select Additional Bundles`. + +#. Select your desired bundles. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-18.png + :scale: 100% + :alt: Bundle Selection + + Figure 18: Bundle Selection + +#. Select :kbd:`Confirm`. + +#. View the bundles that you selected. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-19.png + :scale: 100% + :alt: Select Additional Bundles + + Figure 19: Select Additional Bundles + +Optional: Skip to `Finish installation`_. + +Assign Hostname +=============== + +#. In Advanced Options, select :guilabel:`Assign Hostname`. + +#. In :guilabel:`Hostname`, enter the hostname only (excluding the domain). + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-20.png + :scale: 100% + :alt: Assign Hostname + + Figure 20: Assign Hostname + + .. note:: + + Hostname does not allow empty spaces. Hostname must start with an + alphanumeric character but may also contain hyphens. Maximum length of + 63 characters. + +#. Select :kbd:`Confirm`. + +Optional: Skip to `Finish installation`_. + +Kernel Configuration +==================== + +#. In :guilabel:`Kernel Configuration`, navigate to select your desired + kernel. :guilabel:`Native` is selected by default. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-21.png + :scale: 100% + :alt: Kernel Configuration + + Figure 21: Kernel Configuration + +#. To add arguments, enter the argument in :guilabel:`Add Extra Arguments`. + +#. To remove an argument, enter the argument in :guilabel:`Remove Arguments`. + +#. Select :kbd:`Confirm`. + +Software Updater Configuration +============================== + +#. In Advanced Options, select :guilabel:`Software Updater Configuration`. + +#. In :guilabel:`Mirror URL`, follow the instructions if you wish to + specify a different installation source. + +#. :guilabel:`Enable Auto Updates` is selected by default. If you **do not** + wish to enable automatic software updates, uncheck the box. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-22.png + :scale: 100% + :alt: Software Updater Configuration + + Figure 22: Software Updater Configuration + +#. Select :kbd:`Confirm`. + +Finish installation +******************* + +#. When you are satisfied with your installation configuration, select + :guilabel:`Install`. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-23.png + :scale: 100% + :alt: Assign Hostname + + Figure 23: Finish installation + + .. note: + + All check marks must appear in :guilabel:`Required Options` for the + :guilabel:`Install` button to be enabled. + +#. If you do not enter a selection for all :guilabel:`Required Options`, + the :guilabel:`Install` button remains disabled, as shown + in Figure 24. Return to `Required Options`_ and make selections. + + .. figure:: /_figures/bare-metal-install-desktop/bare-metal-install-desktop-24.png + :scale: 100% + :alt: Required Options - Incomplete + + Figure 24: Required Options - Incomplete + +#. After installation is complete, select :guilabel:`Exit`. + +#. Shut down the target system. + +#. Remove the USB or any installation media. + +#. Power on your system. + + .. note:: + + Allow time for the graphical login to appear. A login prompt shows the administrative user that you created. + +#. Log in as the administrative user. + +Congratulations. You successfully installed |CL|. + +Default partition schema +======================== + +Create partitions per requirements in Table 1. + +.. list-table:: **Table 1. Default partition schema** + :widths: 25, 25, 25, 25 + :header-rows: 1 + + * - FileSystem + - Label + - Mount Point + - Default size + + * - ``VFAT (FAT32)`` + - boot + - /boot + - 150MB + + * - ``ext[234], XFS, or f2fs`` + - root + - / + - *Size depends upon use case/desired bundles.* + +.. note:: + + A 64MiB swapfile is generated by default. The default size may be set + manually with the ``--swap-file-size`` command-line option. + +Troubleshooting +*************** + +:ref:`erase-lvm-troubleshooting-tip` + +Related topics +************** + +* :ref:`install-configfile` + +.. _Downloads: https://clearlinux.org/downloads diff --git a/_sources/get-started/bare-metal-install-server.rst.txt b/_sources/get-started/bare-metal-install-server.rst.txt new file mode 100644 index 00000000..0c345e67 --- /dev/null +++ b/_sources/get-started/bare-metal-install-server.rst.txt @@ -0,0 +1,1042 @@ +.. _bare-metal-install-server: + +Install |CL-ATTR| from the live server +###################################### + +This page explains how to install |CL-ATTR| on bare metal from a bootable USB +drive using a live server image. Alternatively, use a :ref:`YAML configuration file ` to install |CL|. + +.. contents:: + :local: + :depth: 1 + +System requirements +******************* + +Before installing |CL|, verify that the host system supports the +installation: + +* Requires 4 GB or more disk space +* :ref:`system-requirements` +* :ref:`compatibility-check` + +Preliminary steps +***************** + +#. Visit our `Downloads`_ page. + +#. Download the file :file:`clear--live-server.iso`, + also called the |CL| Server. + + .. note:: + + is the latest |CL| auto-numbered release. + +#. Follow your OS instructions to + :ref:`create a bootable usb drive `. + +Install |CL| on your target system +********************************** + +Ensure that your system is configured to boot UEFI. The installation method +described below requires a wired or wireless Internet connection with DHCP. + +Follow these steps to install |CL| on the target system: + +#. Insert the USB drive into an available USB slot. + +#. Power on the system. + +#. Open the system BIOS setup menu by pressing the :kbd:`F2` key. + Your BIOS setup menu entry point may vary. + + .. note:: + |CL| supports UEFI boot. Some hardware may list UEFI and non-UEFI USB + boot entries. In this case, you should select the `UEFI` boot + option. + +#. In the setup menu, enable the UEFI boot and set the USB drive as the first + option in the device boot order. + +#. Save these settings and exit. + +#. Reboot the target system. + +Choose boot menu option +======================= + +#. Choose one of the options shown in Figure 1. + + a. Follow `Verify integrity of installer media (optional)`_. + + #. Select :guilabel:`Clear Linux OS` in the boot menu. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-01.png + :scale: 100% + :alt: Clear Linux OS Installer boot menu + + Figure 1: Clear Linux OS Installer boot menu + + .. note:: + + If no action is taken, the live image starts by default. + +Verify integrity of installer media (optional) +============================================== + +Use :guilabel:`Verify ISO Integrity` to verify the checksum of +the image burned to the installer media. The checksum ensures that the ISO +is uncorrupted (see Figure 1). For every ISO generated, the +:guilabel:`clr-installer` implants checksums, which are verified during +early boot stage as part of :command:`initrd`. + +#. Select :guilabel:`Verify ISO Integrity`. The media will be validated. + +#. If the check passes, it will boot into the live image. Continue in + the next section. + +#. If the check fails, a failure message appears. + + * Restart the process at `Preliminary Steps`_. + +.. _install-clr-server-start: + +Launch the |CL| Installer +************************* + +#. At the :guilabel:`login` prompt, enter :command:`root`. + +#. Follow the onscreen instructions, shown in Figure 2, and + enter a temporary password. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-02.png + :scale: 100% + :alt: root login + + Figure 2: root login + + .. note:: + + If a wireless connection is needed, connect to the network using + :command:`nmtui` before lauching the installer. See the documentation on + :ref:`configuring Wifi with nmtui ` for more details. + +#. At the :guilabel:`root` prompt, enter :command:`clr-installer` and + press :kbd:`Enter`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-03.png + :scale: 100% + :alt: clr-installer command + + Figure 3: clr-installer command + +Minimum installation requirements +********************************* + +To fulfill minimum installation requirements, complete the +`Required options`_. While not required, we encourage you to apply the +`Recommended options`_. `Advanced options`_ are optional. + +.. note:: + + * The :kbd:`Install` button is **only highlighted after** you complete + `Required options`_. + +Main Menu +********* +The |CL| Installer Main Menu appears as shown in Figure 4. + +.. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-04.png + :scale: 100% + :alt: Clear Linux OS Installer + + Figure 4: Clear Linux OS Installer + +The |CL| Installer Main Menu has two tabs: :guilabel:`[R] Required options` +and :guilabel:`[A] Advanced options`. Navigate between tabs using these shortcut keys: + +* :kbd:`Shift+A` for :guilabel:`[A] Advanced options` +* :kbd:`Shift+R` for :guilabel:`[R] Required options` + +To meet the minimum requirements, enter your choices in the +:guilabel:`Required options`. After confirmation, your selections appear +beside the :guilabel:`>>` chevron, below the menu options. + +Navigation +********** + +* Select :kbd:`Tab` or :kbd:`Up/Down` arrows to highlight your choice. + +* Select :kbd:`Enter` or :kbd:`Spacebar` to confirm your choice. + +* Select :kbd:`Cancel` or :kbd:`Esc` to cancel your choice. + +Required options +**************** + +Choose Timezone +=============== + +#. From the Main Menu, navigate to :guilabel:`Choose Timezone`. + `UTC` is the default. + +#. Select :kbd:`Enter`. + +#. In :guilabel:`Select System Timezone`, use :kbd:`Up/Down` arrows + navigate to the desired timezone. + +#. Press :kbd:`Enter` to confirm. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-05.png + :scale: 100% + :alt: Select System Timezone + + Figure 5: Select System Timezone + +Choose Language +=============== + +#. From the Main Menu, navigate to :guilabel:`Choose Language`. + +#. Select :kbd:`Enter`. + +#. In :guilabel:`Select System Language`, navigate to your desired language. + +#. Press :kbd:`Enter` to confirm. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-06.png + :scale: 100% + :alt: Select System Language + + Figure 6: Select System Language + +Configure the Keyboard +====================== + +#. From the Main Menu, select :guilabel:`Configure the Keyboard`. + +#. Select :kbd:`Enter`. + +#. In :guilabel:`Select Keyboard`, navigate to the desired option. + +#. Select :kbd:`Enter` to :kbd:`Confirm`. + +#. Optional: In :guilabel:`Test keyboard`, type text to assure + that the keys map to your keyboard. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-07.png + :scale: 100% + :alt: Select Keyboard menu + + Figure 7: Select Keyboard menu + +Configure Installation Media +============================ + +#. From the Main Menu, select :guilabel:`Configure Installation Media`. + +#. Choose an installation method: + * `Safe Installation`_ + * `Destructive Installation`_ + * `Advanced Installation`_ + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-08.png + :scale: 100% + :alt: Select Installation Media + + Figure 8: Select Installation Media + +#. Select :guilabel:`Rescan Media` to show available installation targets. + +Safe Installation +----------------- + +Use this method to safely install |CL| on media with available space, or +alongside existing partitions, and accept the `Default partition schema`_. +If enough free space exists, safe installation is allowed. See also +`Troubleshooting`_ below. + +Destructive Installation +------------------------ + +Use this method to destroy the contents of the target device, install |CL| +on it, and accept the `Default partition schema`_. + +.. note:: + + From the :guilabel:`Select Installation Media` menu, select + :guilabel:`Enable Encryption` to encrypt the root filesystem for either + option above. See also `Disk encryption`_ for more information. + +Advanced Installation +--------------------- + +Use this method to manually configure partitions using `cgdisk`. +This example uses the `Default partition schema`_. The space you allocate for +your ``root``, or additional partitions, may vary. + +#. Navigate to :guilabel:`Advanced Installation` and press :kbd:`Spacebar`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-09.png + :scale: 100% + :alt: Advanced installation + + Figure 9: Advanced installation + +#. If no target media appears, select :kbd:`Rescan Media`. + +#. Navigate to :guilabel:`Partition` and and press :kbd:`Spacebar` + to launch `cgdisk`. + +Partition codes +--------------- + +* ef00 - EFI System +* 8200 - Linux swap +* 8300 - Linux filesystem + +boot partition +-------------- + +#. With the free space highlighted in the cgdisk utility, select + :guilabel:`[New]`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-10.png + :scale: 100% + :alt: Select New + + Figure 10: Select New + + .. note:: + + The `/boot` partition must be `VFAT(FAT32)`. + +#. Where :guilabel:`First sector` appears, press :kbd:`Enter`. + +#. For :guilabel:`Size in sectors`, type 150M. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-11.png + :scale: 100% + :alt: Size in sectors + + Figure 11: Size in sectors + +#. Press `Enter`. + +#. Enter the hex code `ef00` and press :kbd:`Enter`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-12.png + :scale: 100% + :alt: `ef00` partition code + + Figure 12: `ef00` partition code + +#. For the partition name, enter `CLR_BOOT`, the EFI boot partition. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-13.png + :scale: 100% + :alt: CLR_BOOT + + Figure 13: CLR_BOOT + + .. note:: + + Encryption is not allowed on the CLR_BOOT partition. + +Now follow the same process to configure the remaining partitions. + +swap partition (optional) +------------------------- + +A swapfile is generated by default during installation. However, if you +prefer to create a swap partition, follow the steps below. + +#. Use the :kbd:`Up/Down` arrow to select free space. + +#. Select :guilabel:`[New]` at bottom and press :kbd:`Enter`. + +#. Under :guilabel:`First sector`, press :kbd:`Enter`. + +#. For :guilabel:`Size in sectors`, type 256M, and press :kbd:`Enter`. + +#. Enter the hex code `8200` and press :kbd:`Enter`. + +#. In :guilabel:`Enter new partition name...`, type CLR_SWAP. + +#. Press :kbd:`Enter`. + +root partition +-------------- + +#. Use the :kbd:`Up/Down` arrow to select free space. + +#. Select :guilabel:`[New]` at bottom and press Enter. + +#. Under :guilabel:`First sector`, press Enter. + +#. For :guilabel:`Size in sectors`, type in desired size. + Optionally, press :kbd:`Enter` to use the remaining space available. + +#. Press Enter. + +#. Enter the hex code `8300` and press :kbd:`Enter`. + +#. In :guilabel:`Enter new partition name...`, type: CLR_ROOT. + The `/root` partition must be `ext[234]`, `XFS`, or `f2fs`. + If no filesystem exists, the installer will default to `VFAT(FAT32)` + for `/boot`, and `ext4` for all others. + + .. note:: + + You may also append `_F` to the partition name to force the formatting. + + * `CLR_ROOT_F`: Force the formatting of the root partition prior to + use. + + * `CLR_F_SWAP`: Force the formatting of the swap partition prior to + use; helpful when re-using a partition for swap which was previously formatted for a file system. + + * `CLR_F_MNT_/data`: Force the formatting of the extra data + partition prior to use + +#. Press :kbd:`Enter`. + +#. After all partitions are defined, verify that your partition + configuration is similar to Figure 14. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-14.png + :scale: 100% + :alt: Final partition configuration + + Figure 14: Final partition configuration + +Additional partitions (optional) +-------------------------------- + +#. Use the :kbd:`Up/Down` arrow to select free space. + +#. Now select :guilabel:`[New]` at bottom and press Enter. + +#. Under :guilabel:`First sector`, press Enter. + +#. For :guilabel:`Size in sectors`, type in desired size. + + .. note:: + + If you do not specify a size, it will use the remaining space. + +#. Press :kbd:`Enter`. + +#. Enter the hex code `8300`. Then press :kbd:`Enter`. + +#. In :guilabel:`Enter new partition name...`, type: `CLR_MNT_`. + For example, replace with `/home`, shown in Figure 15. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-15.png + :scale: 100% + :alt: CLR_MNT + + Figure 15: CLR_MNT + + .. note:: + + If formatting is desired, the `_F` **must precede** `_MNT`. + +#. Alternatively, you may create `CLR_MNT_/srv` or other partitions. + +Write configuration to disk +--------------------------- + +#. When you're satisfied with the partition configuration, press the + Right Arrow until :guilabel:`[Write]` is highlighted. + +#. Press :kbd:`Enter`. + +#. When the prompt appears asking if you want to write the partition table + to disk, type "yes". + +#. Finally, select :guilabel:`[Quit]`. + +Disk encryption +=============== + +For greater security, disk encryption is supported using LUKS for the +any partition except `/boot` on |CL|. To encrypt the root partition, see the +example below. Encryption is optional. + +Encryption Passphrase +--------------------- + +|CL| uses a single passphrase for encrypted partitions. Additional keys may +be configured post-installation using the ``cryptsetup`` tool. + +#. Optional: Select :guilabel:`[X] Enable encryption` to encrypt the root + partition, as shown in Figure 16. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-16.png + :scale: 100% + :alt: Encrypt partition + + Figure 16: Encrypt partition + +#. The :guilabel:`Encryption Passphrase` dialog appears. + + .. note:: + + Minimum length is 8 characters. Maximum length is 94 characters. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-17.png + :scale: 100% + :alt: Encryption Passphrase + + Figure 17: Encryption Passphrase + +#. Enter the same passphrase in the first and second field. + +#. Navigate to :guilabel:`Confirm` and press :kbd:`Enter`. + + .. note:: + + :guilabel:`Confirm` is only highlighted if passphrases match. + +Manage User +=========== + +Add New User +------------ + +#. In Required Options, select :guilabel:`Manage User`. + +#. Select :guilabel:`Add New User` as shown in Figure 18. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-18.png + :scale: 100% + :alt: Add New User, User Name + + Figure 18: Add New User + +#. Optional: Enter a :guilabel:`User Name`. + + .. note: + + The User Name must be alphanumeric and can include spaces, commas, + underscores or hyphens. Maximum length is 64 characters. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-19.png + :scale: 100% + :alt: User Name + + Figure 19: User Name + +#. Enter a :guilabel:`Login`. + + .. note:: + + The User Login must be alphanumeric and can include hyphens and underscores. Maximum length is 31 characters. + +#. Enter a :guilabel:`Password`. + + .. note: + + Minimum length is 8 characters. Maximum length is 255 characters. + +#. In :guilabel:`Confirm`, enter the same password. + +#. The :guilabel:`Administrator` checkbox is selected by default. + + .. note:: + + Selecting Administrator enables sudo privileges for the user. For the installation to proceed, at least one user must be assigned as an Administrator. + +#. Select :kbd:`Confirm`. To reset the form, select :guilabel:`Reset`. + +#. In :guilabel:`Manage User`, navigate to :guilabel:`Confirm`. + +#. With :guilabel:`Confirm` highlighted, select :kbd:`Enter`. + +Modify / Delete User +-------------------- + +#. In :guilabel:`Manage User`, navigate to the user you wish + to modify until highlighted, as shown in Figure 20. + +#. Select :kbd:`Enter` to modify the user. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-20.png + :scale: 100% + :alt: Modify User + + Figure 20: Modify User + +#. Modify user details as desired. + +#. Navigate to :kbd:`Confirm` until highlighted. + + .. note:: + + Optional: Select :guilabel:`Reset` to rest the form. + +#. Select :guilabel:`Confirm` to save the changes you made. + +#. Optional: In :guilabel:`Modify User`, to delete the user, navigate to + the :guilabel:`Delete` button and select :kbd:`Enter`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-21.png + :scale: 100% + :alt: Delete User + + Figure 21: Delete User + +You are returned to :guilabel:`Manage User`. + +#. Navigate to :kbd:`Confirm` until highlighted. + +#. Select :guilabel:`Enter` to complete :guilabel:`Manage User` options. + +Telemetry +========= + +:ref:`telem-guide` is a |CL| feature that reports failures and crashes to +the |CL| development team for improvements. + +Select your desired option on whether to participate in telemetry. + +#. In the Main Menu, navigate to :guilabel:`Telemetry` and select + :kbd:`Enter`. + +#. Select :kbd:`Tab` to highlight your choice. + +#. Select :kbd:`Enter` to confirm. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-22.png + :scale: 100% + :alt: Enable Telemetry + + Figure 22: Enable Telemetry + +Recommended options +******************* + +After you complete the `Required options`_, we highly recommend completing +some `Advanced options`_: + +* `Assign Hostname`_ Simplify your development environment + +Skip to finish installation +=========================== + +After selecting values for all :guilabel:`Required options`, you may skip +to `Finish installation`_. + +Otherwise, continue below. In the Main Menu, select +:guilabel:`Advanced options` for additional configuration. + +Advanced options +**************** + +Configure Network Interfaces +============================ + +By default, |CL| is configured to automatically detect the host network +interface using DHCP. However, if you want to use a static IP address or if +you do not have a DHCP server on your network, follow these instructions to +manually configure the network interface. Otherwise, default network +interface settings are automatically applied. + +.. note:: + + If DHCP is available, no user selection may be required. + +#. Navigate to :guilabel:`Configure Network Interfaces` and + select :kbd:`Enter`. + +#. Navigate to the network :guilabel:`interface` you wish to change. + +#. When the desired :guilabel:`interface` is highlighted, select + :guilabel:`Enter` to edit. + + .. note:: Multiple network interfaces may appear. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-23.png + :scale: 100% + :alt: Configure Network Interfaces + + Figure 23: Configure Network Interfaces + +#. Notice :guilabel:`Automatic / dhcp` is selected by default (at bottom). + + Optional: Navigate to the checkbox :guilabel:`Automatic / dhcp` and select + :kbd:`Spacebar` to deselect. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-24.png + :scale: 100% + :alt: Network interface configuration + + Figure 24: Network interface configuration + +#. Navigate to the appropriate fields and assign the desired + network configuration. + +#. To save settings, navigate to :guilabel:`Confirm` and select + :kbd:`Enter`. + + .. note:: + + To revert to previous settings, navigate to the :guilabel:`Cancel` + and select :kbd:`Enter`. + +#. Upon confirming network configuration, the :guilabel:`Testing Networking` + dialog appears. Assure the result shows success. If a failure occurs, + your changes will not be saved. + +#. Upon confirmation, you are returned to :guilabel:`Network interface` + settings. + +#. Navigate to and select :guilabel:`Main Menu`. + +Optional: Skip to `Finish installation`_. + +Proxy +===== + +|CL| automatically attempts to detect proxy settings, as described in +:ref:`autoproxy`. If you need to manually assign proxy settings, follow this +instruction. + +#. From the Advanced options menu, navigate to :guilabel:`Proxy`, and + select :kbd:`Enter`. + +#. Navigate to the field :guilabel:`HTTPS Proxy`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-25.png + :scale: 100% + :alt: Configure the network proxy + + Figure 25: Configure the network proxy + +#. Enter the desired proxy address and port using conventional syntax, + such as: \http://address:port. + +#. Navigate to :guilabel:`Confirm` and select :kbd:`Enter`. + +#. To revert to previous settings, navigate to :guilabel:`Cancel` + and select :guilabel:`Cancel`. + +Optional: Skip to `Finish installation`_. + +Test Network Settings +===================== + +To manually assure network connectivity before installing |CL|, +select :guilabel:`Test Network Settings` and select :guilabel:`Enter`. + +.. note:: + If using the :command:`off-line installer`, this option is not available. + +A progress bar appears as shown in Figure 26. + +.. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-26.png + :scale: 100% + :alt: Testing Networking dialog + + Figure 26: Testing Networking dialog + +.. note:: + + Any changes made to network settings are automatically tested + during configuration. + +Optional: Skip to `Finish installation`_. + +Select Additional Bundles +========================= + +This option is only available with a valid network connection. +Bundle selection is disabled if no network connection exists. + +#. On the Advanced menu, select :guilabel:`Select Additional Bundles`. + +#. Navigate to the desired bundle using :kbd:`Tab` or :kbd:`Up/Down` arrows. + +#. Select :kbd:`Spacebar` to select the checkbox for each desired bundle. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-27.png + :scale: 100% + :alt: Bundle Selection + + Figure 27: Bundle Selection + +#. Optional: To start developing with |CL|, we recommend + adding :file:`os-clr-on-clr`. + +#. Navigate to and select :kbd:`Confirm`. + + You are returned to the :guilabel:`Advanced options` menu. + +Optional: Skip to `Finish installation`_. + +Assign Hostname +=============== + +#. In Advanced Options, select :guilabel:`Assign Hostname`. + +#. In :guilabel:`Hostname`, enter the hostname only (excluding the domain). + + .. note:: + + Hostname does not allow empty spaces. Hostname must start with an + alphanumeric character but may also contain hyphens. Maximum length of + 63 characters. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-28.png + :scale: 100% + :alt: Assign Hostname + + Figure 28: Assign Hostname + +#. Navigate to :kbd:`Confirm` until highlighted. + +#. Select :kbd:`Confirm`. + +Optional: Skip to `Finish installation`_. + +Kernel Command Line +=================== + +For advanced users, |CL| provides the ability to add or remove kernel +arguments. If you want to append a new argument, enter the argument here. +This argument will be used every time you install or update a +new kernel. + +#. In Advanced Options, select :guilabel:`Tab` to highlight + :guilabel:`Kernel Command Line`. + +#. Select :kbd:`Enter`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-29.png + :scale: 100% + :alt: Kernel Command Line + + Figure 29: Kernel Command Line + +#. Choose from the following options. + + * To add arguments, enter the argument in :guilabel:`Add Extra Arguments`. + + * To remove an argument, enter the argument in + :guilabel:`Remove Arguments`. + +#. Select :kbd:`Confirm`. + +Optional: Skip to `Finish installation`_. + +Kernel Selection +================ + +#. Select a kernel option. By default, the latest kernel release is + selected. Native kernel is shown in Figure 30. + +#. To select a different kernel, navigate to it using :guilabel:`Tab`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-30.png + :scale: 100% + :alt: Kernel selection + + Figure 30 Kernel selection + +#. Select :kbd:`Spacebar` to select the desired option. + +#. Navigate to :kbd:`Confirm` and select :kbd:`Enter`. + +Optional: Skip to `Finish installation`_. + +Swupd Mirror +============ + +If you have your own custom mirror of |CL|, you can add its URL. + +#. In Advanced Options, select :guilabel:`Swupd Mirror`. + +#. To add a local swupd mirror, enter a valid URL in :guilabel:`Mirror URL:` + +#. Select :kbd:`Confirm`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-31.png + :scale: 100% + :alt: Swupd Mirror + + Figure 31: Swupd Mirror + +Optional: Skip to `Finish installation`_. + +Automatic OS Updates +==================== + +Automatic OS updates are enabled by default. In the rare case that you +need to disable automatic software updates, follow the onscreen instructions, +shown in Figure 32. + +#. In Advanced Options, select :guilabel:`Automatic OS Updates`. + +#. Select the desired option. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-32.png + :scale: 100% + :alt: Automatic OS Updates + + Figure 32: Automatic OS Updates + +You are returned to the :guilabel:`Main Menu`. + +Save Configuration Settings +=========================== + +#. In Advanced Options, select :guilabel:`Save Configuration Settings`. + +#. A dialog box shows the installation configuration was saved to + :file:`clr-installer.yaml` + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-33.png + :scale: 100% + :alt: Save configuration to YAML file + + Figure 33: Save configuration to YAML file + +#. Use the :file:`clr-installer.yaml` file to install |CL|, with the same + configuration, on multiple targets. + +Finish installation +******************* + +#. When you are satisfied with your installation configuration, navigate to + :guilabel:`Install` and select :kbd:`Enter`. + + .. figure:: /_figures/bare-metal-install-server/bare-metal-install-server-34.png + :scale: 100% + :alt: Select Install + + Figure 34: Select Install + +#. Select :guilabel:`reboot`. + + .. note:: + + If you do not assign an administrative user, upon rebooting, + enter `root` and set the root password immediately. + +#. When the system reboots, remove any installation media present. + +Default partition schema +======================== + +Create partitions per requirements in Table 1. + +.. list-table:: **Table 1. Default partition schema** + :widths: 25, 25, 25, 25 + :header-rows: 1 + + * - FileSystem + - Label + - Mount Point + - Default size + + * - ``VFAT(FAT32)`` + - boot + - /boot + - 150MB + + * - ``ext[234], `XFS`, or f2fs`` + - root + - / + - *Size depends upon use case/desired bundles.* + +.. note:: + + A 64MiB swapfile is generated by default. The default size may be set + manually with the ``--swap-file-size`` command-line option. + +Troubleshooting +*************** + +For Configure Installation Media +================================ + +If a warning message appears that no media or space is available after +entering :guilabel:`Configure Installation Media`: + +- Verify that target media has enough free space. + +- Confirm the USB is properly connected to and mounted on target media. + +- Review the size of existing partitions on the target media: + + - Linux\* OS: :command:`lsblk -a` + - Windows\* OS: :command:`diskpart`, then :command:`list disk` + - macOS\* platform: :command:`diskutil list` + +.. _erase-lvm-troubleshooting-tip: + +Erase LVM Partitions Before Installing |CL| +=========================================== + +If you’re planning to install |CL| on a drive that has LVM partitions, +you must erase them first before using the clr-installer. + +Here is an example of a drive (/dev/sda) with LVMs: + +.. code-block:: console + :emphasize-lines: 6-9 + + NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT + loop0 7:0 0 627.6M 1 loop + sda 8:0 0 335.4G 0 disk + ├─sda1 8:1 0 200M 0 part + ├─sda2 8:2 0 1G 0 part + └─sda3 8:3 0 334.2G 0 part + ├─LVM-root 252:0 0 70G 0 lvm + ├─LVM-home 252:1 0 248.4G 0 lvm + └─LVM-swap 252:2 0 15.7G 0 lvm + +If you do not erase the LVMs first, you will encounter a clr-installer +error like this: + +.. code-block:: console + + root@clr-live~ # clr-installer + + Please report this crash using GitHub Issues: + https://github.com/clearlinux/clr-installer/issues + + Include the following as attachments to enable diagnosis: + /root/pre-install-clr-installer.yaml + /root/clr-installer.log + + You may need to remove any personal data of concern from the attachments. + The Installer will now exit. + exit status 1 + + Error Trace: + errors.Wrap() + errors/errors.go:91 + storage.makeFs() + storage/ops.go:79 + +The quickest and simplest method to erasing the LVMs is to execute these +commands: + +.. code-block:: bash + + sudo sgdisk -Z /dev/ + sudo partprobe + sudo dmsetup remove_all --force + sudo partprobe + +Related topics +************** + +* :ref:`install-configfile` + + +.. _Downloads: https://clearlinux.org/downloads + + diff --git a/_sources/get-started/bootable-usb.rst.txt b/_sources/get-started/bootable-usb.rst.txt new file mode 100644 index 00000000..6a395999 --- /dev/null +++ b/_sources/get-started/bootable-usb.rst.txt @@ -0,0 +1,171 @@ +.. _bootable-usb: + +Create a bootable USB drive using Etcher\* +########################################## + +Use Etcher* software from Balena\* to flash the |CL| image to a USB drive. +An `Advanced: Linux CLI`_ option is also available. + +Prerequisites +************* + +* Download the |CL| Desktop or Server image from the `Downloads`_ page +* Recommended minimum **4GB** USB drive or larger +* Download and install the `Etcher`_ version per your operating system. + +Burn the |CL| image onto a USB drive +==================================== + +.. caution:: + + Burning an image formats the USB drive and destroys all pre-existing + content. Back up your data before proceeding. + +#. Launch Etcher. + + .. rst-class:: dropshadow + + .. figure:: /_figures/bootable-usb/balenaEtcher_Start.PNG + :scale: 100% + :alt: Start screen + + Figure 1: Start screen + +#. Press :guilabel:`Select Image`. + +#. Change directory to where the image resides. + +#. Select the image and click :guilabel:`Open`. + + .. rst-class:: dropshadow + + .. figure:: /_figures/bootable-usb/balenaEtcher_ImageSelect.PNG + :scale: 100% + :alt: In Open, select the image + + Figure 2: In Open, select the image + +#. Plug in the USB drive. + +#. Identify the USB drive or click :guilabel:`Change` to select a + different USB. + + .. note:: + + This shows all USB drives attached to the system. + + .. rst-class:: dropshadow + + .. figure:: /_figures/bootable-usb/balenaEtcher_DriveSlect.PNG + :scale: 100% + :alt: USB drives attached + + Figure 3: USB drives attached + +#. Select the proper device and press :guilabel:`Continue`. + + .. rst-class:: dropshadow + + .. figure:: /_figures/bootable-usb/balenaEtcher_ReadyToFlash.PNG + :scale: 100% + :alt: USB Flash Device selected + + Figure 4: USB Flash Device selected + +#. When ready press the :guilabel:`Flash!` Button. + The dialog shows :guilabel:`Flashing` while in progress. + + .. rst-class:: dropshadow + + .. figure:: /_figures/bootable-usb/balenaEtcher_StartingToFlash.PNG + :scale: 100% + :alt: Starting to flash + + Figure 5: Starting to flash + + .. rst-class:: dropshadow + + .. figure:: /_figures/bootable-usb/balenaEtcher_Flashing.PNG + :scale: 100% + :alt: Flashing, percentage complete + + Figure 6: Flashing, percentage complete + +#. :guilabel:`Flash complete!` shows when the process is finished. + + .. rst-class:: dropshadow + + .. figure:: /_figures/bootable-usb/balenaEtcher_Done.PNG + :scale: 100% + :alt: Flash Complete! + + Figure 7: Flash Complete! + + .. note:: + + The process may take more than a few minutes. When the process completes, close BalenaEtcher. + +Advanced: Linux CLI +=================== + +#. Open a Terminal window. + +#. Change directory to where the image resides. + +#. Plug in the USB drive. + +#. Identify all drives attached to the system. In the example output below, there are 3 drives (`/dev/sda`, `/dev/sdb`, and `/dev/sdc`) attached, where `/dev/sda` is the primary drive and the remaining are USB drives. + + .. code-block:: bash + + lsblk -po NAME,SIZE,VENDOR,MODEL,TRAN,TYPE,PARTLABEL,MOUNTPOINT + + Example output: + + .. code-block:: console + + NAME SIZE VENDOR MODEL TRAN TYPE PARTLABEL MOUNTPOINT + /dev/sda 119.2G ATA SAMSUNG_MZ7PC128HAFU-000 sata disk + ├─/dev/sda1 450M part Basic data partition + ├─/dev/sda2 100M part EFI system partition + ├─/dev/sda3 16M part Microsoft reserved partition + ├─/dev/sda4 97.2G part Basic data partition + ├─/dev/sda5 142M part EFI + ├─/dev/sda6 245M part linux-swap [SWAP] + └─/dev/sda7 21.1G part / / + /dev/sdb 7.5G General UDisk usb disk + └─/dev/sdb1 7.5G part Microsoft Basic Data /run/media/clear/CENA_X64FRE + /dev/sdc 15G Patriot_Memory usb disk + └─/dev/sdc1 15G part /run/media/clear/U + + .. note:: + + Some Linux distros may automatically mount a USB drive when it is plugged in. + +#. Unmount the USB drive you want to use before burning an image onto it. + Use the :command:`umount` command followed by the device identifier/partition. For example, to unmount all ``/dev/sdc`` partitions: + + .. code-block:: bash + + sudo umount /dev/sdc* + +#. Burn the image onto the USB drive. This example burns an image onto + ``/dev/sdc``. The device name of the USB may vary. + + .. code-block:: bash + + sudo dd if=./clear-[version number]-live-[desktop | server].iso of=/dev/sdc oflag=sync bs=4M status=progress + +Eject the |CL| image USB drive +============================== + +.. caution:: + + If you do not properly unmount the USB drive before removing it, it may cause file system checksum errors in it. If this happens, burn the image again, ensuring all the USB drive partitions are unmounted first before removing drive. + +#. Unmount the USB per your OS instructions. + +#. Then eject the USB. + +.. _Downloads: https://clearlinux.org/downloads +.. _Etcher: https://www.balena.io/etcher/ diff --git a/_sources/get-started/cloud-install/aws-web.rst.txt b/_sources/get-started/cloud-install/aws-web.rst.txt new file mode 100644 index 00000000..9d4312c9 --- /dev/null +++ b/_sources/get-started/cloud-install/aws-web.rst.txt @@ -0,0 +1,288 @@ +.. _aws-web: + +|CL-ATTR| on Amazon Web Services\* +################################## + +This tutorial explains how to create and launch a |CL| +:abbr:`AMI (Amazon Machine Image)` instance from the +:abbr:`AWS\* (Amazon Web Services)` console and complete the following tasks: + +#. Locate and select the |CL| OS Basic AMI in the AWS Marketplace. +#. Create a new public and private key pair to allow you to connect to your + |CL| instance securely. +#. Launch the new |CL| instance and connect to it. +#. Update your instance of |CL| using the :command:`swupd` command. +#. Stop the |CL| instance. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +This tutorial assumes the following statements are true: + +* You are using a linux-based system to access AWS and can run :command:`SSH` + to access the remote |CL| AWS image. +* Your browser puts downloaded files in the :file:`$HOME/Downloads` + directory. +* You have already set up an AWS user account and logged into the AWS + console. + +.. note:: + This tutorial uses a |CL| AMI t2.micro instance that is eligible for the + AWS free tier. To learn more about AWS and setting up an account, visit the + AWS website at http://aws.amazon.com. + +Locate, select, and launch the |CL| Basic AMI +********************************************* + +#. Start from your main AWS services console menu in your browser and select + the :guilabel:`EC2` text as shown in Figure 1: + + .. figure:: /_figures/aws/aws-web-1.png + :scale: 50 % + :alt: AWS Console + + Figure 1: :guilabel:`AWS Console` + + This selection brings up your :guilabel:`EC2 Dashboard` menu. + +#. To create a new |CL| instance from the :guilabel:`EC2 Dashboard` menu, + select the :guilabel:`Launch Instance` button as shown in Figure 2: + + .. figure:: /_figures/aws/aws-web-2.png + :scale: 50 % + :alt: EC2 Dashboard + + Figure 2: :guilabel:`EC2 Dashboard` + + This selection takes you to the + :guilabel:`Step 1: Choose an Amazon Machine Image (AMI)` menu. + +#. To find the :guilabel:`Clear Linux OS Basic` AMI in the + :guilabel:`Step 1: Choose an Amazon Machine Image (AMI)` menu, do the + following: + + #. In the lefthand navigation window, select the + :guilabel:`AWS Marketplace` menu item to bring up the search bar to + :guilabel:`Search AWS Marketplace Products`. + + #. In the search bar, type "clear linux os" and press the :kbd:`Enter` key + to search for and locate the :guilabel:`Clear Linux OS Basic` AMI. + + #. Select the :guilabel:`Clear Linux OS Basic` AMI by clicking the + :guilabel:`Select` button as shown in Figure 3: + + .. figure:: /_figures/aws/aws-web-3.png + :scale: 50 % + :alt: Step 1: Choose AMI + + Figure 3: :guilabel:`Step 1: Choose AMI` + + #. A pop-up dialog box appears showing you more information about the + :guilabel:`Clear Linux OS Basic` AMI along with the pricing details for + running |CL| on different platform configurations as shown in Figure 4. + Select the :guilabel:`Continue` button. + + .. figure:: /_figures/aws/aws-web-4.png + :scale: 50 % + :alt: Clear Linux OS Basic + + Figure 4: :guilabel:`Clear Linux OS Basic` + +#. The :guilabel:`Choose Instance Type` menu appears as shown in Figure 5. + + .. figure:: /_figures/aws/aws-web-5.png + :scale: 50 % + :alt: Choose an Instance Type + + Figure 5: :guilabel:`Choose an Instance Type` + + Select the :guilabel:`t2.micro` type by clicking the box on the left side + of the instance and then select the :guilabel:`Review and Launch` button to + move to the :guilabel:`Step 7: Review the Instance Launch` menu. + + .. note:: + + You can configure the instance details, add additional storage, add + tags, and configure the security group before selecting the + :guilabel:`Review and Launch` button if you want to further customize + this |CL| instance. + +#. The :guilabel:`Step 7: Review the Instance Launch` menu, shown in Figure 6, + allows you to :guilabel:`Cancel` the process, return to + the :guilabel:`Previous` screen to change the configuration + or :guilabel:`Launch` the instance defined. + + .. figure:: /_figures/aws/aws-web-6.png + :scale: 50 % + :alt: Step 7: Review the Instance Launch + + Figure 6: :guilabel:`Step 7: Review the Instance Launch` + + #. Select the :guilabel:`Launch` button. A dialog box appears, as shown in + Figure 7, asking you to + :guilabel:`Select an existing key pair or create a new pair`. + + .. figure:: /_figures/aws/aws-web-7.png + :scale: 50 % + :alt: Select an existing key pair or create a new pair + + Figure 7: :guilabel:`Select an existing key pair or create a new pair` + + #. Select the :guilabel:`Create a new key pair` option. + + #. For the :guilabel:`Key pair name` field, enter `AWSClearTestKey`. + + #. Select the :guilabel:`Download Key Pair` button to download the + :file:`AWSClearTestKey.pem` to your browser's defined + :file:`Downloads` directory. + + #. When the file finishes downloading, select the + :guilabel:`Launch Instances` button to proceed to the + :guilabel:`Launch Status` menu shown in Figure 8. + + .. figure:: /_figures/aws/aws-web-8.png + :scale: 50 % + :alt: Launch Status + + Figure 8: :guilabel:`Launch Status` + + #. Once the :guilabel:`Launch Status` page changes to what is shown in + Figure 9, select the :guilabel:`View Instances` button to view your + :guilabel:`Instances` dashboard. + + .. figure:: /_figures/aws/aws-web-9.png + :scale: 50 % + :alt: View Instance + + Figure 9: :guilabel:`View Instance` + +Connect to your Clear Linux OS basic instance +********************************************* + +Your :guilabel:`Instances` Dashboard is shown in Figure 10 with the new |CL| +OS basic instance already selected and in the running state. If there are +other instances available, they are also listed but not selected. + +.. figure:: /_figures/aws/aws-web-10.png + :scale: 50 % + :alt: Instance Dashboard + + Figure 10: :guilabel:`Instance Dashboard` + +#. To connect to your running instance, click the :guilabel:`Connect` button + located at the top of your dashboard. AWS brings up the pop-up dialog + box shown in Figure 11 describing how to connect to your running instance. + +.. _fig-aws-web-11: + +.. figure:: /_figures/aws/aws-web-11.png + :scale: 50 % + :alt: Connect to Your Instance + + Figure 11: :guilabel:`Connect to Your Instance` + +#. Open a terminal on your system. You should be in your :file:`$HOME` + directory. + +#. Copy the previously downloaded keyfile from the :file:`Downloads` + directory to the current directory. + + .. code-block:: console + + cp Downloads/AWSClearTestKey.pem . + +#. Change the attributes of the :file:`AWSClearTestKey.pem` using the + :command:`chmod` command as instructed in the dialog box shown in Figure + 11. + + .. code-block:: console + + chmod 400 AWSClearTestKey.pem + +#. Copy the text highlighted in the :guilabel:`Example:` section that is + shown in :ref:`figure 11`. Paste the copied text into your + terminal, change the text before the `@` sign to the username `clear`, and + press the :kbd:`Enter` key to execute the command. + + .. code-block:: console + + ssh -i "AWSClearTestKey.pem" clear@ec2-34-209-39-184.us-west-2.compute.amazonaws.com + +#. A message appears on the terminal stating the authenticity of the host + can't be established and prompts you with the message: + + .. code-block:: console + + The authenticity of host 'ec2-34-209-39-184.us-west-2.compute.amazonaws.com (34.209.39.184)' can't be established. + ECDSA key fingerprint is SHA256:LrziT5Ar66iBTfia8qmiIsrfBUm/UGam76U8bDR6yJc. + Are you sure you want to continue connecting (yes/no)? + +#. Type `yes` and press the :kbd:`Enter` key. Another warning is printed to + the terminal and you are now at the command prompt of your new |CL| + instance. + + .. code-block:: console + + Warning: Permanently added 'ec2-34-209-39-184.us-west-2.compute.amazonaws.com,34.209.39.184' (ECDSA) to the list of known hosts. + clear@clr-96a8565d0ca54b0c80364a1e5e7b0f88 ~ $ + +Update the |CL| instance +************************ + +Run the :command:`sudo swupd update` command to update the operating +system as shown in Figure 12: + +.. figure:: /_figures/aws/aws-web-12.png + :scale: 50 % + :alt: sudo swupd update + + Figure 12: :guilabel:`sudo swupd update` + +In this example, we updated from version 18940 to 19100. + +Stop the |CL| instance +********************** + +When you are finished using your AWS |CL| instance, you must stop it using +the :guilabel:`Instances` dashboard to stop accruing charges. Complete the +following steps from the :guilabel:`Instances` dashboard to stop your AWS |CL| +instance from running. + +#. Select the :guilabel:`Actions` button to bring up a pull-down menu. + +#. Select the :guilabel:`Instance State` menu item to expand the options. + +#. Select :guilabel:`Stop` menu item to shut down the running instance. + + Figure 13 illustrates these steps. + + .. figure:: /_figures/aws/aws-web-13.png + :scale: 50 % + :alt: Stop Instance + + Figure 13: :guilabel:`Stop Instance` + +#. A pop-up dialog box appears warning you that any ephemeral storage of + your instance will be lost. Select the :guilabel:`Yes, Stop` button to stop + your |CL| instance. + +.. figure:: /_figures/aws/aws-web-14.png + :scale: 50 % + :alt: Stop Instances + + Figure 14: :guilabel:`Stop Instances` + +Congratulations! You are up and running with |CL| on AWS. To see what you +can do with your |CL| instance, visit our :ref:`tutorials ` +section for examples on using your |CL| system. + +Related topics +************** + +* :ref:`azure` +* :ref:`gce` +* :ref:`clr-digitalocean` diff --git a/_sources/get-started/cloud-install/azure.rst.txt b/_sources/get-started/cloud-install/azure.rst.txt new file mode 100644 index 00000000..a6cc1f20 --- /dev/null +++ b/_sources/get-started/cloud-install/azure.rst.txt @@ -0,0 +1,548 @@ +.. _azure: + +|CL-ATTR| on Microsoft\* Azure\* +################################ + +This tutorial shows how to install and use the Azure :abbr:`CLI (Command Line +Interface)` on |CL|. The Azure CLI offers the ability to create and +manage resources in MS Azure from the command line. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +|CL| is available for you to use in the Microsoft Azure marketplace and +is offered with three different images, also known as a +:abbr:`SKU (Stock Keeping Unit)`. + +* |CL| Basic - This SKU consists of a bare-bones system which can be used as a + starting point for those wanting to explore and build out a system with + additional software bundles of their choosing. + +* |CL| Containers - This SKU comes with the containers-basic software bundle + already installed. + +* |CL| Machine-learning - This SKU comes pre-loaded with popular open source + tools for developing machine learning applications. + +Sign in at the `Azure portal`_ to access these images from the Azure dashboard, +or use the MS Azure CLI 2.0. If you do not already have an account set up with +MS Azure, you can sign up for a `MS Azure free account`_ to access the +|CL| :abbr:`VM (Virtual Machine)` images. + +Prerequisites +************* + +To use the MS Azure CLI 2.0 on your |CL| system, your system must have the +following packages installed: + +* Python 2.7 or later + +* libffi + +* OpenSSL 1.0.2 + +You can check to see which versions you have installed on your system by +running the individual commands as follows: + +.. code-block:: bash + + python --version + +.. code-block:: console + + python 2.7.12 + +.. code-block:: bash + + openssl version + +.. code-block:: console + + OpenSSL 1.0.2n 7 Dec 2017 + +.. code-block:: bash + + ls -l /usr/lib64/libffi* + +.. code-block:: console + + lrwxrwxrwx 1 root root 15 Jan 12 2017 /usr/lib64/libffi.so.6 -> libffi.so.6.0.4 + -rwxr-xr-x 1 root root 38792 Jan 12 2017 /usr/lib64/libffi.so.6.0.4 + +If you do not have these packages installed on your |CL| system, install the +sysadmin-basic software bundle using the :command:`swupd` command: + +.. code-block:: bash + + sudo swupd bundle-add sysadmin-basic + +.. note:: + + These instructions are for installing the MS Azure CLI 2.0 tools on a |CL| + system. If you are installing the CLI on another platform, follow the + instructions in the `MS Azure Install Azure CLI tutorial`_ for your + specific operating system. + +Install MS Azure CLI 2.0 on |CL| +******************************** + +#. To install the MS Azure CLI 2.0 on |CL|, use the :command:`curl` command as + follows: + + .. code-block:: bash + + curl -L https://aka.ms/InstallAzureCli | bash + + If you get an error message from :command:`curl` related to the -L + parameter or an error message is generated that includes the text "Object + Moved", use the full URL instead of the aka.ms redirect address: + + .. code-block:: bash + + curl https://azurecliprod.blob.core.windows.net/install | bash + +#. The installation script begins and prompts you several times during + execution for information. + + .. note:: + + The console output from the script displays your username instead of the + **[user]** variable shown on this tutorial. + + .. code-block:: console + + ===> In what directory would you like to place the install? (leave blank to use '/home/[user]/lib/azure-cli'): + + Press the :kbd:`Enter` key to accept the default or chose another + directory in which to install the MS Azure CLI 2.0 tools. + + .. code-block:: console + + ===> In what directory would you like to place the 'az' executable? (leave blank to use '/home/[user]/bin'): + + Press the :kbd:`Enter` key to accept the default or chose another + directory in which to install the :command:`az` executable. + +#. The installation downloads and builds all the required tools. When it is + complete, it prompts you: + + .. code-block:: console + + ===> Modify profile to update your $PATH and enable shell/tab completion now? (Y/n): Y + + Type :kbd:`Y` and press the :kbd:`Enter` key to allow this modification. + + .. code-block:: console + + ===> Enter a path to an rc file to update (leave blank to use '/home/[user]/.bashrc'): + + Press the :kbd:`Enter` key to accept the default or enter the pathname to + your :file:`.bashrc` file. The installation completes with the final output + shown below: + + .. code-block:: console + + -- Backed up '/home/[user].bashrc' to '/home/[user]/.bashrc.backup' + -- Tab completion set up complete. + -- If tab completion is not activated, verify that '/home/[user]/.bashrc' is sourced by your shell. + -- + -- ** Run `exec -l $SHELL` to restart your shell. ** + -- + -- Installation successful. + -- Run the CLI with /home/[user]/bin/az --help + +#. When the installation program finishes, you must restart your shell for + the changes to take effect. When the installation is successful, run the + following command to restart your shell. + + .. code-block:: bash + + exec -l $SHELL + +With the MS Azure CLI 2.0 executable successfully built and installed, run +the :command:`az` command. + +.. code-block:: bash + + az + +The output from the :command:`az` command is shown below: + +.. code-block:: console + + + /\ + / \ _____ _ _ __ ___ + / /\ \ |_ / | | | \'__/ _ \ + / ____ \ / /| |_| | | | __/ + /_/ \_\/___|\__,_|_| \___| + + + Welcome to the cool new Azure CLI! + + Here are the base commands: + + account : Manage Azure subscription information. + acr : Manage Azure Container Registries. + acs : Manage Azure Container Services. + ad : Synchronize on-premises directories and manage Azure Active Directory + resources. + advisor : (PREVIEW) Manage Azure Advisor. + aks : Manage Kubernetes clusters. + appservice : Manage App Service plans. + backup : Commands to manage Azure Backups. + batch : Manage Azure Batch. + batchai : Batch AI. + billing : Manage Azure Billing. + cdn : Manage Azure Content Delivery Networks (CDNs). + cloud : Manage registered Azure clouds. + cognitiveservices: Manage Azure Cognitive Services accounts. + configure : Display and manage the Azure CLI 2.0 configuration. This command is + interactive. + consumption : Manage consumption of Azure resources. + container : (PREVIEW) Manage Azure Container Instances. + cosmosdb : Manage Azure Cosmos DB database accounts. + disk : Manage Azure Managed Disks. + dla : (PREVIEW) Manage Data Lake Analytics accounts, jobs, and catalogs. + dls : (PREVIEW) Manage Data Lake Store accounts and filesystems. + eventgrid : Manage Azure Event Grid topics and subscriptions. + extension : Manage and update CLI extensions. + feature : Manage resource provider features. + feedback : Loving or hating the CLI? Let us know! + find : Find Azure CLI commands. + functionapp : Manage function apps. + group : Manage resource groups and template deployments. + image : Manage custom virtual machine images. + interactive : Start interactive mode. + iot : (PREVIEW) Manage Internet of Things (IoT) assets. + keyvault : Safeguard and maintain control of keys, secrets, and certificates. + lab : Manage Azure DevTest Labs. + lock : Manage Azure locks. + login : Log in to Azure. + logout : Log out to remove access to Azure subscriptions. + managedapp : Manage template solutions provided and maintained by Independent Software + Vendors (ISVs). + monitor : Manage the Azure Monitor Service. + mysql : Manage Azure Database for MySQL servers. + network : Manage Azure Network resources. + policy : Manage resource policies. + postgres : Manage Azure Database for PostgreSQL servers. + provider : Manage resource providers. + redis : Access to a secure, dedicated Redis cache for your Azure applications. + reservations : Manage Azure Reservations. + resource : Manage Azure resources. + role : Manage user roles for access control with Azure Active Directory and service + principals. + sf : Manage and administer Azure Service Fabric clusters. + snapshot : Manage point-in-time copies of managed disks, native blobs, or other + snapshots. + sql : Manage Azure SQL Databases and Data Warehouses. + storage : Manage Azure Cloud Storage resources. + tag : Manage resource tags. + vm : Provision Linux or Windows virtual machines. + vmss : Manage groupings of virtual machines in an Azure Virtual Machine Scale Set + (VMSS). + webapp : Manage web apps. + +Log into your Microsoft Azure account +************************************* + +#. With the :command:`az` command properly installed and functional, login to + your MS Azure account using the :command:`az login` command shown below: + + .. code-block:: bash + + az login + + The output from this command is: + + .. code-block:: console + + To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code BQ9MG442B to authenticate. + +#. Open your browser and enter the page `https://aka.ms/devicelogin` as shown + in figure 1: + + .. figure:: /_figures/azure/azure-1.png + :scale: 50 % + :alt: Microsoft Device Login + + Figure 1: :guilabel:`Microsoft Device Login` + +#. Enter the code `BQ9MG442B` to authenticate your device as shown in figure + 2. The code `BQ9MG442B` is a random authentication code generated for each + session login and will be different each time you log into MS Azure using + the :command:`az login` command. + + .. figure:: /_figures/azure/azure-2.png + :scale: 50 % + :alt: Microsoft Device Login - Azure CLI + + Figure 2: :guilabel:`Microsoft Device Login - Azure CLI` + +#. Once you enter the authentication code, the website displays a screen to + enter your existing Microsoft Azure credentials. + +#. Log in with your existing MS Azure account credentials. The + browser screen shows you have signed into the Microsoft Cross-platform + Command Line Interface application on your device, as shown in figure 3. + You can close the page. + + .. figure:: /_figures/azure/azure-3.png + :scale: 50 % + :alt: Microsoft Azure Cross-platform CLI + + Figure 3: :guilabel:`Microsoft Azure Cross-platform CLI` + +The MS Azure CLI 2.0 interface is now active using your existing MS Azure +account credentials. + +Create a MS Azure resource group +******************************** + +To learn more about MS Azure resource groups, visit the +`Azure Resource Manager overview`_ for an overview and detailed description +of resources within MS Azure. + +#. To create a new resource group, run the :command:`az group create ...` + command shown below to create a resource group named `ClearResourceGroup` + using the `-n` parameter and locate it in the `westus` region using the + `-l` parameter. + + .. code-block:: bash + + az group create -n ClearResourceGroup -l westus + +#. When the command has completed, the output from this command is similar to + the following: + + .. code-block:: console + + { + "id": "/subscriptions/{unique-id}/resourceGroups/ClearResourceGroup", + "location": "westus", + "managedBy": null, + "name": "ClearResourceGroup", + "properties": { + "provisioningState": "Succeeded" + }, + "tags": null + } + +Create and log into the |CL| virtual machine +******************************************** + +For this tutorial, we are using the |CL| Basic SKU for our VM. + +#. To locate the available |CL| Basic VM SKU images in the MS Azure + marketplace, run the :command:`az vm image ...` command: + + .. code-block:: bash + + az vm image list --offer clear-linux --sku basic --all --output table + + This command may take some time to finish. The output lists all available + |CL| Basic images available in the Microsoft Azure marketplace as shown + below: + + .. code-block:: console + + Offer Publisher Sku Urn Version + -------------- ------------------- ---------------- ------------------------------------------------------------- --------- + clear-linux-os clear-linux-project basic clear-linux-project:clear-linux-os:basic:15780.0.0 15780.0.0 + clear-linux-os clear-linux-project basic clear-linux-project:clear-linux-os:basic:15960.0.0 15960.0.0 + clear-linux-os clear-linux-project basic clear-linux-project:clear-linux-os:basic:16050.0.0 16050.0.0 + clear-linux-os clear-linux-project basic clear-linux-project:clear-linux-os:basic:16150.0.0 16150.0.0 + clear-linux-os clear-linux-project basic clear-linux-project:clear-linux-os:basic:16500.0.0 16500.0.0 + clear-linux-os clear-linux-project basic clear-linux-project:clear-linux-os:basic:16810.0.0 16810.0.0 + clear-linux-os clear-linux-project basic clear-linux-project:clear-linux-os:basic:18080.0.0 18080.0.0 + clear-linux-os clear-linux-project basic clear-linux-project:clear-linux-os:basic:18620.0.0 18620.0.0 + clear-linux-os clear-linux-project basic clear-linux-project:clear-linux-os:basic:18860.0.0 18860.0.0 + + .. note:: + + The output list shows current offerings. New versions are added to the + MS Azure marketplace all the time. To reference the latest version of an + image, you can use the version label `latest` when specifying an image. + +#. The information shown in the `Urn` column lists the + `Publisher:Offer:Sku:Version` for each image available. This is the + information we want to create the |CL| Basic VM. Since we are creating a + |CL| Basic VM, highlight the `clear-linux-project:clear-linux-os:basic:` + string and copy it to your clipboard. Use the label + `latest` instead of referencing a specific version. + +#. Create the new |CL| Basic VM. Run the :command:`az vm create ...` + command using the URN `:clear-linux-project:clear-linux-os:basic:latest` + that we copied to the clipboard on the previous step. + + .. code-block:: bash + + az vm create --resource-group ClearResourceGroup --name ClearVM --image clear-linux-project:clear-linux-os:basic:latest --generate-ssh-keys + + .. note:: + + If you have already defined your public/private SSH key pair and they + are stored in your :file:`$HOME/.ssh` directory, you do not need to + include the *--generate-ssh-keys* option. + + The output from this command will look similar to this output, where + [user] is your user name: + + .. code-block:: console + + SSH key files '/home/[user]/.ssh/id_rsa' and '/home/[user]/.ssh/id_rsa.pub' have been generated under ~/.ssh to allow SSH access to the VM. If using machines without permanent storage, back up your keys to a safe location. + + running... + + { + "fqdns": "", + "id": "/subscriptions/{unique-id}/resourceGroups/ClearResourceGroup/providers/Microsoft.Compute/virtualMachines/ClearVM", + "location": "westus", + "macAddress": "00-0D-3A-37-C7-59", + "powerState": "VM running", + "privateIpAddress": "10.0.0.4", + "publicIpAddress": "13.91.4.245", + "resourceGroup": "ClearResourceGroup", + "zones": "" + } + + Take note of the public IP address from the output. + +#. Login into the new |CL| Basic VM, run the :command:`ssh` command with the + public IP address obtained from the previous step as shown: + + .. code-block:: bash + + ssh [user]@13.91.4.245 + + You may see the following message about the authenticity of the host. If + this appears, type `yes` to proceed connecting to your new |CL| VM. + + .. code-block:: console + + The authenticity of host '13.91.4.245 (13.91.4.245)' can't be established. + RSA key fingerprint is SHA256:{unique-number}. + Are you sure you want to continue connecting (yes/no)? yes + Warning: Permanently added '13.91.4.245' (RSA) to the list of known hosts. + + [user]@ClearVM ~ $ + + You are now logged into your new |CL| VM as [user], where [user] is your + user name. To check which software bundles are included with + this VM image, run the :command:`sudo swupd bundle-list` command inside the + VM: + + .. code-block:: bash + + sudo swupd bundle-list + + The output shown should be similar to: + + .. code-block:: console + + swupd-client bundle list 3.14.1 + Copyright (C) 2012-2017 Intel Corporation + + bootloader + editors + kernel-hyperv + network-basic + openssh-server + os-cloudguest-azure + os-core + os-core-update + perl-basic + python-basic + python3-basic + storage-utils + sysadmin-basic + Current OS version: 19600 + + When you are finished using your new |CL| VM, type :command:`exit` to close + the :command:`SSH` terminal and logout. + +Stop and deallocate the |CL| VM and resources +********************************************* + +When you finish using your new |CL| instance, you must stop the VM and +deallocate the resources in your resource group. If you only stop a VM, the OS +image shuts down but the resources associated with it in your resource group +remain allocated and incurring charges. For instance, if you stop and then +later start the VM using the :command:`az vm start...` without deallocating +the resources, the IP address is retained and you can access the VM using that +same IP address. To release the resources associated with the VM and stop +incurring charges for them, you must deallocate the resources as well. + +#. At the command prompt, enter the :command:`az vm stop...` command as + follows: + + .. code-block:: bash + + az vm stop --resource-group ClearResourceGroup --name ClearVM + + This will stop the VM and then output text similar to: + + .. code-block:: console + + { + "endTime": "2017-12-13T23:04:02.346676+00:00", + "error": null, + "name": "{unique-name}", + "startTime": "2017-12-13T23:03:59.018536+00:00", + "status": "Succeeded" + } + +#. Once the VM stops, deallocate the VM resources to stop incurring + charges for the |CL| instance. Enter the following command: + + .. code-block:: console + + az vm deallocate --resource-group ClearResourceGroup --name ClearVM + +**Congratulations!** You are up and running with |CL| on MS Azure using the +Azure CLI 2.0 command line tools. + +Next steps +********** + +To see use cases you can fulfill with your |CL| instance, visit our +:ref:`tutorials ` section. + +For additional information visit the |CL| +`Azure Partner Mini Case Study`_ and the `Azure Partner Datasheet`_. + +To learn more about the MS Azure CLI 2.0 tool and options that are available, +visit the `MS Azure documentation and tutorials`_ website. + +Related topics +************** + +* :ref:`gce` +* :ref:`aws-web` +* :ref:`clr-digitalocean` + +.. _`Azure Portal`: + https://portal.azure.com + +.. _`MS Azure free account`: + https://azure.microsoft.com/en-us/free/ + +.. _`MS Azure documentation and tutorials`: + https://docs.microsoft.com/en-us/cli/azure/overview?view=azure-cli-latest + +.. _`MS Azure Install Azure CLI tutorial`: + https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest + +.. _`Azure Resource Manager overview`: + https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-overview + +.. _Azure Partner Datasheet: + http://download.microsoft.com/download/D/9/E/D9E22342-96D9-4455-BB15-99A1AF514DDD/Microsoft%20Azure%20Partner%20Datasheet%20-%20Intel%20Clear%20Linux.pdf + +.. _Azure Partner Mini Case Study: + http://download.microsoft.com/download/D/9/E/D9E22342-96D9-4455-BB15-99A1AF514DDD/Microsoft%20Azure%20Partner%20Mini%20Case%20Study%20-%20Intel%20Clear%20Linux.pdf diff --git a/_sources/get-started/cloud-install/digitalocean.rst.txt b/_sources/get-started/cloud-install/digitalocean.rst.txt new file mode 100644 index 00000000..142b01bc --- /dev/null +++ b/_sources/get-started/cloud-install/digitalocean.rst.txt @@ -0,0 +1,364 @@ +.. _clr-digitalocean: + +|CL-ATTR| on DigitalOcean\* +########################### + +This guide explains how to import a |CL-ATTR| image to `DigitalOcean`_ +and then deploy a VM instance. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* Set up a DigitalOcean account. + +* Create an SSH key on your client system that you will use to remote + into the VM. You can follow the `DigitalOcean's SSH key creation guide`_. + +Add |CL| Image to DigitalOcean +****************************** + +Before you can deploy a |CL| instance on DigitalOcean, you need to add +an image since it's currently not available in its marketplace. +You can use our pre-built image or you can build your own custom image. + +Use pre-built image +=================== + +.. note:: + Our cloud images (`clear--digitalocean.img.gz`) for + DigitalOcean are considered **Beta** until we finish setting up our + automated testing of the images against the DigitalOcean environment. + Apart from the initial version, `clear-31870-digitalocean.img.gz`_, we + cannot guarantee that future versions and updates to the initial + version is problems-free. + +.. bktan8 - commented out until the images are fully validated by DevOps + and go live on official Downloads page. + Go to the |CL| `downloads` page and copy the URL for the + **Cloud Guest Legacy** image. See Figure 1. + figure:: ../../_figures/digitalocean/01-digitalocean.png + :scale: 100 % + :alt: Cloud Guest Legacy image + Figure 1: Cloud Guest Legacy image + +#. Copy the URL for `clear-31870-digitalocean.img.gz`_. + +#. Skip to the `Upload image`_ section. + +Build custom image +================== + +For this method, you need a |CL| system to generate an image using +the *clr-installer* tool. + +#. Add the *clr-installer* and *gzip* bundles. + + .. code-block:: bash + + sudo swupd bundle-add clr-installer gzip + +#. Create an image configuration YAML file. + See `Installer YAML Syntax`_ for more information on the clr-installer + configuration YAML syntax. + + .. code-block:: bash + + cat > clear-digitalocean.yaml << EOF + #clear-linux-config + + # switch between aliases if you want to install to an actual block device + # i.e /dev/sda + block-devices: [ + {name: "bdevice", file: "clear-digitalocean.img"} + ] + + targetMedia: + - name: \${bdevice} + size: "800M" + type: disk + children: + - name: \${bdevice}1 + fstype: ext4 + options: -O ^64bit + mountpoint: / + size: "800M" + type: part + + bundles: [ + bootloader, + openssh-server, + os-cloudguest, + os-core, + os-core-update, + systemd-networkd-autostart + ] + + autoUpdate: false + postArchive: false + postReboot: false + telemetry: false + legacyBios: true + + keyboard: us + language: en_US.UTF-8 + kernel: kernel-kvm + + version: 0 + EOF + + The settings that are required in order to make the image + work on DigitalOcean are: + + * *os-cloudguest* bundle: Allows DigitalOcean to provision the + image with settings such as hostname, resource (CPU, memory, + storage) sizing, and user creation. + * *legacyBios: true*: The image need to support legacy BIOS to boot + on DigitalOcean. + +#. Generate the image. + + .. code-block:: bash + + sudo clr-installer -c clear-digitalocean.yaml + + The output should be :file:`clear-digitalocean.img`. + +#. Compress the image with *gzip* to save bandwidth and upload time. + + .. code-block:: bash + + gzip clear-digitalocean.img + + The output should be :file:`clear-digitalocean.img.gz`. + + .. note:: + + *bzip2* is the other compression format DigitalOcean accepts. + +Upload image +============ + +#. On DigitalOcean's website, go to :menuselection:`MANAGE --> Images + --> Custom Images`. + + See Figure 1. + + .. figure:: ../../_figures/digitalocean/01-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Upload custom images + + Figure 1: DigitalOcean - Upload custom images + +#. Select an upload method. + + * To import a pre-built image from |CL| `downloads`_, click + :guilabel:`Import via URL`, paste the URL, and click :guilabel:`Next`. + + See Figure 2. + + .. figure:: ../../_figures/digitalocean/02-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Import via URL + + Figure 2: DigitalOcean - Import via URL + + * To import your custom image, click :guilabel:`Upload Image` + and select the image from your client system. + +#. Set the :guilabel:`DISTRIBUTION` type as :guilabel:`Unknown`. + + See Figure 3. + + | + +#. Choose your preferred datacenter region. + +#. Click :guilabel:`Upload Image`. + Wait for the upload to finish before proceeding to the next section. + + .. figure:: ../../_figures/digitalocean/03-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Set image distribution type, region, tag + + Figure 3: DigitalOcean - Set image distribution type, region, tag + +Create and Deploy a |CL| Instance +********************************* + +#. On DigitalOcean's website, go to :menuselection:`MANAGE --> Droplets` + and then click :guilabel:`Create Droplet`. + + See Figure 4. + + .. figure:: ../../_figures/digitalocean/04-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Create Droplet + + Figure 4: DigitalOcean - Create Droplet + +#. Under :guilabel:`Choose an image`, select :guilabel:`Custom images`. + + See Figure 5. + + | + +#. Select your uploaded |CL| image. + + .. figure:: ../../_figures/digitalocean/05-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Choose custom image + + Figure 5: DigitalOcean - Choose custom image + +#. Under :guilabel:`Choose a plan`, select your preferred plan. + + See Figure 6. + + .. figure:: ../../_figures/digitalocean/06-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Choose plan + + Figure 6: DigitalOcean - Choose plan + +#. Under :guilabel:`Choose a datacenter region`, select the region you + want the instance deployed to. + + See Figure 7. + + .. figure:: ../../_figures/digitalocean/07-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Choose datacenter region + + Figure 7: DigitalOcean - Choose datacenter region + +#. Assign SSH key to default *clear* user. + + By default, the user *clear* will be added to the instance and + an SSH key must be assigned to this account. + + a. Under :guilabel:`Authentication`, select :guilabel:`SSH keys` and + click :guilabel:`New SSH Key`. + + See Figure 8. + + .. figure:: ../../_figures/digitalocean/08-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Add SSH key + + Figure 8: DigitalOcean - Add SSH key + + #. Copy and paste your SSH public key in the :guilabel:`SSH key content` + text field. + + See Figure 9. + + | + + #. Give a name for the SSH key. + + #. Click :guilabel:`Add SSH Key`. + + .. figure:: ../../_figures/digitalocean/09-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Add public SSH key + + Figure 9: DigitalOcean - Add public SSH key + + .. note:: + + If you need to add additional users to the instance, you can do that + wth a YAML-formatted *cloud-config* user data script. + For more information on cloud-config scripting for |CL|, see our + subset implementation of cloud-init called `micro-config-drive`_. + + a. Under :guilabel:`Select additional options`, + select :guilabel:`User data`. + + #. Add your YAML-formatted *cloud-config* user data in the field below. + Here is a simple example: + + .. code-block:: console + + #cloud-config + + users: + - name: foobar + gecos: Foo B. Bar + homedir: /home/foobar + ssh-authorized-keys: + - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC65OihS4UP27xKOpqKWgT9 + mgUNwEqhUEpTGGvopjT65Y/KU9Wfj6EYsdGzbHHcMUhFSTxAUAV4POH5d0LR + MzI7sXMe528eCmpm2fTOHDDkVrurP/Jr2bjB9IrfSMkBYS8uRd603xNg/RDq + EH3XzVeEDdEAxoej0mzsJ2UkQSBi+PD1J7JeCbX2lsb55x2yWzaUa+BTai7+ + /TU4UabTRDtFTiXhx2rImSSguofDISVll6W5TTzbGmHdoEI+8DIAFU66ZgC9 + SzL75LQi1YAWlj5XG+dXhN6Ev6KFM34odvWdxeCj0jcx5UIXcieBfOuLujEH + dVybwNLG7hxDy/67BA1j username@mydomain.com + sudo: + - [ "ALL=(ALL) NOPASSWD:ALL" ] + +#. Under :guilabel:`Finalize and create`: + + a. Set the number of instances you want to deploy. + + #. Set the hostname for the instance. + + See Figure 10. + + | + +#. Click :guilabel:`Create Droplet` to deploy the instance. + + .. figure:: ../../_figures/digitalocean/10-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Finalize and create Droplet + + Figure 10: DigitalOcean - Finalize and create Droplet + +Connect to Your |CL| Instance +***************************** + +#. On DigitalOcean's website, go to :menuselection:`MANAGE --> Droplets`. + + See Figure 11. + + | + +#. Get the IP address of your |CL| instance. + + .. figure:: ../../_figures/digitalocean/11-digitalocean.png + :scale: 100 % + :alt: DigitalOcean - Get Droplet IP address + + Figure 11: DigitalOcean - Get Droplet IP address + +#. On your client system, SSH into your instance. + For example: + + .. code-block:: bash + + ssh clear@ -i + + +Related topics +************** + +* :ref:`gce` +* :ref:`azure` +* :ref:`aws-web` + +.. _clear-31870-digitalocean.img.gz: https://cdn.download.clearlinux.org/releases/31870/clear/clear-31870-digitalocean.img.gz + +.. _DigitalOcean: https://www.digitalocean.com/ + +.. _DigitalOcean's SSH key creation guide: https://www.digitalocean.com/docs/droplets/how-to/add-ssh-keys/create-with-openssh/ + +.. _downloads: https://clearlinux.org/downloads + +.. _Installer YAML Syntax: + https://github.com/clearlinux/clr-installer/blob/master/scripts/InstallerYAMLSyntax.md + +.. _micro-config-drive: https://github.com/clearlinux/micro-config-drive diff --git a/_sources/get-started/cloud-install/gce.rst.txt b/_sources/get-started/cloud-install/gce.rst.txt new file mode 100644 index 00000000..431ecb17 --- /dev/null +++ b/_sources/get-started/cloud-install/gce.rst.txt @@ -0,0 +1,265 @@ +.. _gce: + +|CL-ATTR| on Google Cloud Platform\* +#################################### + +This page explains the steps to create a virtual machine instance of +|CL-ATTR| on `Google Cloud Platform`_ (:abbr:`GCP (Google Cloud Platform)`). + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* Set up a Google account and a GCP billing account. + +* Generate and install a user SSH key in the Linux PCs that will connect to + the VMs in GCP. + +Setup |CL| VM on GCP +******************** + +#. Sign in to your Google\* account on the + `Google Cloud Console `_: + + .. figure:: /_figures/gce/00-sign-in.png + :scale: 50 % + :alt: Sign in to Google services + + Figure 1: Google sign in screen + +#. Google Cloud Platform uses **Projects** to manage resources. + Select or create a new project for hosting the |CL| VM. + + .. note:: + + Refer to the + `Quickstart Using a Linux VM `_ + guide to learn about the process of creating VM instances on GCP. + +#. Navigate to the latest |CL| + `release folder `_ + to view the currently released :abbr:`GCE (Google Compute Engine\*)` + image, and download the :file:`clear--gce.tar.gz` + image archive. + + You don't need to uncompress the image archive, the intact file will + be uploaded to the Google Cloud Storage later. + +#. Create a *Storage Bucket* for hosting the |CL| image source archive + downloaded in the previous step: + + * Click the :guilabel:`Navigation menu` icon on the upper left screen menu. + + * Select the :menuselection:`Storage` item from the sidebar on the left. + You will be sent to the Storage Browser tool or the Cloud Storage + overview page. + + .. figure:: /_figures/gce/01-cloud-storage.png + :scale: 50 % + :alt: Browse Google Cloud Storage + + Figure 2: Browse Google Cloud Storage + + .. note:: + You may need to create a billing account and link to this project + before you create a bucket. + + .. figure:: /_figures/gce/02-storage-browser.png + :scale: 50 % + :alt: Cloud Storage Browser tool + + Figure 3: Cloud Storage Browser tool + + * Click the :guilabel:`CREATE BUCKET` button to enter the bucket creation tool. + The bucket name must be unique because buckets in the Cloud Storage share + a single global namespace. + + Leave the remaining options set to the defaults, and click the + :guilabel:`Create` button at the bottom to create a *Bucket*. + + .. figure:: /_figures/gce/03-create-bucket.png + :scale: 50 % + :alt: Set a unique bucket name + + Figure 4: Set bucket name + +#. Once the bucket is created, click the :guilabel:`Upload files` button + on the Bucket details page to upload the |CL| GCE image archive + to the named bucket: + + .. figure:: /_figures/gce/04-bucket-created.png + :scale: 50 % + :alt: Cloud Storage bucket is available for storing objects + + Figure 5: Cloud Storage bucket + + .. figure:: /_figures/gce/10-image-upload.png + :scale: 50 % + :alt: Uploading the image source archive file + + Figure 6: Uploading the image source archive file + + .. figure:: /_figures/gce/11-bucket-uploaded.png + :scale: 50 % + :alt: Image archive imported complete + + Figure 7: Importing complete + +#. Browse the Compute Engine Image library page: + + * Click the :guilabel:`Navigation menu` icon on the upper left screen menu. + + * Select the :menuselection:`Compute Engine --> Images` from the side bar + on the left. + + .. figure:: /_figures/gce/20-gce-image.png + :scale: 50 % + :alt: Go to Google Compute Engine Image library + + Figure 8: Image library + +#. On the Compute Engine Image library page, click the + :guilabel:`[+] CREATE IMAGE` menu item to create a custom image: + + .. figure:: /_figures/gce/20-image-library.png + :scale: 50 % + :alt: Create a Google Compute Engine image + + Figure 9: Create image + +#. In the VM image creation page, change the image source type to + *Cloud Storage file*. + +#. Under :guilabel:`Source`, select :guilabel:`Browse`. + +#. Locate the :file:`clear--gce.tar.gz` file, + and click :guilabel:`Select`. + + .. figure:: /_figures/gce/21-create-image.png + :scale: 50 % + :alt: Create the image using the imported image archive object + + Figure 10: Create image using imported object + + Accept all default options, and click the :guilabel:`Create` button + at the bottom to import the Clear Linux GCE image to the image library. + + .. figure:: /_figures/gce/22-image-list.png + :scale: 50 % + :alt: Clear Linux Compute Engine image is created + + Figure 11: Image is created + +#. After the |CL| image is imported, you can launch a VM instance running + |CL|: + + * Click the :guilabel:`Navigation menu` icon on the upper left screen menu. + + * Select :menuselection:`Compute Engine --> VM Instances` from the side bar + on the left. + + .. figure:: /_figures/gce/30-vm-instances.png + :scale: 50 % + :alt: Go to VM instances catalog + + Figure 12: VM instances catalog + +#. If no VM instance was created in this project, you will be prompted to + create one. + +#. Alternatively, click the :guilabel:`CREATE INSTANCE` button on the VM + instances page to create a VM instance. + + .. figure:: /_figures/gce/30-vm-none.png + :scale: 50 % + :alt: Prompt for VM creation + + Figure 13: VM creation + + .. figure:: /_figures/gce/30-vm-catalog.png + :scale: 50 % + :alt: List of VM instances + + Figure 14: VM instances list + + * Under :guilabel:`Region`, choose a region based on the + `Best practices for Compute Engine regions selection`_. + + * Under :guilabel:`Boot disk`, click the :guilabel:`Change` button. + + .. figure:: /_figures/gce/30-create-vm.png + :scale: 50 % + :alt: Use custom image while creating Clear Linux VM instance + + Figure 15: Use custom image + + * Select the :menuselection:`Custom images` tab for using Clear Linux OS GCE image. + + .. figure:: /_figures/gce/31-select-boot-disk.png + :scale: 50 % + :alt: Select Clear Linux boot disk to create a VM instance + + Figure 16: Select Clear Linux boot disk to create a VM instance + + * Scroll down to the bottom of the VM instance creation page, + expand the :guilabel:`Management, security, disks, networking, sole tenancy` + group. + + .. figure:: /_figures/gce/40-clear-vm-security.png + :scale: 50 % + :alt: Clear Linux requires setting up SSH keys + + Figure 17: Set up SSH keys + + .. note:: + |CL| does not allow SSH login with a root account by default. + As a result, you must configure the VM instance with your + SSH public key, so that you are able to access it remotely. + + Refer to :ref:`security` for more details. + + * Click the :menuselection:`Security` tab, copy and paste your SSH public key: + + .. figure:: /_figures/gce/40-ssh-key.png + :scale: 50 % + :alt: Set SSH key for remote login + + Figure 18: Set SSH key for remote login + + .. note:: + + The username is assigned from characters preceding ``@`` in the email + address, included in the SSH key. + + * Click the :guilabel:`Create` button to create the |CL| VM. + +#. The Clear Linux VM instance is created and assigned a public IP address: + + .. figure:: /_figures/gce/41-vm-created.png + :scale: 50 % + :alt: Clear Linux VM instance is created and started + + Figure 19: Clear Linux VM instance is created and started + +#. You can now SSH login to the VM using the IP address obtained in the + previous step, and the username associated with the SSH public key: + + .. figure:: /_figures/gce/42-ssh-vm.png + :scale: 50 % + :alt: SSH login to the Clear Linux VM + + Figure 20: SSH login to Clear Linux VM + +Related topics +************** + +* :ref:`azure` +* :ref:`aws-web` +* :ref:`clr-digitalocean` + +.. _Google Cloud Platform: https://cloud.google.com/ + +.. _Best practices for Compute Engine regions selection: https://cloud.google.com/solutions/best-practices-compute-engine-region-selection diff --git a/_sources/get-started/cloud-install/import-clr-aws.rst.txt b/_sources/get-started/cloud-install/import-clr-aws.rst.txt new file mode 100644 index 00000000..6433da4f --- /dev/null +++ b/_sources/get-started/cloud-install/import-clr-aws.rst.txt @@ -0,0 +1,511 @@ +.. _import-clr-aws: + +Import Clear Linux Image and Launch Instance on AWS +################################################### + +Clear Linux is available on the AWS marketplace. However, it may not +be the latest version because we only update the marketplace on a +periodic basis, as often as weekly or but maybe monthly as well. +If you want to use the latest release from us or upload your own +custom image, follow this guide. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* You are familiar with AWS and how to use it + +Download or create a |CL| image for AWS +*************************************** + +Obtain an AWS |CL| image using one of these methods. + +Download pre-built image +======================== +#. Go to the `Downloads`_ page and download the + *Amazon\* Web Services (AWS)* image. + +#. Uncompress it. + +Create a custom image using clr-installer +========================================= +#. On a |CL| system, open a terminal. + +#. Install the `clr-installer` bundle. + + .. code-block:: bash + + sudo swupd bundle-add clr-installer + +#. Download a sample `aws.yaml`_ configuration file. + +#. Make changes to the configuration file as needed. + See `Installer YAML Syntax`_ for more information on clr-installer + configuration YAML syntax. + +#. Download the `AWS image post-install script`_ and make it executable. + +#. Produce an image with clr-installer. + + .. code-block:: bash + + clr-installer --template $PWD/aws.yaml + +Create an S3 bucket +******************* + +#. Log into AWS. + +#. Go to :guilabel:`Services`, :guilabel:`Storage`, and select :guilabel:`S3`. + See Figure 1. + + .. figure:: ../../_figures/aws/import-clr-aws-01.png + :scale: 70% + :alt: AWS Services - S3 Management Console + + Figure 1: AWS Services - S3 Management Console + +#. Click :guilabel:`+ Create bucket`. + + .. figure:: ../../_figures/aws/import-clr-aws-02.png + :scale: 70% + :alt: AWS S3 - Create bucket + + Figure 2: AWS S3 - Create bucket + +#. Set a bucket name and select a region. + See Figure 3. + + .. figure:: ../../_figures/aws/import-clr-aws-03.png + :scale: 70% + :alt: AWS S3 - Create bucket - Set bucket name and region + + Figure 3: AWS S3 - Create bucket - Set bucket name and region + +#. Leave the :guilabel:`Configure options` and :guilabel:`Set permissions` + settings as is or configure as desired. See Figure 4 and 5. + + .. figure:: ../../_figures/aws/import-clr-aws-04.png + :scale: 70% + :alt: AWS S3 - Create bucket - Configure options + + Figure 4: AWS S3 - Create bucket - Configure options + + .. figure:: ../../_figures/aws/import-clr-aws-05.png + :scale: 70% + :alt: AWS S3 - Create bucket - Set permissions + + Figure 5: AWS S3 - Create bucket - Set permissions + +#. At the :guilabel:`Review` screen, click :guilabel:`Create bucket`. + + .. figure:: ../../_figures/aws/import-clr-aws-06.png + :scale: 70% + :alt: AWS S3 - Create bucket - Review + + Figure 6: AWS S3 - Create bucket - Review + + The created bucket should appear. See Figure 7. + + .. figure:: ../../_figures/aws/import-clr-aws-07.png + :scale: 70% + :alt: AWS S3 - Created bucket + + Figure 7: AWS S3 - Created bucket + +Upload the |CL| image into the bucket +************************************* + +#. Click on the bucket. + See Figure 8. + + .. figure:: ../../_figures/aws/import-clr-aws-08.png + :scale: 70% + :alt: AWS S3 - Select bucket + + Figure 8: AWS S3 - Select bucket + +#. Click :guilabel:`Upload`. + See Figure 9. + + .. figure:: ../../_figures/aws/import-clr-aws-09.png + :scale: 70% + :alt: AWS S3 - Upload + + Figure 9: AWS S3 - Upload + +#. Click :guilabel:`Add files` and select the |CL| image file to upload. + See Figure 10. + + .. figure:: ../../_figures/aws/import-clr-aws-10.png + :scale: 70% + :alt: AWS S3 - Add files + + Figure 10: AWS S3 - Add files + +#. Click :guilabel:`Next`. Leave remaining settings as is or set as desired. + See Figure 11, Figure 12, and Figure 13. + + .. figure:: ../../_figures/aws/import-clr-aws-11.png + :scale: 70% + :alt: AWS S3 - Add files + + Figure 11: AWS S3 - Add files + + .. figure:: ../../_figures/aws/import-clr-aws-12.png + :scale: 70% + :alt: AWS S3 - Set permissions + + Figure 12: AWS S3 - Set permissions + + .. figure:: ../../_figures/aws/import-clr-aws-13.png + :scale: 70% + :alt: AWS S3 - Set properties + + Figure 13: AWS S3 - Set properties + +#. Click :guilabel:`Upload` to upload the image. + See Figure 14. + + .. figure:: ../../_figures/aws/import-clr-aws-14.png + :scale: 70% + :alt: AWS S3 - Upload + + Figure 14: AWS S3 - Upload + +Add a user to IAM with AWS_CLI privilege +**************************************** + +#. Go to :guilabel:`Services`, :guilabel:`Security, Identity, & Compliance`, + and select :guilabel:`IAM`. + See Figure 15. + + .. figure:: ../../_figures/aws/import-clr-aws-15.png + :scale: 70% + :alt: AWS Services - IAM + + Figure 15: AWS Services - IAM + +#. On the left navigation bar under :guilabel:`Access management`, + select :guilabel:`Users`. + See Figure 16. + + .. figure:: ../../_figures/aws/import-clr-aws-16.png + :scale: 70% + :alt: AWS AIM - Access management + + Figure 16: AWS AIM - Access management + +#. Click :guilabel:`Add user`. + See Figure 17. + + .. figure:: ../../_figures/aws/import-clr-aws-17.png + :scale: 70% + :alt: AWS AIM - Add user + + Figure 17: AWS AIM - Add user + +#. Under the :guilabel:`Set user details` section, enter a user name. + See Figure 18. + + .. figure:: ../../_figures/aws/import-clr-aws-18.png + :scale: 70% + :alt: AWS AIM - Enter user name and select access type + + Figure 18: AWS AIM - Enter user name and select access type + +#. Under the :guilabel:`Select AWS access type` section, + checkmark :guilabel:`Programmatic access`. + See Figure 18. + +#. Click :guilabel:`Next: Permissions`. + +#. Under :guilabel:`Set permissions`, select :guilabel:`Add user to group`. + See Figure 19. + + .. figure:: ../../_figures/aws/import-clr-aws-19.png + :scale: 70% + :alt: AWS AIM - Set user permissions + + Figure 19: AWS AIM - Set user permissions + +#. Under :guilabel:`Add user to group`, enter `AWS_CLI` into search window. + Checkmark :guilabel:`AWS_CLI`. + See Figure 19. + +#. Click :guilabel:`Next: Tags`. + +#. Click :guilabel:`Next: Review`. + +#. Click :guilabel:`Create user`. + See Figure 20. + + .. figure:: ../../_figures/aws/import-clr-aws-20.png + :scale: 70% + :alt: AWS AIM - Create user + + Figure 20: AWS AIM - Create user + +#. After the user is successfully added, save the :guilabel:`Access key ID` + and the :guilabel:`Secret access key`. These will be used when setting up + the AWS CLI tool at a later step. + See Figure 21. + + .. figure:: ../../_figures/aws/import-clr-aws-21.png + :scale: 70% + :alt: AWS AIM - Access key ID and secret access key + + Figure 21: AWS AIM - Access key ID and secret access key + +#. Click :guilabel:`Close`. + +Install and configure the AWS CLI tool on your system +***************************************************** + +#. To install the tool on |CL|, simply run: + + .. code-block:: bash + + sudo swupd bundle-add cloud-api + + .. note: + + If you are using a different OS, follow the + `Installing the AWS CLI version 2`_ guide. + +#. Configure it with your security credentials, default region, + and default output format. See `Configuring the AWS CLI`_ for more information. + + .. code-block:: bash + + aws configure + + Below is an example (using the security credentials that was created in + the previous section): + + .. code-block:: console + + AWS Access Key ID [None]: AKIA5LEGQPQ3EUB3JMS7 + AWS Secret Access Key [None]: EcvbWpWr+Gp7NhBoVEacwR3EifzN7xTTg8B1PHvO + Default region name [None]: us-west-2 + Default output format [None]: json + +#. Verify your credentials are good. + + .. code-block:: bash + + aws iam list-access-keys + + If you get something like the example below, then make sure you set your + system date and time properly. + + .. code-block:: console + + An error occurred (SignatureDoesNotMatch) when calling the ListAccessKeys operation: Signature expired: 20200305T153154Z is now earlier than 20200305T231847Z (20200305T233347Z - 15 min.) + +Import a snapshot of the |CL| image +*********************************** + +#. Create a :file:`container.json` with the description of the image to import. + Specify the name of the S3 bucket that was created earlier for the + `S3Bucket` field and the name of |CL| image that was uploaded to the S3 bucket + for the `S3Key`. + + Here's an example: + + .. code-block:: console + + { + "Description": "My Clear Linux AWS 32400 Image", + "Format": "raw", + "UserBucket": { + "S3Bucket": "my-clearlinux-bucket", + "S3Key": "clear-32400-aws.img" + } + } + +#. Import a snapshot of the image. + + .. code-block:: bash + + aws ec2 import-snapshot \ + --description "My Clear Linux AWS 32400 Snapshot" \ + --disk-container file://container.json + + You should get an output similar this example: + + .. code-block:: console + + { + "Description": "My Clear Linux AWS 32400 Snapshot", + "ImportTaskId": "import-snap-00fa9ccd98e9b8378", + "SnapshotTaskDetail": { + "Description": "My Clear Linux AWS 32400 Snapshot", + "DiskImageSize": 0.0, + "Format": "RAW", + "Progress": "3", + "Status": "active", + "StatusMessage": "pending", + "UserBucket": { + "S3Bucket": "my-clearlinux-bucket", + "S3Key": "clear-32400-aws.img" + } + } + } + +#. Using the `ImportTaskId` from the previous step, check the status + of the import. For example: + + .. code-block:: bash + + snapshot_id=$(aws ec2 describe-import-snapshot-tasks \ + --import-task-ids "import-snap-00fa9ccd98e9b8378" \ + | grep SnapshotId | awk -F '"' '{print $4}') + + Wait for the `Status` field to show `completed` before proceeding. + + The resulting `snapshot_id` will be used to create an AMI in + the next section. + +Create an AMI from the snapshot +******************************* + +There are 2 methods to create an AMI from the snapshot. + +* *AWS CLI Method*: + + .. code-block:: bash + + aws ec2 register-image \ + --name "My-Clear-Linux-32400-AMI" \ + --description "My Clear Linux 32400 AMI" \ + --architecture x86_64 \ + --virtualization-type hvm \ + --ena-support \ + --root-device-name "/dev/sda1" \ + --block-device-mappings "[ + { + \”Deviceame\": \"/dev/sda1\", + \"Ebs\": { + \"SnapshotId\": \"$snapshot_id\" + } + } + ]" + +* *GUI Method*: + + #. Go to :guilabel:`Services`, :guilabel:`Compute`, and select + :guilabel:`EC2`. + See Figure 22. + + .. figure:: ../../_figures/aws/import-clr-aws-22.png + :scale: 70% + :alt: AWS Services - EC2 + + Figure 22: AWS Services - EC2 + + #. Click :guilabel:`Snapshots`. + See Figure 23. + + .. figure:: ../../_figures/aws/import-clr-aws-23.png + :scale: 70% + :alt: AWS Services - Snapshots + + Figure 23: AWS Services - Snapshots + + #. Locate the snaphot using the `Snapshot ID`. + See Figure 24. + + .. figure:: ../../_figures/aws/import-clr-aws-24.png + :scale: 70% + :alt: AWS Services - Snapshots + + Figure 24: AWS Services - Snapshots + + #. Right-click it and select :guilabel:`Create Image`. + + #. Configure as follows: + + * Enter the name in the :guilabel:`Name` field + * Enter the description in the :guilabel:`Description` field + * Set the :guilabel:`Architecture` as `x86_64` + * Set the :guilabel:`Virtualization type` as `Hardware-assisted virtualization` + * Set the :guilabel:`Root device name` as `/dev/sda1` + + See Figure 25. + + .. figure:: ../../_figures/aws/import-clr-aws-25.png + :scale: 70% + :alt: AWS Services - Snapshots + + Figure 25: AWS Services - Snapshots + + #. Click :guilabel:`Create`. + +Launch an instance +****************** + +#. Go to :guilabel:`Services`, :guilabel:`Compute`, and select + :guilabel:`EC2`. + See Figure 26. + + .. figure:: ../../_figures/aws/import-clr-aws-26.png + :scale: 70% + :alt: AWS Services - EC2 + + Figure 26: AWS Services - EC2 + +#. Click the :guilabel:`Launch Instance` dropdown and select + :guilabel:`Launch Instance`. + See Figure 27. + + .. figure:: ../../_figures/aws/import-clr-aws-27.png + :scale: 70% + :alt: AWS Services - Launch instance + + Figure 27: AWS Services - Launch instance + +#. On the left navigation bar, select :guilabel:`My AMIs`. + See Figure 28. + + .. figure:: ../../_figures/aws/import-clr-aws-28.png + :scale: 70% + :alt: AWS Services - Select AMI + + Figure 28: AWS Services - Select AMI + +#. Find your AMI and click :guilabel:`Select`. + +#. From here onward, configure the details of your instance as desired + and launch it. + +Connect to your |CL| instance +***************************** + +#. Follow these steps to `connect to your instance`_. + +Related topics +************** + +* :ref:`azure` +* :ref:`gce` +* :ref:`clr-digitalocean` + +.. _Downloads: + https://clearlinux.org/downloads +.. _aws.yaml: + https://cdn.download.clearlinux.org/current/config/image/aws.yaml +.. _AWS image post-install script: + https://cdn.download.clearlinux.org/current/config/image/aws-disable-root.sh +.. _Installing the AWS CLI version 2: + https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html +.. _Configuring the AWS CLI: + https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html +.. _connect to your instance: + https://docs.01.org/clearlinux/latest/get-started/cloud-install/aws-web.html#connect-to-your-clear-linux-os-basic-instance +.. _Installer YAML Syntax: + https://github.com/clearlinux/clr-installer/blob/master/scripts/InstallerYAMLSyntax.md + diff --git a/_sources/get-started/cloud-install/qingcloud.rst.txt b/_sources/get-started/cloud-install/qingcloud.rst.txt new file mode 100644 index 00000000..3a5c7106 --- /dev/null +++ b/_sources/get-started/cloud-install/qingcloud.rst.txt @@ -0,0 +1,262 @@ +.. _qingcloud: + +|CL-ATTR| on QingCloud\* +########################### + +This tutorial describes how to create and launch a Clear Linux OS instance +from the QingCloud console. + +.. contents:: + :local: + :depth: 1 + + +Prerequisites +************* + +This tutorial assumes that you have completed the following configuration steps: + +* Your environment can run SSH to access remote Clear Linux OS virtual hosts. +* You know the absolute path where the browser downloaded the file. +* You have set up a user account for QingCloud and that the account is + enabled and logged in to the QingCloud console. To learn more about + QingCloud and setting up an account, please visit QingCloud's official + `website `_. + + +Select and start |CL| virtual host with QingCloud console +********************************************************* + +#. Select :guilabel:`计算>主机` (Compute>Host) in the main menu of the + QingCloud console and click the :guilabel:`创建` (Create) option. + + .. figure:: /_figures/qingcloud/QingCloud-1.png + :scale: 50 % + :alt: QingCloud console + +#. On the host creation page, first click the :guilabel:`系统` (System) option, + click the |CL| icon on the far right. Click the :guilabel:`下一步` (Next) + button to continue. + + .. figure:: /_figures/qingcloud/QingCloud-2.png + :scale: 50 % + :alt: Select Clear Linux OS to create virtual host + + Select |CL| to create a virtual host + +#. In the configuration selection interface, you can configure the + number of CPU cores, memory size, and the storage backup method. + For demonstration purposes, we will choose the default configuration. + Click the :guilabel:`下一步` (Next) button to go to the network settings + interface. + + .. figure:: /_figures/qingcloud/QingCloud-3.png + :scale: 50 % + :alt: Configuration selection + + Configuration selection + +#. Select :guilabel:`基础网络` (Basic Network) in the network settings + interface. + + .. figure:: /_figures/qingcloud/QingCloud-4.png + :scale: 50 % + :alt: Network settings + + Network Settings + +#. In the basic information setting interface, you need to enter the virtual + host name and set the SSH key login method. + +Create an SSH key (Optional) +============================ + +#. If you haven't created an SSH key before, click the :guilabel:`创建一个` + (Create one) button to create an SSH key. + + .. figure:: /_figures/qingcloud/QingCloud-6.png + :scale: 50 % + :alt: Create SSH key + + Create SSH Key + +#. In the SSH key creation interface, you can fill in the key name, and select + encryption method you prefer. After confirming that they are correct, click + the :guilabel:`提交` (Submit) button. + + .. figure:: /_figures/qingcloud/QingCloud-6.png + :scale: 50 % + :alt: New SSH key + + New SSH Key + +#. After the download button appears, please download the key within 10 + minutes, and save the key locally for connecting to the virtual host later. + + .. figure:: /_figures/qingcloud/QingCloud-7.png + :scale: 50 % + :alt: Download SSH key + + Download SSH Key + +#. After ensuring that the SSH key has been properly downloaded and saved, + check the basic information of the virtual host. After confirming that they + are correct, click the :guilabel:`创建` (Create) button. + + .. figure:: /_figures/qingcloud/QingCloud-8.png + :scale: 50 % + :alt: Confirm the information and create a virtual host + + Confirm the information and create a virtual host + +QingCloud will now create the Clear Linux OS virtual host. You +can check the state of the virtual host in the new interface. + +Apply for a public IP and add it to the virtual host +**************************************************** + +#. Since QingCloud does not automatically assign a public IP address to a + virtual host created using the default network, we need to manually apply + and add it to the virtual host. Click the :guilabel:`网络与CDN` (Network and + CDN) button on the left side of the navigation bar . + + .. figure:: /_figures/qingcloud/QingCloud-9.png + :scale: 50 % + :alt: Network and CDN + + Network and CDN + +#. In the network and CDN configuration interface, click the :guilabel:`公网IP` + (Public IP) button on the left , and click the :guilabel:`申请` (Apply) + button in the middle to create a public IP. + + .. figure:: /_figures/qingcloud/QingCloud-10.png + :scale: 50 % + :alt: Apply for public IP + + Apply for public IP + + After clicking the apply button, a dialog will pop up. Read it + carefully and click the :guilabel:`继续申请公网IP` (Continue to apply for + public IP) button. + + .. figure:: /_figures/qingcloud/QingCloud-11.png + :scale: 50 % + :alt: Confirmation dialog + + Confirmation dialog + +#. On the public network IP application page, confirm and fill in the + relevant information, including the charging mode and bandwidth upper limit + (the charge-by-bandwidth mode is used in this tutorial and the 2Mbps + bandwidth limit is set). After confirming that they are correct, click + :guilabel:`提交` (Submit) button. + + .. figure:: /_figures/qingcloud/QingCloud-12.png + :scale: 50 % + :alt: Confirmation of Public IP Application + + Confirmation of Public IP Application + +#. After that, click the :guilabel:`计算>网卡` (Compute>Network Card) buttons + in the navigation bar to come to the network card interface. + + .. figure:: /_figures/qingcloud/QingCloud-13.png + :scale: 50 % + :alt: NIC interface + + Network Interface + +#. On the network card interface, select the network card of the |CL| host + that you just created. Click the :guilabel:`更多操作` (More Actions) + button above, and then click the :guilabel:`绑定公网IPv4` (Binding Public + Network IPv4) button. + + .. figure:: /_figures/qingcloud/QingCloud-14.png + :scale: 50 % + :alt: Bind selected + + Bind selected + +#. On the binding public network IP confirmation interface, select the public + IP address that has just been applied for, and click the :guilabel:`提交` + (Submit) button below . After waiting a moment, the state of the |CL| + virtual host will change. + + .. figure:: /_figures/qingcloud/QingCloud-15.png + :scale: 50 % + :alt: Commit binding + + Commit binding + + .. figure:: /_figures/qingcloud/QingCloud-16.png + :scale: 50 % + :alt: Public network IP binding succeeded + + Public network IP binding succeeded + +Connect to |CL| virtual host +********************************* + +Click the :guilabel:`计算>主机` (Compute>Host) buttons on the left side of the +navigation bar to confirm that the current virtual host is running and has a public IP address. + +.. figure:: /_figures/qingcloud/QingCloud-17.png + :scale: 50 % + :alt: Confirm that the virtual host is currently in a normal state + + Confirm that the virtual host is currently in a normal state + +#. Copy the public IP address of the current |CL| virtual host and + connect to it from an SSH client. Here we need to use the previously saved + SSH key. + +#. In this tutorial, the MobaXterm client is used as an example to demonstrate + the login process. Check each item as shown. For the user name, we choose + ``root``. For the key, select the SSH key that was downloaded and saved to + the local computer . + + .. figure:: /_figures/qingcloud/QingCloud-18.png + :scale: 50 % + :alt: SSH login virtual host settings + + SSH login virtual host settings + +#. Click :guilabel:`Login` to log in to the + |CL| virtual host after completing the setup process. + + .. figure:: /_figures/qingcloud/QingCloud-19.png + :scale: 50 % + :alt: SSH login successful + + SSH login successful + +Remove |CL| virtual host +************************ + +This section explains how to delete a |CL| virtual host created on QingCloud. + +On the left navigation bar select :guilabel:`计算>主机` (Compute>Master), find +the |CL| host you just created, and click the checkbox next to it. Select +:guilabel:`更多操作>删除` (More Actions>Delete) to delete the virtual host. + +.. figure:: /_figures/qingcloud/QingCloud-20.png + :scale: 50 % + :alt: Remove Clear Linux OS Virtual Host + + Remove Clear Linux OS Virtual Host + +Delete the applied public IP +**************************** + +Select :guilabel:`网络与CDN>公网IP` (Network and CDN>Public IP) from the +navigation bar on the left , and then find the public IP address just applied. +Select it as shown, then click :guilabel:`更多操作>删除` (More Actions>Delete) +to delete. + +.. figure:: /_figures/qingcloud/QingCloud-21.png + :scale: 50 % + :alt: Delete public network IP address + + Delete public network IP address + \ No newline at end of file diff --git a/_sources/get-started/compatibility-check.rst.txt b/_sources/get-started/compatibility-check.rst.txt new file mode 100644 index 00000000..d4297369 --- /dev/null +++ b/_sources/get-started/compatibility-check.rst.txt @@ -0,0 +1,104 @@ +.. _compatibility-check: + +Check Processor Compatibility +############################# + +Before installing |CL-ATTR|, check your host system's processor compatibility using one of +the following options: + +.. note:: + This does not check other system components (for example: storage and + graphics) for compatibility with |CL|. + +Option 1: Use the :command:`clear-linux-check-config.sh` script on an existing Linux system +******************************************************************************************* + +#. Download the `clear-linux-check-config.sh`_ file. + + If a browser is not available, use: + + .. code-block:: bash + + curl -O https://cdn.download.clearlinux.org/current/clear-linux-check-config.sh + +#. Make the script executable. + + .. code-block:: bash + + chmod +x clear-linux-check-config.sh + +#. Run the script. + + #. Check to see if the host's processor is capable of running |CL|. + + .. code-block:: bash + + ./clear-linux-check-config.sh host + + #. Check to see if the host is capable of running |CL| in a container. + + .. code-block:: bash + + ./clear-linux-check-config.sh container + + The script prints a list of test results similar to the output below. + All items should return a `SUCCESS` status. This example indicates the + host's processor supports running |CL|. + + .. code-block:: console + + Checking if host is capable of running Clear Linux* OS + + SUCCESS: 64-bit CPU (lm) + SUCCESS: Supplemental Streaming SIMD Extensions 3 (ssse3) + SUCCESS: Streaming SIMD Extension v4.1 (sse4_1) + SUCCESS: Streaming SIMD Extensions v4.2 (sse4_2) + SUCCESS: Carry-less Multiplication extensions (pclmulqdq) + +Option 2: Use a |CL| live image on a non-Linux system +===================================================== + +#. `Download`_ either the `Desktop` or `Server` version of the live image ISO. + +#. Follow the instruction to :ref:`bootable-usb`. + +#. Boot up the |CL| live image on the USB. + +#. Check compatibility as follows: + + * *Desktop version:* + + a. Open a terminal. + + #. Check compatibility. + + .. code-block:: bash + + sudo clr-installer --system-check + + * *Server version:* + + a. Log in as `root` and set a password. + + #. Check compatibility. + + .. code-block:: bash + + clr-installer --system-check + + Expected output for a compatible host processor: + + .. code-block:: console + + Checking for required CPU feature: lm [success] + Checking for required CPU feature: sse4_2 [success] + Checking for required CPU feature: sse4_1 [success] + Checking for required CPU feature: pclmulqdq [success] + Checking for required CPU feature: ssse3 [success] + Success: System is compatible + +.. _clear-linux-check-config.sh: + https://cdn.download.clearlinux.org/current/clear-linux-check-config.sh + +.. _Download: + https://clearlinux.org/downloads diff --git a/_sources/get-started/containers/container-images.rst.txt b/_sources/get-started/containers/container-images.rst.txt new file mode 100644 index 00000000..848dfb1f --- /dev/null +++ b/_sources/get-started/containers/container-images.rst.txt @@ -0,0 +1,79 @@ +.. _container-images: + +|CL-ATTR| container images +########################## + +|CL| can run inside of a container on top of any operating system as long as +it is hosting a containerized environment, such as Docker* or Kubernetes*. A +|CL| base image is available for standalone use as well as variations of +popular application images built from the |CL| base image. + +Browse all |CL| container images on `the Docker Hub* website +`_. Find the +Dockerfiles used to build |CL| container images `on GitHub +`_. + +See the `containers `_ page for +the benefits of using |CL| containers and using |CL| as a container host. + +Container image types +********************* + +|CL| base image +=============== + +The `Clear Linux OS base container `_ is +an official image on Docker Hub*. The |CL| base container image can be used to +run a standalone or as a `parent image +`_ for building other +container images. + +On a Docker host simply use the command :command:`docker run clearlinux` to +pull and start a |CL| container. + + +|CL|-based runtime images +========================= + +|CL| container images for programming languages and their runtimes are +available on Docker Hub*. These can be used by developers to create and run +applications using these popular runtimes. + +Below are some popular |CL|-based runtime images: + +* `clearlinux/golang `_ +* `clearlinux/node `_ +* `clearlinux/numpy `_ +* `clearlinux/python `_ +* `clearlinux/perl `_ +* `clearlinux/r-base `_ + +More |CL|-based images can be found on Docker Hub: +https://hub.docker.com/u/clearlinux. + + +|CL|-based application images +============================= + +|CL| container images for common applications are available on Docker Hub. +These can be used to create and deploy containerized services. + +Below are some popular |CL|-based runtime images: + +* `clearlinux/nginx `_ +* `clearlinux/mariadb `_ +* `clearlinux/postgres `_ +* `clearlinux/redis `_ +* `clearlinux/tensorflow `_ +* `clearlinux/wordpress `_ + + +More |CL|-based images can be found on Docker Hub: +https://hub.docker.com/u/clearlinux. + +Related topics +============== +* :ref:`container-image-new` +* :ref:`container-image-modify` +* :ref:`docker` +* :ref:`kata` diff --git a/_sources/get-started/index.rst.txt b/_sources/get-started/index.rst.txt new file mode 100644 index 00000000..151a0ff7 --- /dev/null +++ b/_sources/get-started/index.rst.txt @@ -0,0 +1,69 @@ +.. _get-started: + +Get started +########### + +The Get Started section guides you through the requirements and installation +of |CL-ATTR|. Follow these step-by-step instructions to get started with |CL|, +fast. + +Pre-install +*********** + +There are a couple of things to take care of before you install. + +* :ref:`system-requirements` +* :ref:`compatibility-check` +* :ref:`bootable-usb` + +When installing |CL-ATTR| in a VM, consider which kernel to use. + +* :ref:`Compatible VM kernels ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + compatibility-check + bootable-usb + +Install +******* + +.. toctree:: + :maxdepth: 1 + + bare-metal-install-desktop + bare-metal-install-server + install-configfile + ipxe-install + +.. _virtual-machine-install: + +Install in a virtual machine +**************************** + +.. toctree:: + :maxdepth: 1 + :glob: + + virtual-machine-install/* + ../../guides/maintenance/increase-virtual-disk-size.rst + +Deploy to the cloud +******************* + +.. toctree:: + :maxdepth: 1 + :glob: + + cloud-install/* + +Containers +********** + +.. toctree:: + :maxdepth: 1 + :glob: + + containers/* diff --git a/_sources/get-started/install-configfile.rst.txt b/_sources/get-started/install-configfile.rst.txt new file mode 100644 index 00000000..f6e70377 --- /dev/null +++ b/_sources/get-started/install-configfile.rst.txt @@ -0,0 +1,197 @@ +.. _install-configfile: + +Install using clr-installer and a configuration file +#################################################### + +In addition to the interactive GUI and text-based modes, +:command:`clr-installer` also supports an unattended mode where you +simply provide it a YAML configuration file. + +This guide shows you two examples of how to use its unattended mode. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +For installation onto bare metal, ensure that your target system +supports these requirements: + +* :ref:`system-requirements` +* :ref:`compatibility-check` + +Download and make bootable USB of the live server image +******************************************************* + +See :ref:`bootable-usb`. + +Example 1: Fresh installation onto bare metal +********************************************* + +This example uses a YAML configuration file to perform a new installation. + +#. Boot up the |CL| Live Server USB thumb drive. + +#. Select :guilabel:`Clear Linux OS` from the menu. + +#. In the console window, log in as `root` and set a password. + +#. Verify you have a network connection to the Internet and configure proxy + settings if you're working behind a firewall. + +#. Download a sample YAML configuration file. For example, if you want to + install |CL| with a desktop GUI, you might want to use :file:`live-desktop.yaml`. + Or you can use the :file:`live-server.yaml` if you want to install a non-GUI version + of |CL|. + + * *Desktop:* + + .. code-block:: bash + + curl -O https://cdn.download.clearlinux.org/current/config/image/live-desktop.yaml + + * *Server:* + + .. code-block:: bash + + curl -O https://cdn.download.clearlinux.org/current/config/image/live-server.yaml + +#. Edit the YAML configuration file and change the settings as needed. + + Commonly-changed settings include (refer to the example below): + + a. Under *block-devices* (line 15), set your target media. For example: ``file: "/dev/sda"``. + #. Under *targetMedia* (line 34), set the third partition size to “0” to use the entire disk space. + #. Under *bundles* (line 37), add additional bundles as needed. + #. Delete the *post-install* section unless you have post-installation scripts. + #. Under *Version* (line 50), set a version number. To use the latest version, set to “0”. + + See `Installer YAML Syntax`_ for more details. + + .. code-block:: console + :linenos: + :emphasize-lines: 14,15,34,37,50 + + #clear-linux-config + + # c-basic-offset: 2; tab-width: 2; indent-tabs-mode: nil + # vi: set shiftwidth=2 tabstop=2 expandtab: + # :indentSize=2:tabSize=2:noTabs=true: + + # File: developer-live-server.yaml + # Use Case: Live Image which boots into login prompt + # Optionally allows for installing Clear Linux OS + # using the TUI clr-installer by running clr-installer + + # switch between aliases if you want to install to an actual block device + # i.e /dev/sda + block-devices: [ + {name: "bdevice", file: "/dev/sda"} + ] + + targetMedia: + - name: ${bdevice} + type: disk + children: + - name: ${bdevice}1 + fstype: vfat + mountpoint: /boot + size: "150M" + type: part + - name: ${bdevice}2 + fstype: swap + size: "32M" + type: part + - name: ${bdevice}3 + fstype: ext4 + mountpoint: / + size: "0" + type: part + + bundles: [os-core, os-core-update, NetworkManager, clr-installer, vim] + + autoUpdate: false + postArchive: false + postReboot: false + telemetry: false + iso: true + keepImage: true + + keyboard: us + language: en_US.UTF-8 + kernel: kernel-native + + version: 30010 + +#. Start the unattended installation using the `--config` option. + + .. code-block:: bash + + clr-installer --config live-server.yaml + +#. Reboot your system after installation is completed. + +Example 2: Replicate a previous installation +******************************************** + +This example uses a saved configuration file from a previous installation, +which you can use to easily clone the installation on additional machines +, ideally with the same hardware configuration. + +.. warning:: + + Be aware of the following when applying a saved configuration on a new machine: + + * Make sure the target media on the new machine matches up + + * The users' credentials will be replicated as well + +#. On a system where |CL| was installed, open a terminal window. + +#. Get root privilege. + + .. code-block:: bash + + sudo su + +#. Copy the :file:`clr-installer.yaml` from :file:`/root` to a USB thumb drive. + + .. code-block:: bash + + cp /root/clr-installer.yaml + +#. Install on target system. + + a. Boot up the |CL| Live Server USB thumb drive. + + #. Select :guilabel:`Clear Linux OS` from the menu. + + #. In the console window, log in as `root` and set a password. + + #. Verify you have a network connection to the Internet and configure proxy + settings if you're working behind a firewall. + + #. Plug in and mount the USB thumb drive containing the retrieved + :file:`clr-installer.yaml` configuration file. + + #. Doublecheck to make sure the target media in the saved configuration file + matches with the target system's. + + #. Start the installation. + + .. code-block:: bash + + clr-installer --config clr-installer.yaml + + #. Reboot your system after installation is completed. + +References +********** + +* `Clear Linux Installer`_ +* `Installer YAML Syntax`_ + +.. _Clear Linux Installer: https://github.com/clearlinux/clr-installer +.. _Installer YAML Syntax: https://github.com/clearlinux/clr-installer/blob/master/scripts/InstallerYAMLSyntax.md diff --git a/_sources/get-started/ipxe-install.rst.txt b/_sources/get-started/ipxe-install.rst.txt new file mode 100644 index 00000000..466991f5 --- /dev/null +++ b/_sources/get-started/ipxe-install.rst.txt @@ -0,0 +1,589 @@ +.. _ipxe-install: + +Install |CL| Over the Network with iPXE +####################################### + +PXE :abbr:`PXE (Pre-boot Execution Environment)` is an industry standard +that describes client-server interaction with network-boot software and +uses the DHCP and TFTP protocols. iPXE, a fork of gPXE, is an open-source +version of PXE. It enables computers without built-in PXE capability to +network-boot using protocols such as HTTP, :abbr:`iSCSI (Internet Small +Computer Systems Interface)`, :abbr:`AoE (ATA over Ethernet\*)`, and +:abbr:`FCoE (Fiber Channel over Ethernet\*)`. + +This guide demonstrates how to setup an iPXE server to install |CL-ATTR| +over the network. + +Figure 1 depicts the flow of information between an iPXE server and a +PXE client. + +.. figure:: ../_figures/ipxe/ipxe-install-1.png + :alt: PXE information flow + + Figure 1: PXE information flow + +.. caution:: + + The |CL| PXE image that boots through the iPXE process automatically + erases all data and partitions on the PXE client system and performs + a fresh installation according to a clr-installer YAML configuration + file. + +Prerequisites +************* + +Your iPXE server must have: + +* Ethernet/LAN boot option +* At least two network adapters +* Connection to a public (WAN) network +* Secure Boot option disabled in BIOS + +Your clients must have: + +* Ethernet/LAN boot option +* One network adapter +* Secure Boot option disabled in BIOS +* The minimum requirements to run |CL|. Review the :ref:`compatibility-check`. + +Connect the iPXE server and clients to a network switch on a private +(LAN) network, as shown in Figure 2. + +.. figure:: ../_figures/ipxe/ipxe-install-2.png + :alt: Network topology + + Figure 2: Network topology + +Install |CL| on server +********************** + +#. Install |CL| on the system that will serve as the iPXE server. + We recommend using the `server` version. + +#. Open a terminal window. + +#. Add the :command:`pxe-server` bundle to your |CL| system. + The bundle contains all the necessary apps (web server, iPXE firmwares, + dnsmasq which provides TFTP, DNS, DHCP functionalities) to run an + iPXE server. + + .. code-block:: bash + + sudo swupd bundle-add pxe-server + +#. Define the following variables used for setting up the iPXE server. + Be sure to substitute the value for the WAN_INTERFACE and + LAN_INTERFACE variables with your LAN and WAN interfaces names. + Use :command:`ip a` to list your network devices and get their + names. + + .. code-block:: bash + + IPXE_APP_NAME=ipxe + IPXE_PORT=50000 + WEB_ROOT_DIR=/var/www + IPXE_ROOT_DIR=${WEB_ROOT_DIR}/${IPXE_APP_NAME} + TFTP_ROOT_DIR=/srv/tftp + CLR_INSTALLER_CONF_DIR=clr-installer-configs + WAN_INTERFACE=eno1 + LAN_INTERFACE=eno2 + IPXE_SUBNET=192.168.100 + IPXE_LAN_IP=${IPXE_SUBNET}.1 + IPXE_SUBNET_MASK_IP=255.255.255.0 + IPXE_SUBNET_BITMASK=16 + +Setup nginx web server to host iPXE +*********************************** + +#. Set up an nginx web server to serve the |CL| PXE image to clients + using these steps: + + .. code-block:: bash + + # setup nginx + sudo mkdir -p /etc/nginx/conf.d + sudo cp /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf + + # grant $USER permission to run the web server + sudo tee -a /etc/nginx/nginx.conf << EOF + user $USER; + EOF + + # web server config + sudo tee -a /etc/nginx/conf.d/${IPXE_APP_NAME}.conf << EOF + server { + listen ${IPXE_PORT}; + server_name localhost; + + # directory to store ipxe + location /${IPXE_APP_NAME}/ { + root ${WEB_ROOT_DIR}/${IPXE_APP_NAME}; + rewrite ^/${IPXE_APP_NAME}(/.*)$ \$1 break; + } + + # directory to store clr-installer configs + location /${CLR_INSTALLER_CONF_DIR}/ { + root ${WEB_ROOT_DIR}/${CLR_INSTALLER_CONF_DIR}; + rewrite ^/${CLR_INSTALLER_CONF_DIR}(/.*)$ \$1 break; + } + } + EOF + +#. Set nginx to start automatically on boot and then start it. + + .. code-block:: bash + + sudo systemctl enable nginx --now + +Configure iPXE +************** + +#. Download the latest |CL| PXE image and extract the files into the iPXE root. + + .. code-block:: bash + + sudo curl -o /tmp/clear-pxe.tar.xz \ + https://cdn.download.clearlinux.org/current/clear-$(curl \ + https://cdn.download.clearlinux.org/latest)-pxe.tar.xz + sudo mkdir -p ${IPXE_ROOT_DIR} + sudo tar -xJf /tmp/clear-pxe.tar.xz -C ${IPXE_ROOT_DIR} + sudo ln -sf $(ls ${IPXE_ROOT_DIR} | grep 'org.clearlinux.*') ${IPXE_ROOT_DIR}/linux + + .. note:: + + Ensure that the initial ramdisk file is named :file:`initrd` and + the kernel file is named :file:`linux`, which is a symbolic link to the + actual kernel file. + +#. Create an iPXE boot script. The script presents a menu of bootable images to + download, boot, and install |CL|, according to a designated clr-installer + YAML configuration file. + + .. code-block:: bash + + sudo tee -a ${IPXE_ROOT_DIR}/ipxe_boot_script.ipxe << EOF + #!ipxe + + set menu-timeout 5000 + set submenu-timeout \${menu-timeout} + isset \${menu-default} || set menu-default clr-server + + :menu + menu Select a version of Clear Linux OS to install + item clr-desktop Clear Linux OS (Desktop) + item clr-server Clear Linux OS (Server) + item ipxe-shell iPXE Shell + item reboot Reboot + + choose --timeout \${menu-timeout} --default \${menu-default} selected || goto cancel + set menu-timeout 0 + goto \${selected} + + :clr-desktop + echo Booting and installing Clear Linux OS (Desktop)... + kernel linux quiet init=/usr/lib/systemd/systemd-bootchart initcall_debug \\ + tsc=reliable no_timer_check noreplace-smp rw initrd=initrd \\ + clri.descriptor=http://${IPXE_LAN_IP}:${IPXE_PORT}/${CLR_INSTALLER_CONF_DIR}/clr-desktop.yaml + initrd initrd + boot || goto failed + + :clr-server + echo Booting and installing Clear Linux OS (Server)... + kernel linux quiet init=/usr/lib/systemd/systemd-bootchart initcall_debug \\ + tsc=reliable no_timer_check noreplace-smp rw initrd=initrd \\ + clri.descriptor=http://${IPXE_LAN_IP}:${IPXE_PORT}/${CLR_INSTALLER_CONF_DIR}/clr-server.yaml + initrd initrd + boot || goto failed + + :cancel + echo Menu canceled, going to iPXE shell + + :ipxe-shell + echo Type 'exit' to return to the menu + shell + set menu-timeout 0 + set submenu-timeout 0 + goto menu + + echo Booting + :failed + echo Booting failed, going to iPXE shell + goto shell + + :reboot + echo Rebooting... + sleep 1 + reboot + EOF + + .. note:: + + The `clri.discriptor` option tells clr-installer where to download a YAML + configuration file to use. Without this option, the |CL| PXE image will + simply boot and not perform any installation. + +Add clr-installer YAML configuration files +****************************************** + +After the |CL| PXE image boot, clr-installer downloads the YAML configuration file +specified in the kernel command-line and installs accordingly. + +See `Installer YAML Syntax`_ for more information on clr-installer configuration +YAML syntax. + +#. Create the directory to store the configuration files. + + .. code-block:: bash + + sudo mkdir -p ${WEB_ROOT_DIR}/${CLR_INSTALLER_CONF_DIR} + +#. Create this sample `Desktop` configuration called :file:`clr-desktop.yaml`. + + .. code-block:: bash + + sudo tee -a ${WEB_ROOT_DIR}/${CLR_INSTALLER_CONF_DIR}/clr-desktop.yaml << EOF + #clear-linux-config + + # switch between aliases if you want to install to an actuall block device + # i.e /dev/sda + block-devices: [ + {name: "bdevice", file: "/dev/sda"} + ] + + targetMedia: + - name: \${bdevice} + type: disk + children: + - name: \${bdevice}1 + fstype: vfat + mountpoint: /boot + size: "150M" + type: part + - name: \${bdevice}2 + fstype: swap + size: "250M" + type: part + - name: \${bdevice}3 + fstype: ext4 + mountpoint: / + size: "0" # Use remaining disk space + type: part + + bundles: [ bootloader, os-core, os-core-update, desktop-autostart, libreoffice, + vlc, c-basic, git, openssh-server, vim ] + + autoUpdate: true + postArchive: false + postReboot: true + telemetry: false + hostname: clrlinux-desktop + keyboard: us + language: en_US.UTF-8 + kernel: kernel-native + + users: + - login: clrlinux + username: Clear Linux + # Password is "clear123" + password: $6$SJJMfnInWQg.CvMA$m2F8dJGj71zvi9mSNMktHMsPH3qhBm8pgXDNdaBe2yFfgi479JXvEqWkvQ6OxIUgGNQ5YXFIF0tCn.hEXB90G/ + admin: true + - login: root + username: Root Root + # Password is "clear123" + password: $6$SJJMfnInWQg.CvMA$m2F8dJGj71zvi9mSNMktHMsPH3qhBm8pgXDNdaBe2yFfgi479JXvEqWkvQ6OxIUgGNQ5YXFIF0tCn.hEXB90G/ + admin: true + + pre-install: [ + {cmd: "curl -o /tmp/add-issue.sh http://${IPXE_LAN_IP}:${IPXE_PORT}/${CLR_INSTALLER_CONF_DIR}/add-issue.sh"}, + {cmd: "chmod +x /tmp/add-issue.sh"} + ] + + post-install: [ + {cmd: "echo PermitRootLogin yes > \${chrootDir}/etc/ssh/sshd_config"}, + {cmd: "/tmp/add-issue.sh \${chrootDir}"} + ] + EOF + + +#. Create this sample `Server` configuration called :file:`clr-server.yaml`. + + .. code-block:: bash + + sudo tee -a ${WEB_ROOT_DIR}/${CLR_INSTALLER_CONF_DIR}/clr-server.yaml << EOF + #clear-linux-config + + # switch between aliases if you want to install to an actuall block device + # i.e /dev/sda + block-devices: [ + {name: "bdevice", file: "/dev/sda"} + ] + + targetMedia: + - name: \${bdevice} + type: disk + children: + - name: \${bdevice}1 + fstype: vfat + mountpoint: /boot + size: "150M" + type: part + - name: \${bdevice}2 + fstype: swap + size: "250M" + type: part + - name: \${bdevice}3 + fstype: ext4 + mountpoint: / + size: "0" # Use remaining disk space + type: part + + bundles: [ bootloader, os-core, os-core-update, vim ] + + autoUpdate: true + postArchive: false + postReboot: true + telemetry: false + hostname: clrlinux-server + keyboard: us + language: en_US.UTF-8 + kernel: kernel-native + + users: + - login: clrlinux + username: Clear Linux + # Password is "clear123" + password: \$6\$SJJMfnInWQg.CvMA\$m2F8dJGj71zvi9mSNMktHMsPH3qhBm8pgXDNdaBe2yFfgi479JXvEqWkvQ6OxIUgGNQ5YXFIF0tCn.hEXB90G/ + admin: true + - login: root + username: Root Root + # Password is "clear123" + password: \$6\$SJJMfnInWQg.CvMA\$m2F8dJGj71zvi9mSNMktHMsPH3qhBm8pgXDNdaBe2yFfgi479JXvEqWkvQ6OxIUgGNQ5YXFIF0tCn.hEXB90G/ + admin: true + + pre-install: [ + {cmd: "curl -o /tmp/add-issue.sh http://${IPXE_LAN_IP}:${IPXE_PORT}/${CLR_INSTALLER_CONF_DIR}/add-issue.sh"}, + {cmd: "chmod +x /tmp/add-issue.sh"} + ] + + post-install: [ + {cmd: "echo PermitRootLogin yes > \${chrootDir}/etc/ssh/sshd_config"}, + {cmd: "/tmp/add-issue.sh \${chrootDir}"} + ] + EOF + +#. Add following content to the :file:`add-issue.sh` script, which will be + used by the above two YAML configuration files: + + .. code-block:: bash + + sudo tee -a ${WEB_ROOT_DIR}/${CLR_INSTALLER_CONF_DIR}/add-issue.sh << EOF + #!/bin/bash + echo "Creating custom issue file for \$1" + + echo "Welcome to the Clear Linux* OS + + * Documentation: https://clearlinux.org/documentation + * Community Support: https://community.clearlinux.org + + " >> \$1/etc/issue + + exit 0 + EOF + +Configure network +***************** + +#. The DNS server, included with the `pxe-server` bundle, + conflicts with the DNS stub listener provided in `systemd-resolved`. + Disable the DNS stub listener and temporarily stop `systemd-resolved`. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd + sudo tee -a /etc/systemd/resolved.conf << EOF + [Resolve] + DNSStubListener=no + EOF + + sudo systemctl stop systemd-resolved + +#. Disable NetworkManager. The base installation of |CL| comes with two + network managers, systemd-networkd and NetworkManager, with the latter + being the default. systemd-networkd is recommended for a server use case, + so we will disable NetworkManager. + + .. code-block:: bash + + sudo systemctl mask --now NetworkManager + +#. Assign a static IP address to the LAN side network adapter + and restart `systemd-networkd`. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/network + sudo tee -a /etc/systemd/network/70-internal-static.network << EOF + [Match] + Name=${LAN_INTERFACE} + [Network] + DHCP=no + Address=${IPXE_LAN_IP}/${IPXE_SUBNET_BITMASK} + EOF + + sudo systemctl enable systemd-networkd + sudo systemctl restart systemd-networkd + +Setup NAT +********* + +#. Configure :abbr:`NAT (Network Address Translation)` to route traffic from + the LAN to the WAN network so clients can download upstream bundles for + installation. And to make these changes persistent during reboots, save the + changes to the firewall. + + .. code-block:: bash + + sudo iptables -t nat -F POSTROUTING + sudo iptables -t nat -A POSTROUTING -o ${WAN_INTERFACE} -j MASQUERADE + sudo systemctl enable iptables-save.service + sudo systemctl restart iptables-save.service + sudo systemctl enable iptables-restore.service + sudo systemctl restart iptables-restore.service + +#. Configure the kernel to forward network packets to different interfaces. + Otherwise, NAT will not work. + + .. code-block:: bash + + sudo mkdir -p /etc/sysctl.d + sudo tee -a /etc/sysctl.d/80-nat-forwarding.conf << EOF + net.ipv4.ip_forward=1 + EOF + + sudo tee -a /proc/sys/net/ipv4/ip_forward << EOF + 1 + EOF + +Setup dnsmaq for DHCP, DNS, and TFTP functionalities +**************************************************** + +#. Create a configuration file for `dnsmasq` to listen on a dedicated IP address + for TFTP, DNS, and DHCP functions. PXE clients on the LAN network will talk to + this IP address. + + .. code-block:: bash + + sudo tee -a /etc/dnsmasq.conf << EOF + listen-address=${IPXE_LAN_IP} + EOF + +#. Add the options to serve iPXE firmware images to clients over TFTP to + the :file:`dnsmasq` configuration file. + + .. code-block:: bash + + sudo tee -a /etc/dnsmasq.conf << EOF + enable-tftp + tftp-root=${TFTP_ROOT_DIR} + EOF + +#. Add the options to host a DHCP server for clients to the :file:`dnsmasq` + configuration file. + + .. code-block:: bash + + sudo tee -a /etc/dnsmasq.conf << EOF + dhcp-leasefile=/var/db/dnsmasq.leases + + dhcp-authoritative + dhcp-option=option:router,${IPXE_LAN_IP} + dhcp-option=option:dns-server,${IPXE_LAN_IP} + + dhcp-match=set:ipxeclient,60,IPXEClient* + dhcp-range=tag:ipxeclient,${IPXE_SUBNET}.2,${IPXE_SUBNET}.253,${IPXE_SUBNET_MASK_IP},15m + dhcp-range=tag:!ipxeclient,${IPXE_SUBNET}.2,${IPXE_SUBNET}.253,${IPXE_SUBNET_MASK_IP},6h + + dhcp-match=set:ipxeboot,175 + dhcp-boot=tag:ipxeboot,http://${IPXE_LAN_IP}:${IPXE_PORT}/${IPXE_APP_NAME}/ipxe_boot_script.ipxe + dhcp-boot=tag:!ipxeboot,undionly.kpxe,${IPXE_LAN_IP} + EOF + + The configuration provides the following important functions: + + * Directs clients without an iPXE implementation to the TFTP server + to acquire architecture-specific iPXE firmware images that allow them + to perform an iPXE boot. + * Activates only on the network adapter that has an IP address on the + defined subnet. + * Directs clients to the DNS server. + * Directs clients to the iPXE server for routing via NAT. + * Divides the private network into two pools of IP addresses. One pool + is for network boot and one pool is used after boot. Each pool has + their own lease times. + +#. Create a file for `dnsmasq` to record the IP addresses it provides + to clients. + + .. code-block:: bash + + sudo mkdir -p /var/db + sudo touch /var/db/dnsmasq.leases + +#. Create a TFTP hosting directory and populate it with the iPXE firmware. + + .. code-block:: bash + + sudo mkdir -p ${TFTP_ROOT_DIR} + sudo ln -sf /usr/share/ipxe/undionly.kpxe ${TFTP_ROOT_DIR}/undionly.kpxe + +#. Start `dnsmasq` and enable startup on boot. + + .. code-block:: bash + + sudo systemctl daemon-reload + sudo systemctl enable dnsmasq + sudo systemctl restart dnsmasq + +#. Start `systemd-resolved`. + + .. code-block:: bash + + sudo systemctl start systemd-resolved + + .. note:: + + `systemd-resolved` dynamically updates the list of DNS servers for the + LAN network if you use the `dnsmasq` DNS server. The setup creates a + pass-through DNS server that relies on the DNS servers listed in + :file:`/etc/resolv.conf`. + +Verify setup +************ + +Verify you can access these URLs before deploying: + +* \http://{$IPXE_LAN_IP}:{$IPXE_PORT}/${IPXE_APP_NAME}/ipxe_boot_script.ipxe +* \http://{$IPXE_LAN_IP}:{$IPXE_PORT}/${CLR_INSTALLER_CONF_DIR}/clr-desktop.yaml +* \http://{$IPXE_LAN_IP}:{$IPXE_PORT}/${CLR_INSTALLER_CONF_DIR}/clr-server.yaml +* \http://{$IPXE_LAN_IP}:{$IPXE_PORT}/${CLR_INSTALLER_CONF_DIR}/add-issue.sh + +Deploy +****** + +#. Connect your client system to the LAN network. + +#. Power on the client. + +#. Set your client to network boot. It should get an IP address and download + the iPXE script. + +#. When presented with the iPXE menu, select one of the options. The client + will then download and boot the |CL| image. Once booted, clr-installer will + download the assigned YAML configuration file and begin to install |CL|. + After installation, the client will reboot to |CL|. + +.. _iPXE: + http://ipxe.org/ + +.. _Installer YAML Syntax: + https://github.com/clearlinux/clr-installer/blob/master/scripts/InstallerYAMLSyntax.md diff --git a/_sources/get-started/virtual-machine-install/hyper-v.rst.txt b/_sources/get-started/virtual-machine-install/hyper-v.rst.txt new file mode 100644 index 00000000..0feb9fba --- /dev/null +++ b/_sources/get-started/virtual-machine-install/hyper-v.rst.txt @@ -0,0 +1,152 @@ +.. _hyper-v: + +|CL-ATTR| on Microsoft Hyper-V\* +################################ + +This page explains how to run a |CL-ATTR| :abbr:`VM (virtual machine)` on a +Microsoft\* Hyper-V\* hypervisor. + +.. contents:: + :local: + :depth: 1 + + +Overview +******** + +Hyper-V is a type 1 bare-metal hypervisor that runs directly on system +hardware. It is available for `Windows\* server`_ and client operating +systems, including `Windows 10`_. + +|CL| provides a virtual disk image for Hyper-V, which also includes +a :ref:`Hyper-V specific kernel ` and drivers. + +Prerequisites +************* + +* Enable virtualization on the host system from EFI/BIOS, such as: + + * `Intel® Virtualization Technology`_ (Intel® VT) + * `Intel® Virtualization Technology for Directed I/O`_ (Intel® VT-d) + +* Install Hyper-V on the appropriate Windows operating system: + + * `Install the Hyper-V role on Windows Server`_ + * `Install Hyper-V on Windows 10`_ + +* Configure the appropriate virtual networking in Hyper-V: + + * `Create a virtual network on Windows Server`_ + * `Create a virtual network on Windows 10`_ + + +Download the |CL| disk image for Hyper-V +**************************************** + +#. Download the :file:`clear-[VERSION]-azure-hyperv.vhd.gz` for Microsoft* + Hyper-V from the `downloads`_ website. + +#. Verify and extract the image using these instructions: + :ref:`download-verify-decompress`. + +#. Extract the compressed file using software such as the + 7-Zip\* tool or the WinZip\* tool. + + After extraction, the file should be named :file:`clear-[VERSION]-azure-hyperv.vhd`. + +Create and configure new VM +**************************** + +#. Open the **Hyper-V Manager** from the Start menu. + + .. figure:: figures/hyper-v/hyper-v-01.png + :scale: 100% + :alt: Hyper-V Manager from the Start menu + + Figure 1: Hyper-V Manager from the Start menu + + .. note:: + + You may need to manually enable Hyper-V on a Windows\* machine. Review + ``Windows Features``. + +#. Create a *New Virtual Machine* by clicking the :guilabel:`Action` menu, + then selecting :guilabel:`New` and :guilabel:`Virtual Machine...`. + + .. figure:: figures/hyper-v/hyper-v-02.png + :scale: 100% + :alt: New Virtual Machine in Hyper-V Manager + + Figure 2: New Virtual Machine in Hyper-V Manager + +#. Follow the *New Virtual Machine Wizard* to create a new virtual machine + specifying the options below: + + - **Name**: Choose name (for example, ClearLinuxOS-VM) + - **Specify Generation**: Generation 1 + - **Startup memory**: 2048 MB or more + - **Configure Networking**: Change :guilabel:`Connection` to `Default Switch` + - **Connect Virtual Hard Disk**: Select :guilabel:`Use an existing virtual + hard disk` and browse to find the + :file:`clear-[VERSION]-azure-hyperv.vhd` file. + + After finishing the wizard, the VM will be created but not powered on. + +#. Configure the VM by right-clicking it in the Hyper-V Manager and selecting + :guilabel:`Settings...`. Figure 3 shows the Settings page after configuration selections. + + **Optional** + + - If you wish to `Encrypt state and virtual machine traffic, under + :guilabel:`Security`, select :guilabel:`Add Key Storage Drive`. + + - Under :guilabel:`Processor`, consider increasing the number of virtual + processors assigned to the |CL| VM to improve performance. + + .. figure:: figures/hyper-v/hyper-v-03.png + :scale: 100% + :alt: |CL| VM Settings in Hyper-V Manager + + Figure 3: |CL| VM Settings page after configuration + +#. Click :guilabel:`Apply` at the bottom of the VM Settings screen. + +#. Click :guilabel:`OK` at the bottom of the VM Settings screen. + + +Start the VM +************ + +#. Start the |CL| VM by right-clicking the VM in Hyper-V Manager and + selecting :guilabel:`Start`. + +#. Connect to the VM console by right-clicking the VM in Hyper-V Manager and + selecting :guilabel:`Connect...`. A new *Virtual Machine Connection* + window is displayed. + +#. After |CL| is booted, log in to the console with user *root*. You are + prompted to set a new password immediately. + + .. code-block:: console + + > User: root + +|CL-ATTR| on Microsoft Hyper-V\* is ready for use. + +Related topics +************** + +* :ref:`increase-virtual-disk-size` + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _`Windows\* Server`: https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/hyper-v-on-windows-server +.. _`Windows 10`: https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/index +.. _`Intel® Virtualization Technology`: http://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html +.. _`Intel® Virtualization Technology for Directed I/O`: https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhancing-intel-platforms-for-efficient-virtualization-of-io-devices +.. _`Install the Hyper-V role on Windows Server`: https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/get-started/install-the-hyper-v-role-on-windows-server +.. _Install Hyper-V on Windows 10: https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v +.. _`Create a virtual network on Windows Server`: https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v/get-started/create-a-virtual-switch-for-hyper-v-virtual-machines +.. _`Create a virtual network on Windows 10`: https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/connect-to-network +.. _downloads: https://clearlinux.org/downloads + diff --git a/_sources/get-started/virtual-machine-install/kvm.rst.txt b/_sources/get-started/virtual-machine-install/kvm.rst.txt new file mode 100644 index 00000000..2b73fc4f --- /dev/null +++ b/_sources/get-started/virtual-machine-install/kvm.rst.txt @@ -0,0 +1,220 @@ +.. _kvm: + +|CL-ATTR| on KVM +################ + +This page explains how to run |CL-ATTR| in a virtualized environment using +:abbr:`KVM (Kernel-based Virtual Machine)`. + +.. contents:: + :local: + :depth: 1 + +Install QEMU-KVM +**************** + +#. Enable the `Intel® Virtualization Technology`_ (Intel® VT) and the + `Intel® Virtualization Technology for Directed I/O`_ (Intel® VT-d) in the + host machine’s BIOS. + +#. Log in and open a terminal emulator. + +#. Install `QEMU*-KVM` on the host machine. Below are some examples using + different distros: + + ==================== ========================================= + Host OS Installation command + ==================== ========================================= + |CL| :command:`sudo swupd bundle-add kvm-host` + -------------------- ----------------------------------------- + Ubuntu\* and Mint\* :command:`sudo apt-get install qemu-kvm` + -------------------- ----------------------------------------- + Fedora :command:`dnf install qemu-kvm` + ==================== ========================================= + +Download and launch the virtual machine image +********************************************* + +#. Download the latest pre-built |CL| KVM image file from + the `image `_ directory. Look for + ``clear--kvm.img.xz``. You can also use this command: + + .. code-block:: bash + + curl -o clear.img.xz https://cdn.download.clearlinux.org/image/$(curl https://cdn.download.clearlinux.org/image/latest-images.json | grep -o 'clear-[0-9]*-kvm.img.xz' | head -1) + +#. Uncompress the downloaded image: + + .. code-block:: bash + + xz -dv clear.img.xz + +#. Download the 3 OVMF files (`OVMF.fd`, `OVMF_CODE.fd`, `OVMF_VARS.fd`) that + provides UEFI support for virtual machines. + + .. code-block:: bash + + curl -O https://cdn.download.clearlinux.org/image/OVMF.fd + curl -O https://cdn.download.clearlinux.org/image/OVMF_CODE.fd + curl -O https://cdn.download.clearlinux.org/image/OVMF_VARS.fd + + .. note:: + + The default OVMF files from |CL| may not work for some non-|CL| distro + version(s). You may get an `ASSERT` in the `debug.log` file when + starting the VM. If you encounter this, use the distro-specific OVMF + files instead. + + +#. Download the `start_qemu.sh`_ script from the + `image `_ directory. This script + will launch the |CL| VM and provide console interaction within the same + terminal emulator window. + + .. code-block:: bash + + curl -O https://cdn.download.clearlinux.org/image/start_qemu.sh + +#. Start the |CL| KVM virtual machine: + + .. code-block:: bash + + sudo bash ./start_qemu.sh clear.img + +#. Log in as ``root`` user and set a new password. + +Optional: Enable SSH access +*************************** + +To interact with the |CL| VM remotely through SSH instead of the console it +was launched from, follow these steps. + +#. Enable and configure SSH in the |CL| VM to allow root login as described in + :ref:`openssh-server`. + + +#. SSH into the |CL| VM using the port `10022`. This port number is set in + :file:`start_qemu.sh` and passed through to the SSH service running on port + 22 + + .. code-block:: bash + + ssh -p 10022 root@ + + +Optional: Install a graphical user interface (GUI) +************************************************** + +To add :abbr:`GDM (GNOME Display Manager)` to the |CL| VM, follow these steps: + +#. Shutdown the active |CL| VM. + + .. code-block:: bash + + poweroff + +#. Install the Spice viewer on the localhost or remote system. Below are some + examples using different distros: + + ==================== ========================================= + Host OS Installation command + ==================== ========================================= + |CL| :command:`sudo swupd bundle-add virt-viewer` + -------------------- ----------------------------------------- + Ubuntu\* and Mint\* :command:`sudo apt-get install virt-viewer` + -------------------- ----------------------------------------- + Fedora :command:`dnf install virt-viewer` + ==================== ========================================= + + +#. Modify the :file:`start_qemu.sh` script to increase memory (`-m`), add + graphics driver (`-vga`), and add Spice (`-spice`, `-usb`, and + `-device`) support. + + .. code-block:: console + :emphasize-lines: 5-10 + + qemu-system-x86_64 \ + -enable-kvm \ + ${UEFI_BIOS} \ + -smp sockets=1,cpus=4,cores=2 -cpu host \ + -m 4096 \ + -vga qxl \ + -nographic \ + -spice port=5924,disable-ticketing \ + -usb \ + -device usb-tablet,bus=usb-bus.0 \ + -drive file="$IMAGE",if=virtio,aio=threads,format=raw \ + -netdev user,id=mynet0,hostfwd=tcp::${VMN}0022-:22,hostfwd=tcp::${VMN}2375-:2375 \ + -device virtio-net-pci,netdev=mynet0 \ + -debugcon file:debug.log -global isa-debugcon.iobase=0x402 $@ + +#. Due to changes in the :file:`start_qemu.sh` script from the previous step, + having previously used OVMF files will result in the VM not booting + properly and you returning to the UEFI shell. The easiest way to avoid this + is to delete the 3 OVMF files and reobtain originals before relaunching the + VM: + + .. code-block:: bash + + rm -v OVMF*.fd + curl -O https://cdn.download.clearlinux.org/image/OVMF.fd + curl -O https://cdn.download.clearlinux.org/image/OVMF_CODE.fd + curl -O https://cdn.download.clearlinux.org/image/OVMF_VARS.fd + +#. Increase the size of the VM by 10GB to accommodate the GDM installation: + + .. code-block:: bash + + qemu-img resize -f raw clear--kvm.img +10G + +#. Relaunch the |CL| VM: + + .. code-block:: bash + + sudo ./start_qemu.sh clear.img + +#. Determine the IP address of the host on which you will launch the VM. + Substitute in the next step with this information. + + .. code-block:: bash + + ip a + + +#. From the local host or remote system, open a new terminal emulator window + and connect into the |CL| VM using the Spice viewer: + + .. code-block:: bash + + remote-viewer spice://:5924 + +#. Log in as `root` user into the |CL| VM. + +#. Follow these steps from :ref:`increase-virtual-disk-size` to resize the + partition of the virtual disk of the VM. + +#. Add GDM to the |CL| VM: + + .. code-block:: bash + + swupd bundle-add desktop-autostart + +#. Reboot the |CL| VM to start GDM: + + .. code-block:: bash + + reboot + +#. Go through the GDM out-of-box experience (OOBE). + +#. The default aspect ratio of the GDM GUI for the |CL| VM is 4:3. To change + it, use GDM's `Devices > Displays` setting tool (located at the top-right + corner). + + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _Intel® Virtualization Technology: https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html +.. _Intel® Virtualization Technology for Directed I/O: https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhancing-intel-platforms-for-efficient-virtualization-of-io-devices +.. _start_qemu.sh: https://cdn.download.clearlinux.org/image/start_qemu.sh diff --git a/_sources/get-started/virtual-machine-install/parallels.rst.txt b/_sources/get-started/virtual-machine-install/parallels.rst.txt new file mode 100644 index 00000000..eeaf8f67 --- /dev/null +++ b/_sources/get-started/virtual-machine-install/parallels.rst.txt @@ -0,0 +1,132 @@ +.. _parallels: + +|CL-ATTR| on Parallels\* Desktop for Mac\* +########################################## + +This page explains how to run |CL| Server in :abbr:`CLI (command-line interface)` mode as a guest OS in Parallels Desktop 14 for Mac. + +Parallels Desktop for Mac is virtualization software that allows other +operating systems, such as Linux, to run side-by-side with macOS\*. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* Install Parallels Desktop 14 for Mac. + +Download ISO image +****************** + +#. Download a live-server ISO installation file from https://clearlinux.org/downloads. + This guide uses |CL| Server 30140 as its example. + +#. Unzip the ISO image with the command: + + .. code-block:: bash + + gunzip clear-30140-live-server.iso.xz + +Initialize new VM +***************** + +Start Parallels and initialize your :abbr:`VM (Virtual Machine)` with the +following steps. + +#. Go to :menuselection:`File > New`. + +#. In the opening dialog window, select + :guilabel:`Install Windows or another OS from a DVD or image`, then click + :guilabel:`Continue`. (See Figure 1.) + + .. figure:: /_figures/parallels/parallels-01.png + :alt: Parallels opening dialog + + Figure 1: Parallels opening dialog + +#. On the next screen, select :guilabel:`Image File`, then click + :guilabel:`Select a file...` as shown in Figure 2. + + .. figure:: /_figures/parallels/parallels-02.png + :alt: Dialog to select source for VM + + Figure 2: Dialog to select source for VM + +#. Select your ISO file. The system displays the warning message "Unable to + detect operating system", as shown in Figure 3. This message is expected and + can be ignored. Click :guilabel:`Continue`. + + .. figure:: /_figures/parallels/parallels-03.png + :alt: Warning that OS is not detected + + Figure 3: Warning that OS is not detected + +#. You are prompted to select your OS, as shown in Figure 4. Select + :menuselection:`More Linux > Other Linux` from the drop-down menu and click + :guilabel:`Continue`. + + .. figure:: /_figures/parallels/parallels-04.png + :alt: Select OS from drop-down menu + + Figure 4: Select OS from drop-down menu + +#. Name your VM and check :guilabel:`Customize settings before installation`. + (See Figure 5.) + + .. figure:: /_figures/parallels/parallels-05.png + :alt: Name and Location screen + + Figure 5: Name and Location screen + +#. Click :guilabel:`Create`. The Configuration window for the new VM opens, as + shown in Figure 6. + + Select :menuselection:`Hardware > Boot Order`. + + .. figure:: /_figures/parallels/parallels-06.png + :alt: VM Configuration window + + Figure 6: VM Configuration window + +#. Expand :guilabel:`Advanced Settings`. Set :guilabel:`BIOS` to “EFI 64-bit” + and in the :guilabel:`Boot flags` field, enter “vm.bios.efi=1” as shown in + Figure 7. + + .. figure:: /_figures/parallels/parallels-07.png + :alt: Advanced configuration settings + + Figure 7: Advanced configuration settings + +#. Close the Configuration window and click :guilabel:`Continue`. + + If camera and microphone access restriction warnings are displayed, you can + ignore them. + +Install |CL| on VM +****************** + +#. Follow the prompts and install |CL| using the text-based installer as shown + in Figure 8. + + Refer to :ref:`bare-metal-install-server` for additional installation + instructions. + + .. figure:: /_figures/parallels/parallels-08.png + :alt: On screen instructions from text-based installer + + Figure 8: On screen instructions from text-based installer + +#. After installation, reboot the VM. You are prompted to log in, as shown + in Figure 9. Log in with the credentials you used when you installed |CL| + on the VM. + + .. figure:: /_figures/parallels/parallels-09.png + :alt: Log in prompt + + Figure 9: Log in prompt + + +Congratulations! You have successfully set up a |CL| VM using Parallels +Desktop for Mac. diff --git a/_sources/get-started/virtual-machine-install/proxmox.rst.txt b/_sources/get-started/virtual-machine-install/proxmox.rst.txt new file mode 100644 index 00000000..c37a0ec9 --- /dev/null +++ b/_sources/get-started/virtual-machine-install/proxmox.rst.txt @@ -0,0 +1,209 @@ +.. _proxmox: + +|CL-ATTR| on Proxmox\* Virtual Environment +########################################## + +This guide explains how to create a new VM in Proxmox VE 6.1-3, install and run |CL| on as a guest OS. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* Proxmox VE 6.1-3 server already set up and you have familiarity with how + to use it. + +Download the Latest |CL| Live Server Image +****************************************** + +#. Visit our `Downloads`_ page. + +#. Download the file :file:`clear--live-server.iso`, + also called the |CL| Server. + + .. note:: + + is the latest |CL| auto-numbered release. + +Upload |CL| Live Server Image to Promox Server +********************************************** + +#. Connect to your Proxmox server and log into an account with sufficient + permission to create and manage VMs. + +#. Under the :guilabel:`Server View` window, select the :guilabel:`local` + storage. See Figure 1. + +#. On the right window, click :guilabel:`Upload`. + + .. figure:: ../../_figures/proxmox/proxmox-01.png + :scale: 100% + :alt: Proxmox - Upload ISO + + Figure 1: Proxmox - Upload ISO + +#. Set the :guilabel:`Content` as `ISO image`. See Figure 2. + +#. Click :guilabel:`Select File...` and select the |CL| ISO. + +#. Click :guilabel:`Upload`. + + .. figure:: ../../_figures/proxmox/proxmox-02.png + :scale: 100% + :alt: Proxmox - Select ISO to upload + + Figure 2: Proxmox - Select ISO to upload + + The ISO should now appear in the :guilabel:`Content` list. See Figure 3. + + .. figure:: ../../_figures/proxmox/proxmox-03.png + :scale: 100% + :alt: Proxmox - Content list + + Figure 3: Proxmox - Content list + +Create VM on Proxmox +******************** + +#. Under the :guilabel:`Server View` window, select your Proxmox node. + See Figure 4. + +#. On the right window, click :guilabel:`Create VM`. + + .. figure:: ../../_figures/proxmox/proxmox-04.png + :scale: 100% + :alt: Proxmox - Create VM + + Figure 4: Proxmox - Create VM + +#. In the :guilabel:`General` tab: + | See Figure 5. + + a. Check the :guilabel:`Advanced` checkbox. + + #. In the :guilabel:`Name` field, give the VM a name. + + .. figure:: ../../_figures/proxmox/proxmox-05.png + :scale: 100% + :alt: Proxmox - Create VM - General settings + + Figure 5: Proxmox - Create VM - General settings + +#. In the :guilabel:`OS` tab: + See Figure 6. + + a. Select :guilabel:`Use CD/DVD disc image file (iso)`. + + #. For :guilabel:`Storage`, select :guilabel:`local`. + + #. For :guilabel:`ISO image`, select the |CL| ISO you uploaded earlier. + + #. Set the :guilabel:`Type` to :guilabel:`Linux`. + + #. Set the :guilabel:`Version` to :guilabel:`5.x - 2.6 kernel`. + + .. figure:: ../../_figures/proxmox/proxmox-06.png + :scale: 100% + :alt: Proxmox - Create VM - OS settings + + Figure 6: Proxmox - Create VM - OS settings + +#. In the :guilabel:`System` tab: + See Figure 7. + + a. For :guilabel:`BIOS`, select :guilabel:`OVMF (UEFI)`. + + #. For :guilabel:`Storage`, select an appropriate location. + + #. For :guilabel:`Machine`, select :guilabel:`q35`. + + .. figure:: ../../_figures/proxmox/proxmox-07.png + :scale: 100% + :alt: Proxmox - Create VM - System settings + + Figure 7: Proxmox - Create VM - System settings + +#. In the :guilabel:`Hard Disk` tab: + See Figure 8. + + a. For :guilabel:`Disk size (GiB)`, set the desired disk size for your VM. + A minimum of 4GB is required for |CL|. + + .. figure:: ../../_figures/proxmox/proxmox-08.png + :scale: 100% + :alt: Proxmox - Create VM - Hard Disk settings + + Figure 8: Proxmox - Create VM - Hard Disk settings + +#. In the :guilabel:`CPU` tab: + See Figure 9. + + a. Set the :guilabel:`Type` to :guilabel:`host`. + + #. For the :guilabel:`Extra CPU Flags`, scroll to the bottom and turn on the + :guilabel:`aes` setting by clicking the :guilabel:`+` radio button. + + .. figure:: ../../_figures/proxmox/proxmox-09.png + :scale: 100% + :alt: Proxmox - Create VM - CPU settings + + Figure 9: Proxmox - Create VM - CPU settings + +#. In the :guilabel:`Memory` tab: + See Figure 10. + + a. For :guilabel:`Memory (MiB)`, set a desired value. + + .. figure:: ../../_figures/proxmox/proxmox-10.png + :scale: 100% + :alt: Proxmox - Create VM - Memory settings + + Figure 10: Proxmox - Create VM - Memory settings + +#. In the :guilabel:`Network` tab: + See Figure 11. + + a. For :guilabel:`Model`, select :guilabel:`E1000`. + + .. figure:: ../../_figures/proxmox/proxmox-11.png + :scale: 100% + :alt: Proxmox - Create VM - Network settings + + Figure 11: Proxmox - Create VM - Network settings + +#. In the :guilabel:`Confirm` tab: + See Figure 12. + + a. Confirm the settings. + + #. Click :guilabel:`Finish` to create the VM. The new VM should appear + under the :guilabel:`Server View` window. + + .. figure:: ../../_figures/proxmox/proxmox-12.png + :scale: 100% + :alt: Proxmox - Create VM - Confirm settings + + Figure 12: Proxmox - Create VM - Confirm settings + +Start VM and Install |CL| on Promox +*********************************** + +#. Under the :guilabel:`Server View` window, select your newly-created VM. + See Figure 13. + +#. On the right window, click :guilabel:`Start`. + +#. Click :guilabel:`Console` button to bring up a console and interact with it. + + .. figure:: ../../_figures/proxmox/proxmox-13.png + :scale: 100% + :alt: Proxmox - Start VM + + Figure 13: Proxmox - Start VM + +#. Follow the instructions in the :ref:`bare-metal-install-server` guide + starting at the `Launch the Clear Linux OS Installer` section. + +.. _Downloads: https://clearlinux.org/downloads diff --git a/_sources/get-started/virtual-machine-install/virt-manager.rst.txt b/_sources/get-started/virtual-machine-install/virt-manager.rst.txt new file mode 100644 index 00000000..cdca51dc --- /dev/null +++ b/_sources/get-started/virtual-machine-install/virt-manager.rst.txt @@ -0,0 +1,225 @@ +.. _virt-manager: + +|CL-ATTR| using virt-manager +############################ + +This page explains how to create a |CL-ATTR| virtual machine using the +`virt-mgr`_ desktop application with |CL| as the guest operating system. +These instructions support the |CL| live-server installer to create the |CL| +:abbr:`VM (Virtual Machine)`. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +#. Enable virtualization, such as `Intel® Virtualization Technology`_ + (Intel® VT), on the host system from the UEFI firmware setup. + +#. Install the software bundles kvm-host and virt-manager-gui using + :command:`swupd`: + + .. code-block:: bash + + sudo swupd bundle-add kvm-host virt-manager-gui + +#. Add your userid to the `kvm` and `libvirt` groups. + + .. code-block:: bash + + sudo usermod -G kvm -a $USER + sudo usermod -G libvirt -a $USER + +#. Enable the `libvirtd` daemon and reboot the system to complete the + process. + + .. code-block:: bash + + sudo systemctl enable libvirtd + sudo reboot + + +Download the |CL| installer ISO +******************************* + +There are several options available to set up and use a |CL| VM with +:command:`virt-manager`. You can either download the `KVM` image and run it +as-is or download the installer ISO and run it to create a new installation of +|CL|. + +This example uses the live-server-installer ISO to create a new installation. + +#. Download the `Clear Linux* OS Server` from the `Downloads`_ page. + +#. (Optional) Validate the integrity of the downloaded image by checking the + file hash and signatures. Refer to :ref:`validate-signatures` for detailed + steps. + +Launch and set up virt-manager +****************************** + +Virt-manager is a GUI-based virtual machine manager that runs in your desktop +environment. This example uses the Gnome\* desktop. + +#. Launch the Virtual Machine Manager from the applications window. The + application window opens as shown in Figure 1. + + .. figure:: figures/virtmgr/virt-manager-01.png + :scale: 100% + :alt: Virtual Machine Manager + + Figure 1: Virtual Machine Manager + +#. In the `Name` field, select and highlight the `QEMU/KVM` item, then select + :menuselection:`Edit > Connection Details`. A dialog box with + `QEMU/KVM Connection Details` opens as shown in Figure 2. + + .. figure:: figures/virtmgr/virt-manager-02.png + :scale: 100% + :alt: QEMU/KVM Connection Details + + Figure 2: QEMU/KVM Connection Details + +#. On the `Overview` tab, check the `Autoconnect` field. Select the `Virtual + Networks` tab and in the lower left of the dialog window, select the + :guilabel:`+` key to add a new network connection. The `Create a new virtual + network` dialog window opens as shown in Figure 3. To accept the default + values, select the :guilabel:`Finish` button. + + .. figure:: figures/virtmgr/virt-manager-03.png + :scale: 100% + :alt: Create a new virtual network + + Figure 3: Create a new virtual network + +#. Close the `QEMU/KVM Connection details` dialog box and return to the Virtual + Machine Manager main console. You are ready to create your VM. + +Create a new virt-manager virtual machine +***************************************** + +In the Virtual Machine Manager main console, either select +:menuselection:`File > New Virtual Machine` or click the `Create a +new virtual machine` icon. This launches the `New VM` wizard, shown in Figure 4. + +.. figure:: figures/virtmgr/virt-manager-04.png + :scale: 100% + :alt: New VM + + Figure 4: New VM dialog box, step 1 + +#. Select `Local install media (ISO image or CDROM)` and select the + :guilabel:`Forward` button. + +#. In step 2 of the `New VM` wizard, you can choose ISO or CDROM install + media. + + a. Uncheck `Automatically detect from the installation media / source` + field and select the :guilabel:`Browse...` button as shown in Figure 5. + + .. figure:: figures/virtmgr/virt-manager-05.png + :scale: 100% + :alt: New VM + + Figure 5: New VM dialog box, step 2: Choose media + + #. In the `Choose Storage Volume` dialog, select the + :guilabel:`Browse Local` button as shown in Figure 6. Browse to + the ISO image that you downloaded earlier and open it. + + .. figure:: figures/virtmgr/virt-manager-06.png + :scale: 100% + :alt: Choose storage volume + + Figure 6: Choose storage volume dialog box + + #. In the `Choose the operating system you are installing` search field, + type `generic` and select the `Generic default` value when it is displayed. + Select the :guilabel:`Forward` button as shown in Figure 7. + + .. figure:: figures/virtmgr/virt-manager-07.png + :scale: 100% + :alt: New VM + + Figure 7: New VM dialog box, step 2: Choose operating system + + .. note:: + + A message may be displayed that says the emulator does not have + search permissions for the ISO image path. Select :guilabel:`Yes` to + proceed to the next step. + +#. Step 3 of the `New VM` wizard allocates the memory and CPUs for + the new VM. Choose settings that are valid for the resources on your host + system. This example sets `Memory` to 2048GB and `CPUs` to 1. Once complete, + select the :guilabel:`Forward` button as shown in Figure 8. + + .. figure:: figures/virtmgr/virt-manager-08.png + :scale: 100% + :alt: New VM Choose Memory and CPU settings dialog box + + Figure 8: New VM dialog box, step 3: Choose Memory and CPU settings + +#. Step 4 of the `New VM` wizard sets up the storage media for your VM. You + can create a new disk image or use an existing image. This example selects + `Enable storage for this virtual machine` and creates a 20GB image for it. + Once complete, select the :guilabel:`Forward` button as shown in Figure 9. + + .. figure:: figures/virtmgr/virt-manager-09.png + :scale: 100% + :alt: New VM Enable storage dialog box + + Figure 9: New VM dialog box, step 4: Enable storage + +#. Step 5 of the `New VM` wizard displays the selections you made and allows + you to customize the configuration before running the installation. Select the + `Customize configuration before install` checkbox and select the + :guilabel:`Finish` button as shown in Figure 10. + + .. figure:: figures/virtmgr/virt-manager-10.png + :scale: 100% + :alt: New VM Ready to begin the installation dialog box + + Figure 10: New VM dialog box, step 5: Ready to begin the installation + +#. Customize the installation process by changing the firmware from `BIOS` to + `UEFI x86_64`. |CL| requires UEFI firmware. In the `Firmware` field, select + the :file:`UEFI x86_64:/usr/share/qemu/OVMF.fd` entry as shown in Figure 11 + and select the :guilabel:`Apply` button. + + .. figure:: figures/virtmgr/virt-manager-11.png + :scale: 100% + :alt: vm1 on QEMU/KVM dialog box + + Figure 11: vm1 on QEMU/KVM dialog box + +#. Begin the installation by selecting the :guilabel:`Begin Installation` in + the upper left corner of the `vm1 on QEMU/KVM` dialog box. + +Install |CL| in the virt-manager VM +*********************************** + +To install |CL| in your VM, follow the instructions in the getting started +guide :ref:`bare-metal-install-server`. + +.. note:: + + You do not need to set up the network as described in the installation + guide, because you already downloaded the ISO image and connected to your + VM. Your network will show up as a wired connection. + +Congratulations! You have successfully installed |CL| in your new VM and can +begin using it immediately. The `virt-manager` tool is maintained on GitHub\* +at `virt-manager-github`_. + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _virt-mgr: https://www.virt-manager.org + +.. _Downloads: https://clearlinux.org/downloads + +.. _virt-manager-github: https://github.com/virt-manager/virt-manager + +.. _Intel® Virtualization Technology: https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html diff --git a/_sources/get-started/virtual-machine-install/virtualbox-cl-installer.rst.txt b/_sources/get-started/virtual-machine-install/virtualbox-cl-installer.rst.txt new file mode 100644 index 00000000..9a164281 --- /dev/null +++ b/_sources/get-started/virtual-machine-install/virtualbox-cl-installer.rst.txt @@ -0,0 +1,407 @@ +.. _virtualbox-cl-installer: + +|CL-ATTR| on VirtualBox\* +######################### + +This page explains how to create a virtual machine on the `VirtualBox`_ +hypervisor with |CL-ATTR| as the guest operating system. These instructions +support the |CL| live-server installer to create the |CL| virtual machine (VM). + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +#. Enable virtualization, such as `Intel® Virtualization Technology `_ + (Intel® VT), on the host system from EFI/BIOS. + +#. Download and install |VB| **version 6.0 or later** from + `VirtualBox`_ using the `VirtualBox Installation Instructions`_ for your + platform. + +Download and extract the |CL| installer ISO +******************************************* + +#. Download the :file:`clear--live-server.iso.xz` of + |CL| on the `Downloads`_ page. + +#. Validate the integrity of the downloaded image by checking the file hash + and signatures. Refer to :ref:`validate-signatures` for detailed steps. + +#. Decompress the downloaded image. + + - On Windows, you can use `7zip`_ to extract the file by right-clicking the + file to *Extract Here* (in the same directory) + + .. figure:: figures/vbox/virtualbox-cl-installer-01.png + :scale: 100% + :alt: 7zip extract here command + + Figure 1: 7zip extract here command + + - On Linux : + + .. code-block:: bash + + xz -d clear--live-server.iso.xz + +#. Delete the originally downloaded compressed file. + +Create a new |VB| virtual machine +********************************* + +A new :abbr:`VM (Virtual Machine)` needs to be created in |VBM| where |CL| +will be installed. General instructions for creating a virtual machine and +details about using different settings are available in the VirtualBox manual section `Creating Your First Virtual Machine`_. + +#. Launch the |VBM| from your host system. + +#. Click the :guilabel:`New` button to create a new VM. + +#. Choose :guilabel:`Expert mode`. + +#. On the :guilabel:`Create Virtual Machine` screen, enter the following settings: + + - **Name**: Choose name (e.g. ClearLinuxOS-VM). + - **Type**: Linux + - **Version**: **Linux 2.6 / 3.x / 4.x (64-bit)** + - **Hard disk**: `Create a virtual hard disk now` + - **Memory size default**: 2048 MB (Adjust appropriately.) + + .. note:: + Later, if you want to change the amount of RAM allocated, power down your VM. Return to :file:`Settings > System` and change + :guilabel:`Base Memory` to the desired size. + + .. figure:: figures/vbox/virtualbox-cl-installer-02.png + :scale: 100% + :alt: Create Virtual Machine + + Figure 2: Create Virtual Machine + +#. Click :guilabel:`Create`. + +#. On the :guilabel:`Create Virtual Hard Disk` screen, select: + + - **File location** + - **File size**: **32.00 GB**. Adjust size to your needs. + - **Hard disk file type**: `VDI (VirtualBox Disk Image)` + - **Storage on physical hard disk**: `Dynamically allocated` + + .. figure:: figures/vbox/virtualbox-cl-installer-03.png + :scale: 100% + :alt: Create Virtual Hard Disk + + Figure 3: Create Virtual Hard Disk + +#. Click :guilabel:`Create`. + + A new virtual machine will be created and appear in the |VBM|. + +#. Click :guilabel:`Settings` to configure the |CL| VM. + +#. In the left-hand menu, navigate to the :menuselection:`System` menu. + +#. On the :guilabel:`Motherboard` tab, select the :guilabel:`Chipset` menu, and + then select :menuselection:`ICH9`. See Figure 4. + + .. note:: + + You can select which chipset will be presented to the virtual machine. + Consult the `VM VirtualBox User Manual`_ for more details. + +#. In :guilabel:`Enabled Features`, check these boxes: + + - **Enable I/O APIC** + - **Enable EFI (special OSes only)** + + .. figure:: figures/vbox/virtualbox-cl-installer-04.png + :scale: 100% + :alt: Settings > System + + Figure 4: Settings > System + + .. note:: + + By default, only 1 virtual CPU is allocated to the new VM. Consider + increasing the number of virtual processors allocated to the virtual + machine under Settings > System > Processor for increased + performance. + +#. Click :guilabel:`OK`. + +Install |CL| on the |VB| VM +*************************** + +|CL| is ready to be installed. + +Mount the installation ISO +========================== + +The |CL| installer ISO needs to be mounted as a virtual CD-ROM on the VM +before powering the VM on. + +#. From the *ClearLinux-OS* :guilabel:`Settings` menu at left, select + :guilabel:`Storage`. + +#. From :guilabel:`Storage Devices`, middle column, click the blue + disk labeled :guilabel:`Empty`. + +#. From the :guilabel:`Attributes` menu, click the blue CD disk next to + the :guilabel:`Optical Drive` drop down menu and click + :guilabel:`Choose Virtual Optical Disk File...` + + .. figure:: figures/vbox/virtualbox-cl-installer-05.png + :scale: 100% + :alt: Choose Virtual Optical Disk Drive + + Figure 5: Choose Virtual Optical Disk Drive + +#. Where there appears :guilabel:`Please choose a virtual optical disk file`, + select the ISO file and click *Open*. + + .. figure:: figures/vbox/virtualbox-cl-installer-06.png + :scale: 100% + :alt: Mounting an ISO + + Figure 6: Mounting an ISO + +#. Click :guilabel:`OK` to exit and return to the main |VBM|. + +Install |CL| with live-server installer +======================================= + +#. In the |VBM|, select virtual machine you created and click :guilabel:`Start`. + + .. figure:: figures/vbox/virtualbox-cl-installer-07.png + :scale: 100% + :alt: Start the installer + + Figure 7: Start the installer + + .. note:: + + To release the mouse cursor from the VM console window, press the right + :kbd:`Ctrl` key on the keyboard. + +#. When :guilabel:`Clear Linux Installer` in boot manager appears, + select :kbd:`Enter`. Do not install the bundle `desktop-autostart`. + +#. Follow the steps in :ref:`bare-metal-install-server` to + install |CL| onto the VM virtual disk. Note: + + #. In :guilabel:`Configure Installation Media`, navigate top + VBOX HARDDISK, and then select :guilabel:`Confirm`. + + #. In :menuselection:`Advanced options --> Manage User`, create an + administrative user. + + #. Do not install the bundle `desktop-autostart`. + +#. When |CL| installation is complete, click :guilabel:`Exit`. + +#. At the prompt, enter: + + .. code-block:: bash + + shutdown now + +Unmount the ISO +=============== + +The |CL| installer ISO needs to be unmounted to allow the VM to boot from the +virtual hard disk. + +#. Return to the |VBM|. + +#. Click :guilabel:`Settings` to configure the |CL| VM. + +#. From the VM :guilabel:`Settings` window, navigate to the :guilabel:`Storage` + pane in the left menu. + +#. From the middle :guilabel:`Storage Devices` column, click the blue CD disk + labeled :guilabel:`clear--live-server.iso` under the + :guilabel:`Controller: IDE`. + +#. From the :guilabel:`Attributes` column on the right, in :guilabel:`Optical Drive`, + select the blue CD icon beside and click + :guilabel:`Remove Disk from Virtual Drive`. + + .. figure:: figures/vbox/virtualbox-cl-installer-08.png + :scale: 100% + :alt: Remove Disk from Virtual Drive + + Figure 8: Remove Disk from Virtual Drive + +#. Click :guilabel:`OK` to exit the :guilabel:`VM Settings` menu and return to + the main |VBM|. + +Install |VB| Linux Guest Additions +================================== + +|CL| provides Linux Guest Additions drivers for full compatibility using an +install script in the **kernel-lts** (Long Term Support) bundle by |CL|. + +#. From the |VBM| select the |CL| VM, and select :guilabel:`Start`. + +#. In the VM Console, log in as the administrative user previously created. + + .. note:: + A message may appear: "A kernel update is available: you may wish + to reboot the system." + + To update the kernel, enter: + + .. code-block:: bash + + sudo reboot + + At initial login, enter the administrative user's password and continue. + +#. Validate the installed kernel is **kernel-lts** by checking the output + of the :command:`uname -r` command. It should end in **.lts** or **.lts2018**. + + .. code-block:: bash + + uname -r + .lts + + If the running kernel is not **lts**: install the LTS kernel manually, + update the bootloader, and check again: + + .. code-block:: bash + + sudo swupd bundle-add kernel-lts + clr-boot-manager set-kernel $(basename $(realpath /usr/lib/kernel/default-lts)) + clr-boot-manager update + reboot + +#. Remove any kernel bundles that do not end in *-lts* or *kernel-install* + to simplify and avoid conflicts: + + .. code-block:: bash + + sudo swupd bundle-list | grep kernel + sudo swupd bundle-remove + +#. In the VM Console top menu, click :guilabel:`Devices`, and select + :guilabel:`Insert Guest Additions CD image...` to mount the |VB| driver + installation to the |CL| VM. + + .. figure:: figures/vbox/virtualbox-cl-installer-09.png + :scale: 100% + :alt: Insert Guest Additions CD image + + Figure 9: Insert Guest Additions CD image + +#. If a dialogue appears, "VBx_GAs_6.0.8... Would you like to run it?", + select :guilabel:`Cancel`. + + Instead, we provide a script to patch and install |VB| drivers on |CL|. + +#. Open a Terminal and enter the script: + + .. code-block:: bash + + sudo install-vbox-lga + + .. note:: + + Successful installation shows: "Guest Additions installation complete". + If drivers are already installed, don't re-install them. + +#. Shut down the system. Select :menuselection:`Machine --> ACPI Shutdown`. + + .. figure:: figures/vbox/virtualbox-cl-installer-10.png + :scale: 100% + :alt: Powering off a VirtualBox VM + + Figure 10: Powering off a VirtualBox VM + +#. Select :guilabel:`Settings`, :guilabel:`Display`. + +#. In :guilabel:`Graphics Controller`, select :guilabel:`VBoxSVGA` + to adjust screen size dynamically. + + .. figure:: figures/vbox/virtualbox-cl-installer-11.png + :scale: 100% + :alt: Remove Disk from Virtual Drive + + Figure 11: VirtualBox hardware acceleration error + +#. In the |VBM|, select :guilabel:`Start`. + +#. In the VM console, login and verify the |VB| drivers are loaded: + + .. code-block:: bash + + lsmod | grep ^vbox + + You should see drivers loaded with names beginning with **vbox**: + (e.g., vboxvideo, vboxguest). + +#. Add `desktop-autostart` for a full desktop experience. + + .. code-block:: bash + + sudo swupd bundle-add desktop-autostart + +#. Reboot the VM and log in with the administrative user. + + .. code-block:: bash + + sudo reboot + +The |CL| VM running on |VB| is ready to develop and explore. + +Troubleshooting +*************** + +#. **Problem:** On a Microsoft\* Windows\* OS, |VB| encounters an error when + trying to start a VM indicating *VT-X/AMD-v hardware acceleration is not + available on your system.* + + .. figure:: figures/vbox/virtualbox-cl-installer-12.png + :scale: 100% + :alt: Remove Disk from Virtual Drive + + Figure 12: VirtualBox hardware acceleration error + + **Solution:** First, double check the `Prerequisites`_ section to make + sure *Hardware accelerated virtualization* extensions have been enabled + in the host system's EFI/BIOS. + + *Hardware accelerated virtualization*, may get disabled for |VB| when + another hypervisor, such as *Hyper-V* is enabled. + + To disable *Hyper-V* execute this command in an + **Administrator: Command Prompt or Powershell**, and reboot the system: + + .. code-block:: bash + + bcdedit /set {current} hypervisorlaunchtype off + + To enable Hyper-V again, execute this command in an + **Administrator: Command Prompt or Powershell**, and reboot the system: + + .. code-block:: bash + + bcdedit /set {current} hypervisorlaunchtype Auto + + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _VirtualBox Installation Instructions: https://www.virtualbox.org/manual/ch02.html + +.. _VirtualBox: https://www.virtualbox.org + +.. _Downloads: https://clearlinux.org/downloads + +.. _`Creating Your First Virtual Machine`: https://www.virtualbox.org/manual/UserManual.html#gui-createvm + +.. _7zip: http://www.7-zip.org/ + +.. _Intel® Virtualization Technology: https://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html + +.. _VM VirtualBox User Manual: https://docs.oracle.com/cd/E97728_01/E97727/html/settings-system.html \ No newline at end of file diff --git a/_sources/get-started/virtual-machine-install/vmw-player.rst.txt b/_sources/get-started/virtual-machine-install/vmw-player.rst.txt new file mode 100644 index 00000000..b222340e --- /dev/null +++ b/_sources/get-started/virtual-machine-install/vmw-player.rst.txt @@ -0,0 +1,422 @@ +.. _vmw-player: + +|CL-ATTR| on VMware\* Workstation Player +######################################## + +This guide explains how to set up the VMware\* Workstation Player 15.5.1 +hypervisor and instantiate a VM instance of |CL| by installing it using +an ISO or using a pre-built image. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +`VMware Workstation Player`_ is a type 2 hypervisor. It runs on top of +Windows\* or Linux\* operating systems. With VMware Workstation Player, +you can create, configure, manage, and run |CL-ATTR| +:abbr:`VMs (Virtual Machines)` on your local system. + +VMware offers a type 1 hypervisor called `VMware ESXi`_ designed for the +cloud environment. For information on how to install |CL| as guest OS on +it, see :ref:`vmware-esxi-install-cl`. + +.. note:: + + The screenshots in this document show the Windows version of the + VMware Workstation Player 15.5.1. The menus and prompts are similar to those + in other versions and for the Linux version, save some minor wording + differences. + +Install the VMware Workstation Player hypervisor +************************************************ + +#. Enable Intel® Virtualization Technology (Intel® VT) and + Intel® Virtualization Technology for Directed I/O (Intel® VT-d) in + your system's BIOS. + +#. `VMware Workstation Player`_ is available for Windows and Linux. + Download your preferred version. + +#. Install VMware Workstation Player by following the instructions + appropriate for your system's OS: + + * On supported Linux distros: + + a. Ensure your Linux distro is running a GUI desktop. + #. Start a terminal emulator. + #. Start the installer by issuing the command below and follow the + guided steps. + + .. code-block:: console + + sudo sh ./VMware-Player-.x86_64.bundle + + * On Windows: + + a. Start the installer. + #. Follow the setup wizard. + +For additional help, see the `VMware Workstation Player Documentation`_. + +Create a blank VM +***************** + +#. Start the ``VMware Workstation Player`` app. + +#. On the home screen, click :guilabel:`Create a New Virtual Machine`. See + Figure 1. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-01.png + :scale: 100% + :alt: VMware Workstation Player - Create a new virtual machine + + Figure 1: VMware Workstation Player - Create a new virtual + machine + +#. Select :guilabel:`I will install the operating system later`. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-02.png + :scale: 100% + :alt: I will install the operating system later. + + Figure 2: I will install the operating system later. + +#. Click the :guilabel:`Next` button. + +#. On the :guilabel:`Select a Guest Operating System` window, set the + :guilabel:`Guest operating system` setting to :guilabel:`Linux`. See + Figure 3. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-03.png + :scale: 100% + :alt: VMware Workstation Player - Select guest operating system type + + Figure 3: VMware Workstation Player - Select guest operating system + type + +#. Set the :guilabel:`Version` setting to + :guilabel:`Other Linux 5.x or later kernel 64-bit`. + +#. Click the :guilabel:`Next` button. + +#. On the :guilabel:`Name the Virtual Machine` screen, name the new VM. See + Figure 4. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-04.png + :scale: 100% + :alt: VMware Workstation Player - Name virtual machine + + Figure 4: VMware Workstation Player - Name virtual machine + +#. Click the :guilabel:`Next` button. + +#. On the :guilabel:`Specify Disk Capacity` screen, set the VM's maximum disk + size. If you're planning to use a pre-built image, just use the default + size for now. See Figure 5. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-05.png + :scale: 100% + :alt: VMware Workstation Player - Set disk capacity + + Figure 5: VMware Workstation Player - Set disk capacity + + .. note:: + + For optimal performance with the |CL| Desktop image, we recommend 32GB + of drive space. See :ref:`system-requirements` for more details. + +#. Click the :guilabel:`Next` button. + +#. On the :guilabel:`Ready to Create Virtual Machine` screen, click the + :guilabel:`Customize Hardware...` button. See Figure 6. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-06.png + :scale: 100% + :alt: VMware Workstation Player - Customize hardware + + Figure 6: VMware Workstation Player - Customize hardware + +#. Select :guilabel:`Memory` and set a desired value. See Figure 7. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-07.png + :scale: 100% + :alt: VMware Workstation Player - Set memory size + + Figure 7: VMware Workstation Player - Set memory size + + .. note:: + + The |CL| live installer ISO needs a minimum of 1GB of RAM. + After completing installation, |CL| can run on as little as + 128MB of RAM. Thus, you can reduce the memory size if needed. + See :ref:`system-requirements` for more details. + +#. Under the :guilabel:`Device` list, select :guilabel:`Processors`. See + Figure 8. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-08.png + :scale: 100% + :alt: VMware Workstation Player - Set virtualization engine option + + Figure 8: VMware Workstation Player - Set virtualization engine + option + +#. Under :guilabel:`Processors` and :guilabel:`Number of processor cores`, + enter the desired number of cores. + +#. Under the :guilabel:`Virtualization engine` section, + check the :guilabel:`Virtualize Intel VT-x/EPT or AMD-V/RVI` box. + +#. Click the :guilabel:`Close` button. + +#. Click the :guilabel:`Finish` button. + +Enable UEFI boot support +************************ + +|CL| needs UEFI support to boot and work properly. To enable it: + +#. Close the ``VMware Workstation Player`` app. + +#. Add the following line to the end of your VM's :file:`.vmx` file. + + .. code-block:: console + + firmware = "efi" + + .. note:: + + Depending on the OS, you can typically find the VMware VM files under: + + * On Linux distros: :file:`/home/username/vmware` + * On Windows: :file:`C:\\Users\\username\\Documents\\Virtual Machines` + +Instantiate |CL| +**************** + +If you want to install |CL| from scratch, following the instructions +in the **Install |CL| using ISO** tab. Otherwise, follow the +**Use |CL| pre-built VMware image** tab to use our pre-built image. + +.. tabs:: + + .. tab:: Install |CL| using ISO + + #. Navigate to the |CL| `Downloads`_ page and download either the ``Server`` + or ``Desktop`` ISO image. After the download is complete, you will + attach this image. + + #. Start the ``VMware Workstation Player`` app. + + #. Select the VM that was created in section `Create a blank VM`_. + See Figure 9. + + #. Click :guilabel:`Edit virtual machine settings`. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-09.png + :scale: 100% + :alt: VMware Workstation Player - Edit virtual machine settings + + Figure 09: VMware Workstation Player - Edit virtual machine settings + + #. In the :guilabel:`Virtual Machine settings` window, + under :guilabel:`Hardware`, select guilabel:`CD/DVD (IDE)`. + See Figure 10. + + #. Under :guilabel:`Connection` at the right, select + :guilabel:`Use ISO image file`. + + #. Click :guilabel:`Browse` and select the + |CL| installer ISO. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-10.png + :scale: 100% + :alt: VMware Workstation Player - Select |CL| installer ISO + + Figure 10: VMware Workstation Player - Select |CL| installer ISO + + #. Click :guilabel:`OK` to close the :guilabel:`Virtual Machine settings` + window. + + #. Start the VM by clicking :guilabel:`Play virtual machine`. + + #. Follow one of these guides to complete the installation of |CL|. + + * *Desktop* version: :ref:`install-clr-desktop-start` + * *Server* version: :ref:`install-clr-server-start` + + #. Reboot the VM after the installation completes. + + #. Install the ``os-cloudguest-vmware`` bundle, the open source + VMware Tools for Linux\* guest operating systems, which enables + new features and improves general performance. + + .. code-block:: bash + + sudo swupd bundle-add os-cloudguest-vmware + sudo systemctl enable --now open-vm-tools + + More information is available on the `VMWare Tools Product Documentation`_ + site. + + .. tab:: Use |CL| pre-built VMWare image + + #. Navigate to the |CL| `Downloads`_ page and download the ``VMware`` + image. + + #. Decompress the downloaded file and move it to the + directory where your newly-created VM files reside. + + .. note:: + + Depending on the OS, you can typically find the VMware VM + files under: + + * Linux distros :file:`/home/username/vmware` + * Windows :file:`C:\\Users\\username\\Documents\\Virtual Machines` + + #. Start the ``VMware Workstation Player`` app. + + #. Select the VM that was created in section `Create a blank VM`_. + See Figure 9. + + #. Click :guilabel:`Edit virtual machine settings`. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-09.png + :scale: 100% + :alt: VMware Workstation Player - Edit virtual machine settings + + Figure 9: VMware Workstation Player - Edit virtual machine settings + + #. Under :guilabel:`Hardware` and :guilabel:`Device` list, select + :guilabel:`Hard Disk (SCSI)`. See Figure 11. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-11.png + :scale: 100% + :alt: VMware Workstation Player - Remove hard drive + + Figure 11: VMware Workstation Player - Remove hard drive + + #. Click the :guilabel:`Remove` button. + + #. To add a new hard disk and attach the pre-built |CL| + VMware image, click the :guilabel:`Add` button. See Figure 12. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-12.png + :scale: 100% + :alt: VMware Workstation Player - Add new device + + Figure 12: VMware Workstation Player - Add new device + + #. Under the :guilabel:`Hardware types` section, select + :guilabel:`Hard Disk`. See Figure 13. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-13.png + :scale: 100% + :alt: VMware Workstation Player - Add hard drive + + Figure 13: VMware Workstation Player - Add hard drive + + #. Click the :guilabel:`Next` button. + + #. Select your preferred :guilabel:`Virtual disk type`. + See Figure 14. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-14.png + :scale: 100% + :alt: VMware Workstation Player - Select virtual disk type + + Figure 14: VMware Workstation Player - Select virtual disk type + + #. Select the :guilabel:`Use an existing virtual disk` option. + See Figure 15. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-15.png + :scale: 100% + :alt: VMware Workstation Player - Use existing virtual disk + + Figure 15: VMware Workstation Player - Use existing virtual disk + + #. Click the :guilabel:`Browse` button and select the + pre-built |CL| VMware image file. See Figure 16. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/vmw-player/vmw-player-16.png + :scale: 100% + :alt: VMware Workstation Player - Select pre-built VMware |CL| image file + + Figure 16: VMware Workstation Player - Select pre-built VMware |CL| + image file + + #. Click the :guilabel:`Finish` button. + + .. note:: + + When asked to convert the existing virtual disk to a newer format, + selecting either option works. + + #. Click the :guilabel:`OK` button. + + #. Start the VM by clicking :guilabel:`Play virtual machine`. + + .. note:: + + If you need to increase the disk size of the pre-built |CL| image, see + :ref:`increase-virtual-disk-size`. + +Related topics +************** + +For other guides on using the VMWare Player and ESXi, see: + +* :ref:`vmware-esxi-install-cl` + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _VMware ESXi: https://www.vmware.com/products/esxi-and-esx.html + +.. _VMware Workstation Player: + https://www.vmware.com/products/workstation-player.html + +.. _VMware Workstation Player Documentation: + https://docs.vmware.com/en/VMware-Workstation-Player/index.html + +.. _Downloads: https://clearlinux.org/downloads + +.. _VMWare Tools Product Documentation: https://docs.vmware.com/en/VMware-Tools/10.1.0/com.vmware.vsphere.vmwaretools.doc/GUID-8B6EA5B7-453B-48AA-92E5-DB7F061341D1.html diff --git a/_sources/get-started/virtual-machine-install/vmware-esxi-install-cl.rst.txt b/_sources/get-started/virtual-machine-install/vmware-esxi-install-cl.rst.txt new file mode 100644 index 00000000..1828cf2f --- /dev/null +++ b/_sources/get-started/virtual-machine-install/vmware-esxi-install-cl.rst.txt @@ -0,0 +1,278 @@ +.. _vmware-esxi-install-cl: + +|CL-ATTR| on VMware\* ESXi +########################## + +This page explains how to create a new :abbr:`VM (Virtual Machine)` and +manually install |CL-ATTR| on the new VM with VMware ESXi 6.5. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +`VMware ESXi`_ is a type 1 bare-metal hypervisor that runs directly on top +of server hardware. With VMware ESXi, you can create, configure, manage, and +run |CL| virtual machines in the cloud. + +Manually installing |CL| on a new VM gives additional configuration flexibility +during installation. For example: alternate disk sizes, number of partitions, +pre-installed bundles, etc. + +.. note:: + + VMware also offers a type 2 hypervisor designed for the desktop environment, called `VMware Workstation Player`_. Refer to + :ref:`vmw-player` for more information. + + Visit :ref:`image-types` to learn more about all available images. + +Download the latest |CL| installer ISO +************************************** + +Get the latest |CL| installer ISO image from the `image`_ repository. +Look for :file:`clear-[version number]-installer.iso.xz`. + +We also provide instructions for downloading and verifying a Clear Linux ISO. +For more information, refer to :ref:`download-verify-decompress`. + +Upload the |CL| installer ISO to the VMware server +************************************************** + +#. Connect to the VMware server and log into an account with sufficient + permission to create and manage VMs. +#. Under the :guilabel:`Navigator` window, select :guilabel:`Storage`. + See Figure 1. +#. Under the :guilabel:`Datastores` tab, click the :guilabel:`Datastore browser` + button. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-1.png + :scale: 100 % + :alt: VMware ESXi - Navigator > Storage + + Figure 1: VMware ESXi - Navigator > Storage + +#. Click the :guilabel:`Create directory` button and name the directory `ISOs`. + See Figure 2. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-2.png + :scale: 100 % + :alt: VMware ESXi - Datastore > Create directory + + Figure 2: VMware ESXi - Datastore > Create directory + +#. Select the newly-created directory and click the :guilabel:`Upload` button. + See Figure 3. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-3.png + :scale: 100 % + :alt: VMware ESXi - Datastore > Upload ISO + + Figure 3: VMware ESXi - Datastore > Upload ISO + +#. Select the decompressed |CL| installer ISO file :file:`clear-[version number]-installer.iso` + and upload it. + +Create and configure a new VM +***************************** + +In this section, you will create a new VM, configure its basic parameters such +as drive size, number of CPUs, memory size, and then attach the |CL| installer ISO. + +#. Under the :guilabel:`Navigator` window, select :guilabel:`Virtual Machines`. + See Figure 4. +#. In the right window, click the :guilabel:`Create / Register VM` button. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-4.png + :scale: 100 % + :alt: VMware ESXi - Navigator > Virtual Machines + + Figure 4: VMware ESXi - Navigator > Virtual Machines + +#. On the :guilabel:`Select creation type` step: + + #. Select the :guilabel:`Create a new virtual machine` option. + See Figure 5. + #. Click the :guilabel:`Next` button. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-5.png + :scale: 100 % + :alt: VMware ESXi - Create a new virtual machine + + Figure 5: VMware ESXi - Create a new virtual machine + +#. On the :guilabel:`Select a name and guest OS` step: + + #. Give the new VM a name in the :guilabel:`Name` field. See Figure 6. + #. Set the :guilabel:`Compatibility` option to :guilabel:`ESXi 6.5 virtual machine`. + #. Set the :guilabel:`Guest OS family` option to :guilabel:`Linux`. + #. Set the :guilabel:`Guest OS version` option to :guilabel:`Other 3.x or later Linux (64-bit)`. + #. Click the :guilabel:`Next` button. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-6.png + :scale: 100 % + :alt: VMware ESXi - Give a name and select guest OS type + + Figure 6: VMware ESXi - Give a name and select guest OS type + +#. On the :guilabel:`Select storage` step: + + #. Accept the default option. + #. Click the :guilabel:`Next` button. + +#. On the :guilabel:`Customize settings` step: + + #. Click the :guilabel:`Virtual Hardware` button. See Figure 7. + #. Expand the :guilabel:`CPU` setting and enable :guilabel:`Hardware virtualization` by + checking :guilabel:`Expose hardware assisted virtualization to the guest OS`. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-7.png + :scale: 100 % + :alt: VMware ESXi - Enable hardware virtualization + + Figure 7: VMware ESXi - Enable hardware virtualization + + #. Set :guilabel:`Memory` size to 2048MB (2GB). See Figure 8. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-8.png + :scale: 100 % + :alt: VMware ESXi - Set memory size + + Figure 8: VMware ESXi - Set memory size + + .. note:: + + The |CL| installer ISO needs a minimum of 2GB of RAM to work properly. + You can reduce the memory size after the installation completes if you want, + because a minimum |CL| installation can function on as little as 128MB of RAM. + See :ref:`system-requirements` for more details. + + #. Set :guilabel:`Hard disk 1` to the desired capacity. See Figure 9. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-9.png + :scale: 100 % + :alt: VMware ESXi - Set hard disk size + + Figure 9: VMware ESXi - Set hard disk size + + .. note:: + + A minimum |CL| installation can exist on 600MB of drive space. + See :ref:`system-requirements` for more details. + + #. Attach the |CL| installer ISO. For the :guilabel:`CD/DVD Drive 1` setting, + click the drop-down list to the right of it and select the :guilabel:`Datastore ISO file` + option. Then select the |CL| installer ISO :file:`clear-[version number]-installer.iso` + that you previously uploaded to the VMware server. See Figure 10. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-10.png + :scale: 100 % + :alt: VMware ESXi - Set CD/DVD to boot installer ISO + + Figure 10: VMware ESXi - Set CD/DVD to boot installer ISO + +#. Click the :guilabel:`Next` button. +#. Click the :guilabel:`Finish` button. + +Install |CL| into the new VM +**************************** + +#. Power on the VM. + + #. Under the :guilabel:`Navigator` window, select :guilabel:`Virtual Machines`. + See Figure 11. + #. In the right window, select the newly-created VM. + #. Click the :guilabel:`Power on` button. + #. Click on the icon representing the VM to bring it into view and maximize + its window. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-11.png + :scale: 100 % + :alt: VMware ESXi - Navigator > Virtual Machines > Power on VM + + Figure 11: VMware ESXi - Navigator > Virtual Machines > Power on VM + +#. Follow the :ref:`install-on-target-start` guide to complete the installation of + |CL|. +#. After the installation is complete, follow the |CL| instruction to reboot it. + This will restart the installer again. + +Reconfigure the VM's settings to boot the newly-installed |CL| +************************************************************** + +After |CL| has been installed using the installer ISO, it must be detached so +it will not run again. Also, in order to boot the newly-installed |CL|, you must +enable UEFI support. + +#. Power off the VM. + + #. Click the :guilabel:`Actions` button - located on the top-right corner + of the VM's windows - and go to the :guilabel:`Power` setting and + select the :guilabel:`Power off` option. See Figure 12. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-12.png + :scale: 100 % + :alt: VMware ESXi - Actions > Power off + + Figure 12: VMware ESXi - Actions > Power off + +#. Edit the VM settings. + + #. Click the :guilabel:`Actions` button again and select :guilabel:`Edit settings`. + See Figure 13. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-13.png + :scale: 100 % + :alt: VMware ESXi - Actions > Edit settings + + Figure 13: VMware ESXi - Actions > Edit settings + +#. Disconnect the CD/DVD to stop it from booting the |CL| installer ISO again. + + #. Click the :guilabel:`Virtual Hardware` button. See Figure 14. + #. For the :guilabel:`CD/DVD Drive 1` setting, uncheck the + :guilabel:`Connect` checkbox. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-14.png + :scale: 100 % + :alt: VMware ESXi - Disconnect the CD/DVD drive + + Figure 14: VMware ESXi - Disconnect the CD/DVD drive + +#. |CL| needs UEFI support in order to boot. Enable it. + + #. Click the :guilabel:`VM Options` button. See Figure 15. + #. Expand the :guilabel:`Boot Options` setting. + #. For the :guilabel:`Firmware` setting, click the drop-down list to the right + of it and select the :guilabel:`EFI` option. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-15.png + :scale: 100 % + :alt: VMware ESXi - Set boot firmware to EFI + + Figure 15: VMware ESXi - Set boot firmware to EFI + +#. Click the :guilabel:`Save` button. + +Power on the VM and boot |CL| +***************************** + +After configuring the settings above, power on the VM. + +#. Under the :guilabel:`Navigator` window, select :guilabel:`Virtual Machines`. + See Figure 16. +#. In the right window, select the VM. +#. Click the :guilabel:`Power on` button. +#. Click on the icon representing the VM to bring it into view and maximize + its window. + + .. figure:: figures/vmware-esxi/vmware-esxi-install-cl-16.png + :scale: 100 % + :alt: VMware ESXi - Navigator > Virtual Machines > Power on VM + + Figure 16: VMware ESXi - Navigator > Virtual Machines > Power on VM + +.. _VMware ESXi: https://www.vmware.com/products/esxi-and-esx.html +.. _VMware Workstation Player: https://www.vmware.com/products/workstation-player.html +.. _image: https://cdn.download.clearlinux.org/image/ diff --git a/_sources/guides/clear/autoproxy.rst.txt b/_sources/guides/clear/autoproxy.rst.txt new file mode 100644 index 00000000..f6ed1082 --- /dev/null +++ b/_sources/guides/clear/autoproxy.rst.txt @@ -0,0 +1,168 @@ +.. _autoproxy: + +Autoproxy +######### + +Autoproxy is provided to enable |CL-ATTR| to work smoothly behind a +corporate proxy. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +Autoproxy tries to detect a Proxy Auto-Config (PAC) script and use it to +automatically resolve the proxy needed for a given connection. With +Autoproxy, you can use |CL| inside any proxy environment without having to +manually configure the proxies. + +Corporate and private networks can be very complex, needing to restrict and +control network connections for security reasons. The typical side effects +are limited or blocked connectivity, and require manual configuration of +proxies to perform the most mundane tasks, such as cloning a repo or checking +for updates. With |CL|, all of the work is done behind the scenes to +effortlessly use your network and have connections “just work”. + +This feature removes severe complications with network connectivity due to +proxy issues. You can automate tasks, such as unit testing, without worrying +about the proxy not being set, and you can remove unset proxies from the +equation when dealing with network unavailability across systems. + +How it works +************ + +We designed Autoproxy around tools provided by most Linux\* +distributions with a few minor additions and modifications. We leveraged the +DHCP and network information obtained from systemd and created a +PAC-discovery daemon. The daemon uses the information to resolve a URL for a +PAC file. The daemon then passes the URL into PACrunner\*. PACrunner +downloads the PAC file and uses the newly implemented Duktape\* engine to +parse it. + +.. figure:: figures/autoproxy_0.png + :width: 400px + + Figure 1: Autoproxy Flow + +From that point on, any cURL\* or network requests query PACrunner for the +correct proxy to use. We modified the cURL library to communicate with +PACrunner over DBus. However, cURL will ignore PACrunner and run normally if +no PAC file is loaded or if you manually set any proxies. Thus, your +environment settings are respected and no time is wasted trying to resolve a +proxy. All these steps happen in the background with no user interaction. + +Troubleshooting +*************** + +Autoproxy allows |CL| to operate seamlessly behind a proxy +because :ref:`swupd ` and other |CL| tools are implemented on +top of libcurl. Tools that do not use libcurl, like git, must +be configured independently. + +If you are familiar with PAC files and WPAD, you can use +:command:`pacdiscovery` and :command:`FindProxyForURL` to +troubleshoot problems with autoproxy. + +.. note:: + + Learn more about WPAD, PAC files, and PAC functions at `findproxyforurl`_. + +.. _findproxyforurl: http://findproxyforurl.com/ + + +Run :command:`pacdiscovery` with no arguments to indicate |br| + +#. if there is a problem resolving the :command:`WPAD` host name resolution: + + .. code-block:: bash + + sudo pacdiscovery + + Sample output: + + .. code-block:: console + + failed getaddrinfo: No address associated with hostname + Unable to find wpad host + +#. or if the :command:`pacrunner` service is disabled (masked). + + .. code-block:: bash + + sudo pacdiscovery + + Sample output: + + .. code-block:: console + + PAC url: http://autoproxy.your.domain.com/wpad.dat + Failed to create proxy config: Unit pacrunner.service is masked. + +Unmask the :command:`pacrunner` service by running: + +.. code-block:: bash + + sudo systemctl unmask pacrunner.service + + + +Use :command:`FindProxyForURL` with :command:`busctl` to indicate |br| + +#. the URL and port of the proxy server when an external URL and host are + provided as arguments: + + .. code-block:: bash + + busctl call org.pacrunner /org/pacrunner/client org.pacrunner.Client FindProxyForURL ss "http://www.google.com" "google.com" + + Sample output showing proxy was found: + + .. code-block:: console + + s "PROXY proxy.your.domain.com:" + +#. if the :command:`pacrunner.service` is masked: + + .. code-block:: bash + + busctl call org.pacrunner /org/pacrunner/client org.pacrunner.Client FindProxyForURL ss "http://www.google.com" "google.com" + + Sample output: + + .. code-block:: console + + Unit pacrunner.service is masked. + dig wpad, dig wpad. + +#. if a proxy server is not available, or if :command:`pacrunner` is running + without a PAC file: + + .. code-block:: bash + + busctl call org.pacrunner /org/pacrunner/client org.pacrunner.Client FindProxyForURL ss "http://www.google.com" "google.com" + + Sample output, indicating connection made directly, without proxy: + + .. code-block:: console + + s "DIRECT" + +Once :command:`pacdiscovery` is able to look up :command:`WPAD`, restart the +:command:`pacrunner` service: + +.. code-block:: bash + + sudo systemctl stop pacrunner + sudo systemctl restart pacdiscovery + +.. note:: + + A "domain" or "search" entry in :file:`/etc/resolv.conf` is required + for short name lookups to resolve. The :file:`resolv.conf` man page has + additional details. + +.. |br| raw:: html + +

\ No newline at end of file diff --git a/_sources/guides/clear/autospec.rst.txt b/_sources/guides/clear/autospec.rst.txt new file mode 100644 index 00000000..0727392e --- /dev/null +++ b/_sources/guides/clear/autospec.rst.txt @@ -0,0 +1,611 @@ +.. _autospec: + +autospec +######## + +**autospec** is a tool used to assist with the automated creation and +maintenance of RPM packaging in |CL-ATTR|. Where a standard +:abbr:`RPM (RPM Package Manager)` build process using :command:`rpmbuild` +requires a tarball and :file:`.spec` file to start, autospec requires only a +tarball and package name to start. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +The autospec tool attempts to infer the requirements of the :file:`.spec` +file by analyzing the source code and :file:`Makefile` information. It +continuously runs updated builds based on new information discovered from +build failures until it has a complete and valid :file:`.spec` file. If +needed, you can influence the behavior of autospec and customize the build by providing optional `control files`_ to the autospec tool. + +autospec uses **mock** as a sandbox to run the builds. Visit the `mock wiki`_ +for additional information on using mock. + +For a general understanding of how an RPM works, visit +the `rpm website`_ or the `RPM Packaging Guide`_. + +.. raw:: html + + + +How it works +************ + +Learn the autospec tool set up and process. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +============= + +The setup for building source in |CL| must be completed before using the +autospec tool. + +Refer to `Setup environment to build source`_ for instructions on completing +the setup. + +Create an RPM +============= + +The basic autospec process is described in the following steps: + +#. The :command:`make autospec` command generates a :file:`.spec` file based + on the analysis of code and existing control files. + + Any control files should be located in the same directory as the resulting + :file:`.spec` file. View the `autospec README`_ for more information on `control files`_. + +#. autospec creates a build root with mock config. + +#. autospec attempts to build an RPM from the generated :file:`.spec`. + +#. autospec detects any missed declarations in the :file:`.spec`. + +#. If build errors occur, autospec scans the build log to try to detect + the root cause. + +#. If autospec detects the root cause and knows how to continue, it restarts + the build automatically at step 1 with updated build instructions. + +#. Otherwise, autospec stops the build for user inspection to resolve the + errors. Respond to the build process output by fixing source code issues + and/or editing control files to resolve issues, which may include + dependencies or exclusions. See `autospec README`_ for more information on + control files. + + The user resumes the process at step 1 after errors are resolved. + + If a binary dependency doesn't exist in |CL|, you must build it + before running autospec again. + +Following these steps, autospec continues to rebuild the package, based on +new information discovered from build failures, until it has a valid +:file:`.spec`. If no build errors occur, RPM packages are successfully built. + +Examples +******** + +Complete `Setup environment to build source`_ before using these examples. + +.. contents:: + :local: + :depth: 1 + +Example 1: Build RPM with an existing spec file +=============================================== + +This example shows how to build a RPM from a pre-packaged upstream package +with an existing spec file. The example uses the ``dmidecode`` package. + +#. Navigate to the autospec workspace and clone the ``dmidecode`` package: + + .. code-block:: bash + + cd ~/clearlinux + make clone_dmidecode + + .. note:: + + You can clone all package repos at once using the following command: + + .. code-block:: bash + + make [-j NUM] clone-packages + + The optional NUM is the number of threads to use. + + For a list of available packages, view the + :file:`~/clearlinux/projects/common/packages` file. + +#. Navigate to the local copy of the ``dmidecode`` package and build it: + + .. code-block:: bash + + cd ~/clearlinux/packages/dmidecode/ + make build + +#. The resulting RPMs are in :file:`./rpms`. Build logs and additional RPMs + are in :file:`./results`. + +Example 2: Build a new RPM +========================== + +This example shows how to build a new RPM with no spec file. The example will +create a simple helloclear RPM. + +#. Navigate to the autospec workspace and build the helloclear RPM. The + :file:`Makefile` provides a :command:`make autospecnew` that can + automatically generate an RPM package using the autospec tool. You must + pass the URL to the source tarball and the NAME of the RPM you wish to + create: + + .. code-block:: bash + + cd ~/clearlinux + make autospecnew URL="https://github.com/clearlinux/helloclear/archive/helloclear-v1.0.tar.gz" NAME="helloclear" + + The resulting RPMs are in :file:`./packages/helloclear/rpms`. Build logs and additional RPMs are in :file:`./packages/helloclear/results`. + +Example 3: Generate a new spec file with a pre-defined package +============================================================== + +This example shows how to modify an existing package to create a custom RPM. +In this example you will make a simple change to the ``dmidecode`` package +and rebuild the package. + +#. Navigate to the autospec workspace and clone the ``dmidecode`` package: + + .. code-block:: bash + + cd ~/clearlinux + make clone_dmidecode + +#. Navigate into the *dmidecode* directory: + + .. code-block:: bash + + cd packages/dmidecode + +#. Open the :file:`excludes` file with an editor and add these lines: + + .. code-block:: console + + /usr/bin/biosdecode + /usr/bin/ownership + /usr/bin/vpddecode + /usr/share/man/man8/biosdecode.8 + /usr/share/man/man8/ownership.8 + /usr/share/man/man8/vpddecode.8 + + .. note:: + + These files aren't needed by dmidecode, so we can remove them without + any issues. + +#. In the :file:`dmidecode` directory, build the modified ``dmidecode`` + package: + + .. code-block:: bash + + make autospec + +#. The resulting RPMs are in :file:`./rpms`. Logs are in :file:`./results`. + +Example 4: Provide control files to autospec +============================================ + +This example shows how to modify control files to correct build failures that +autospec is unable to resolve. In this example, you will add a missing +license and dependencies so autospec can complete a successful build. + +#. Navigate to the autospec workspace: + + .. code-block:: bash + + cd ~/clearlinux + +#. If you have not already, clone all upstream package repos: + + .. code-block:: bash + + make [-j NUM] clone-packages + + The optional NUM is the number of threads to use. + + .. note:: + + In a later step of this example, we will search the cloned package + repos for a missing dependency. + +#. Build the opae-sdk RPM: + + .. code-block:: bash + + make autospecnew URL="https://github.com/OPAE/opae-sdk/archive/0.13.0.tar.gz" NAME="opae-sdk" + + This results in an error for a missing license file: + + .. code-block:: console + + [FATAL] Cannot find any license or opae-sdk.license file! + +#. Navigate to the package with build failures: + + .. code-block:: bash + + cd packages/opae-sdk + +#. Add one or more valid license identifiers from the + `SPDX License List `_. + In the example below, two different licenses are appropriate based on the + opae-sdk project licensing: + + .. code-block:: bash + + echo "BSD-3-Clause MIT" > opae-sdk.license + +#. Run autospec again: + + .. code-block:: bash + + make autospec + + This results in a generic error: + + .. code-block:: console + + [FATAL] Build failed, aborting + +#. Open the build log to view the error details: + + .. code-block:: bash + + cat ./results/build.log + + The build log contains details for the specific failures. In this + instance, there are missing dependencies: + + .. code-block:: console + + CMake Error: The following variables are used in this project, but + they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: + + CJSON_LIBRARY + linked by target "opae-c++-utils" in directory /builddir/build/BUILD/opae-sdk-0.13.0/tools/c++utilslib + json-c_LIBRARIES + linked by target "opae-c" in directory /builddir/build/BUILD/opae-sdk-0.13.0/libopae + libuuid_LIBRARIES + linked by target "opae-c" in directory /builddir/build/BUILD/opae-sdk-0.13.0/libopae + +#. Search the spec files of upstream |CL| packages to see if the json-c + library is available. In this case, it does exist and we'll add the json-c 'dev' package into the buildreq_add: + + .. code-block:: bash + + grep 'json-c\.so$' ~/clearlinux/packages/*/*.spec + echo "json-c-dev" >> buildreq_add + + .. note:: + + This search step works only if the user cloned all of the upstream package repos. In this example, upstream package repos were cloned in a previous step. + +#. Search the spec files of upstream |CL| packages to see if the libuuid + library is available. In this case, it exists in the util-linux package, so we'll add util-linux-dev package into the buildreq_add: + + .. code-block:: bash + + grep 'libuuid\.so$' ~/clearlinux/packages/*/*.spec + echo "util-linux-dev" >> buildreq_add + +#. Run autospec again and find the successfully-generated RPMs in the + :file:`rpms` directory: + + .. code-block:: bash + + make autospec + + .. note:: + + If you need a dependency that does not exist in the |CL| repo, you must first build it manually (see `Example 2: Build a new RPM`_), then add the repo so that autospec knows the package exists. For example: + + .. code-block:: bash + + cd ~/clearlinux/packages/ + make repoadd + make repostatus + + You only need to add the dependency to the :file:`buildreq_add` control + file if autospec is not able to automatically find the correct dependency + on its own. + +.. TODO: Document how to set up a license server for use with autospec. +.. TODO: Demonstrate control file management. Establish specific use cases. + +Example 5: Update an existing package +===================================== + +The |CL| team prefers to carry no patches and seeks to make the latest +releases work. If we do need patches, we use :command:`autospec` to add, +remove, or manage patches. The :command:`autospec` control files are +integral to the patch management process. Developers can expect a more +streamlined approach to managing a large collection of packages with +:command:`autospec`. + +Adding and submitting patches +----------------------------- + +* To add patches to |CL| upstream, follow `patching source code`_. + +* To submit a patch to upstream, follow + `contributing to an existing software package`_. + +If you maintain a downstream derivative of |CL| and you want to integrate +new or patched packages into your mix, follow the process in :ref:`mixer`. + +Assuming you have followed the above process, :command:`autospec` has +generated a new spec file. + +Refresh a package and inspect +----------------------------- + +In this example, we use autospec to refresh the :command:`m4` package and +recreate RPM files. + +#. Navigate to the top-level directory of the workspace + + .. code-block:: bash + + cd clearlinux + + - where :command:`clearlinux` is the top level of the tooling workspace + +#. Run the make_clone command and then navigate to the package. + + .. code-block:: bash + + make clone_m4 + + cd packages/m4 + +#. Make desired changes to the package, its control files, or + other files. + +#. Finally, run: + + .. code-block:: bash + + make autospec + +#. To view spec file changes, run: + + .. code-block:: bash + + git show m4.spec + + The output shows: + + .. code-block:: console + + m4: Autospec creation for version 1.4.18 + + diff --git a/m4.spec b/m4.spec + index f76c78d..97b846a 100644 + --- a/m4.spec + +++ b/m4.spec + @@ -6,15 +6,14 @@ + # + Name : m4 + Version : 1.4.18 + -Release : 88 + +Release : 89 + URL : http://mirrors.kernel.org/gnu/m4/m4-1.4.18.tar.xz + Source0 : http://mirrors.kernel.org/gnu/m4/m4-1.4.18.tar.xz + -Source99 : http://mirrors.kernel.org/gnu/m4/m4-1.4.18.tar.xz.sig + +Source1 : http://mirrors.kernel.org/gnu/m4/m4-1.4.18.tar.xz.sig + Summary : No detailed summary available + Group : Development/Tools + ... + +#. The following commands provide a more complete view of the changes. + + * :command:`git log -p` + * :command:`gitk` + +Test packaged software +********************** + +After software has been packaged with autospec, the resulting RPMs can be +tested for functionality before being integrated and deployed into a |CL| +image with the :ref:`Mixer tool `. + +The |CL| development tooling offers two ways to quickly test autospec +generated RPMs. + +.. note:: + The methods outlined below should only be used for temporary testing on + development systems. + + +Test in a |CL| virtual machine +============================== + +The |CL| development tooling includes a method to install RPMs into a |CL| +virtual machine running on the KVM hypervisor. Using a :abbr:`VM (Virtual +Machine)` allows testing in a completely isolated environment. + +To test an autospec-created package inside a VM: + +#. Download the |CL| KVM image into the :file:`~/clearlinux` directory as + :file:`clear.img`. The location and name :file:`clear.img.xz` is important + for the tooling to work: + + .. code-block:: bash + + cd ~/clearlinux + curl -o clear.img.xz https://download.clearlinux.org/image/$(curl https://download.clearlinux.org/image/latest-images | grep '[0-9]'-kvm) + +#. Extract the downloaded |CL| KVM image: + + .. code-block:: bash + + unxz -v clear.img.xz + +#. Copy the QEMU start script and virtual firmware needed for KVM into the + :file:`~/clearlinux` directory: + + .. code-block:: bash + + cp ~/clearlinux/projects/common/start_qemu.sh . + cp /usr/share/qemu/OVMF.fd . + +#. Run :command:`make install` from the package's autospec directory. The + :command:`make install` command mounts the downloaded |CL| KVM image and + installs the autospec-created RPM into it: + + .. code-block:: bash + + cd ~/clearlinux/packages/ + make install + + The code that makes this possible can be viewed by searching for the + *install:* target in the `Makefile.common`_ file on GitHub. + +#. Return to the :file:`~/clearlinux` directory and start the |CL| VM: + + .. code-block:: bash + + cd ~/clearlinux/ + sudo ./start_qemu.sh clear.img + +#. A new |CL| VM will launch in the console. Log into the VM as *root* and set + a new password for the VM. + +#. Check that the software is installed in the |CL| VM as expected and perform + any relevant tests. + +#. After testing has been completed, the |CL| VM can be powered off and + deleted: + + .. code-block:: bash + + poweroff + rm clear.img + + +Test directly on a development machine +====================================== + +The |CL| development tooling also includes a method to extract +autospec-created RPMs locally onto a |CL| development system for testing. +Extracting an RPM directly onto a system offers quicker testing; however +conflicts may occur and responsibility to remove the software after testing is +up to the developer. + +To test an autospec created package directly on the |CL| development system: + +#. Run :command:`make install-local` from the package's autospec directory. + The :command:`make install-local` command extracts the RPM directly onto + the filesystem of the running |CL| system: + + .. code-block:: bash + + cd ~/clearlinux/packages/ + make install-local + + The code that makes this possible can be viewed by searching for the + *install-local:* target in the `Makefile.common`_ file on GitHub. + +#. Check that the software is installed as expected and perform any relevant + tests. + +#. After testing has been completed, the software and any related files must + be identified and deleted. The :command:`swupd repair --picky` + command can help restore the state of the :file:`/usr` directory (see + :ref:`swupd `) however any other files must be cleaned up + manually. + + +References +********** + +Reference the `autospec README`_ for details regarding `autospec` commands and options. + +Setup environment to build source +================================= + +.. _install-tooling-after-header: + +Setup of the workspace and tooling used for building source in |CL| is mostly +automated for you with a setup script. It uses tools from the +:command:`os-clr-on-clr` bundle. + +The setup script creates a workspace in the :file:`clearlinux` folder, with the +subfolders :file:`Makefile`, :file:`packages`, and :file:`projects`. The +:file:`projects` folder contains the main tools used for making packages in +|CL| :file:`autospec` and :file:`common`. + +Follow these steps to setup the workspace and tooling for building source: + +#. Install the :command:`os-clr-on-clr` bundle: + + .. code-block:: bash + + sudo swupd bundle-add os-clr-on-clr + +#. Download the :file:`user-setup.sh` script: + + .. code-block:: bash + + curl -O https://raw.githubusercontent.com/clearlinux/common/master/user-setup.sh + +#. Make :file:`user-setup.sh` executable: + + .. code-block:: bash + + chmod +x user-setup.sh + +#. Run the script as an unprivileged user: + + .. code-block:: bash + + ./user-setup.sh + +#. After the script completes, log out and log in again to complete the setup + process. + +#. Set your Git user email and username for the repos on your system: + + .. code-block:: bash + + git config --global user.email "you@example.com" + git config --global user.name "Your Name" + + This global setting is used by |CL| tools that make use of Git. + +.. _install-tooling-end: + +Related topics +************** + +* :ref:`Mixer tool ` +* :ref:`Proxy Configuration ` + +.. _contributing to an existing software package: https://github.com/clearlinux/distribution/blob/master/contributing.md#contributing-to-an-existing-software-package + +.. _patching source code: https://github.com/clearlinux/distribution/blob/master/contributing.md#patching-source-code + +.. _`Makefile.common`: https://github.com/clearlinux/common/blob/master/Makefile.common +.. _autospec README: https://github.com/clearlinux/autospec +.. _control files: https://github.com/clearlinux/autospec#control-files +.. _mock wiki: https://github.com/rpm-software-management/mock/wiki +.. _rpm website: http://rpm.org +.. _RPM Packaging Guide: https://rpm-packaging-guide.github.io/ + + +.. TODO: Add link to how to submit a new package: https://github.com/clearlinux/distribution/blob/master/contributing.md#contributing-a-new-software-package diff --git a/_sources/guides/clear/bundles.rst.txt b/_sources/guides/clear/bundles.rst.txt new file mode 100644 index 00000000..52cc194d --- /dev/null +++ b/_sources/guides/clear/bundles.rst.txt @@ -0,0 +1,27 @@ +.. _bundles-guide: + +Bundles +####### + +Linux-based operating systems contain the code of several hundred, if +not thousands, of open source projects. To make this manageable, +distributions use a concept called "packages" to configure and compile +the source code of these projects into binaries. + +Many distributions then split the content of these compiled packages +into so-called sub-packages, which are the granularity at which these +distributions deploy their software. With those kinds of distributions, +system administrators can then install and update sub-packages +individually or as a set, using tools such as "yum" and "apt-get." + +The |CL-ATTR| takes a slightly different approach. While we also use the +concept of packages to manage compiling source code into binaries, we do not +use the package concept to deploy software. Instead, we provide software +"bundles" that are installed and managed using :ref:`swupd`. +Each bundle contains as many or as few open source projects needed to provide a +complete functionality. + +Related topics +************** + +* :ref:`swupd-guide` \ No newline at end of file diff --git a/_sources/guides/clear/compatible-kernels.rst.txt b/_sources/guides/clear/compatible-kernels.rst.txt new file mode 100644 index 00000000..5882459b --- /dev/null +++ b/_sources/guides/clear/compatible-kernels.rst.txt @@ -0,0 +1,72 @@ +.. _compatible-kernels: + +Kernels +####### + +The |CL-ATTR| provides the following Linux kernels with a respective bundle. +This document describes the specific use cases these `bundles`_ serve +and provides links to their source code. + +Bare metal only +*************** + +Kernel native + The *kernel-native* bundle focuses on the bare metal platforms. It is + optimized for fast booting and performs best on the Intel® Architecture Processors + described on the :ref:`supported hardware list`. The + optimization patches are found in our `Linux`_ GitHub\* repo. + +.. _vm-kernels: + +Also compatible with VMs +************************ + +Kernel LTS + The *kernel-lts* bundle focuses on the bare metal platforms but uses the + latest :abbr:`LTS (Long Term Support)` Linux kernel. It is optimized for + fast booting and performs best on the Intel® Architecture Processors described + on the :ref:`supported hardware list`. Additionally, this + kernel includes the VirtualBox\* kernel modules, see our + :ref:`instructions on using Virtualbox` for more + information. The optimization patches are found in our `Linux-LTS`_ GitHub + repo. + +VM only +******* + +Kernel KVM + The *kernel-kvm* bundle focuses on the Linux + :abbr:`KVM (Kernel-based Virtual Machine)`. It is optimized for fast + booting and performs best on Virtual Machines running on the Intel® Architecture + Processors described on the + :ref:`supported hardware list`. Use this kernel when + running |CL| as the guest OS on top of *qemu/kvm*. Use this kernel with + **cloud orchestrators** using *qemu/kvm* internally as their **hypervisor** + . This kernel can be used as a standalone |CL| VM, see our + :ref:`instructions on using KVM` for more information. The + optimization patches are found in our `Linux-KVM`_ GitHub repo. + +Kernel Hyper-V\* + The *kernel-hyperv* bundle focuses on running Linux on Microsoft\* + Hyper-V. It is optimized for fast booting and performs best on Virtual + Machines running on the Intel® Architecture Processors described on the + :ref:`supported hardware list`. + Use this kernel when running |CL| as the guest OS of **Cloud Instances** in + projects such as Microsoft `Azure`_\*. This kernel can be used in a + standalone |CL| VM, see our :ref:`instructions on using Hyper-V` + for more information. The optimization patches are found in our + `Linux-HyperV`_ GitHub repo. + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _Linux: https://github.com/clearlinux-pkgs/linux +.. _Linux-LTS: https://github.com/clearlinux-pkgs/linux-lts +.. _Linux-KVM: https://github.com/clearlinux-pkgs/linux-kvm +.. _Linux-HyperV: https://github.com/clearlinux-pkgs/linux-hyperv +.. _Linux-HyperV-LTS: https://github.com/clearlinux-pkgs/linux-hyperv-lts +.. _Linux-Container: https://github.com/clearlinux-pkgs/linux-container +.. _bundles: https://github.com/clearlinux/clr-bundles +.. _CIAO: https://github.com/01org/ciao +.. _Azure: + https://azuremarketplace.microsoft.com/en-us/marketplace/apps/clear-linux-project.clear-linux-os + diff --git a/_sources/guides/clear/debug.rst.txt b/_sources/guides/clear/debug.rst.txt new file mode 100644 index 00000000..bb984767 --- /dev/null +++ b/_sources/guides/clear/debug.rst.txt @@ -0,0 +1,91 @@ +.. _debug: + +Debug system +############ + +|CL-ATTR| introduces a novel approach to system software debugging using +*clr-debug-info*. On the client side, the |CL| debug system obtains any +necessary debug information on-the-fly over a network during a debugging +session. On the server side, the system curates and compresses debug +information into small pieces for efficient downloading. + +For developers, this avoids the interruption during debugging that usually +happens when debug information is missing. This can be especially useful on +systems where storage is limited. + + +.. contents:: :local: + :depth: 2 + + +Background +---------- + +Software that is compiled and packaged for general usage in an operating +system typically only contains components that are used to execute the +program, such as binaries and libraries. Extra developer data, such as the +actual source code and symbol information, are separated and excluded for +efficiency. + +The debug information helps relate binary code to human readable source code +lines and variables. Most of the time, this auxiliary information +is not needed; +however without it, debugging a program results in limited visibility. + + +Usage +----- + +The clr-debug-info system is integrated into |CL| and seamlessly engages once +installed. + +#. Install the *dev-utils* bundle. + + .. code:: bash + + sudo swupd bundle-add dev-utils + + .. note:: + + The *telemetrics* and *performance-tools* bundles also include + clr-debug-info. + + +#. Start a debugging session against a program using a debugger, such as GDB. + For example, to debug *gnome-control-center* execute the following + command: + + .. code:: bash + + gdb /usr/bin/gnome-control-center + +As you step through the program and debug information is needed, the +clr_debug_daemon obtains it in the background. + + +Implementation +-------------- + +The implementation of the |CL| debug system is open source and available on +GitHub at: https://github.com/clearlinux/clr-debug-info/ + +.. figure:: figures/debug-diagram.png + :width: 400px + :alt: Debug system communication flow + + Figure 1: The communication flow of the |CL| debug system + +The |CL| debug system implements a :abbr:`FUSE (filesystem in userspace)` +filesystem mounted at :file:`/usr/lib/debug` and :file:`/usr/src/debug`. The +FUSE filesystem starts automatically. You can verify its status by executing +:command:`systemctl status clr_debug_fuse.service`. + +The *clr_debug_daemon* is responsible for fetching the appropriate package +debug content from the server and making it available for any debugging +programs that need it. It is socket activated whenever a request to the local +FUSE filesystem occurs. You can verify its status with :command:`systemctl +status clr_debug_daemon.service`. + + +|CL| hosts debuginfo content packaged for consumption by |CL| debug clients at +https://download.clearlinux.org/debuginfo/ diff --git a/_sources/guides/clear/k8s-migration.rst.txt b/_sources/guides/clear/k8s-migration.rst.txt new file mode 100644 index 00000000..cb863fe1 --- /dev/null +++ b/_sources/guides/clear/k8s-migration.rst.txt @@ -0,0 +1,321 @@ +.. _kubernetes-migration: + +Kubernetes\* migration +###################### + +This guide describes how to migrate `Kubernetes container orchestration system`_ on |CL-ATTR| from 1.17.x to 1.19.x. + +.. contents:: + :local: + :depth: 1 + +Background +********** + +The version of Kubernetes\* was bumped from 1.17.7 to 1.19.4 in |CL-ATTR| +release 34090. This guide and the |CL| bundle `k8s-migration` were created +to help facilitate migration of a cluster from 1.17.x to the latest 1.19.x . + +The new |CL| bundle `k8s-migration` was added in |CL-ATTR| release 34270. + +Prerequisites +************* + +* Make sure you check any updates to kubernetes upgrade doc for caveats related to the version that is running in the cluster. +* Make sure ALL the nodes are in Ready state. Without that, the cluster cannot be upgraded. + Either fix the broken nodes or remove them from the cluster. + +.. contents:: + :local: + :depth: 1 + +Upgrade 1.17.x ---> 1.18.15 +*************************** + +#. Upgrade Control Node to 1.18.15 first + + First step would be to upgrade one of the main control node and + update kubernetes components on them. You will need to have a newer + version of :command:`kubeadm` for the upgrade to work. Please consult + `kubeadm upgrade guide`_ + for any caveats from your current version to the new one. + + Update |CL| to the latest release to update the kubernetes version. + + .. code-block:: bash + + sudo -E swupd update + + .. note:: + Note: PLEASE DO NOT REBOOT YOUR SYSTEM AT THIS TIME. |CL| is awesome and + your stuff will work just fine. + +#. Add the new Kubernetes migration bundle which contains the 1.18.15 binaries. + + .. code-block:: bash + + sudo -E swupd bundle-add k8s-migration + +#. Find the upgrade version of kubeadm that can used. This should be 1.18.15. + + This command will show the command and possible jumps that can be made from the current kubernetes version. + + .. code-block:: bash + + sudo -E /usr/k8s-migration/bin/kubeadm upgrade plan + + Sample output: + + .. code-block:: console + + [upgrade/config] Making sure the configuration is correct: + [upgrade/config] Reading configuration from the cluster... + [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' + [preflight] Running pre-flight checks. + [upgrade] Running cluster health checks + [upgrade] Fetching available versions to upgrade to + [upgrade/versions] Cluster version: v1.17.17 + [upgrade/versions] kubeadm version: v1.18.15 + I0209 21:12:49.868786 832739 version.go:252] remote version is much newer: v1.20.2; falling back to: stable-1.18 + [upgrade/versions] Latest stable version: v1.18.15 + [upgrade/versions] Latest stable version: v1.18.15 + [upgrade/versions] Latest version in the v1.17 series: v1.17.17 + [upgrade/versions] Latest version in the v1.17 series: v1.17.17 + + Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply': + COMPONENT CURRENT AVAILABLE + Kubelet 3 x v1.17.7 v1.18.15 + + Upgrade to the latest stable version: + + COMPONENT CURRENT AVAILABLE + API Server v1.17.17 v1.18.15 + Controller Manager v1.17.17 v1.18.15 + Scheduler v1.17.17 v1.18.15 + Kube Proxy v1.17.17 v1.18.15 + CoreDNS 1.6.5 1.6.7 + Etcd 3.4.3 3.4.3-0 + + You can now apply the upgrade by executing the following command: + + kubeadm upgrade apply v1.18.15 + +#. Upgrade the node to the intermediate 1.18.15 version of Kubernetes. + + .. code-block:: bash + + sudo -E /usr/k8s-migration/bin/kubeadm upgrade apply v1.18.15 + + .. note:: + Note: Do **not** reboot the system yet. + +#. Upgrade Additional Control Nodes to 1.18.15 + + In multi-node control plane, verify all the control plane nodes are updated prior to upgrading the worker nodes/SUTs. + +#. Upgrade Other Nodes to 1.18.15 + + For each of the other nodes: + + a. Update |CL| to the latest release to update the kubernetes version. + + .. code-block:: bash + + sudo -E swupd update + + #. Add the new Kubernetes migration bundle which contains the 1.18.15 binaries. + + .. code-block:: bash + + sudo -E swupd bundle-add k8s-migration + + #. On the **Admin node**, drain the Client node *FIRST* + + .. code-block:: bash + + /usr/k8s-migration/bin/kubectl drain --ignore-daemonsets --delete-local-data + + #. Back on the **Client node**, upgrade Kubernetes on the Client + + .. code-block:: bash + + sudo -E /usr/k8s-migration/bin/kubeadm upgrade node + + #. On the **Admin node**, re-enable the Client + + .. code-block:: bash + + /usr/k8s-migration/bin/kubectl uncordon + + + #. Back on the **Client node**, restart Kubernetes on the Client + + .. code-block:: bash + + sudo -E systemctl restart kubelet + +#. Restart Kubernetes on the Admin node(s) to finish the 1.18.x upgrade + + .. code-block:: bash + + sudo -E systemctl restart kubelet + + .. note:: + Note: Wait for all nodes to be Ready and showing the 1.19.x version. + This version will now show as it is the released version the + service files will see and use, but the Nodes are *not* upgraded yet. + +Upgrade 1.18.15 ---> 1.19.x +*************************** + +#. Upgrade Control Node to 1.19.x + + Now that systems are upgraded to the intermediate release of 1.18.15 + each of the nodes can be upgraded to the latest 1.19.x release. + +#. Find the upgrade version of kubeadm that can used. This should be 1.19.x. + + This command will show the command and possible jumps that can be made from the current kubernetes version. + + .. code-block:: bash + + sudo -E kubeadm upgrade plan + + Sample output: + + .. code-block:: console + + [upgrade/config] Making sure the configuration is correct: + [upgrade/config] Reading configuration from the cluster... + [upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml' + [preflight] Running pre-flight checks. + [upgrade] Running cluster health checks + [upgrade] Fetching available versions to upgrade to + [upgrade/versions] Cluster version: v1.18.15 + [upgrade/versions] kubeadm version: v1.19.7 + I0209 23:08:23.810900 925910 version.go:252] remote version is much newer: v1.20.2; falling back to: stable-1.19 + [upgrade/versions] Latest stable version: v1.19.7 + [upgrade/versions] Latest stable version: v1.19.7 + [upgrade/versions] Latest version in the v1.18 series: v1.18.15 + [upgrade/versions] Latest version in the v1.18 series: v1.18.15 + + Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply': + COMPONENT CURRENT AVAILABLE + kubelet 3 x v1.17.7 v1.19.7 + + Upgrade to the latest stable version: + + COMPONENT CURRENT AVAILABLE + kube-apiserver v1.18.15 v1.19.7 + kube-controller-manager v1.18.15 v1.19.7 + kube-scheduler v1.18.15 v1.19.7 + kube-proxy v1.18.15 v1.19.7 + CoreDNS 1.6.7 1.7.0 + etcd 3.4.3-0 3.4.13-0 + + You can now apply the upgrade by executing the following command: + + kubeadm upgrade apply v1.19.7 + + The table below shows the current state of component configs as understood by this version of kubeadm. + Configs that have a "yes" mark in the "MANUAL UPGRADE REQUIRED" column require manual config upgrade or + resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually + upgrade to is denoted in the "PREFERRED VERSION" column. + + API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED + kubeproxy.config.k8s.io v1alpha1 v1alpha1 no + kubelet.config.k8s.io v1beta1 v1beta1 no + +#. Upgrade the node to the latest 1.19.x version of Kubernetes. + + .. code-block:: bash + + sudo -E /usr/bin/kubeadm upgrade apply v1.19.7 + + .. note:: + + Note: Do **not** reboot the system yet. + +#. Upgrade Additional Control Nodes to 1.19.x + + In multi-node control plane, verify all the control plane nodes are updated prior to upgrading the worker nodes/SUTs. + +#. Upgrade Other Nodes to 1.19.x + + For each of the other nodes: + + a. On the **Admin node**, drain the Client *FIRST* + + .. code-block:: bash + + kubectl drain --ignore-daemonsets + + #. Back on the **Client node**, upgrade Kubernetes on the Client + + .. code-block:: bash + + sudo -E kubeadm upgrade node + + #. On the **Admin node**, re-enable the Client + + .. code-block:: bash + + kubectl uncordon + + #. Back on the **Client node**, if you wish reboot the Client, it is now safe to do so. + + .. code-block:: bash + + sudo reboot + +#. Reboot the Control Node (optional) + + *If you wish reboot the nodes, it is now safe to do so.* + + .. code-block:: bash + + sudo reboot + +**Congratulations!** + +You've successfully installed and set up Kubernetes in |CL| using CRI-O and kata-runtime. You are now ready to follow on-screen instructions to deploy a pod network to the cluster and join worker nodes with the displayed token and IP information. + +Clean up: Remove the migration bundle for each node + +.. code-block:: bash + + sudo -E swupd bundle-remove k8s-migration + +Related topics +************** + +Read the Kubernetes documentation to learn more about: + +* `Kubernetes tutorial `_ + +* `Kubernetes best practices `_ + +* Deploying Kubernetes with a `cloud-native-setup`_ + +* `Understanding basic Kubernetes architecture`_ + +* `Deploying an application to your cluster`_ + +* Installing a `pod network add-on`_ + +* `Joining your nodes`_ + + +.. _kubeadm upgrade guide: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/ + +.. _Kubernetes container orchestration system: https://kubernetes.io/ + +.. _Understanding basic Kubernetes architecture: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-3 + +.. _Deploying an application to your cluster: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-2 + +.. _pod network add-on: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network + +.. _Joining your nodes: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#join-nodes + +.. _cloud-native-setup: https://github.com/clearlinux/cloud-native-setup/tree/master/clr-k8s-examples diff --git a/_sources/guides/clear/mixer.rst.txt b/_sources/guides/clear/mixer.rst.txt new file mode 100644 index 00000000..b384dda9 --- /dev/null +++ b/_sources/guides/clear/mixer.rst.txt @@ -0,0 +1,1110 @@ +.. _mixer: + +mixer +##### + +The |CL-ATTR| team uses **mixer** to generate official update content and +releases. The update content generated by mixer is then consumed by swupd on +a downstream client. The same mixer tool is available to those who wish to create customized update content and releases. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +mixer uses the following sources as inputs to generate update content: + +* Upstream |CL| bundles with their corresponding RPM packages +* Locally-defined bundles with their corresponding local RPM packages +* Locally-defined bundles with upstream RPM packages +* Locally-defined bundles with non-RPM content + +Using the mixer tool, you select which content from these sources that +becomes part of your update. Your selection of sources produces a unique +combination of functionality for your custom update content, known as +a **mix**. + +The update content that mixer generates consists of various pieces of OS +content, update metadata, as well as a complete image. The OS content +includes all files in an update, as well as zero- and delta-packs for +improved update performance. The update metadata, stored as manifests, +describes all of the bundle information for the update. Update content +produced by mixer is then published to a web server and consumed by clients +via :command:`swupd`. Refer to :ref:`swupd ` for additional +information regarding updates and update content. + +How it works +************ + +Learn the mixer tool set up and workflow. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +============= + +* :command:`mixer` bundle + + Add the mixer tool by installing the :command:`mixer` bundle. Refer to + :ref:`swupd-guide` for more information on installing bundles. + +* If you're working behind a corporate proxy, configure proxy settings using + the :ref:`General proxy settings for many applications ` steps. + +* Location to host the update content and images + + In order for :command:`swupd` to make use of your mix, the update content for your mix must be hosted on a web server. Your mix will be configured with an update location URL, which :command:`swupd` will use to pull down updates. + + Refer to `Set up a nginx web server for mixer`_ for an simple example of + setting up an update location. + +Mix setup +========== + +Follow these steps to create and initialize the mixer workspace. Complete +the setup before you create a mix. + +#. Create workspace. + + The mixer tool uses a simple workspace to contain all input and output in a + basic directory structure. The workspace is simply an empty folder from + which you execute the mixer commands. Each mix uses its own separate + workspace. + +#. Initialize the workspace and mix. + + Before you create a mix, you must explicitly initialize the mixer workspace. + During initialization, the mixer workspace is configured and the base for + your mix is defined. By default, your mix is based on the latest + upstream version and starts with the minimum set of bundles. Your first custom + mix version number starts at 10. Alternatively, you can select other + versions or bundle sets from which to start. + + Initialization creates the directory structure within the workspace and adds + the :file:`builder.conf` file, which is used to configure the mixer tool. + + View the `mixer.init man page`_ for more information on mixer + initialization. + + View the list of suitable `releases`_ from which to mix. + +#. Edit builder.conf. + + :file:`builder.conf` tells the mixer tool how to configure the mix. For + example, it allows you to configure where mixer output is located and where swupd update content will be located. + + At minimum, set the URL of your update server so your custom OS knows where to get update content. + + Refer to the `builder.conf`_ section for more information. + +Create a mix +============ + +A mix is created with the following steps: + +#. Add custom RPMs and set up local repo (optional). + + If you are adding custom RPMs to your mix, you must add the RPMs to + your mix workspace and set up a corresponding local repository. + + Go to the :ref:`autospec` guide to learn to build RPMs from + scratch. If the RPMs are not built on |CL|, make sure your + configuration and toolchain builds them correctly for |CL|. Otherwise there + is no guarantee they will be compatible. + + Refer to the :ref:`autospec` guide for more information on using autospec to + build RPMs. + +#. Update and build bundles. + + Add, edit, or remove bundles that will be part of your content and build + them. mixer automatically updates the :file:`mixbundles` file when you + update the bundles in your mix. + + View the `mixer.bundle man page`_ for more information on configuring bundles + in a mix. + + View the `mixer.build man page`_ for more information on building bundles. + + View the `Bundles`_ section for more information on how mixer manages + bundles. + +#. Create the update content. + + mixer creates update content with this step. Zero-packs are created + automatically, and delta-packs can be optionally created at the same time + (for all builds after version 0). + + A zero-pack is the full set of content needed to go from mix version 0 + (nothing) to the mix version for which you just built content. + + A delta-pack provides the content *delta* between a `PAST_VERSION` to a + `MIX_VERSION` that allows the transition from one mix version to another. + + View :ref:`swupd-guide` for more information on update content. + +#. Create image. + + mixer creates a bootable image from your updated content using + the `clr-installer`_ tool. In this step you can specify which bundles you want + *preinstalled* in the image. Users can later install other bundles available + in your mix. + +#. Make update available. + + Deploy update content and images to your update server. + + View the `Example 5: Deploy updates to target`_ for a simple deployment + scenario. + +Maintain or modify mix +====================== + +Update or modify your content to a new version by following the steps to +create a mix. Increment the mix version number for the next mix. + +Examples +******** + +The following examples are designed to work together and in order. The examples +use: + +* A stock installation of |CL|. +* A web server that comes with |CL| to host the content updates. +* A simple VM that updates against the locally produced content created in + Example 2. + +Complete all `Prerequisites`_ before using these examples. + +Example 1: Mix set up +===================== + +This example shows the basic steps for the first-time setup of +mixer for a new mix. + +#. Create a directory to use as a workspace for mixer: + + .. code-block:: bash + + mkdir ~/mixer + +#. In your mixer workspace, generate an initial mix based on the latest + upstream |CL| version, with minimum bundles. In the initialization + output, be aware that your initial mix version is set to 10 and that the + minimum bundles have been added. + + .. code-block:: bash + + cd ~/mixer + mixer init + + .. note:: + + If you want to add all upstream bundles in your mix, + initialize your mix as shown below. + + .. code-block:: bash + + mixer init --all-upstream + +#. Look up your IP address: + + .. code-block:: bash + + networkctl status + +#. Copy the IP “Address”, from above, for the next step. + + .. note:: + + In this example, we put `mixer` and `nginx` on the same system. In a production environment, they would likely reside on different systems. + +#. Edit :file:`builder.conf`. Paste the IP address from the previous step + as the value after \http:// for CONTENTURL and VERSIONURL. For example: + + .. code-block:: console + + CONTENTURL="http://192.168.25.52" + VERSIONURL="http://192.168.25.52" + +#. `Set up a nginx web server for mixer`_. + + +Example 2: Create a simple mix +============================== + +This example shows how to create a simple custom mix using upstream content. +We'll create an image for a QEMU virtual machine that we can use later to +test our mix. + +We can use the default bundles that were added during initialization, but +these include the :command:`native-kernel` bundle that is intended to be +used on a bare metal system instead of a VM. So we will modify the default +bundle set to get a smaller kernel image, which will also be faster to load. + +.. note:: + + The only bundles available to :command:`swupd` for a given release are + those that were added to the mix during build time. A mix doesn’t + automatically inherit all upstream bundles. + +#. Ensure that you have run `mixer init`, shown in Example 1. + +#. Update bundles in mix: + + .. code-block:: bash + + mixer bundle remove kernel-native + mixer bundle add kernel-kvm + + .. note:: + The mixer bundle commands operate on the bundle description files but not on the bundle contents. To remove bundle contents and their tracking completely, follow `Example 6: Remove a bundle from client system`_, Advanced. + +#. In this case, we will add the `editors` bundle from upstream, but we will + remove the :command:`joe` editor. + + .. code-block:: bash + + mixer bundle add editors + mixer bundle edit editors + +#. Use an editor and manually remove `joe` from the bundle definition. + + .. code-block:: bash + + $EDITOR ./local-bundles/editors + +#. List the bundles in the mix again to confirm removal of :command:`joe`. + + .. code-block:: bash + + mixer bundle list --tree + +#. Build bundles: + + .. code-block:: bash + + sudo mixer build bundles + +#. First, browse to web server from Example 1. The web page appears yet + has no update content. Build the update content: + + .. code-block:: bash + + sudo mixer build update + + After that is completed, on your web server, you can see the update + content for mix version 10. + +Example 3: Create an update for your mix +======================================== + +Next, let’s create a new version of the mix. We’ll add a new bundle. + +#. Create a new version of your mix, for the live image to + update to. Increment your mix version by 10: + + .. code-block:: bash + + mixer versions update + +#. Add the upstream :command:`curl` bundle to version 20 of the mix: + + .. code-block:: bash + + mixer bundle add curl + +#. Build your next mix version that incorporates the new bundle. + + .. code-block:: bash + + sudo mixer build bundles + sudo mixer build update + +#. Optionally, you can build delta-packs, which help reduce client update + time: + + .. code-block:: bash + + sudo mixer build delta-packs --from 10 --to 20 + +Refresh your web server to see the update content for mix version 20. + +You can also look in ~/mixer/update/www/ to see the update +content in your workspace. + +Example 4: Build an image +========================= + +This example shows how to build a bootable image containing the +:command:`kernel-kvm`, :command:`os-core`, and the :command:`os-core-update` +bundles from `Example 2: Create a simple mix`_. Complete that example before starting this one. + +Underneath, mixer uses `clr-installer`_ to generate the image. + +#. Change directory into your mix. + +#. Configure image. + + Create a YAML configuration file to specify aspects of your image + such as image name, target media, bundles, etc. See `Installer YAML Syntax`_ + for more information on clr-installer configuration YAML syntax. + + For this example, we will download a sample YAML and modify it. + + .. code-block:: bash + + curl -O https://raw.githubusercontent.com/clearlinux/clr-installer/master/scripts/kvm.yaml + + Make the following revisions to :file:`kvm.yaml`: + + * Reduce overall image size and root partition size by 5GB. + * Remove these bundles from the image: ``editors``, ``network-basic``, + ``openssh-server``, ``sysadmin-basic`` + * Add ``version: 10`` to tell :command:`mixer` to generate an image based + on mix version 10 + + .. note:: + + When creating an image, it is not necessary to include all of the bundles + that are in your entire mix. Once you have a working image, you can use + :command:`swupd` to add them as needed. + + Your :file:`kvm.yaml` should look like below: + + .. code-block:: console + :linenos: + :emphasize-lines: 11,26,29-33,44 + + #clear-linux-config + + # switch between aliases if you want to install to an actuall block device + # i.e /dev/sda + block-devices: [ + {name: "bdevice", file: "kvm.img"} + ] + + targetMedia: + - name: ${bdevice} + size: "3.54G" + type: disk + children: + - name: ${bdevice}1 + fstype: vfat + mountpoint: /boot + size: "512M" + type: part + - name: ${bdevice}2 + fstype: swap + size: "32M" + type: part + - name: ${bdevice}3 + fstype: ext4 + mountpoint: / + size: "3G" + type: part + + bundles: [ + bootloader, + os-core, + os-core-update, + ] + + autoUpdate: false + postArchive: false + postReboot: false + telemetry: false + + keyboard: us + language: en_US.UTF-8 + kernel: kernel-kvm + + version: 10 + +#. Build the image. + + .. code-block:: bash + + sudo mixer build image --template $PWD/kvm.yaml + + The output from this step will be :file:`kvm.img`, which is a live + image. + +Example 5: Deploy updates to target +=================================== + +The image created in Example 4 is directly bootable in QEMU. In this example, +we'll boot the image and verify it. Then we'll update the image from +mix version 10 to mix version 20. + +#. Set up the QEMU environment. + + Install the :command:`kvm-host` bundle to your |CL|: + + .. code-block:: bash + + sudo swupd bundle-add kvm-host + +#. Get the virtual EFI firmware, download the image launch script, and make + it executable: + + .. code-block:: bash + + curl -O https://download.clearlinux.org/image/OVMF.fd + curl -O https://download.clearlinux.org/image/start_qemu.sh + chmod +x start_qemu.sh + +#. Start your VM image (created in Example 4): + + .. code-block:: bash + + sudo ./start_qemu.sh kvm.img + +#. Log in as root and set a password. + +#. By default, the :command:`swupd` client is designed to communicate with an + HTTPS server. For development purposes, the swupd client can talk to + an HTTP server if you add the flag ``allow-insecure-http``. + + To avoid adding a flag each time when invoking :command:`swupd`, enter: + + .. code-block:: bash + + mkdir -p /etc/swupd + cat > /etc/swupd/config << EOF + [GLOBAL] + allow_insecure_http=true + EOF + +#. Try out your mix. + + a. Show the version and update URLs + + .. code-block:: bash + + swupd info + + #. List the bundles installed in your mix: + + .. code-block:: bash + + swupd bundle-list + + #. List available bundles on your update server. + + .. code-block:: bash + + swupd bundle-list -a + + #. Now we will add the :command:`editors` bundle that we modified. + + .. code-block:: bash + + swupd bundle-add editors + + #. Try to start the :command:`joe` editor. + + .. code-block:: bash + + joe + + It should not work because we removed it from the original + :command:`editors` bundle. + + #. Next we will update from version 10 to 20 to capture the + newly-available bundles. + + .. code-block:: bash + + swupd check-update + swupd update + swupd bundle-list -a + + #. Now your mix should be at version 20 and :command:`curl` is available. + Try using :command:`curl`. This will fail because it is not yet installed. + + .. code-block:: console + + curl: command not found + To install curl use: swupd bundle-add curl + + #. Add the new bundle from your update server to your VM. Retry :command:`curl`. + It works! + + .. code-block:: bash + + swupd bundle-add curl + curl -O https://download.clearlinux.org/image/start_qemu.sh + +#. Shutdown your VM: + + .. code-block:: bash + + poweroff + +Example 6: Remove a bundle from client system +============================================= + +Removing a bundle in a future release requires more steps than deleting the +bundle description file, as shown in Example 2. After a bundle is built in +the mix, you must assure all of the files that are part of the bundle are +removed from the client where that bundle is installed. To do this, create a +version of this bundle in which all of its content is marked for deletion. + +In the following example, we show how to remove the contents of the `editors` +bundle that we added to our mix in Example 2. + +#. First update your mix version. This will set the mix to the next version. + + .. code-block:: bash + + mixer versions update + + .. note:: + Run this command every time that you want to build a new version. + +#. Navigate to local-bundles: + + .. code-block:: bash + + cd local-bundles + +#. Open the `editors` bundle with an editor and delete + **all lines** that follow after the `[MAINTAINERS]` line. + +#. Afterward, it should look like this: + + .. code-block:: console + + # [TITLE]: editors + # [DESCRIPTION]: Run popular terminal text editors. + # [STATUS]: Active + # [CAPABILITIES]: + # [TAGS]: Tools and Utilities, Editor + # [MAINTAINER]: Developer + +#. Save and exit. + +#. Next, run a build to capture recently edited bundles and update your mix. + + .. code-block:: bash + + sudo mixer build all + + .. note:: + :command:`mixer build all` runs both :command:`mixer build bundles` and :command:`mixer build update` in one step. + +At this point the new mix, version 30, is complete. All the content of the +editors bundles is marked as deleted. If any clients of this mix upgraded to +mix build version 30, the content of the editors bundle would be removed. +Note that the bundle still exists and is being tracked by :command:`swupd`, +but it contains no files. + +Example 7: Execute a format bump +================================ + +As a maintainer of your mix, you must execute a format bump if you wish to: + +* Track upstream’s format bump on your downstream derivative +* Delete any custom bundles that were added + +Follow the appropriate use case below depending on your needs. + +Basic +----- + +If you maintain your own downstream derivative and you want to track +upstream, you need to do a format bump when one occurs on upstream. This +method helps you track the latest changes on upstream; however, it does not +change any local content that was added or deleted. For example, if you +deprecated bundles, this method will **not remove the bundle tracking**. +Refer to `Advanced`_ for help on managing your local mix and removing bundle +tracking. + +In this example, we show a mix version that was initialized to upstream +version 29740 (format 27). You need to update your mix to upstream version +30700 (format 28). To do so, you will go through a format bump. + +#. Change to your mix location and verify the current version of the mix and + its format. + + .. code-block:: bash + + mixer versions + +#. Update to upstream version, which has a newer format. + + .. code-block:: bash + + mixer versions update --upstream-version 30700 + + The output will look like this: + + .. code-block:: console + + Old mix: 10 + Old upstream: 29740 (format: 27) + + New mix: 20 + New upstream: 30700 (format: 28) + [...] + + Read the output carefully: + + * The Old mix shows the current version (10) of your mix. + + * The Old upstream shows the version and format (27) on which it’s based. + + * The New mix shows the new version (20) of your mix. + + * The New upstream shows the version and format (28) on which it’s based. + +#. Given that the format in the output differs, you need to run a + format bump: + + .. code-block:: bash + + sudo mixer build upstream-format --new-format 28 + + .. note:: + + You specify the :command:`--new-format` to indicate the format (28) to which you transition. + +#. Your mix is now synchronized with the new format (28); however, you must + still advance to the desired or latest version. + + .. code-block:: bash + + mixer versions update --upstream-version 30700 + +Advanced +-------- + +To properly remove a bundle from being tracked by :command:`swupd`, +do a manual format bump. This process can also be used to perform +customizations during the update, such as: + +* Adjustment in the command parameters + +* Change the content of the chroot + +Follow the `afb.sh reference script`_ to learn how to do a manual format bump. The `afb.sh reference script`_ shows an example of how to: + +* Create a mix + +* Add a bundle + +* Deprecate a bundle + +* Do a format bump to remove the deprecated bundle + +References +********** + +Reference the `mixer man page`_ for details regarding mixer commands and options. + +.. contents:: + :local: + :depth: 1 + +.. rst-class:: content-collapse + +builder.conf +============ + +mixer initialization creates a :file:`builder.conf` that stores the basic +configuration for the mixer tool. The items of primary interest are CONTENTURL +and VERSIONURL, which will be used by systems updating against your custom +content. + +.. code-block:: console + + #builder.conf + + #VERSION 1.2 + + [Builder] + CERT = "/home/clr/mix/Swupd_Root.pem" + SERVER_STATE_DIR = "/home/clr/mix/update" + VERSIONS_PATH = "/home/clr/mix" + YUM_CONF = "/home/clr/mix/.yum-mix.conf" + + [Swupd] + BUNDLE = "os-core-update" + CONTENTURL = "" + VERSIONURL = "" + COMPRESSION = ["external-xz"] + UPSTREAM_BUNDLES_URL = "https://github.com/clearlinux/clr-bundles/archive/" + + [Server] + DEBUG_INFO_BANNED = "true" + DEBUG_INFO_LIB = "/usr/lib/debug" + DEBUG_INFO_SRC = "/usr/src/debug" + + [Mixer] + LOCAL_BUNDLE_DIR = "/home/clr/mix/local-bundles" + LOCAL_REPO_DIR = "/home/clr/mix/local-yum" + LOCAL_RPM_DIR = "/home/clr/mix/local-rpms" + OS_RELEASE_PATH = "" + +Additional explanation of variables in :file:`builder.conf` is provided in +Table 1. + +.. list-table:: **Table 1**: Variables in builder.conf + :widths: 50, 50 + :header-rows: 1 + + * - **Variable** + - **Description** + + * - `CERT` + - Sets the path where mixer stores the certificate file used to sign + content for verification. mixer automatically generates the + certificate if you do not provide the path to an existing one, and + signs the :file:`Manifest.MoM` file to provide security for the + updated content you create. + + chroot-builder uses the certificate file to sign the root :file:` + Manifest.MoM` file to provide security for content verification. + swupd uses this certificate to verify the :file:`Manifest.MoM` file's + signature. + + For now, we strongly recommend that you do not modify this variable, + as swupd expects a certificate with a very specific configuration to sign and verify properly. + + * - `CONTENTURL` and `VERSIONURL` + - Set these variables to the IP address of the web server hosting the + update content. + + VERSIONURL is the IP address where the swupd client + looks to determine if a new version is available. + + CONTENTURL is the location from which swupd pulls content updates. If + the web server is on the same machine as the SERVER_STATE_DIR + directory, you can create a symlink to the directory in your web + server's document root to easily host the content. + + These URLs are embedded in the images created by mixer. + + * - `LOCAL_BUNDLE_DIR` + - Sets the path where mixer stores the local bundle definition files. + The bundle definition files include any new, original bundles you + create, along with any edited versions of upstream bundles. + + * - `SERVER_STATE_DIR` + - Sets the path to which mixer outputs content. By default, mixer + automatically sets the path. + + * - `VERSIONS_PATH` + - Sets the path for the mix version and upstream version's two state + files: :file:`mixversion` and :file:`upstreamversion`. mixer creates + both files for you when you set up the workspace. + + * - `YUM_CONF` + - Sets the path where mixer automatically generates the + :file:`.yum-mix.conf` file. The yum configuration file points the + chroot-builder to where the RPMs are stored. + ++-------------------------------+----------------------------------------------------------+ +| **Variable** | **Explanation** | ++-------------------------------+----------------------------------------------------------+ +| `CERT` | Sets the path where mixer stores the certificate file | +| | used to sign content for verification. mixer | +| | automatically generates the certificate if you do not | +| | provide the path to an existing one, and signs the | +| | :file:`Manifest.MoM` file to provide security for the | +| | updated content you create. | +| | | +| | chroot-builder uses the certificate file to sign | +| | the root :file:`Manifest.MoM` file to provide | +| | security for content verification. | +| | | +| | swupd uses this certificate to verify the | +| | :file:`Manifest.MoM` file's signature. | +| | | +| | For now, we strongly recommend that you do not modify | +| | this variable, as swupd expects a certificate with a | +| | very specific configuration to sign and verify | +| | properly. | ++-------------------------------+----------------------------------------------------------+ +| `CONTENTURL` and `VERSIONURL` | Set these variables to the IP address of the web server | +| | hosting the update content. | +| | | +| | VERSIONURL is the IP address where the swupd client | +| | looks to determine if a new version is available. | +| | | +| | CONTENTURL is the location from which swupd pulls | +| | content updates. | +| | | +| | If the web server is on the same machine as the | +| | SERVER_STATE_DIR directory, you can create a symlink to | +| | the directory in your web server's document root to | +| | easily host the content. | +| | | +| | These URLs are embedded in the images created by mixer. | ++-------------------------------+----------------------------------------------------------+ +| `LOCAL_BUNDLE_DIR` | Sets the path where mixer stores the local bundle | +| | definition files. The bundle definition files include | +| | any new, original bundles you create, along with any | +| | edited versions of upstream bundles. | ++-------------------------------+----------------------------------------------------------+ +| `SERVER_STATE_DIR` | Sets the path to which mixer outputs content. By | +| | default, mixer automatically sets the path. | ++-------------------------------+----------------------------------------------------------+ +| `VERSIONS_PATH` | Sets the path for the mix version and upstream version's | +| | two state files: :file:`mixversion` and | +| | :file:`upstreamversion`. mixer creates both files for | +| | you when you set up the workspace. | ++-------------------------------+----------------------------------------------------------+ +| `YUM_CONF` | Sets the path where mixer automatically generates the | +| | :file:`.yum-mix.conf` file. | +| | | +| | The yum configuration file points the chroot-builder to | +| | where the RPMs are stored. | ++-------------------------------+----------------------------------------------------------+ +| **Table 1**: *Variables in builder.conf* | ++-------------------------------+----------------------------------------------------------+ + +Format version +-------------- + +Compatible versions of an OS are tracked with an OS *compatibility epoch*. +Versions of an OS within an epoch are fully compatible and can update to any +other version within that epoch. The compatibility epoch is set as the +`Format` variable in the :file:`mixer.state` file. Variables in the +:file:`mixer.state` are used by mixer between executions and should not be +manually changed. + +Format bump +----------- + +Mixer needs to produce content that is consumable by swupd. For swupd to +consume the content, it needs a consistent protocol that describes the +requirements of the Manifest. + +If the `Format` increments to a new epoch (a "format bump"), the underlying +`swupd` protocol has changed such that updating from one build version in an +old format to a new build version in a new format is **only** allowed if one +performs a corresponding format bump. + +Format bumps are “checkpoints” (see Figure 1). The first release (20) is +built on the previous format with a `swupd` that is capable of interpreting +the next format. The second release (30) has the same content, but it’s +built in the new format. + +Suppose you have build version 10, but you need the tools in build version +40. Whereas version 10 belongs to Format 27, version 40 belongs to Format +28. The swupd client needs to follow formats sequentially. First, you must +update to version 20, which effectively enables a format bump to version 30. +Doing a format bump bridges the gap so your mix can progress to build +version 40. + +.. figure:: ../../_figures/mixer/format-bump.png + :alt: Format bump + + Figure 1: Format bump + +.. note:: + if you update to build 20 and then check which format of the distro is + used, the new build version will show 30, and the new format will show 28. + +.. rst-class:: content-collapse + +Bundles +======= + +mixer stores information about the bundles included in a mix in a flat file +called :file:`mixbundles`, which is located in the path set by the +VERSIONS_PATH variable in :file:`builder.conf`. :file:`mixbundles` is +automatically created when the mix is initiated. mixer will refresh the file +each time you change the bundles in the mix. + +Bundles belong in one of two categories: upstream or local. Upstream +bundles are those provided by |CL|. Local bundles are either modified upstream bundles or new local bundles. + +Upstream bundles +---------------- + +mixer automatically downloads and caches upstream bundle definition files. +These definition files are stored in the upstream-bundles directory in the +workspace. Do not modify the files in this directory. This directory is +simply a mirror for mixer to use. mixer will automatically delete the +contents of this directory before repopulating it on-the-fly if a new +version must be downloaded. + +The mixer tool automatically caches the bundles for the |CL| version +configured in the :file:`upstreamversion` file. :command:`mixer` also +cleans up old versions once they are no longer needed. + +Local bundles +------------- + +Local bundles are bundles that you create, or are edited versions of upstream +bundles. Local bundle definition files are stored in the local-bundles +directory in the workspace. The LOCAL_BUNDLE_DIR variable sets the path of this directory in the :file:`builder.conf` file. + +*mixer always checks for local bundles first and the upstream bundles +second.* So bundles in the local-bundles directory will always take +precedence over any upstream bundles that have the same name. This +precedence enables you to copy upstream bundles locally, and edit into a +local variation. + + +Bundle definition files +----------------------- + +A ``bundle definition`` file consists of a header, followed by a list +of packages and directives. The header holds important meta-data, like +the TITLE, DESCRIPTION, and STATUS. Other meta-data include TAGS, which +define a bundle's function in the ecosystem, and MAINTAINER, which gives +contact information. + +Following the header are the directives, shown in Table 2. + +.. list-table:: **Table 2**: Bundle directives + :widths: 50,50 + :header-rows: 1 + + * - **Directive** + - **Description** + + * - ``include()`` + - Add with this bundle + + * - ``also-add()`` + - Add unless the option ``--skip-optional`` is used with ``swupd bundle-add``. + + * - ``content()`` + - Add the non-packaged content to the bundle. Refer to :ref:`swupd-3rd-party` for usage of this directive. + +Following is `cluster-tools`, an upstream bundle definition file. The +directives are highlighted, and the rest are packages. + +.. code-block:: bash + :emphasize-lines: 8-12 + + [TITLE]: cluster-tools + [DESCRIPTION]: Utilities to manage computer clusters + [STATUS]: Active + [CAPABILITIES]: HPC + [TAGS]: Tools and Utilities + [MAINTAINER]: Juro Bystricky + + include(curl) + include(libglib) + include(libX11client) + also-add(openmpi) + also-add(modules) + munge + pmix + pdsh + slurm + +Bundle configuration +-------------------- + +mixer provides commands to configure the bundles for a mix, such as to add a +bundle to a mix, to create a new bundle for a mix, or to remove a bundle from a mix. View the `mixer.bundle man page`_ for a full list of commands and more information on configuring bundles in a mix. + +Editing an existing local bundle is as simple as opening the bundle definition +file in your favorite editor, making the desired edits, and saving your changes. + +.. note:: + + Removing bundles from a mix: By default, removing a bundle will only + remove the bundle from the mix. The local bundle definition file will + still remain. To completely remove a bundle, including its local bundle definition file, use the :command:`--local` flag. + + If you remove the bundle definition file for a local, edited version of an + upstream bundle in a mix, the mix reverts to reference the original upstream version of the bundle. + +.. rst-class:: content-collapse + +.. _set-up-nginx-web-server-start: + +Set up a nginx web server for mixer +=================================== + +A web server is needed to host your update content. In this example, +the nginx web server is used. + +#. Install the :command:`nginx` bundle. + + .. code-block:: bash + + sudo swupd bundle-add nginx + +#. Create a symbolic link to the mixer update content directory. + + .. code-block:: bash + + sudo mkdir -p /var/www + + sudo ln -sf $HOME/mixer/update/www /var/www/mixer + +#. Set up nginx configuration files. + + .. code-block:: bash + + sudo mkdir -p /etc/nginx/conf.d + + sudo cp -f /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf + +#. Grant ``$USER`` permission to run the web server. + + .. code-block:: bash + + sudo tee -a /etc/nginx/nginx.conf << EOF + user $USER; + EOF + +#. Configure the mixer update server. + + .. code-block:: bash + + sudo tee -a /etc/nginx/conf.d/mixer-server.conf << EOF + server { + server_name localhost; + location / { + root /var/www/mixer; + autoindex on; + } + } + EOF + +#. Restart the daemon, enable nginx on boot, and start the service. + + .. code-block:: bash + + sudo systemctl daemon-reload + + sudo systemctl enable nginx --now + +#. Verify the web server is running at \http://. + If there's no mix content yet, the expected response from nginx will be + a ``404 Not Found``. + +.. _set-up-nginx-web-server-end: + +Related topics +************** + +* :ref:`autospec` +* :ref:`bundles-guide` +* :ref:`swupd-guide` +* :ref:`swupd-3rd-party` + +.. _mixer man page: https://github.com/clearlinux/mixer-tools/blob/master/docs/mixer.1.rst +.. _mixer.init man page: https://github.com/clearlinux/mixer-tools/blob/master/docs/mixer.init.1.rst +.. _mixer.bundle man page: https://github.com/clearlinux/mixer-tools/blob/master/docs/mixer.bundle.1.rst +.. _mixer.build man page: https://github.com/clearlinux/mixer-tools/blob/master/docs/mixer.build.1.rst +.. _releases: https://github.com/clearlinux/clr-bundles/releases +.. _afb.sh reference script: https://github.com/clearlinux/mixer-tools/blob/master/afb.sh +.. _clr-installer: https://github.com/clearlinux/clr-installer +.. _Installer YAML Syntax: + https://github.com/clearlinux/clr-installer/blob/master/scripts/InstallerYAMLSyntax.md + diff --git a/_sources/guides/clear/performance.rst.txt b/_sources/guides/clear/performance.rst.txt new file mode 100644 index 00000000..5caeaded --- /dev/null +++ b/_sources/guides/clear/performance.rst.txt @@ -0,0 +1,241 @@ +.. _performance: + +Performance +########### + +|CL-ATTR| is built with optimizations across the whole stack for improved +performance. |CL| achieves its performance through a variety of design decisions +and software building techniques. + +.. contents:: :local: + :depth: 1 + +Overview +******** + +The |CL| philosophy is to do everything with performance in mind. The |CL| team +applies this philosophy in the project's codebase and operating culture. + +Below are some examples of the |CL| philosophy: + +**Consider performance holistically.** + Performance optimizations are considered across hardware and software. |CL| + shows the performance potential of a holistic approach on Linux, using Intel® + architecture with optimizations across the full stack. + +**Optimize for runtime performance.** + In general, |CL| will trade the one-time cost of longer build time and larger + storage footprint for the repeated benefit of improved runtime performance. + |CL| users benefit from the optimized software but aren't affected by the + increased build time because the |CL| team builds the software before + distributing it to |CL| clients. + +**Optimize performance for server and cloud use cases first.** + Design decisions that optimize performance for server and cloud also benefit + other use cases, such as IoT devices and desktop clients. + +|CL| has become well-known for the performance it can deliver. +`Phoronix publishes +Linux performance comparisons `_ +that include |CL|. + +Software build toolchain +************************ + +|CL| uses many techniques in its software build toolchain to improve software +performance, such as aggressive compiler flags and CPU-specific optimizations. +If maintained manually, these techniques can become complex to support due to +the volume of packages and the potential for technical drift of package +performance configurations. The |CL| team built the :ref:`autospec` tool to +manage this complexity and to apply the techniques used in the software build +toolchain across the entire project. autospec is available as part of the OS for +developers to use when they build their own projects on |CL|. + +Latest versions of compilers and low-level libraries +==================================================== + +|CL| is a rolling release distribution and follows upstream software +repositories, including compilers and libraries, for updates. |CL| includes +upstream source-level optimizations as soon as they're available. + +A benchmark approach to compiler performance +============================================ + +|CL| chooses the compiler used to build each software package on a case-by-case +basis to maximize performance. Typically, |CL| uses the open source `GNU Compiler +Collection `_ (GCC) with the standard low-level +libraries `Glibc `_ and +`libstdc++ `_ for C and C++ +programming languages. If there is a performance advantage, |CL| will build +packages with `Clang / LLVM `_. + +|CL| uses patched compilers and low-level libraries for exact control of the +software build. Patches include changes that default to more aggressive +optimizations or optimizations that haven't yet been merged upstream. + +View the full list of patches in the autospec repositories on GitHub: + +* https://github.com/clearlinux-pkgs/gcc +* https://github.com/clearlinux-pkgs/glibc +* https://github.com/clearlinux-pkgs/llvm + +Aggressive compiler flags +========================= + +|CL| uses aggressive +`compiler flags `_ to +optimize software builds for runtime performance. Some significant flags that +|CL| often implements are: + +`mtune and march `_ + Options used to tune generated code with optimized instructions for specific + CPU types instead of creating generic code for maximum compatibility. + + |CL| defines its minimum hardware requirements to be second-generation + Intel® microarchitecture code name Westmere (released in 2010) or later. + This enables compiler optimizations that are available only on newer + architectures. Whenever possible, |CL| tunes code for the Haswell generation + processors or newer. + + |CL| sets :command:`march=westmere` and :command:`mtune=haswell`. + + .. note:: + |CL| doesn't require Advanced Encryption Standard (AES), so it should + run on some Intel CPUs from the first generation of Intel® microarchitecture code name Nehalem (released in 2008). Refer to the + `recommended minimum system requirements `_ for specific requirements. + +`O3 `_ + The largest preset of compiler options optimizations for performance. O3 + favors runtime performance. + + View the "Optimize Options" section of the GCC man page for additional + information: :command:`man gcc` + +`LTO `_ + Link-time optimization that performs an optimization between compiled object + files and creation of executable binaries by adding extra information to the + compiled object to help the linker. + +`PGO `_ + Profile guided optimization or field guided optimization performs + optimization based on information sampled during the execution of the program. + + +Compiler flags are set at different levels in the |CL| build environment: + +User flags + The set of default flags used by |CL| when a user compiles software + from source. The flags are exported as system-wide environment variables from + the + `/usr/share/defaults/etc/profile `_ file to the user’s shell by default. These are the + standard variables read by the compiler, named :command:`*FLAGS`, depending + on the compiler. + + .. note:: + Source code may come with software build systems that + override these values. This will cause a difference in expected flags. + The |CL| autospec tooling will attempt to ignore these overrides, but + the build system may still need patching. A manual build will not ignore + the build system override values if they exist. + +Global flags + Compiler flags applied at a global level for all packages. The |CL| RPM + configuration (`clr-rpm-config `_) + contains global compiler flags. Search the :file:`macros` file for + :command:`global_cflags` and search the :file:`rpmrc` file for + :command:`optflags`. Global compiler flags may be overridden. + + .. note:: + |CL| doesn't use RPMs to install software. |CL| + distributes software in the form of :ref:`bundles-guide`. The RPM format + is only used during the |CL| build process as a way to resolve + dependencies. + +Per-package flags + Compiler flags applied at a per-package level. The package's autospec + repository contains the package-specific compiler flags. Search the + :file:`.spec` file for the + section starting with :command:`export CFLAGS`. + +Multiple builds of libraries with CPU-specific optimizations +============================================================ + +To fully use the capabilities in different generations of CPU hardware, |CL| +will perform multiple builds of libraries with CPU-specific optimizations. For +example, |CL| builds libraries with Intel® Advanced Vector Extensions 2 (Intel® +AVX2) and Intel® Advanced Vector Extensions 512 (Intel® AVX-512). |CL| can then +dynamically link to the library with the newest optimization based on the +processor in the running system. Runtime libraries used by ordinary applications +benefit from these CPU specific optimizations. + +The autospec repository for Python* shows an example of this optimization: +https://github.com/clearlinux-pkgs/python3 + +Kernel +****** + +A modern kernel with variants optimized for different platforms +=============================================================== + +|CL| is a rolling release distribution that uses the newest upstream Linux +kernel. The Linux kernel has frequent updates which can include performance +enhancements. It's a policy of the |CL| team to try to upstream any performance +enhancements in the Linux kernel for all to use. + +|CL| `builds different kernel variants `_ for compatibility with specific platforms. +For example, kernels meant to run on virtual machines skip support for much of +the physical hardware that doesn’t show up in VM environments and will slow down +boot. + +View the kernel configuration and patches to the default native kernel in the +autospec repository: https://github.com/clearlinux-pkgs/linux/ + +Utility to enforce kernel runtime parameters +============================================ + +The Linux kernel exposes parameters for tuning the behavior of drivers and +devices such as certain buffers and resource management strategies. |CL| uses a +small utility, `clr-power-tweaks `_, +to set and enforce kernel parameter values weighted towards performance upon +boot. View the set performance values by running :command:`sudo clr_power --debug`. + +Operating system +**************** + +Operating system and software build-time optimizations set the stage for high +performance. Decisions made after the installation of |CL| are equally as +important. + + +CPU performance governor +======================== + +|CL| uses the performance CPU governor which calls for the CPU to operate at +maximum clock frequency. In other words, P-state P0. The idea behind prioritizing +maximum CPU performance is that the faster a program finishes execution, the +faster the CPU can return to a low energy idle state. See the `CPU Power and +Performance documentation `_ +for further details. + +Restructured boot sequence +========================== + +To optimize boot speed, |CL| uses a restructured order for boot processes that +minimizes the time services wait on slow operations and the time boot processes +wait on each other. + +Systemd-bootchart is a tool for graphing the boot sequence and writes logs to a +file under :file:`/run/log`. The tool and corresponding log file make diagnosing slow +boot problems easier. All |CL| systems have `systemd-bootchart `_ enabled by default for every boot. systemd-bootchart configuration is +non-blocking to not materially slow down boot performance. + +Related topics +************** + +* :ref:`cpu-performance` +* `A Linux* OS for Linux Developers `_ +* `The Performance Race `_ +* `Boosting Python* from profile-guided to platform-specific optimizations `_ +* `Transparent use of library packages optimized for Intel® architecture `_ + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* diff --git a/_sources/guides/clear/security.rst.txt b/_sources/guides/clear/security.rst.txt new file mode 100644 index 00000000..0f125b24 --- /dev/null +++ b/_sources/guides/clear/security.rst.txt @@ -0,0 +1,158 @@ +.. _security: + +OS Security +########### + +|CL-ATTR| aims to make systemic and layered security-conscious decisions +that are both performant and practical. This security philosophy is rooted +within the project's codebase and operating culture. + +.. contents:: :local: + :depth: 1 + +Security in updates +******************* + +The |CL| team believes in the benefits of software security through open +sourcing, incremental updates, and rapidly resolving known security advisories. + +The latest Linux\* codebase +=========================== + +|CL| uses the newest version of the Linux kernel which allows the operating +system to leverage the latest features from the upstream Linux kernel, +including security fixes. + +Automated effective updating +============================ + +|CL| is incrementally updated multiple times per day. + +This `rolling release`_ model allows |CL| to consume the latest security fixes +of software packages as soon as they become available. There is no waiting for +major or minor releases on |CL|. + +An update is not effective if it is just simply downloaded onto a system. +It needs to be obtained *AND* ensured that the new patched copy is being +used; not an older copy loaded into memory. |CL| will let you know when a +service needs to be rebooted or do it for your automatically after +a software update, if desired. + +In |CL| updates are delivered automatically, efficiently, and effectively. For +more information about software updates in |CL|, refer to the :ref:`swupd-guide` +guide. + +Automated CVE scanning and remediation +====================================== + +The sheer number of software packages and security vulnerabilities is growing +exponentially. Repositories of Common Vulnerabilities and Exposures (CVEs) +and their fixes, if known, are published by :abbr:`NIST` in a +National Vulnerability Database \ |NVD|\ and at \ |MITRE|\ . + +|CL| employs a proactive and measured approach to addressing known +and fixable :abbr:`CVEs (Common Vulnerabilities and Exposures)`. +Packages are automatically scanned against CVEs daily, and security +patches are deployed as soon as they are available. + +These combined practices minimize the amount of time |CL| systems are exposed to unnecessary security risk. + +Security in software +********************* + +Minimized attack surface +======================== + +|CL| removes legacy, unneeded, or redundant standards and components as much as +possible to enable the use of best known security standards. Below are some +examples: + +* `RC4`, `SSLv3`, `3DES`, and `SHA-1` ciphers which have had known + vulnerabilities, have been explicitly disabled within many |CL| packages to + avoid their accidental usage. + +* Services and subsystems which expose sensitive system information + have been removed such as the `finger` and `tcpwrappers`. + +* `SFTP` has been disabled by default due to security considerations. + +Verified trust +============== + +|CL| encourages the use of secure practices such as encryption +and digital signature verification throughout the system and discourages blind +trust. Below are some examples: + +* All update operations from swupd are transparently encrypted and checked + against the |CL| maintainers' public key for authenticity. + More information about swupd security can be found in the + `Security for software update in Clear Linux* OS`_ blog post. + +* Before being built, packages available from |CL| verify checksums and + signatures provided by third party project codebases and maintainers. + +* |CL| features a unified certificate store, `clrtrust`_ which comes + ready to work with well-known Certificate Authorities out of the box. + clrtrust also offers an easy to use command line interface for managing + system-wide chains of trust, instead of ignoring foreign certificates. + +Compiled with secure options +============================ + +While |CL| packages are optimized for performance on Intel® architecture, +security conscious kernel and compiler options are sensibly taken advantage of. +Below are some examples: + +* Kernels shipped with |CL| are signed and disallow the usage of + custom kernel modules to maintain verifiable system integrity. + +* `Address space layout randomization (ASLR)`_ and + `Kernel address space layout randomization (KASLR)`_ are kernel features + which defend against certain memory based attacks. + More information about PIE executables can be found in the + `Recent GNU* C library improvements`_ blog post. + +Security in system design +************************* + +Simple, yet effective, techniques are used throughout the |CL| system design to +defend against common attack vectors and enable good security hygiene. Below are +some examples: + +* Full disk encryption using :abbr:`LUKS (Linux Unified Key Setup)` is available + during installation. Refer to `cryptsetup`_ for additional information about + LUKS. + +* |CL| uses the PAM cracklib module to harden user login and password + security resulting in: + + - No default username or root password set out of the box with + |CL|, you will be asked to set your own password immediately. + + - Simple password schemes, which are known to be easily compromised, + cannot be set in |CL|. + + - A password blacklist, to avoid system passwords being set to + passwords which have been compromised in the past. + +* `Tallow`_, a lightweight service which monitors and blocks suspicious SSH + login patterns, is installed with the :command:`openssh-server` bundle. + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _`Security for software update in Clear Linux* OS`: https://clearlinux.org/blogs/security-software-update-clear-linux-os-intel-architecture +.. _`Recent GNU* C library improvements`: https://clearlinux.org/blogs/recent-gnu-c-library-improvements +.. _`rolling release`: https://en.wikipedia.org/wiki/Rolling_release +.. _`clrtrust`: https://github.com/clearlinux/clrtrust +.. _`Address space layout randomization (ASLR)`: https://en.wikipedia.org/wiki/Address_space_layout_randomization +.. _`Kernel address space layout randomization (KASLR)`: https://lwn.net/Articles/569635/ +.. _`cryptsetup`: https://gitlab.com/cryptsetup/cryptsetup/ +.. _`Tallow`: https://github.com/clearlinux/tallow + +.. |NVD| raw:: html + + https://nvd.nist.gov/ + +.. |MITRE| raw:: html + + https://cve.mitre.org/ diff --git a/_sources/guides/clear/stateless.rst.txt b/_sources/guides/clear/stateless.rst.txt new file mode 100644 index 00000000..0c157854 --- /dev/null +++ b/_sources/guides/clear/stateless.rst.txt @@ -0,0 +1,149 @@ +.. _stateless: + +Stateless +######### + +In most operating systems, user data, system data, and configuration files +can become intermingled, which can make them challenging to manage. + +.. figure:: figures/stateless-1.png + :scale: 45% + :align: center + :alt: Stateless: User and system files mixed + + Figure 1: Without stateless, user and system files become mixed on the filesystem over time. + +|CL-ATTR| has a stateless design philosophy with the goal to provide an +:abbr:`OS (operating system)` that functions without excessive user +configuration or customization. Stateless in this context does *not* mean +ephemeral or non-persistent. + +.. contents:: :local: + :depth: 1 + +File-level separation +********************* + +To accomplish a stateless design, the |CL| filesystem hierarchy is separated +between user-owned areas and |CL|-owned areas. + +.. figure:: figures/stateless-2.png + :scale: 45% + :align: center + :alt: Stateless: User and system files separation + + Figure 2: With stateless, user and system files are separated on the filesystem. + +System area +=========== +Files under the :file:`/usr` directory are managed by |CL| as system files +(except :file:`/usr/local`). +Files written under the :file:`/usr` directory by users can get removed +through system updates with :ref:`swupd `. This operating +assumption allows |CL| to verify and maintain integrity of system files. + +User areas +========== +Files under the :file:`/usr/local`, :file:`/etc/`, :file:`/opt`, :file:`/home`, +and :file:`/var` directories are owned and managed by the user. A freshly +installed |CL| system will only have a minimal set of files in the +:file:`/etc/` directory and software installed by |CL| does not write to +:file:`/etc`. This operating assumption allows |CL| users to clearly identify +the configuration that makes their system unique. + + +Software configuration +********************** + +With stateless separation, default software configurations are read in order +from predefined source code, |CL| provided defaults, and user-provided +configuration. + +Default configurations +====================== + +Software in |CL| provides default configuration values so that it is +immediately functional, except for some that require additional configuration. + +If an upstream software puts default configurations in multiple locations +such as :file:`/usr/` and :file:`/etc`, it will be modified by the |CL| +distro to comply with the stateless design. Also, some default configurations +may be modified to close security loopholes. Defaults will reside +under :file:`/usr/share/defaults`. These files can be referenced as +templates for customization. + +For example, after installing the `httpd` bundle for Apache web server, its +default configurations appear in the :file:`/usr/share/defaults/httpd/` directory. + +Overriding configurations +========================= + +If a configuration needs to be changed, the appropriate file should be +modified by the user under :file:`/etc/`. If the configuration file does not +already exist, it can be created in the appropriate location. + +User-defined configuration files should contain the minimal set of desired +changes and rely on default configuration for the rest. + +For example, a customized Apache configuration can be used instead by: + +#. Install the Apache web server bundle. + + .. code-block:: bash + + sudo swupd bundle-add httpd + +#. Create the destination directory for the configuration. + + .. code-block:: bash + + sudo mkdir /etc/httpd + +#. Copy the default configuration as a reference template. + + .. code-block:: bash + + sudo cp /usr/share/defaults/httpd/httpd.conf /etc/httpd/ + +#. Make any desired modifications to the configurations. + + .. code-block:: bash + + sudoedit /etc/httpd/httpd.conf + +#. Reload the service or reboot the system to pickup any changes. + + .. code-block:: bash + + systemctl daemon-reload httpd && systemctl restart httpd + +This pattern can be used to modify the configurations of other programs too. +The `stateless man page`_ has application-specific examples. + +System reset +************ + +One advantage of the stateless design is that the system defaults can be +easily restored by simply deleting everything under :file:`/etc/` and +:file:`/var`. + +Running the commands below effectively performs a system reset as if it was +just installed: + +.. code-block:: bash + + sudo rm -rf /etc + sudo rm -rf /var + +In other Linux distributions, this can be a catastrophic action that may render +a system unable to boot and/or inaccessible. + +Additional information +********************** + +* `stateless man page`_ +* :ref:`firmware` + +.. _`stateless man page`: https://github.com/clearlinux/clr-man-pages/blob/master/stateless.7.rst + + diff --git a/_sources/guides/clear/swupd-3rd-party.rst.txt b/_sources/guides/clear/swupd-3rd-party.rst.txt new file mode 100644 index 00000000..aa87709e --- /dev/null +++ b/_sources/guides/clear/swupd-3rd-party.rst.txt @@ -0,0 +1,281 @@ +.. _swupd-3rd-party: + +swupd 3rd-party +############### + +Upstream |CL| offers a plethora of `bundles`_ to choose from.  + +For users who want access to additional software outside of the distro, +|CL| provides support for 3rd-party bundles. + +There are two components to 3rd-party bundles: + +* Use :command:`mixer` to create 3rd-party bundles + +* Use the :command:`swupd` subcommand :command:`3rd-party` to manage repos, + consume and manage bundles + +Follow this guide to set up a web server to host 3rd-party bundles, +build an example 3rd-party bundle, install and manage the bundle on a +client system. + +.. contents:: + :local: + :depth: 1 + +Also, see our `general guidelines`_ on sharing 3rd-party bundles. + +Prerequisite +************* + +* Familiarity with :command:`mixer` +* Familiarity with :command:`swupd` +* You must be running |CL| version 32570 or higher + +.. include:: ./mixer.rst + :start-after: set-up-nginx-web-server-start: + :end-before: set-up-nginx-web-server-end: + +Create directory to hold 3rd-party app +************************************** + +#. Create a top-level directory to hold all apps. + + .. code-block:: bash + + mkdir ~/my-3rd-party-apps && pushd $_ + +#. Create a directory for each app and put the content of your software in it. + + In this example, the `helloclear.sh` app simply prints + "Hello Clear!" when invoked. + + .. code-block:: bash + + # make helloclear directory + mkdir -p helloclear/usr/bin && pushd $_ + + # create helloclear.sh script + cat > helloclear.sh << EOF + #!/bin/bash + echo "Hello Clear!" + EOF + + # make script executable + chmod +x helloclear.sh + + popd + + .. note:: + + * You can put whatever you want in your app's directory. All of the content + within each directory will get copied onto the client system under + :file:`/opt/3rd-party/bundles//`. + + * To use a 3rd-party RPM, it is recommended to extract the content of the RPM + into a directory. Use :command:`rpm2cpio | sudo cpio -idv`. + +Create bundle of 3rd-party app with mixer +***************************************** + +Next, use :command:`mixer` to create a bundle for each of the apps from the previous +section. + +#. Install the mixer tool. + + .. code-block:: bash + + sudo swupd bundle-add mixer + +#. Create a mixer workspace. + + .. code-block:: bash + + mkdir ~/mixer && cd $_ + +#. Initialize a mix without any default bundles. + + .. code-block:: bash + + mixer init --no-default-bundles + +#. Configure :file:`builder.conf` to set the default bundle, CONTENTURL, and VERSIONURL. + For the "URL"s in this example, it will be IP address of the web server that was + set up earlier. + Substitute with the IP address of your host. + + .. code-block:: bash + + mixer config set Swupd.BUNDLE "os-core" + mixer config set Swupd.CONTENTURL "http://" + mixer config set Swupd.VERSIONURL "http://" + +#. Create an empty local `os-core` bundle. :command:`swupd` client expects the + `os-core` bundle to exist in a mix even if it’s empty. + + .. code-block:: bash + + mixer bundle create os-core --local + +#. Using the `helloclear` app as an example, create the `helloclear` bundle + and use the `content()` directive with the path to the `helloclear` directory in + the bundle definition. + + Refer to `bundle definition`_ for addition information on how to define a bundle. + + .. code-block:: bash + + mixer bundle create helloclear --local + echo "content($HOME/my-3rd-party-apps/helloclear/)" >> local-bundles/helloclear + +#. Add both bundles to the mix. + + .. code-block:: bash + + mixer bundle add os-core + mixer bundle add helloclear + +#. Build the bundles and generate the update content. + + .. code-block:: bash + + sudo mixer build bundles + sudo mixer build update + +Install and manage 3rd-party bundle on client system +**************************************************** + +Finally, use the :command:`swupd` client tool to install and +manage the bundle created with :command:`mixer` earlier. +All installed 3rd-party bundles reside in :file:`/opt/3rd-party/bundles//`. + +#. First, add a repo link to the web server. + The `os-core` bundle will be added automatically when adding a repo. + It contains items that mixer injected into the mix such as version information, + format, CONTENTURL, VERSIONURL, and certificate. + + .. code-block:: bash + + sudo swupd 3rd-party add my-3rd-party-repo \ + http:// --allow-insecure-http + + .. note:: + + By default, the :command:`swupd` client is designed to communicate + with an HTTPS server. For development purposes, the swupd client + can talk to an HTTP server if you add the flag ``--allow-insecure-http``. + + To avoid adding this flag each time when invoking :command:`swupd`, enter: + + .. code-block:: bash + + sudo mkdir -p /etc/swupd + + sudo tee -a /etc/swupd/config << EOF + [GLOBAL] + allow-insecure-http=true + EOF + +#. Query the list of bundles from the repo. + + .. code-block:: bash + + sudo swupd 3rd-party bundle-list -a + +#. Add the `helloclear` bundle. + + .. code-block:: bash + + sudo swupd 3rd-party bundle-add helloclear + +#. List installed 3rd-party bundles. + + .. code-block:: bash + + sudo swupd 3rd-party bundle-list + +#. Look in :file:`/opt/3rd-party` to confirm they were installed there. + + .. code-block:: bash + + tree /opt/3rd-party + + Example output: + + .. code-block:: console + + /opt/3rd-party/ + ├── bin + │   └── helloclear.sh + ├── bundles + │   └── my-3rd-party-repo + │   └── usr + │   ├── bin + │   │   └── helloclear.sh + │   ├── lib + │   │   └── os-release + │   └── share + │   ├── clear + │   │   ├── bundles + │   │   │   ├── helloclear + │   │   │   └── os-core + │   │   ├── update-ca + │   │   │   └── Swupd_Root.pem + │   │   ├── version + │   │   └── versionstamp + │   └── defaults + │   └── swupd + │   ├── contenturl + │   ├── format + │   └── versionurl + └── repo.ini + + 12 directories, 12 files + +Create more bundles and add to client +************************************* + +From here on, to add new bundles to your mix, follow these steps: + +#. Follow the steps above to add a new directory for each app and put content into it. + +#. In the mixer workspace, run :command:`mixer versions update`. + +#. Follow the remaining mixer process to add and build bundles. + +On the client side: + +#. Run :command:`sudo swupd 3rd-party update` to update to the latest version of your mix. + + .. note:: + + If `swupd autoupdate` is enabled, 3rd-party repositories will update + automatically as well during regular swupd update. + +#. Now, you can see and add the new bundles. + +Some limitations of 3rd-party bundles +************************************* + +#. You cannot upload your bundles to a shared community repo because bundles + are tied to your particular mix with its own certificate. + You have to host your own and share your repo. + +#. As with upstream bundles, 3rd-party bundles installation is simply the unpacking + of files onto your system. It cannot perform pre or post-installation actions such as + adding a favorite shortcut to the Gnome desktop dock, for example. + +Related topics +************** + +* :ref:`autospec` +* :ref:`mixer` +* :ref:`bundles` +* :ref:`swupd-guide` + +.. _bundles: + https://clearlinux.org/software +.. _bundle definition: + https://docs.01.org/clearlinux/latest/guides/clear/mixer.html#id16 +.. _general guidelines: + https://community.clearlinux.org/t/about-the-3rd-party-sw-category/4072 diff --git a/_sources/guides/clear/swupd.rst.txt b/_sources/guides/clear/swupd.rst.txt new file mode 100644 index 00000000..6a686aec --- /dev/null +++ b/_sources/guides/clear/swupd.rst.txt @@ -0,0 +1,353 @@ +.. _swupd-guide: + +swupd +##### + +:command:`swupd` links a |CL-ATTR| installation with upstream updates and +software. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +:command:`swupd` has two main functions: + +#. Manage software and replace APT or YUM, by installing bundles + rather than packages. +#. Check for system updates and install them. + +:command:`swupd` manages overlapping dependencies behind the scenes, ensuring +that all software is compatible across the system. It can be used to verify +the OS, clean cached files, and fix issues. + +:ref:`Bundles ` contain everything needed to deliver a software +capability. They are the smallest granularity component that is +managed by |CL|. A bundle comes with all of its dependencies rather than +downloading a cascade of package dependencies when installing a piece of +software. + +Versioning +========== + +Using package managers to monitor software version compatibility or +compare multiple systems on many Linux distributions can be cumbersome. + +With |CL| :command:`swupd`, versioning happens at the individual file level. +This means |CL| generates an entirely new OS version with any set of software +changes to the system, including software downgrades or removals. This +rolling release versioning model is similar to :command:`git` internal version +tracking, where any of the individual file commits are tracked and move the +pointer forward when changed. + +A number that represents the **current** release of the OS describes the +versions of all the software on the OS. Each build is composed of a specific +set of bundles made from a particular version of packages. On a daily basis, +this matters to system administrators who need to determine which of their +systems do not have the latest security fixes, or which combinations of +software have been tested. Every release of the same number is guaranteed to +contain the same versions of software, so there's no ambiguity between two +systems running the same version of |CL|. + +Updating +======== + +|CL| enforces regular updating of the OS by default and automatically +checks for updates against a version server. The content server provides the +file and metadata content for all versions and can be the same as the +version server. The content url server provides metadata in the form of +*manifests*, which list and describe file contents, symlinks, +and directories. Additionally, the actual content is provided to clients +in the form of archive files. + +Software updates with |CL| are also efficient. Unlike package-based +distributions, :command:`swupd` only updates files that have changed, rather +than entire packages. For example, it is quite common for an OS security +patch to be as small as 15 KB. Using binary deltas, |CL| is able to +apply only what is needed. + +For details on how to generate update content for |CL|, see the +:ref:`mixer ` tool. + +How it works +************ + +Prerequisites +============= + +* The device is on a well-connected network. +* The device is able to connect to an update server. The default server is: + https://cdn.download.clearlinux.org/update/ + +Updates +======= + +|CL| updates are automatic by default, but can be set to occur only on +demand. :command:`swupd` makes sure that regular updates are simple and +secure. It can also check the validity of currently installed files and +software, and can correct any problems. + +Manifests +--------- + +The |CL| software update content consists of data and metadata. The data is +the files that end up in the OS. The metadata contains relevant information to +properly provision the data to the OS file system, as well as update the +system and add or remove additional content to the OS. + +The manifests are mostly long lists of hashes that describe content. +Each bundle gets its own manifest file. There is a master manifest +file that describes all manifests to tie it all together. + +Fullfiles, packs, and delta packs +--------------------------------- + +To speed up updates and optimize content delivery, update data provisioned to +a system is obtained by one of the following methods: + +* *Fullfiles* are always generated for every file in every release. This + allows any |CL| to obtain the exact copy of the content + for each version directly. This is used if the OS verification + needs to replace a single file, for instance. + +* *Packs* are available for some releases. They combine many files to speed + up the creation of installation media and large updates. + +* *Delta packs* are an optimized version of packs that only contain updates + (binary diffs). They cannot be used without having the original file content. + +Bundle search +============= + +:command:`swupd` searches download manifest data for +bundles that match the term. Enter only one term, or hyphenated term, per +search. Use the command :command:`man swupd` to learn more. + +Only the base bundle is returned. Bundles can contain other bundles via +includes. For more details, see `Bundle Definition Files`_ and its +subdirectory bundles. + +Bundles that are already installed are marked **(installed)** in search +results. + +Optionally, you can review our `bundles`_ on GitHub\*. + +Examples +******** + +Example 1: Disable and enable automatic updates +=============================================== + +|CL| updates are automatic by default, but can be set to occur only +on demand. + +#. Verify your current auto-update setting. + + .. code-block:: bash + + sudo swupd autoupdate + + Output: + + .. code-block:: console + + Enabled + +#. Disable automatic updates. + + .. code-block:: bash + + sudo swupd autoupdate --disable + + Output: + + .. code-block:: console + + Warning: disabling automatic updates may take you out of compliance with your IT policy + + Running systemctl to disable updates + Created symlink /etc/systemd/system/swupd-update.service → /dev/null. + Created symlink /etc/systemd/system/swupd-update.timer → /dev/null. + +#. Check manually for updates. + + .. code-block:: bash + + sudo swupd check-update + +#. Install an update after identifying one that you need. + + .. code-block:: bash + + sudo swupd update --version + +#. Re-enable automatic installs. + + .. code-block:: bash + + sudo swupd autoupdate --enable + +.. _swupd-guide-example-install-bundle: + +Example 2: Find and install Kata Containers\* +============================================= + +Kata Containers is a popular container implementation. Unlike other +container implementations, each Kata Container has its own +kernel instance and runs on its own :abbr:`VM (Virtual Machine)` for +improved security. + +|CL| makes it very easy to install, since you only need to add +one bundle to use `Kata Containers`_: `containers-virt`_, despite a +number of dependencies. Also, check out our tutorial: :ref:`kata`. + +#. Find the correct bundle. + + To return all possible matches for the search string, enter + :command:`swupd search`, followed by 'kata': + + .. code-block:: bash + + sudo swupd search kata + + The output should be similar to: + + .. code-block:: console + + Bundle with the best search result: + + containers-virt - Run container applications from Dockerhub in + lightweight virtual machines + + This bundle can be installed with: + + swupd bundle-add containers-virt + + Alternative bundle options are + + cloud-native-basic - Contains ClearLinux native software for Cloud + + .. note:: + + If your search does not produce results with a specific term, shorten + the search term. For example, use *kube* instead of *kubernetes*. + +#. Add the bundle. + + .. code-block:: bash + + sudo swupd bundle-add containers-virt + + .. note:: + + To add multiple bundles, add a space followed by the bundle name. + + The output of a successful installation should be similar to: + + .. code-block:: console + + Downloading packs... + + Extracting containers-virt pack for version 24430 + ...50% + Extracting kernel-container pack for version 24430 + ...100% + Starting download of remaining update content. This may take a while... + ...100% + Finishing download of update content... + Installing bundle(s) files... + ...100% + Calling post-update helper scripts. + Successfully installed 1 bundle + +Example 3: Verify and correct system file mismatch +================================================== + +:command:`swupd` can determine whether system directories and files have +been added to, overwritten, removed, or modified (e.g., permissions). + +.. code-block:: bash + + sudo swupd diagnose + +All directories that are watched by :command:`swupd` are verified according +to the manifest data. Hash mismatches are flagged as follows: + +.. code-block:: console + + Verifying version 23300 + Verifying files + ...0% + Hash mismatch for file: /usr/bin/chardetect + ... + ... + Hash mismatch for file: /usr/lib/python3.6/site-packages/urllib3/util/wait.py + ...100% + Inspected 237180 files + 423 files did not match + Verify successful + +In this case, Python\* packages that were installed on top of the default +install were flagged as mismatched. :command:`swupd` can be directed to +ignore or fix issues based on command line options. + +:command:`swupd` can correct any issues it detects. Additional directives +can be added including a white list of directories to be ignored. + +The following command repairs issues, removes unknown items, and +ignores files or directories matching :file:`/usr/lib/python`: + +.. code-block:: bash + + sudo swupd repair --picky --picky-whitelist=/usr/lib/python + +.. _swupd-quick-ref: + +Quick reference +*************** + +swupd info + Returns the currently installed version and update servers. + +swupd update + Updates to a specific version or updates to latest version if no + arguments are used. + +swupd bundle-list [--all] + Lists installed bundles. + +swupd search + Finds a bundle that contains your search term. + +swupd bundle-add + Adds a bundle. + +swupd bundle-remove + Removes a bundle. + +swupd --help + Lists additional :command:`swupd` commands. + +man swupd + Opens the :command:`swupd` man page. + +Refer to `swupd source documentation`_ on GitHub for more details. + +Related topics +************** + +* :ref:`autospec` +* :ref:`mixer` +* :ref:`bundles` + +.. _swupd source documentation: https://github.com/clearlinux/swupd-client/blob/master/docs/swupd.1.rst + +.. _Kata Containers: https://clearlinux.org/downloads/containers + +.. _containers-virt: https://github.com/clearlinux/clr-bundles/blob/master/bundles/containers-virt + +.. _Bundle Definition Files: https://github.com/clearlinux/clr-bundles + +.. _bundles: https://github.com/clearlinux/clr-bundles/tree/master/bundles diff --git a/_sources/guides/clear/telemetrics.rst.txt b/_sources/guides/clear/telemetrics.rst.txt new file mode 100644 index 00000000..42d152b8 --- /dev/null +++ b/_sources/guides/clear/telemetrics.rst.txt @@ -0,0 +1,894 @@ +.. _telem-guide: + +Telemetrics +########### + +This guide describes the |CL-ATTR| telemetry solution. + +.. important:: + + Telemetry in |CL| is **opt-in**. The telemetry client is **not** active + and sends **no** data until you explicitly enable it. + +.. note:: + + The telemetry functionality adheres to + `Intel privacy policies `_ + regarding the collection and use of :abbr:`PII (Personally Identifiable Information)` and is open source. + + No intentionally identifiable information about the user or system owner is + collected. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +Telemetrics in |CL| is a client and server solution used to collect +data from running |CL| systems to help quickly identify and fix bugs in the +OS. Both client and server are customizable, and an API is available on the +client side for instrumenting your code for debug and analysis. + +Telemetry, one of the key features of |CL|, enables developers to observe and +proactively address issues in the OS before end users are impacted. + +Telemetrics is a +`portmanteau word `_ made from: + +* Telemetry, which is sensing and reporting data. +* Analytics, which is using visualization and statistical inferencing to make + sense of the reported data. + +|CL| telemetry reports system-level debug/crash information using specialized +probes. The probes monitor system tasks such as swupd, kernel oops, machine +error checks, and the BIOS error report table for unhandled hardware +failures. Telemetry enables real-time issue reporting to allow system +developers to focus quickly on an issue and monitor corrective actions. + +|CL| telemetry is fully customizable and can also be used during software +development for debugging purposes. You can use the libtelemetry library in +your code to create custom telemetry records. You can also use the +telem-record-gen utility in script files for light-touch record creation +where instrumenting code files doesn't make sense. For more information on +configuring the telemetry client, refer to section `Client Configuration`_. + +The |CL| telemetrics solution is an **opt-in** choice on the client side. +By default, the telemetry client is disabled until you choose to enable it. +Enabling the client is covered in this guide. + +Architecture +============ + +|CL| telemetry has two fundamental components, which are shown in Figure 1: + +* Client, which generates and delivers records to the backend server via the + network. + +* Backend, which receives records sent from the client and displays the + cumulative content through a specialized web interface. + +.. figure:: /_figures/telemetrics/telemetry-e2e.png + :alt: Figure 1, Telemetry Architecture + + Figure 1: :guilabel:`|CL| Telemetry Architecture` + +The telemetry client provides the front end of the telemetrics solution and +includes the following components: + +* telemprobd, which is a daemon that receives and prepares telemetry records + from probes and spools them to disk. +* telempostd, which is a daemon that manages spooled telemetry records and + delivers these records according to configurable settings. +* probes, which collect specific types of data from the operating system. +* libtelemetry, which is the API that telemetrics probes use to create + records. + +The telemetry backend provides the server-side component of the telemetrics +solution and consists of: + +* Nginx web server. +* Two Flask apps: + + * Collector, which is an ingestion web app for records received from client + probes. + * TelemetryUI, which is a web app that exposes different views to visualize + the telemetry data. +* PostgreSQL as the underlying database server. + +.. note:: + + The default telemetry backend server is hosted by the Intel |CL| + development team and is not viewable outside the Intel firewall. To + collect your own records, you must set up your own telemetry backend + server. + +How to use +********** + +From a workflow perspective, the |CL| telemetrics system is straightforward. +On the client side, the main decisions after installation and enabling +telemetry involve what to do with the record data generated by the probes. +You can send the data to the default telemetry server or a custom backend +server, keep the data local to the system, or both. The backend server has a +more complex setup, but once it's running, it is simple to configure and use. + +This section describes some of the possible scenarios for configuring +the |CL| telemetrics system, and suggests which ones make sense according to +your needs. + +For more information on configuring the telemetry client, refer to section +`Client Configuration`_. + +Scenarios +========= + +#. Enable telemetry: + + You must opt-in and start telemetry before probes can generate records. + You can configure the client before starting telemetry by creating a + custom :file:`telemetrics.conf` file that you place in the + :file:`/etc/telemetrics` directory. If you choose to use the built-in + default settings, records will be sent to the telemetrics backend server + managed by the |CL| development team at Intel. + +#. Save record data locally: + + You can configure the telemetry client to save records locally. This is + convenient when you want instant feedback during a development cycle, or + to track system issues if you believe there is a machine-specific problem. + The client can be set not to send records at all or to both keep the + records locally and send to the backend server. + +#. Set up a server to collect data: + + Whether you are managing a network of |CL| systems or you don't want to + send records to the default telemetry server, you can set up a backend + server to collect your records. The backend server can be installed on any + Linux system and provides the same dashboard as the default server. + + +#. Instrument your code with the libtelemetry API: + + The :command:`telemetrics` bundle includes the libtelemetry C library, + which exposes an API used by the telemprobd and telempostd daemons. You + can use these in your applications as well. The API documentation is + located in the :file:`telemetry.h` file in `Telemetrics client`_ + repository. + + +Examples +******** + +.. contents:: + :local: + :depth: 1 + +Enable or disable telemetry +=========================== + +#. Enabling during installation: + + During the initial installation of |CL|, you are requested to join the + stability enhancement program and allow |CL| to collect anonymous reports + to improve system stability. If you choose not to join this program, then + the telemetry software bundle is not added to your system. If you do + choose to join the program, the installer will automatically enable + telemetry on your system by installing the telemetrics bundle, creating + the file :file:`/etc/telemetrics/opt-in`, and enabling the telemetrics + systemd services to run after installation is complete and the system is + restarted. + +#. Enabling after install: + + To install telemetry on your system, run the following commands: + + .. code-block:: bash + + sudo swupd bundle-add telemetrics + sudo telemctl opt-in + sudo telemctl start + + This installs the necessary software, enables telemetry by creating the + file :file:`/etc/telemetrics/opt-in`, and starts the :command:`telemprobd` + and :command:`telempostd` daemons. Your system will begin to send + telemetry data to the backend server. + +#. Disabling after install: + + To disable both of the telemetry daemons, run the following command: + + .. code-block:: bash + + sudo telemctl stop + +#. Opt in to telemetry: + + To opt-in to the telemetry services, simply enter the opt-in command: + + .. code-block:: bash + + sudo telemctl opt-in + sudo telemctl start + + This creates the :file:`/etc/telemetrics/opt-in` file, if it doesn't + already exist. You will need to explicitly start the telemetry services + after you have opted in. + +#. Opt out of telemetry: + + To stop sending telemetrics data from your system, opt out of the + telemetry service: + + .. code-block:: bash + + sudo telemctl opt-out + + This removes the file :file:`/etc/telemetrics/opt-in` and stops the + telemetry services. + + +Saving data locally +=================== + +This example requires |CL| to be installed and telemetry to be enabled on the +system. + +To change how records are managed, copy the default +:file:`/usr/share/defaults/telemetrics/telemetrics.conf` file to +:file:`/etc/telemetrics/telemetrics.conf` and edit it. The changes in the +:file:`/etc/telemetrics/telemetrics.conf` file will override the built-in +defaults referenced in the +:file:`/usr/share/defaults/telemetrics/telemetrics.conf` file. +You will need root permissions to create and edit files in :file:`/etc`. For +each example, and for any time you make changes to the configuration file, +you must restart the client daemons to pick up the changes: + +.. code-block:: bash + + sudo telemctl restart + + +The :command:`telemctl journal` command gives you access to features and +options of the telemetry journal to assist with system analytics and debug. +:command:`telemctl journal` has a number of options to help filter records. +Use :command:`-h` or :command:`--help` to view usage options. + + +#. Keep a local copy and send records to backend server: + + To keep a local copy of the telemetry record and also send it on to the + backend server, we will need to change the + :guilabel:`record_retention_enabled` configuration key value to + :guilabel:`true`. + +#. Keep all records -- don't send to backend server: + + To keep records on the system without sending them to a backend server, set + the :guilabel:`record_server_delivery_enabled` key value to + :guilabel:`false`. Note that you will also need to ensure the + :guilabel:`record_retention_enabled` configuration key value is set to + :guilabel:`true` or the system will not keep local copies. + +#. Keep and send records to custom server: + + This assumes you have set up a custom server according to the next example. + + The server is identified by the :guilabel:`server` setting, and by default + records are sent to the |CL| server + :guilabel:`server=https://clr.telemetry.intel.com/v2/collector`. To change + this, you can use an IP address or fully qualified domain name. + + +Set up a backend server to collect telemetry records +==================================================== + +For this example, start with a clean installation of |CL| on a new system +using the :ref:`bare-metal-install-server` getting started guide and: + +#. Join the :guilabel:`Stability Enhancement Program` to install and + enable the telemetrics components. + +#. Select the manual installation method with the following settings: + + * Set the hostname to :guilabel:`clr-telem-server`, + * Create an administrative user named :guilabel:`clear` and add this user + to sudoers + +#. Log in with your administrative user, from your :file:`$HOME` directory, + run :command:`git` to clone the :guilabel:`telemetrics-backend` repository + into the :file:`$HOME/telemetrics-backend` directory: + + .. code-block:: console + + git clone https://github.com/clearlinux/telemetrics-backend + + .. note:: + + You may need to set up the :envvar:`https_proxy` environment variable if + you have issues reaching github.com. + +#. Change your current working directory to + :file:`telemetrics-backend/scripts`. + +#. Before you install the telemetrics backend with the :file:`deploy.sh` + script file in the next step, here is an explanation of the options to be + specified: + + * :command:`-a install` to perform an install + * :command:`-d clr` to install to a |CL| distro + * :command:`-H localhost` to set the domain to localhost + + .. caution:: + The :file:`deploy.sh` shell script has minimal error checking and makes + several changes to your system. Be sure that the options you define on + the cmdline are correct before proceeding. + +#. Run the shell script from the :file:`$HOME/telemetrics-backend/scripts` + directory: + + .. code-block:: console + + ./deploy.sh -H localhost -a install -d clr + + The script starts and lists all the defined options and prompts you for + the :guilabel:`PostgreSQL` database password. + + .. code-block:: console + + Options: + host: localhost + distro: clr + action: install + repo: https://github.com/clearlinux/telemetrics-backend + source: master + type: git + DB password: (default: postgres): + +#. For the :guilabel:`DB password:`, press the :kbd:`Enter` key to accept the + default password `postgres`. + + .. note:: + + The :file:`deploy.sh` script uses :command:`sudo` to run commands and + you may be prompted to enter your user password at any time while the + script is executing. If this occurs, enter your user password to + execute the :command:`sudo` command. + + +#. After all the server components have been installed, you are prompted to + enter the :guilabel:`PostgreSQL` database password to change it as + illustrated below: + + .. code-block:: console + + Enter password for 'postgres' user: + New password: + Retype new password: + passwd: password updated successfully + + Enter `postgres` for the current value of the password and then enter a new + password. Retype it to verify the new password and the + :guilabel:`PostgreSQL` database password will be updated. + +#. After the installation is complete, you can use your web browser to view + the new server by opening the browser on the system and typing in + :command:`localhost` in the address bar. You should see a web page similar + to the one shown in Figure 2 below. + + .. figure:: /_figures/telemetrics/telemetry-backend-1.png + :alt: Telemetry UI + + Figure 2: :guilabel:`Telemetry UI` + +Create records with telem-record-gen +==================================== + +The :command:`telemetrics` bundle provides a record generator tool called +`telem-record-gen`. This tool can be used to create records from shell +scripts or the command line when it is not desirable to write a probe in C. +Records are sent to the backend server, and can also be echoed to stdout. + +There are three ways to supply the payload to the record: + +#. On the command line, use the :command:`-p ` option: + + .. code-block:: bash + + telem-record-gen -c a/b/c -n -o -p 'payload goes here' + + .. code-block:: console + + record_format_version: 4 + classification: a/b/c + severity: 1 + machine_id: FFFFFFFF + creation_timestamp: 1539023189 + arch: x86_64 + host_type: innotek GmbH|VirtualBox|1.2 + build: 25180 + kernel_version: 4.14.71-404.lts + payload_format_version: 1 + system_name: clear-linux-os + board_name: VirtualBox|Oracle Corporation + cpu_model: Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz + bios_version: VirtualBox + event_id: 2236710e4fc11e4a646ce956c7802788 + + payload goes here + +#. Specify a file that contains the payload with the option + :command:`-P path/to/file`. + + .. code-block:: bash + + telem-record-gen -c a/b/c -n -o -P ./payload_file.txt + + .. code-block:: console + + record_format_version: 4 + classification: a/b/c + severity: 1 + machine_id: FFFFFFFF + creation_timestamp: 1539023621 + arch: x86_64 + host_type: innotek GmbH|VirtualBox|1.2 + build: 25180 + kernel_version: 4.14.71-404.lts + payload_format_version: 1 + system_name: clear-linux-os + board_name: VirtualBox|Oracle Corporation + cpu_model: Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz + bios_version: VirtualBox + event_id: d73d6040afd7693cccdfece479df9795 + + payload read from file + +#. If the :command:`-p` or :command:`-P` options are absent, the tool reads + from stdin so you can use it in a :file:`heredoc` in scripts. + + .. code-block:: bash + + #telem-record-gen -c a/b/c -n -o << HEOF + payload read from stdin + HEOF + + .. code-block:: console + + record_format_version: 4 + classification: a/b/c + severity: 1 + machine_id: FFFFFFFF + creation_timestamp: 1539023621 + arch: x86_64 + host_type: innotek GmbH|VirtualBox|1.2 + build: 25180 + kernel_version: 4.14.71-404.lts + payload_format_version: 1 + system_name: clear-linux-os + board_name: VirtualBox|Oracle Corporation + cpu_model: Intel(R) Core(TM) i7-4650U CPU @ 1.70GHz + bios_version: VirtualBox + event_id: 2f070e8e71679f2b1f28794e3a6c42ee + + payload read from stdin + + +Set a static machine id +======================= + +The machine id reported by the telemetry client is rotated every three days +for privacy reasons. If you wish to have a static machine id for testing +purposes, you can opt in by creating a file named +:file:`opt-in-static-machine-id` in the directory :file:`/etc/telemetrics/`. + +#. Create a directory :file:`telemetrics`. + + .. code-block:: bash + + sudo mkdir -p /etc/telemetrics + + +#. Create the file and replace the "unique machine ID" with your desired + static machine ID. + + .. code-block:: bash + + echo "unique machine ID" | sudo tee /etc/telemetrics/opt-in-static-machine-id + +.. note:: + + The machine ID is different from the system hostname. + +Instrument your code with the libtelemetry API +============================================== + +Prerequisites +------------- + +Confirm that the telemetrics header file is located on the system at +:file:`usr/include/telemetry.h`. The `latest version`_ of the file can also +be found on github for reference, but installing the :command:`telemetrics` +bundle will install the header file that matches your |CL| version. + +#. Includes and variables: + + You must include the following headers in your code to use the API: + + .. code-block:: console + + #define _GNU_SOURCE + #include + #include + #include + #include + + + Use the following code to create the variables needed to hold the data for + the record to be created: + + .. code-block:: console + + uint32_t severity = 1; + uint32_t payload_version = 1; + char classification[30] = "org.clearlinux/hello/world"; + struct telem_ref *tm_handle = NULL; + char *payload; + int ret = 0; + + + + Severity: + Type: uint32_t + Value: Severity field value. Accepted values are in the range 1-4, with + 1 being the lowest severity and 4 being the highest severity. Values + provided outside of this range are clamped to 1 or 4 [low, med, high, + crit]. + + Payload_version: + Type: uint32_t + Value: Payload format version. The only currently supported value is 1, + which indicates that the payload is a freely-formatted (unstructured) + string. Values greater than 1 are reserved for future use. + + Classification: + Type: char array + Value: It should have the form, DOMAIN/PROBENAME/REST: DOMAIN is the + reverse domain to use as a namespace for the probe (e.g. org.clearlinux), + PROBENAME is the name of the probe, and REST is an arbitrary value that + the probe should use to classify the record. The maximum length for the + classification string is 122 bytes. Each sub-category may be no longer + than 40 bytes long. Two \'/\' delimiters are required. + + Tm_handle: + Type: Telem_ref struct pointer + Value: Struct pointer declared by the caller. The struct is initialized + if the function returns success. + + Payload: + Type: char pointer + Value: The payload to set. + +#. For this example, we'll set the payload to “hello” by using + :command:`asprintf()`: + + .. code-block:: console + + if (asprintf(&payload, "hello\n") < 0) { + exit(EXIT_FAILURE); + } + + The functions :command:`asprintf()` and :command:`vasprintf()` are analogs + of :command:`sprintf(3)` and :command:`vsprintf(3)`, except that they + allocate a string large enough to hold the output including the + terminating null byte ('\0'), and return a pointer to it via the first + argument. This pointer should be passed to :command:`free(3)` to release + the allocated storage when it is no longer needed. + +#. Create the new telemetry record: + + The function :command:`tm_create_record()` initializes a telemetry + record and sets the severity and classification of that record, as well as + the payload version number. The memory needed to store the telemetry + record is allocated and should be freed with :command:`tm_free_record()` + when no longer needed. + + .. code-block:: console + + if ((ret = tm_create_record(&tm_handle, severity, classification, payload_version)) < 0) { + printf("Failed to create record: %s\n", strerror(-ret)); + ret = 1; + goto fail; + } + +#. Set the payload field of a telemetrics record: + + The function :command:`tm_set_payload()` attaches the provided telemetry + record data to the telemetry record. The current maximum payload size is + 8192b. + + .. code-block:: console + + if ((ret = tm_set_payload(tm_handle, payload)) < 0) { + printf("Failed to set record payload: %s\n", strerror(-ret)); + ret = 1; + goto fail; + } + free(payload); + + The :command:`free()` function frees the memory space pointed to by `ptr`, + which must have been returned by a previous call to :command:`malloc()`, + :command:`calloc()`, or :command:`realloc()`. Otherwise, or if + :command:`free(ptr)` has already been called before, undefined behavior + occurs. If `ptr` is NULL, no operation is performed. + +#. Send a record to the telemetrics daemon: + + The function :command:`tm_send_record()` delivers the record to the local + :command:`telemprobd(1)` service. Since the telemetry record was allocated + by the program it should be freed with :command:`tm_free_record()` when it + is no longer needed. + + .. code-block:: console + + if ((ret = tm_send_record(tm_handle)) < 0) { + printf("Failed to send record to daemon: %s\n", strerror(-ret)); + ret = 1; + goto fail; + } else { + printf("Successfully sent record to daemon.\n"); + ret = 0; + } + fail: + tm_free_record(tm_handle); + tm_handle = NULL; + + return ret; + + +#. A full sample application with compiling flags: + + Create a new file :file:`test.c` and add the following code: + + .. code-block:: console + + #define _GNU_SOURCE + #include + #include + #include + #include + + int main(int argc, char **argv) + { + uint32_t severity = 1; + uint32_t payload_version = 1; + char classification[30] = "org.clearlinux/hello/world"; + struct telem_ref *tm_handle = NULL; + char *payload; + + int ret = 0; + + if (asprintf(&payload, "hello\n") < 0) { + exit(EXIT_FAILURE); + } + + if ((ret = tm_create_record(&tm_handle, severity, classification, payload_version)) < 0) { + printf("Failed to create record: %s\n", strerror(-ret)); + ret = 1; + goto fail; + } + + if ((ret = tm_set_payload(tm_handle, payload)) < 0) { + printf("Failed to set record payload: %s\n", strerror(-ret)); + ret = 1; + goto fail; + } + + free(payload); + + if ((ret = tm_send_record(tm_handle)) < 0) { + printf("Failed to send record to daemon: %s\n", strerror(-ret)); + ret = 1; + goto fail; + } else { + printf("Successfully sent record to daemon.\n"); + ret = 0; + } + fail: + tm_free_record(tm_handle); + tm_handle = NULL; + + return ret; + } + + + + Compile with the gcc compiler, using this command: + + .. code-block:: bash + + gcc test.c -ltelemetry -o test_telem + + + Test to ensure the program is working: + + .. code-block:: bash + + ./test_telem + Successfully sent record to daemon. + + .. note:: + + A full example of the `heartbeat probe`_ in C is documented in the + source code. + +Reference +********* + +.. contents:: + :local: + :depth: 1 + +The telemetry API +================= + +Installing the :command:`telemetrics` bundle includes the libtelemetry C +library, which exposes an API used by the telemprobd and telempostd daemons. +You can use these in your applications as well. The API documentation is found +in the :file:`telemetry.h` file in `Telemetrics client`_ repository. + +Client configuration +==================== + +The telemetry client will look for the configuration file located at +:file:`/etc/telemetrics/telemetrics.conf` and use it if it exists. If the +file does not exist, the client will use the default configuration defined +at build time. There is a sample configuration file located at +:file:`/usr/share/defaults/telemetrics/telemetrics.conf` and represents the +default values that are used when the programs are built. To modify or +customize the configuration, copy the file from +:file:`/usr/share/defaults/telemetrics/telemetrics.conf` to the file +:file:`/etc/telemetrics/telemetrics.conf` and edit it to add your +customizations. + +.. code-block:: bash + + sudo mkdir -p /etc/telemetrics + cp /usr/share/defaults/telemetrics/telemetrics.conf /etc/telemetrics/telemetrics.conf + +.. note:: + + Telemetrics configuration is a layered mechanism since the defaults are + defined at build time and each field can be overwritten individually. + Therefore you only need to add the specific field that you want to change + from the default value to your customized value in the + :file:`/etc/telemetrics/telemetrics.conf` file. + +Configuration options +--------------------- + +The client can use the following configuration options from the config file: + +server + This specifies the web server to which telempostd sends the telemetry + records. +socket_path + This specifies the path of the unix domain socket on which telemprobd + listens for connections from the probes. +spool_dir + This configuration option is related to spooling. If the daemon is not + able to send the telemetry records to the backend server due to reasons + such as the network availability, then it stores the records in a spool + directory. This option specifies the path of the spool directory. This + directory should be owned by the same user as the daemon. +record_expiry + This is the time, in minutes, after which the records in the spool + directory are deleted by the daemon. +spool_process_time + This specifies the time interval, in seconds, that the daemon waits + before checking the spool directory for records. The daemon picks up the + records in the order of modification date and tries to send the record to + the server. It sends a maximum of 10 records at a time. If it was able to + send a record successfully, it deletes the record from the spool. If the + daemon finds a record older than the "record_expiry" time, then it deletes + that record. The daemon looks at a maximum of 20 records in a single spool + run loop. +rate_limit_enabled + This determines whether rate-limiting is enabled or disabled. When + enabled, there is a threshold on both records sent within a window of + time, and record bytes sent within a window a time. +record_burst_limit + This is the maximum amount of records allowed to be passed by the daemon + within the record_window_length of time. If set to -1, the rate-limiting + for record bursts is disabled. +record_window_length + The time, in minutes (0-59), that establishes the window length for the + record_burst_limit. For example, if record_burst_window=1000 and + record_window_length=15, then no more than 1000 records can be passed + within any given fifteen-minute window. +byte_burst_limit + This is the maximum amount of bytes that can be passed by the daemon + within the byte_window_length of time. If set to -1, the rate-limiting + for byte bursts is disabled. +byte_window_length + This is the time, in minutes (0-59), that establishes the window length + for the byte_burst_limit. +rate_limit_strategy + This is the strategy chosen once the rate-limiting threshold has been + reached. Currently the options are 'drop' or 'spool', with spool being the + default. If spool is chosen, records will be spooled and sent at a later + time. +record_retention_enabled + When this key is enabled (true), the daemon saves a copy of the payload on + disk from all valid records. To avoid the excessive use of disk space, + only the latest 100 records are kept. The default value for this + configuration key is false. +record_server_delivery_enabled + This key controls the delivery of records to the server; when enabled + (default value), the record will be posted to the address in the + configuration file. If this configuration key is disabled (false), + records will not be spooled or posted to backend. This configuration key + can be used in combination with record_retention_enabled to keep copies + of telemetry records locally only. + + .. note:: + + Configuration options may change as the telemetry client evolves. + Please use the comments in the default file itself as the most accurate + reference for configuration. + + +Client run-time options +======================= + +The |CL| telemetry client provides an admin tool called :guilabel:`telemctl` +for managing the telemetry services and probes. The tool is located in +:file:`/usr/bin`. Running it with no argument results in the following: + +.. code-block:: bash + + sudo telemctl + +.. code-block:: console + + /usr/bin/telemctl - Control actions for telemetry services + stop Stops all running telemetry services + start Starts all telemetry services + restart Restarts all telemetry services + is-active Checks if telemprobd and telempostd are active + opt-in Opts in to telemetry, and starts telemetry services + opt-out Opts out of telemetry, and stops telemetry services + journal Prints telemetry journal contents. Use -h argument for more + options + +start/stop/restart +------------------ + +The commands to start, stop, and restart the telemetry services manage all +required services and probes on the system. There is no need to separately +start/stop/restart the two client daemons telemprobd and telempostd. +The :command:`restart` command option will call :command:`telemctl stop` +followed by :command:`telemctl start` . + +is-active +--------- + +The :command:`is-active` option reports whether the two client daemons are +active. This is useful to verify that the :command:`opt-in` and +:command:`opt-out` options have taken effect, or to ensure that telemetry is +functioning on the system. Note that both daemons are verified. + +.. code-block:: bash + + sudo telemctl is-active + +.. code-block:: console + + telemprobd : active + telempostd : active + + +.. _Telemetrics client: https://github.com/clearlinux/telemetrics-client/ +.. _latest version: https://github.com/clearlinux/telemetrics-client/tree/master/src +.. _heartbeat probe: https://github.com/clearlinux/telemetrics-client/tree/master/src/probes/hello.c diff --git a/_sources/guides/index.rst.txt b/_sources/guides/index.rst.txt new file mode 100644 index 00000000..53c3a583 --- /dev/null +++ b/_sources/guides/index.rst.txt @@ -0,0 +1,49 @@ +.. _guides: + +Guides +###### + +The following guides provide step-by-step instructions on using |CL|. + +.. note:: + + As of 22 May 2019 :file:`mixin` is no longer supported. + +.. _cl-guides: + +Clear Linux +=========== + +.. toctree:: + :maxdepth: 1 + :glob: + + clear/* + +Maintenance +=========== + +.. toctree:: + :maxdepth: 1 + :glob: + + maintenance/* + +Network +======= + +.. toctree:: + :maxdepth: 1 + :glob: + + network/* + +Kernel +======= + +.. toctree:: + :maxdepth: 1 + :glob: + + kernel/* + diff --git a/_sources/guides/kernel/change-kernel-boot.rst.txt b/_sources/guides/kernel/change-kernel-boot.rst.txt new file mode 100644 index 00000000..860464a9 --- /dev/null +++ b/_sources/guides/kernel/change-kernel-boot.rst.txt @@ -0,0 +1,234 @@ +.. _change-kernel-boot: + +Change Kernel Boot +######################## + +This tutorial explains the process of change kernel boot entry |CL-ATTR|. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +For this tutorial, you will modify your kernel list to boot with the kernel you want to use. This process is valid when you cannot compile third-party kernel modules and need to come back to old, or if you compile your custom kernel. + + +Get the current boot status +*************************** + +.. code-block:: bash + + bootctl status + +This is an example output: + +.. code-block:: bash + + System: + Firmware: UEFI 2.70 (HP 265.256) + Firmware Arch: x64 + Secure Boot: disabled + TPM2 Support: yes + Measured UKI: no + Boot into FW: supported + + Current Boot Loader: + Product: systemd-boot 255 + Features: ✓ Boot counting + ✓ Menu timeout control + ✓ One-shot menu timeout control + ✓ Default entry control + ✓ One-shot entry control + ✓ Support for XBOOTLDR partition + ✓ Support for passing random seed to OS + ✓ Load drop-in drivers + ✓ Support Type #1 sort-key field + ✓ Support @saved pseudo-entry + ✓ Support Type #1 devicetree field + ✓ Enroll SecureBoot keys + ✓ Retain SHIM protocols + ✓ Menu can be disabled + ✓ Boot loader sets ESP information + ESP: /dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5 + File: └─/EFI/org.clearlinux/loaderx64.efi + + Random Seed: + System Token: set + Exists: yes + + Available Boot Loaders on ESP: + ESP: /boot (/dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5) + File: ├─/EFI/systemd/systemd-bootx64.efi (systemd-boot 255) + └─/EFI/BOOT/BOOTX64.EFI (systemd-boot 255) + + Boot Loaders Listed in EFI Variables: + Title: Linux bootloader + ID: 0x0007 + Status: active, boot-order + Partition: /dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5 + File: └─/EFI/org.clearlinux/bootloaderx64.efi + + Title: Linux Boot Manager + ID: 0x0001 + Status: active, boot-order + Partition: /dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5 + File: └─/EFI/systemd/systemd-bootx64.efi + + Title: Windows Boot Manager + ID: 0x0000 + Status: active, boot-order + Partition: /dev/disk/by-partuuid/48d8a9eb-d84d-4a62-8302-edff383290e5 + File: └─/EFI/Microsoft/Boot/bootmgfw.efi + + Boot Loader Entries: + $BOOT: /boot (/dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5) + token: clear-linux-os + + Default Boot Loader Entry: + type: Boot Loader Specification Type #1 (.conf) + title: Clear Linux OS (Clear-linux-native-6.8.10-1434.conf) + id: Clear-linux-native-6.8.10-1434.conf + source: /boot//loader/entries/Clear-linux-native-6.8.10-1434.conf + linux: /boot//EFI/org.clearlinux/kernel-org.clearlinux.native.6.8.10-1434 + initrd: /boot//EFI/org.clearlinux/freestanding-00-early-ucode.cpio + /boot//EFI/org.clearlinux/initrd-org.clearlinux.native.6.8.10-1434 + /boot//EFI/org.clearlinux/freestanding-clr-init.cpio.gz + /boot//EFI/org.clearlinux/freestanding-i915-firmware.cpio + options: root=UUID=67e7ac9a-f7a1-4d5e-bbd6-012f5fa81cb5 rd.luks.uuid=abe6aaf2-3425-4eb1-b7f5-3f36746426fa quiet console=tty0 console=ttyS0,115200n8 cryptomgr.notests init=/usr/bin/initra-desktop initcall> + lines 20-71/71 (END) + ✓ Support @saved pseudo-entry + ✓ Support Type #1 devicetree field + ✓ Enroll SecureBoot keys + ✓ Retain SHIM protocols + ✓ Menu can be disabled + ✓ Boot loader sets ESP information + ESP: /dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5 + File: └─/EFI/org.clearlinux/loaderx64.efi + + Random Seed: + System Token: set + Exists: yes + + Available Boot Loaders on ESP: + ESP: /boot (/dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5) + File: ├─/EFI/systemd/systemd-bootx64.efi (systemd-boot 255) + └─/EFI/BOOT/BOOTX64.EFI (systemd-boot 255) + + Boot Loaders Listed in EFI Variables: + Title: Linux bootloader + ID: 0x0007 + Status: active, boot-order + Partition: /dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5 + File: └─/EFI/org.clearlinux/bootloaderx64.efi + + Title: Linux Boot Manager + ID: 0x0001 + Status: active, boot-order + Partition: /dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5 + File: └─/EFI/systemd/systemd-bootx64.efi + + Title: Windows Boot Manager + ID: 0x0000 + Status: active, boot-order + Partition: /dev/disk/by-partuuid/48d8a9eb-d84d-4a62-8302-edff383290e5 + File: └─/EFI/Microsoft/Boot/bootmgfw.efi + + Boot Loader Entries: + $BOOT: /boot (/dev/disk/by-partuuid/ea2e4278-5c0c-4498-bc99-dfc48a71ceb5) + token: clear-linux-os + + Default Boot Loader Entry: + type: Boot Loader Specification Type #1 (.conf) + title: Clear Linux OS (Clear-linux-native-6.8.10-1434.conf) + id: Clear-linux-native-6.8.10-1434.conf + source: /boot//loader/entries/Clear-linux-native-6.8.10-1434.conf + linux: /boot//EFI/org.clearlinux/kernel-org.clearlinux.native.6.8.10-1434 + initrd: /boot//EFI/org.clearlinux/freestanding-00-early-ucode.cpio + /boot//EFI/org.clearlinux/initrd-org.clearlinux.native.6.8.10-1434 + /boot//EFI/org.clearlinux/freestanding-clr-init.cpio.gz + /boot//EFI/org.clearlinux/freestanding-i915-firmware.cpio + options: root=UUID=67e7ac9a-f7a1-4d5e-bbd6-012f5fa81cb5 rd.luks.uuid=abe6aaf2-3425-4eb1-b7f5-3f36746426fa quiet console=tty0 console=ttyS0,115200n8 cryptomgr.notests init=/usr/bin/initra-desktop initcall_debug intel_iommu=igfx_off kvm-intel.nested=1 no_timer_check noreplace-smp page_alloc.shuffle=1 rcupdate.rcu_expedited=1 rootfstype=ext4,btrfs,xfs,f2fs tsc=reliable rw module.sig_unenforce rootflags=x-systemd.device-timeout=0 + +Get the kernel list installed +***************************** + +.. code-block:: bash + + bootctl list + +And example output: + +.. code-block:: bash + + type: Boot Loader Specification Type #1 (.conf) + title: Clear Linux OS (Clear-linux-preempt_rt-6.1.38-105.conf) + id: Clear-linux-preempt_rt-6.1.38-105.conf + source: /boot//loader/entries/Clear-linux-preempt_rt-6.1.38-105.conf + linux: /boot//EFI/org.clearlinux/kernel-org.clearlinux.preempt_rt.6.1.38-105 + initrd: /boot//EFI/org.clearlinux/freestanding-00-early-ucode.cpio + /boot//EFI/org.clearlinux/freestanding-clr-init.cpio.gz + /boot//EFI/org.clearlinux/freestanding-i915-firmware.cpio + options: root=UUID=67e7ac9a-f7a1-4d5e-bbd6-012f5fa81cb5 rd.luks.uuid=abe6aaf2-3425-4eb1-b7f5-3f36746426fa quiet console=tty0 console=ttyS0,115200n8 cryptomgr.notests init=/usr/bin/initra-desktop initcall_debug intel_iommu=igfx_off kvm-intel.nested=1 no_timer_check noreplace-smp page_alloc.shuffle=1 rcupdate.rcu_expedited=1 rootfstype=ext4,btrfs,xfs t> + + type: Boot Loader Specification Type #1 (.conf) + title: Clear Linux OS (Clear-linux-native-6.9.1-1436.conf) + id: Clear-linux-native-6.9.1-1436.conf + source: /boot//loader/entries/Clear-linux-native-6.9.1-1436.conf + linux: /boot//EFI/org.clearlinux/kernel-org.clearlinux.native.6.9.1-1436 + initrd: /boot//EFI/org.clearlinux/freestanding-00-early-ucode.cpio + /boot//EFI/org.clearlinux/initrd-org.clearlinux.native.6.9.1-1436 + /boot//EFI/org.clearlinux/freestanding-clr-init.cpio.gz + /boot//EFI/org.clearlinux/freestanding-i915-firmware.cpio + options: root=UUID=67e7ac9a-f7a1-4d5e-bbd6-012f5fa81cb5 rd.luks.uuid=abe6aaf2-3425-4eb1-b7f5-3f36746426fa quiet console=tty0 console=ttyS0,115200n8 cryptomgr.notests init=/usr/bin/initra-desktop initcall_debug intel_iommu=igfx_off kvm-intel.nested=1 no_timer_check noreplace-smp page_alloc.shuffle=1 rcupdate.rcu_expedited=1 rootfstype=ext4,btrfs,xfs,f> + + type: Boot Loader Specification Type #1 (.conf) + title: Clear Linux OS (Clear-linux-native-6.8.10-1434.conf) (default) (selected) + id: Clear-linux-native-6.8.10-1434.conf + source: /boot//loader/entries/Clear-linux-native-6.8.10-1434.conf + linux: /boot//EFI/org.clearlinux/kernel-org.clearlinux.native.6.8.10-1434 + initrd: /boot//EFI/org.clearlinux/freestanding-00-early-ucode.cpio + /boot//EFI/org.clearlinux/initrd-org.clearlinux.native.6.8.10-1434 + /boot//EFI/org.clearlinux/freestanding-clr-init.cpio.gz + /boot//EFI/org.clearlinux/freestanding-i915-firmware.cpio + options: root=UUID=67e7ac9a-f7a1-4d5e-bbd6-012f5fa81cb5 rd.luks.uuid=abe6aaf2-3425-4eb1-b7f5-3f36746426fa quiet console=tty0 console=ttyS0,115200n8 cryptomgr.notests init=/usr/bin/initra-desktop initcall_debug intel_iommu=igfx_off kvm-intel.nested=1 no_timer_check noreplace-smp page_alloc.shuffle=1 rcupdate.rcu_expedited=1 rootfstype=ext4,btrfs,xfs,f> + + type: Automatic + title: Reboot Into Firmware Interface + id: auto-reboot-to-firmware-setup + source: /sys/firmware/efi/efivars/LoaderEntries-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f + +Set default kernel to boot +************************** + +You can check the id from the latest command: + +.. code-block:: bash + + bootctl list |grep id: |cut -f 2 -d ":" + id: Clear-linux-preempt_rt-6.1.38-105.conf + id: Clear-linux-native-6.9.1-1436.conf + id: Clear-linux-native-6.8.10-1434.conf + id: auto-reboot-to-firmware-setup + + +Set the kernel + +.. code-block:: bash + + sudo bootctl set-default ID + +For example to set 6.9.1 entry: + +.. code-block:: bash + + sudo bootctl set-default Clear-linux-native-6.9.1-1436.conf + +Just reboot + +.. code-block:: bash + + sudo systemctl reboot + +You will boot with the kernel set before. diff --git a/_sources/guides/kernel/firmware.rst.txt b/_sources/guides/kernel/firmware.rst.txt new file mode 100644 index 00000000..7ae7a825 --- /dev/null +++ b/_sources/guides/kernel/firmware.rst.txt @@ -0,0 +1,107 @@ +.. _firmware: + +Firmware +######## + +This guide shows how |CL-ATTR| handles firmware and microcode loading. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +Many devices and system components require firmware or microcode, software +that runs directly on the device, to function correctly. Because firmware +loading requires privileged hardware access, the kernel is involved in the +process. + +Firmware does not typically come with source code. Instead, firmware is +provided as binary blobs which are licensed for free or non-free use. + +In |CL| firmware is loaded during device initialization which typically +happens at boot time. + +.. _firmware-included-begin: + +Included firmware +***************** + +The Linux kernel project contains a repository for firmware binaries that are +licensed to allow free redistribution. The Linux kernel's firmware repository +can be found here: +https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git + +If the firmware for your device is included upstream, no action is required +for it to be automatically detected and loaded. + +|CL| packages these firmware binaries +in the `linux-firmware bundles +`_ and +automatically includes them with the kernel. + +You can double-check the linux-firmware bundle is installed with the commands below: + + .. code-block:: bash + + sudo swupd bundle-add linux-firmware + find /lib/firmware/ + + + + +Additional firmware loading +*************************** + +Some device hardware manufacturers have a license that limits redistribution +of firmware. This means |CL| is unable to distribute those firmware and you +must manually obtain them from the manufacturer or another source. + +You can place additional firmware in :file:`/etc/firmware`. |CL| reads this +directory for additional firmware files in conjunction with the typical +:file:`/lib/firmware` path to provide a :ref:`stateless design `. + + +#. Create the :file:`/etc/firmware` directory + + .. code-block:: bash + + sudo mkdir -p /etc/firmware + +#. Obtain the additional firmware binary from a trusted source. + +#. Copy the firmware files including any subdirectories to + :file:`/etc/firmware`. It is important to place the firmware files in + expected path for proper loading. + + .. code-block:: bash + + sudo cp -Rv /. /etc/firmware + + +CPU microcode loading +********************* + +Microcode is low level code for processors loaded during the boot process that +contain stability and security updates. + +Microcode updates can be updated by motherboard firmware however this is not +always feasible or does not happen in a timely fashion. The `Linux microcode +loader`_ included in the Linux kernel allows for more flexibility and more +frequent updates. + +|CL| uses the *early loading* mechanism described in the `Linux microcode +loader`_ documented by which the CPU microcode is loaded as early as possible +in the boot process by using an initial RAM disk (initrd). + + +Troubleshooting +*************** + +Look at the output of :command:`sudo dmesg` to see device initialization and +expected firmware paths + + + +.. _`Linux microcode loader`: https://www.kernel.org/doc/Documentation/x86/microcode.txt diff --git a/_sources/guides/kernel/kernel-boot-msg.rst.txt b/_sources/guides/kernel/kernel-boot-msg.rst.txt new file mode 100644 index 00000000..061bba82 --- /dev/null +++ b/_sources/guides/kernel/kernel-boot-msg.rst.txt @@ -0,0 +1,69 @@ +.. _kernel-boot-msg: + +Capture Kernel Boot Messages in the Journal +########################################### + +By default |CL| does not capture kernel boot messages in the journal logs, +where they're reported as "Missed" messages. This design decision was made +to provide a faster boot performance. On the other hand, if you wish to +see the messages, follow this guide. + +Here's an example a journal log with "Missed" messages: + +.. code-block:: console + :linenos: + :emphasize-lines: 4 + + -- Reboot -- + Apr 10 19:55:43 kernel systemd-journald[300]: Journal started + Apr 10 19:55:43 kernel systemd-journald[300]: Runtime Journal (/run/log/journal/d01862ca79d1064ea379cd715cfdd53a) is 5.8M, max 47.0M, 41.1M free. + Apr 10 19:55:43 kernel systemd-journald[300]: Missed 2233 kernel messages + Apr 10 19:55:43 kernel systemd[1]: Started Journal Service. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* `systemd-journald` version 245 and higher + +Enable journaling of kernel boot messages +***************************************** + +#. Open a terminal window. + +#. Create a base journald configuration file. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/journald.conf.d + sudo cp /usr/lib/systemd/journald.conf.d/clear.conf /etc/systemd/journald.conf.d/ + +#. Append :command:`BootKMsg=true` to it. + + .. code-block:: bash + + echo "BootKMsg=true" | sudo tee -a /etc/systemd/journald.conf.d/clear.conf + +#. Reboot. + +.. tip:: + + If you need to increase the kernel buffer length (for example, 1M), do this: + + .. code-block:: bash + + sudo mkdir -p /etc/kernel/cmdline.d/ + echo "log_buf_len=1M" | sudo tee /etc/kernel/cmdline.d/log_buf_len.conf + sudo clr-boot-manager update + +Alternative +*********** + +An alternative is to use :command:`dmesg`. + +.. code-block:: bash + + sudo dmesg diff --git a/_sources/guides/kernel/kernel-development.rst.txt b/_sources/guides/kernel/kernel-development.rst.txt new file mode 100644 index 00000000..9049e031 --- /dev/null +++ b/_sources/guides/kernel/kernel-development.rst.txt @@ -0,0 +1,472 @@ +.. _kernel-development: + +Kernel development +################## + +This guide shows how to obtain and compile a Linux\* kernel source using +|CL-ATTR| development tooling. + +.. contents:: + :local: + :depth: 1 + :backlinks: top + +Overview +******** + +The :ref:`compatible-kernels` available in |CL| aim to be performant and +practical. In some cases, it may be necessary to modify the kernel to suit +your specific needs or test new kernel code as a developer. + +`Source RPMs (SRPMS)`_ are also available for all |CL| kernels, and can be +used for development instead. + +Request changes be included with the |CL| kernel +************************************************ + +If the kernel modification you need is already open source and likely to be +useful to others, consider submitting a request to include it in the +|CL| kernels. If your change request is accepted, you do not need to maintain +your own modified kernel. + +Make enhancement requests to the |CL| `Distribution Project`_ on GitHub\*. + +Set up kernel development environment +************************************* + +In some cases, it may be necessary to modify the kernel to suit your specific +needs or to test new kernel code. + +You can build and install a custom kernel; however you must: + +* Disable Secure Boot +* Maintain any updates to the kernel going forward + +To create a custom kernel, start with the |CL| development environment. +Then make changes to the kernel, build it, and install it. + + +Install the |CL| development tooling framework +============================================== + +.. include:: ../clear/autospec.rst + :start-after: install-tooling-after-header: + :end-before: install-tooling-end: + +Clone the kernel package +======================== + +Clone the existing kernel package repository from |CL| as a starting point. + +#. Clone the Linux kernel package from |CL|. Using the + :command:`make clone_` command in the + :file:`clearlinux/` directory clones the package from the + `clearlinux-pkgs`_ repo on GitHub. + + .. code-block:: bash + + cd ~/clearlinux + make clone_linux + +#. Navigate into the cloned package directory. + + .. code-block:: bash + + cd ~/clearlinux/packages/linux + + +The "linux" package is the kernel that comes with |CL| in the +:command:`kernel-native` bundle. Alternatively, you can use a different kernel +variant as the base for modification. For a list of kernel package names which +you can clone instead, see the `clearlinux-pkgs`_ repo on GitHub. + +.. note:: + + The latest version of the |CL| kernel package is pulled as a starting + point. An older version can pulled by switching to different git tag by using + :command:`git checkout tag/`. + +Change the kernel version +========================= + +|CL| tends to use the latest kernel available from `kernel.org`_, the Linux +upstream. The kernel version that will be built can be changed in the +RPM SPEC file. While most packages in Clear Linux are typically packaged +using :ref:`autospec`, the kernel is not. This means control files +provided by autospec are not available and changes must be made manually. + +#. Open the Linux kernel package RPM SPEC file in an editor. + + .. code-block:: bash + + $EDITOR linux.spec + +#. Modify the Version, Release, and Source0 URL entries at the top of the + file to change the version of Linux kernel that will be compiled. + + A list of current and available kernel release can be found on + `kernel.org`_. + + .. code-block:: spec + :linenos: + :emphasize-lines: 1-3,12 + + Name: linux + Version: 4.20.8 + Release: 696 + License: GPL-2.0 + Summary: The Linux kernel + Url: http://www.kernel.org/ + Group: kernel + Source0: https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.20.8.tar.xz + Source1: config + Source2: cmdline + + %define ktarget native + + .. note:: + - Consider changing the Name from *linux* in the RPM spec file to easily + identify a modified kernel. + + - Consider changing the ktarget from *native* in the RPM spec file to + easily identify a modified kernel. + +#. Commit and save the changes to the file. + +.. _pull-copy-kernel-source: + +Pull a copy of the Linux kernel source code +=========================================== + +Obtain a local copy of the source code to make modifications against. + +#. Run make sources to pull the kernel source code specified in the RPM + SPEC file. In the example, it downloads the :file:`linux-4.20.8.tar.xz` + file. + + .. code-block:: bash + + make sources + + +#. Extract the kernel source code archive. This will create a working copy + of the Linux source that you can modify. + + .. code-block:: bash + + tar -xvf linux-4.20.8.tar.xz + +#. Navigate to the extracted directory. In this example, it has been + extracted into a :file:`linux-4.20.8` directory. + + .. code-block:: bash + + cd linux-4.20.8/ + + +Customize the Linux kernel source +********************************* + +After the kernel sources have been obtained, customizations to the kernel +configuration or source code can be made for inclusion with the kernel +build. These customizations are optional. + +Modify kernel configuration +=========================== + +The kernel source has many configuration options available to pick support for +different hardware and software features. + +These configuration values must be provided in the :file:`.config` file at +compile time. You will need to make modifications to the :file:`.config` +file, and include it in the kernel package. + + +#. Make sure you have followed the steps to :ref:`pull-copy-kernel-source` + and are in the kernel source working directory. + + +#. If you have an existing :file:`.config` file from an old kernel, copy it + into the working directory as :file:`.config` for comparison. + Otherwise, use the |CL| kernel configuration file as template + + .. code-block:: bash + + cp ~/clearlinux/packages/linux/config .config + + +#. Make any desired changes to the :file:`.config` using a kernel + configuration tool. Below are some popular options: + + - :command:`$EDITOR .config` - the .config file can be directly edited + for simple changes with names that are already known. + + - :command:`make config` - a text-based tool that asks questions + one-by-one to decide configuration options. + + - :command:`make menuconfig` - a terminal user interface that provides + menus to decide configuration options. + + - :command:`make xconfig` - a graphical user interface that provides + tree views to decide configuration options. + + + More configuration tools can be found by looking at the make help: + :command:`make help | grep config` + +#. Commit and save the changes to the :file:`.config` file. + +#. Copy the :file:`.config` file from the kernel source directory into + the kernel package directory as :file:`config` for inclusion in the build. + + .. code-block:: bash + + cp .config ../config + +Modify kernel source code +========================= + +Changes to kernel code are applied with patch files. Patch files are +formatted git commits that can be applied to the main source code. + +You will need to obtain a copy of the source code, +make modifications, generate patch file(s), and add them to the RPM SPEC +file for inclusion during the kernel build. + +If you have a large number of patches or a more complex workflow, +consider using a patch management tool in addition to Git such as +`Quilt`_. + + +#. Make sure you have followed the steps to :ref:`pull-copy-kernel-source` and + are in the kernel source working directory. + + +#. Initialize the kernel source directory as a new git repo and create a + commit with all the existing source files to begin tracking changes. + + .. code-block:: bash + + git init + git add -A + git commit -m "Initial commit of Linux kernel source" + + +#. Apply patches provided by the |CL| kernel package to the kernel source + in the working directory. + + .. code-block:: bash + + git am ../*.patch + + +#. Make any of your desired code changes to the Linux source code files. + + +#. Track and commit your changes to the local git repo. + + .. code-block:: bash + + git add + git commit -m "My patch for driver A" + + +#. Generate a patch file based on your git commits. + represents the number of local commits to create patch file. + See the `git-format-patch`_ documentation for detailed information + on using :command:`git format-patch` + + .. code-block:: bash + + git format-patch - + +#. Copy the patch files from the patches directory in the linux + source tree to the RPM build directory. + + .. code-block:: bash + + cp *.patch ~/clearlinux/packages/linux/ + + +#. Navigate back to the RPM build directory. + + .. code-block:: bash + + cd ~/clearlinux/packages/linux/ + +#. Open the Linux kernel package RPM SPEC file in an editor. + + .. code-block:: bash + + $EDITOR linux.spec + +#. Locate the section of the SPEC file that contains existing patch + variable definitions and append your patch file name. Ensure the + patch number does not collide with an existing patch. + In this example, the patch file is called + :file:`2001-my-patch-for-driver-A.patch` + + .. code-block:: spec + :linenos: + :emphasize-lines: 13 + + # + # Small Clear Linux Tweaks + # + Patch0501: 0501-zero-extra-registers.patch + Patch0502: 0502-locking-rwsem-spin-faster.patch + + #Serie1.name WireGuard + #Serie1.git https://git.zx2c4.com/WireGuard + #Serie1.tag 00bf4f8c8c0ec006633a48fd9ee746b30bb9df17 + Patch1001: 1001-WireGuard-fast-modern-secure-kernel-VPN-tunnel.patch + #Serie1.end + + Patch2001: 2001-my-patch-for-driver-A.patch + + +#. Locate the section of the SPEC file further down that contains + patch application and append your patch file number used in the step above. + In this example, patch2001 is added. + + .. code-block:: spec + :linenos: + :emphasize-lines: 11 + + # + # Small tweaks + # + %patch0501 -p1 + %patch0502 -p1 + + #Serie1.patch.start + %patch1001 -p1 + #Serie1.patch.end + + %patch2001 -p1 + + +#. Commit and save the changes to the RPM SPEC file. + +Modify kernel boot parameters +============================= +The kernel boot options are passed from the bootloader to the kernel with +command-line parameters. + +While temporary changes can be made to kernel parameters on a running +system or on a during boot, you can also modify the default parameters that +are persistent and distributed with a customized kernel. + + +#. Open the kernel :file:`cmdline` file in an editor. + + .. code-block:: bash + + $EDITOR cmdline + + +#. Make any desired change to the kernel parameters. + For example, you can remove the :command:`quiet` parameter to see more + verbose output of kernel log messages during the boot process. + +#. Commit and save the changes to the :file:`cmdline` file. + +See the `kernel parameters`_ documentation for a list of available +parameters. + +Build and install the kernel +**************************** +After changes have been made to the kernel source and RPM SPEC file, +the kernel is ready to be compiled and packaged into an RPM. + +The |CL| development tooling makes use of :command:`mock` environments to +isolate building of packages in a sanitized workspace. + +#. Start the compilation process by issuing the :command:`make build` + command. This process is typically resource intensive and will take a + while. + + .. code-block:: bash + + make build + + .. note:: + The mock plugin `ccache`_ can be enabled to help speed up any future + rebuilds of the kernel package by caching compiler outputs and reusing + them. + + +#. The result will be multiple :file:`.rpm` files in the :file:`rpms` + directory as output. + + .. code-block:: bash + + ls rpms/ + + The kernel RPM will be named + :file:`linux--.x86_64.rpm` + + +#. The kernel RPM file can be input to the :ref:`mixer` to create a + custom bundle and mix of |CL|. + +Alternatively, the kernel RPM bundle can be installed manually on a local +machine for testing. This approach works well for individual development or +testing. For a more scalable and customizable approach, consider using the +:ref:`mixer` to provide a custom kernel with updates. + +#. Install the kernel onto the local system by extracting the RPM with the + :command:`rpm2cpio` command. + + .. code-block:: bash + + rpm2cpio linux--.x86_64.rpm | (cd /; sudo cpio -i -d -u -v); + +#. Optionally, increase the bootloader timeout to make interrupting the boot + process and choosing a different kernel easier.This can be helpful to if + you encounter a kernel that does not boot gracefully. + + .. code-block:: bash + + sudo clr-boot-manager set-timeout 20 + + +#. Update the |CL| boot manager to use the new kernel using + :command:`clr-boot-manager` and reboot. + + .. code-block:: bash + + sudo clr-boot-manager update + sudo clr-boot-manager list-kernels + sudo clr-boot-manager set-kernel org.clearlinux..- + + sudo reboot + + +#. After a reboot, verify the customized kernel is running. + + .. code-block:: bash + + uname -a + +Related topics +************** + +* :ref:`kernel-modules` +* :ref:`mixer` + +.. _Distribution Project: https://github.com/clearlinux/distribution/issues/new/choose + +.. _Source RPMs (SRPMS): https://cdn.download.clearlinux.org/current/source/SRPMS/ + +.. _Quilt: http://savannah.nongnu.org/projects/quilt + +.. _clearlinux-pkgs: https://github.com/clearlinux-pkgs + +.. _kernel.org: https://www.kernel.org/ + +.. _kernel parameters: https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt + +.. _ccache: https://fedoraproject.org/wiki/Mock/Plugin/CCache?rd=Subprojects/Mock/Plugin/CCache + +.. _git-format-patch: https://git-scm.com/docs/git-format-patch diff --git a/_sources/guides/kernel/kernel-modules-dkms.rst.txt b/_sources/guides/kernel/kernel-modules-dkms.rst.txt new file mode 100644 index 00000000..3d7ea95d --- /dev/null +++ b/_sources/guides/kernel/kernel-modules-dkms.rst.txt @@ -0,0 +1,316 @@ +.. _kernel-modules-dkms: + +Add kernel modules with DKMS +############################ + +This guide describes how to add kernel modules with +:abbr:`DKMS (Dynamic Kernel Module System)`. + +.. contents:: :local: + :depth: 1 + :backlinks: top + +Overview +******** + +Certain kernel modules are enabled by default in |CL-ATTR|. To use additional +kernel modules that are not part of the Linux source tree, you may need to +build out-of-tree kernel modules. Use this guide to add kernel modules with +DKMS or refer to :ref:`kernel-modules`. + +Description +*********** + +Kernel modules are additional pieces of software capable of being inserted +into the Linux kernel to add functionality, such as a hardware driver. +Kernel modules may already be part of the Linux source tree (in-tree) or may +come from an external source, such as directly from a vendor (out-of-tree). + +DKMS is a framework that facilitates the building and installation of kernel +modules. DKMS allows |CL| to provide hooks that automatically rebuild modules +against new kernel versions. + +.. include:: kernel-modules.rst + :start-after: kernel-modules-availability-begin: + :end-before: kernel-modules-availability-end: + +Install DKMS +************ + +.. _kernel-modules-dkms-install-begin: + +The :command:`kernel-native-dkms` bundle provides the DKMS program and Linux +kernel headers, which are placed under :file:`/usr/lib/modules/$(uname +-r)/build/include/` and are required to compile kernel modules. + +The :command:`kernel-native-dkms` bundle also: + +* Adds a `systemd` update trigger + (:file:`/usr/lib/systemd/system/dkms-new-kernel.service`) to automatically + run DKMS to rebuild modules after a kernel upgrade occurs with :ref:`swupd + update `. + +* Disables kernel module signature verification by appending a kernel + command-line parameter (:command:`module.sig_unenforce`) from the + :file:`/usr/share/kernel/cmdline.d/clr-ignore-mod-sig.conf` file. + +* Adds a notification to the Message of the Day (MOTD) indicating kernel + module signature verification is disabled. + +.. warning:: + + We recommend that you always review the :command:`swupd update` output + to make sure kernel modules were successfully rebuilt against the new + kernel. This is especially important for systems where a successful boot + relies on a kernel module. + +.. _kernel-modules-dkms-install-begin-alt: + +Install the :command:`kernel-native-dkms` or :command:`kernel-lts-dkms` +bundle. + +#. Determine which kernel variant is running on |CL|. Only the *native* + and *lts* kernels are enabled to build and load out-of-tree kernel modules + with DKMS. + + .. code-block:: console + + $ uname -r + 5.XX.YY-ZZZZ.native + + Ensure *.native* or *.lts* is in the kernel name. + +#. Install the DKMS bundle corresponding to the installed kernel. Use + :command:`kernel-native-dkms` for the native kernel or + :command:`kernel-lts-dkms` for the lts kernel. + + .. code-block:: bash + + sudo swupd bundle-add kernel-native-dkms + + or + + .. code-block:: bash + + sudo swupd bundle-add kernel-lts-dkms + + +#. Update the |CL| bootloader and reboot, and + ensure that you can start the new kernel. + + .. code-block:: bash + + sudo clr-boot-manager update + reboot + +.. _kernel-modules-dkms-install-end: + +Build, install, and load an out-of-tree module +********************************************** + +Follow the steps in this section if you are an individual user or testing, +and you need an out-of-tree kernel module that is not available through +|CL|. For a more scalable and customizable approach, we recommend using +:ref:`mixer` to provide a custom kernel and updates. + +Prerequisites +============= + +Before you begin, you must: + +* Disable Secure Boot in UEFI/BIOS. The loading of new out-of-tree modules + modifies the signatures that Secure Boot relies on for trust. + +* Obtain a kernel module package in the form of source code or + pre-compiled binaries. + +Obtain kernel module source +=========================== + +A required :file:`dkms.conf` file inside of the kernel module's source code +directory informs DKMS how the kernel module should be compiled. + +Kernel modules may come packaged as: + +- Source code without a :file:`dkms.conf` file +- Source code with a premade :file:`dkms.conf` file +- Source code with a premade :file:`dkms.conf` file and precompiled module + binaries +- Precompiled module binaries only (without source code) + +Of the package types listed above, only precompiled kernel module binaries +will not work, because |CL| requires kernel modules to be built against the +same kernel source tree before they can be loaded. If you are only able to +obtain source code without a :file:`dkms.conf` file, you must manually create +a :file:`dkms.conf` file, described later in this document. + +#. Download the kernel module's source code. + + * Review the available download options. Some kernel modules provide + separate archives that are specifically enabled for DKMS support. + + * Review the README documentation, because it often provides required + information to build the module with DKMS support. + + .. code-block:: bash + + curl -O http://.tar.gz + tar -xvf .tar.gz + cd / + cat README + +Build kernel module with an existing dkms.conf +============================================== + +If the kernel module maintainer packaged the source archive with the +:command:`dkms mktarball` command, the entire archive can be passed to the +:command:`dkms ldtarball` which completes many steps for you. + +The archive contains the required :file:`dkms.conf` file, and may contain +a :file:`dkms_source_tree` directory and a :file:`dkms_binaries_only` +directory. + +#. Run the :command:`dkms ldtarball` command against the kernel + module archive. + + .. code-block:: bash + + dkms ldtarball .tar.gz + + + :command:`dkms ldtarball` places the kernel module source under + :file:`/usr/src/-/`, builds it if necessary, + and adds the module into the DKMS tree. + + +#. Verify the kernel module is detected by checking the output of the + :command:`dkms status` command. + + .. code-block:: bash + + dkms status + + +#. Install the kernel module. + + .. code-block:: bash + + dkms install -m -v + +Build kernel module without an existing dkms.conf +================================================= + +If the kernel module source does not contain a :file:`dkms.conf` file or the +:command:`dkms ldtarball` command encounters errors, you must manually +create the file. + +Review the kernel module README documentation for guidance on what needs to be +in the :file:`dkms.conf` file, including special variables that may be +required to build successfully. + +Here are some additional resources that can be used for reference: + +* DKMS manual page (:command:`man dkms`) shows detailed syntax in the + DKMS.CONF section. + +* Ubuntu community wiki entry for the `Kernel DKMS Package`_ shows an example + where a single package contains multiple modules. + +* Sample `dkms.conf`_ file in the GitHub\* repository for the DKMS project. + +.. note:: + + :command:`AUTOINSTALL=yes` must be set in the dkms.conf for the module to + be automatically recompiled with |CL| updates. + +The instructions below show a generic example: + +#. Create or modify the :file:`dkms.conf` file inside of the extracted source + code directory. + + .. code-block:: ShellSession + + $ EDITOR dkms.conf + + MAKE="make -C src/ KERNELDIR=/lib/modules/${kernelver}/build" + CLEAN="make -C src/ clean" + BUILT_MODULE_NAME=custom_module + BUILT_MODULE_LOCATION=src/ + PACKAGE_NAME=custom_module + PACKAGE_VERSION=1.0 + DEST_MODULE_LOCATION=/kernel/drivers/other + AUTOINSTALL=yes + + This example identifies a kernel module named *custom_module* with version + *1.0*. + +#. Copy the kernel module source code into the :file:`/usr/src/` directory. + + .. code-block:: bash + + sudo mkdir /usr/src/- + sudo cp -Rv . /usr/src/- + + .. note:: + + ** and ** must match the entries in the + :file:`dkms.conf` file. + +#. Add the kernel module to the DKMS tree so that it is tracked by DKMS. + + .. code-block:: bash + + sudo dkms add -m + +#. Build the kernel module using DKMS. If the build encounters errors, + you may need to edit the :file:`dkms.conf` file. + + .. code-block:: bash + + sudo dkms build -m -v + +#. Install the kernel module using DKMS. + + .. code-block:: bash + + sudo dkms install -m -v + +Load kernel module +================== + +By default, DKMS installs modules "in-tree" under :file:`/lib/modules` so the +:command:`modprobe` command can be used to load them. + +#. Load the installed module with the :command:`modprobe` command. + + .. code-block:: bash + + sudo modprobe + +#. Validate the kernel module is loaded. + + .. code-block:: bash + + lsmod | grep + +Examples +******** + +.. include:: kernel-modules.rst + :start-after: kernel-modules-autoload-begin: + :end-before: kernel-modules-autoload-end: + +Related topics +************** + +* `Dynamic Kernel Module System (DKMS)`_ + +* `Dell Linux Engineering Dynamic Kernel Module Support: From Theory to Practice `_ + +* `Linux Journal: Exploring Dynamic Kernel Module Support `_ + +.. _Dynamic Kernel Module System (DKMS): https://github.com/dell/dkms + +.. _Kernel DKMS Package: https://help.ubuntu.com/community/Kernel/DkmsDriverPackage#Configure_DKMS + +.. _dkms.conf: https://github.com/dell/dkms/blob/master/sample.conf diff --git a/_sources/guides/kernel/kernel-modules.rst.txt b/_sources/guides/kernel/kernel-modules.rst.txt new file mode 100644 index 00000000..992767f9 --- /dev/null +++ b/_sources/guides/kernel/kernel-modules.rst.txt @@ -0,0 +1,228 @@ +.. _kernel-modules: + +Add kernel modules manually +########################### + +This guide describes how to add kernel modules manually. + +.. contents:: :local: + :depth: 1 + :backlinks: top + +Overview +******** + +Certain kernel modules are enabled by default in |CL-ATTR|. To use additional +kernel modules that are not part of the Linux source tree, you may need to +build out-of-tree kernel modules. Use this guide to add kernel modules +manually, or refer to :ref:`kernel-modules-dkms`. + +Description +*********** + +Kernel modules are additional pieces of software capable of being inserted +into the Linux kernel to add functionality, such as a hardware driver. +Kernel modules may already be part of the Linux source tree (in-tree) or may +come from an external source, such as directly from a vendor (out-of-tree). + + +.. _kernel-modules-availability-begin: + +Kernel module availability +************************** + +|CL| comes with many upstream kernel modules available for use. Using an +existing module is significantly easier to maintain and retains signature +verification of the |CL| kernel. For more information on |CL| security +practices, see the :ref:`security` page. + +Before continuing, check if the kernel module you're looking for is already +available in |CL| or submit a request to add the module. + + +Check if the module is already available +======================================== + + +You can search for kernel module file names, which end with the :file:`.ko` +file extension, using the :command:`swupd search` command, as shown in the +following example. See :ref:`swupd-guide` for more information. + +.. code-block:: bash + + sudo swupd search ${module_name}.ko + + +Submit a request to add the module +================================== + +If the kernel module you need is already open source (for example, in the +Linux kernel upstream) and likely to be useful to others, consider submitting +a request to add or enable it in the |CL| kernel. + +Make enhancement requests to the |CL| `Distribution Project on GitHub +`_. + +.. _kernel-modules-availability-end: + + +Build, install, and load an out-of-tree module +********************************************** + +Follow the steps in this section if you are an individual user or testing, and +you need an out-of-tree kernel module that is not available through |CL|. For +a more scalable and customizable approach, we recommend using the +:ref:`mixer` to provide a custom kernel and updates. + + +Prerequisites +============= + +Before you begin, you must: + +* Disable Secure Boot. +* Disable kernel module integrity checking. +* Have a kernel module package in the form of source code. +* Rebuild the module against new versions of the Linux kernel. + +.. note:: + + Any time the kernel is upgraded on your Clear Linux system, you must + rebuild your out-of-tree modules. + + +Build and install kernel module +=============================== + +#. Determine which kernel variant is running on |CL|. In the example below, + the *native* kernel is in use. + + .. code-block:: bash + + $ uname -r + 5.XX.YY-ZZZZ.native + +#. Install the kernel dev bundle corresponding to the installed kernel. The + kernel dev bundle contains the kernel headers, which are placed under + :file:`/usr/lib/modules/$(uname -r)/build/include/` and are required to + compile kernel modules. For example: + + * :command:`linux-dev` for developing against the native kernel. + * :command:`linux-lts-dev` for developing against the LTS kernel. + + .. code-block:: bash + + sudo swupd bundle-add linux-dev + + + +#. Follow instructions from the kernel module source code to compile the + kernel module. For example: + + .. code-block:: bash + + curl -O http://.tar.gz + tar -xvf .tar.gz + cd / + cat README + + + +Load kernel module +================== + +#. Disable Secure Boot in your system's UEFI settings, if you have enabled + it. The loading of new out-of-tree modules modifies the signatures that + Secure Boot relies on for trust. + +#. Disable signature checking for the kernel by modifying the kernel boot + parameters and reboot the system. + + All kernel modules from |CL| have been signed to enforce kernel security. + However, out-of-tree modules break this chain of trust so this mechanism + needs to be disabled. + + .. code-block:: bash + + sudo mkdir -p /etc/kernel/cmdline.d + echo "module.sig_unenforce" | sudo tee /etc/kernel/cmdline.d/allow-unsigned-modules.conf + +#. Update the boot manager and reboot the system to implement the changed + kernel parameters. + + .. code-block:: bash + + sudo clr-boot-manager update + sudo reboot + + .. note:: + + If successful, the :command:`clr-boot-manager update` command does not + return any console output. + +#. After rebooting, manually load out-of-tree modules using the + :command:`insmod` command. + + .. code-block:: bash + + sudo insmod + +Examples +******** + +.. _kernel-modules-autoload-begin: + +Optional: Specify module options and aliases +============================================ + +Use the :command:`modprobe` command to load a module and set options. + +:command:`modprobe` may add or remove more than one module due to module +interdependencies. You can specify which options to use with individual +modules, by using configuration files under the :file:`/etc/modprobe.d` +directory. + +.. code-block:: bash + + sudo mkdir /etc/modprobe.d + +All files underneath the :file:`/etc/modprobe.d` directory that end with the +:file:`.conf` extension specify module options to use when loading. You can +use :file:`.conf` files to create convenient aliases for modules or to +override the normal loading behavior altogether for those with special +requirements. + +Learn more about :command:`modprobe` on the modprobe.d manual page: + +.. code-block:: bash + + man modprobe.d + +Optional: Configure kernel modules to load at boot +================================================== + +Use the :file:`/etc/modules-load.d` configuration directory to specify kernel +modules to load automatically at boot. + +.. code-block:: bash + + sudo mkdir /etc/modules-load.d + +All files underneath the :file:`/etc/modules-load.d` directory that end with +the :file:`.conf` extension contain a list of module names of aliases (one per +line) to load at boot. + +Learn more about module loading in the modules-load.d manual page: + +.. code-block:: bash + + man modules-load.d + + +.. _kernel-modules-autoload-end: + +Related topic +************* + +* :ref:`kernel-modules-dkms` + diff --git a/_sources/guides/maintenance/architect-lifecycle.rst.txt b/_sources/guides/maintenance/architect-lifecycle.rst.txt new file mode 100644 index 00000000..3f3b9e69 --- /dev/null +++ b/_sources/guides/maintenance/architect-lifecycle.rst.txt @@ -0,0 +1,78 @@ +.. _architect-lifecycle: + +Architect the life-cycle of |CL-ATTR| +##################################### + +This guide describes the basic, recommended infrastructure and workflow for +maintaining a |CL-ATTR| derivative. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* A repository with software RPM artifacts and a CI/CD system with a |CL| + machine for building `mixes` + +* Experience using :ref:`mixer ` to create a |CL|-based distro + +* Experience using :ref:`swupd ` for maintaining the |CL| + build environment + +* Familiarity with |CL| architecture and reuse of its content in releases + +Description +*********** + +Maintaining a |CL| derivative requires: + +* Monitoring upstream |CL| for new releases +* Building software packages and staging +* Employing CI/CD automation for building releases +* Integrating Quality Assurance for testing and validation + +Coordinated infrastructure is deployed to automate the life-cycle +of your |CL| derivative. We divide deployment of this infrastructure in two +parts: *Content Workflow*; and *Release Workflow*, shown in Figure 1. + +.. figure:: figures/architect-lifecycle-1.png + :scale: 100% + :alt: Architect the life-cycle + + Figure 1: Architect the life-cycle + +Content Workflow +**************** + +The Content Workflow (Figure 1) orchestrates the processes used to manage +the creation of content for the distribution. This includes everything from +detecting a new release in a custom software repository to generating RPM +package files. The RPM files serve as intermediary artifacts that track software +dependencies and provide file-level data consumed in a Release Workflow. The +`Watcher Pipeline`_ checks |CL| and a content provider, such as Koji, to +determine if a new release is necessary. + +Release Workflow +**************** + +The Release Workflow (Figure 1) gathers the content of the RPMs and +ensures it can be consumed by :ref:`mixer `. A content web server +hosts the |CL| derivative, to which targets connect for updating their OSes. +As an integral part of this toolchain, the `Release Pipeline`_ enables these +derivatives to incorporate |CL| content into their own custom +content. The Watcher Pipeline triggers the Release Pipeline to create +new releases. + +Implementation +************** + +The |CL| Distro Factory manages the Release Workflow. For detailed information +about Distro Factory deployment, refer to the `clr-distro-factory`_ GitHub\* repo. + +.. _clr-distro-factory: https://github.com/clearlinux/clr-distro-factory/wiki#clear-linux-distro-factory + +.. _Release Pipeline: https://github.com/clearlinux/clr-distro-factory/wiki/Release + +.. _Watcher Pipeline: https://github.com/clearlinux/clr-distro-factory/wiki/Watcher diff --git a/_sources/guides/maintenance/configure-hugepages.rst.txt b/_sources/guides/maintenance/configure-hugepages.rst.txt new file mode 100644 index 00000000..4925ba20 --- /dev/null +++ b/_sources/guides/maintenance/configure-hugepages.rst.txt @@ -0,0 +1,74 @@ +.. _enable-huge-pages: + +Configure Huge Pages +######################## + +In |CL| hugepages are enabled by default. The default hugepage size is 2MB. +The total number of hugepages is set to 0 by default. This guide shows you how +to add hugepages to the system and how to change the default hugepage size. + +#. To check the enabled state, run the following command. + + .. code-block:: bash + + cat /sys/kernel/mm/transparent_hugepage/enabled + + The output will look like + + .. code-block:: console + + [always] madvise never + + .. note:: + + The active option is enclosed in brackets. In this case, always is active, + which means hugepages are enabled for every process. The `madvise` + option means that hugepages are enabled for processes that explicitly + call `madvise`_. + +#. To check the size of hugepages, run the below command. + + .. code-block:: bash + + cat /proc/meminfo | grep Huge + + The output should look similar to the following. Although hugepages is + enabled, there are no hugepages available to allocate. + + .. code-block:: console + + AnonHugePages: 624640 kB + ShmemHugePages: 0 kB + FileHugePages: 0 kB + HugePages_Total: 0 + HugePages_Free: 0 + HugePages_Rsvd: 0 + HugePages_Surp: 0 + Hugepagesize: 2048 kB + Hugetlb: 0 kB + +#. If you need 1GB (1,048,576 bytes) of hugepages enabled, then `HugePages_Total` + should be set to 512. Enable it temporarily using the following. + + .. code-block:: bash + + echo 512 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + + To view the change, run the command in the previous setp again. + +#. Changing hugepages from the default 2MB to 1GB must be done at system boot + through kernel boot parameters. In this example, we configure the size and + number of allocatable huge pages at boot. + + .. code-block:: bash + + sudo mkdir -p /etc/kernel/cmdline.d + cat << EOF | sudo tee -a /etc/kernel/cmdline.d/hugepages.conf + default_hugepagesz=1G + hugepagesz=1G + hugepages=10 + EOF + sudo clr-boot-manager update + sudo reboot + + .. _madvise: https://linux.die.net/man/2/madvise diff --git a/_sources/guides/maintenance/container-image-modify.rst.txt b/_sources/guides/maintenance/container-image-modify.rst.txt new file mode 100644 index 00000000..8aed8d9c --- /dev/null +++ b/_sources/guides/maintenance/container-image-modify.rst.txt @@ -0,0 +1,469 @@ +.. _container-image-modify: + +Modify a |CL|-based container image +################################### + +This guide describes how to customize |CL-ATTR|-based container +`images on Docker Hub`_, which include popular applications and runtimes. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +Most of these images utilize a Docker build feature called a `multi-stage +build to reduce image size`_ while some use single-stage build Dockerfiles. An +official base `clearlinux image on Docker Hub`_ is also available. To create a +generic |CL| container image, see :ref:`our guide `. + +Prerequisites +************* + +* Set up a functional Docker environment as described in :ref:`docker`. + +* Download the |CL| microservice Dockerfile repo with the following + command: + + .. code-block:: bash + + git clone https://github.com/clearlinux/dockerfiles.git + +* Navigate to and operate from the cloned :file:`dockerfiles` directory. + + .. code-block:: bash + + cd dockerfiles/ + + +Example 1: Add a bundle +*********************** + +In this example, we add :command:`wget` to the **clearlinux/redis** +Dockerfile. + +#. Enter :command:`swupd search wget` to discover which |CL| bundle includes + the software. The output should tell you that :command:`wget` is available + in the *wget* bundle. + +#. Open a an editor to modify the Dockerfile. + + .. code-block:: bash + + $EDITOR redis/Dockerfile + +#. Append the :command:`wget` bundle to the :command:`--bundles=` parameter + of the :command:`swupd os-install` command. + +#. Run :command:`git diff`. + + The output shows the edits made after adding :command:`wget` in the + clearlinux/redis Dockerfile. + + .. code-block:: diff + + diff --git a/redis/Dockerfile b/redis/Dockerfile + index af977cb..b1effab 100644 + --- a/redis/Dockerfile + +++ b/redis/Dockerfile + @@ -15,7 +15,7 @@ RUN source /os-release && \ + mkdir /install_root \ + && swupd os-install -V ${VERSION_ID} \ + --path /install_root --statedir /swupd-state \ + - --bundles=redis-native,findutils,su-exec --no-boot-update + + --bundles=redis-native,findutils,su-exec,wget --no-boot-update + +#. Build the Dockerfile and apply a unique tag name. In this this example, + we use :command:`wget_added` and add proxies. + + .. code-block:: bash + + docker build \ + --no-cache \ + --build-arg http_proxy=$http_proxy \ + --build-arg https_proxy=$https_proxy \ + --tag clearlinux/redis:wget_added \ + redis/ + +#. Run the Dockerfile with the `wget --version` command to verify that + :command:`wget` has been added to the image. + + .. code-block:: bash + + docker run clearlinux/redis:wget_added wget --version + +#. The output shows: + + .. code-block:: console + + GNU Wget 1.20.3 built on linux-gnu. + + -cares +digest -gpgme +https +ipv6 -iri +large-file -metalink +nls + -ntlm +opie -psl +ssl/openssl + +Example 2: Change |CL| version (single-stage build) +*************************************************** + +This example shows how to rebuild single-stage containers against a specific +OS version, :file:``, by adding a new argument to the Docker build +command line. + +#. Rebuild the :file:`clearlinux/machine-learning-ui`. Add an extra build + argument :command:`swupd_args="-m "`; in this case, the build + version is 31110. + + .. code-block:: bash + :linenos: + :emphasize-lines: 5 + + docker build \ + --no-cache \ + --build-arg http_proxy=$http_proxy \ + --build-arg https_proxy=$https_proxy \ + --build-arg swupd_args="-m 31110" \ + --tag clearlinux/machine-learning-ui:31110 \ + machine-learning-ui/ + +#. Run the docker container image: + + .. code-block:: bash + + docker run clearlinux/machine-learning-ui:31110 swupd info + +#. Sample output shows: + + .. code-block:: console + + Distribution: Clear Linux OS + Installed version: 31110 + Version URL: https://cdn.download.clearlinux.org/update + Content URL: https://cdn.download.clearlinux.org/update + + +Example 3: Change |CL| version (multi-stage build) +************************************************** + +This example shows how to rebuild the cgit Dockerfile to use a specific |CL| +version. The clearlinux/cgit Dockerfile has a multi-stage build with multiple +layers: *os-core*, *httpd*, and *cgit*. This can be used as reference for +building other multi-stage images with any number of layers. + + +.. important:: + + All upper layers of multi-stage Dockerfiles inherit the |CL| version from + the base layer. Rebuild the all underlying base layers against the desired + OS version. In this example, four base layers must be rebuilt. + + +First layer: os-core +-------------------- + +#. Rebuild the first layer, *os-core*. Add an extra build argument + :command:`swupd_args="-m "`; in this case, the build + version is 31110. + + .. code-block:: bash + :linenos: + :emphasize-lines: 5 + + docker build \ + --no-cache \ + --build-arg http_proxy=$http_proxy \ + --build-arg https_proxy=$https_proxy \ + --build-arg swupd_args="-m 31110" \ + --tag clearlinux/os-core:31110 \ + os-core/ + +#. Verify the version-specific image is available: + + .. code-block:: bash + + docker images clearlinux/os-core:31110 + + +Second layer: httpd +------------------- + +The next layer is :file:`clearlinux/httpd`. + +#. Change the :file:`httpd/Dockerfile` to use the version-specific + *os-core:31110* image that was previously built. + + .. code-block:: bash + + $EDITOR httpd/Dockerfile + +#. Run :command:`git diff`. + + The output shows a diff of a modified :file:`clearlinux/httpd` Dockerfile + that uses the previously built clearlinux/os-core:31110. + + .. code-block:: diff + + diff --git a/httpd/Dockerfile b/httpd/Dockerfile + index 6b2a6bf..9df89e4 100644 + --- a/httpd/Dockerfile + +++ b/httpd/Dockerfile + @@ -7,7 +7,7 @@ RUN swupd update --no-boot-update $swupd_args + + # Grab os-release info from the minimal base image so + # that the new content matches the exact OS version + -COPY --from=clearlinux/os-core:latest /usr/lib/os-release / + +COPY --from=clearlinux/os-core:31110 /usr/lib/os-release / + + # Install additional content in a target directory + # using the os version from the minimal base + @@ -26,7 +26,7 @@ COPY --from=clearlinux/os-core:latest / / + os_core_install/ + RUN cd / && \ + find os_core_install | sed -e 's/os_core_install/install_root/' | xargs rm -d &> /dev/null || true + + -FROM clearlinux/os-core:latest + +FROM clearlinux/os-core:31110 + +#. Build Dockerfile. + + .. code-block:: bash + + docker build \ + --no-cache \ + --build-arg http_proxy=$http_proxy \ + --build-arg https_proxy=$https_proxy \ + --tag clearlinux/httpd:31110 \ + httpd/ + +Third layer: cgit +----------------- + +The next layer is :file:`clearlinux/cgit`. + +#. Change the :file:`cgit/Dockerfile` to use the desired OS + version; in this case, the build version is 31110. + + .. code-block:: bash + + $EDITOR cgit/Dockerfile + +#. Run :command:`git diff`. + + The output shows: + + .. code-block:: diff + + diff --git a/cgit/Dockerfile b/cgit/Dockerfile + index 9a3796d..59260fe 100644 + --- a/cgit/Dockerfile + +++ b/cgit/Dockerfile + @@ -7,7 +7,7 @@ RUN swupd update --no-boot-update $swupd_args + + # Grab os-release info from the minimal base image so + # that the new content matches the exact OS version + -COPY --from=clearlinux/httpd:latest /usr/lib/os-release / + +COPY --from=clearlinux/httpd:31110 /usr/lib/os-release / + + # Install additional content in a target directory + # using the os version from the minimal base + @@ -22,11 +22,11 @@ RUN source /os-release && \ + # file exists on different layers. To minimize docker + # image size, remove the overlapped files before copy. + RUN mkdir /os_core_install + -COPY --from=clearlinux/httpd:latest / /os_core_install/ + +COPY --from=clearlinux/httpd:31110 / /os_core_install/ + RUN cd / && \ + find os_core_install | sed -e 's/os_core_install/install_root/' | xargs rm -d &> /dev/null || true + + -FROM clearlinux/httpd:latest + +FROM clearlinux/httpd:31110 + +#. Build Dockerfile. + + .. code-block:: bash + + docker build \ + --no-cache \ + --build-arg http_proxy=$http_proxy \ + --build-arg https_proxy=$https_proxy \ + --tag clearlinux/cgit:31110 \ + cgit/ + +#. Verify the installed OS version by noting the :command:`VERSION_ID` value + in the :file:`/usr/lib/os-release` file in the container filesystem. + + .. code-block:: bash + :linenos: + :emphasize-lines: 6 + + docker run clearlinux/cgit:31110 cat /usr/lib/os-release + NAME="Clear Linux OS" + VERSION=1 + ID=clear-linux-os + ID_LIKE=clear-linux-os + VERSION_ID=31110 + PRETTY_NAME="Clear Linux OS" + ANSI_COLOR="1;35" + HOME_URL="https://clearlinux.org" + SUPPORT_URL="https://clearlinux.org" + BUG_REPORT_URL="mailto:dev@lists.clearlinux.org" + PRIVACY_POLICY_URL=http://www.intel.com/privacy + + +Example 4: Customize an application image at runtime +**************************************************** + +This section describes how to modify a published |CL| container at runtime. +In this example, we add Tensorflow\* into a :command:`clearlinux/python` +container. This approach can help accelerate the feature development process. + +In this example, three separate console windows are used to easily interact +inside and outside of the container. + +First console: Start the container +---------------------------------- + +#. Launch the clearlinux/python container. + + .. code-block:: bash + + docker run -it --rm clearlinux/python + Python 3.7.3 (default, Jun 17 2019, 00:47:04) + [GCC 9.1.1 20190616 gcc-9-branch@272336] on linux + Type "help", "copyright", "credits" or "license" for more information. + +#. Try to import Tensorflow inside the container using the command: + :command:`import tensorflow as tf`. The example below shows the expected + error message because the Docker image does not yet include the Tensorflow + module. + + .. code-block:: bash + + >>> import tensorflow as tf + Traceback (most recent call last): + File "", line 1, in + ModuleNotFoundError: No module named 'tensorflow' + >>> + +Second console: Add a bundle +---------------------------- + +#. In another console, find the :command:`` of + clearlinux/python launched. This example Container ID is d4ce9d526fa6. + + .. code-block:: bash + + docker ps + +#. The output shows: + + .. code-block:: console + + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + d4ce9d526fa6 clearlinux/python python3 About a minute ago Up About a minute amazing_villani + +#. Connect to the running clearlinux/python container. + + .. code-block:: bash + + docker exec -it d4ce9d526fa6 /usr/bin/bash + root@d4ce9d526fa6/ # + + +#. Use :command:`swupd` to install the machine-learning-tensorflow bundle. + + .. code-block:: bash + + root@d4ce9d526fa6/ # swupd bundle-add machine-learning-tensorflow + Loading required manifests... + Downloading packs (692.32 Mb) for: + - machine-learning-tensorflow + … … + ...100% + Finishing packs extraction... + No extra files need to be downloaded + Installing bundle(s) files... + ...100% + Calling post-update helper scripts. + Successfully installed 1 bundle + +#. After the machine-learning-tensorflow bundle is installed in the + container, in the first console, import Tensorflow, which will be + successful now. You could also save the updated container using the + command :command:`docker commit `. + + .. code-block:: bash + + >>> import tensorflow as tf + >>> tf.__version__ + '1.13.1' + +Third console: Save the modified container +------------------------------------------ + +#. In a third console, save the container with a new tag. Our example uses + the tag `tensorflow_added` to identify our modified container. + + .. code-block:: bash + + docker commit d4ce9d526fa6 clearlinux/python:tensorflow_added + +#. Launch the modified container, and then import Tensorflow with success. + + .. code-block:: bash + + docker run -it clearlinux/python:tensorflow_added + Python 3.7.3 (default, Jun 17 2019, 00:47:04) + [GCC 9.1.1 20190616 gcc-9-branch@272336] on linux + Type "help", "copyright", "credits" or "license" for more information. + + .. code-block:: bash + + >>> import tensorflow as tf + >>> tf.__version__ + '1.13.1' + >>> + +Background +********** + +Multi-stage Dockerfiles contain more than one :command:`FROM` directive. All +of the multi-stage Clear Linux OS Dockerfiles share a common base layer +called :command:`clearlinux/os-core:latest`. All of the higher level layers +inherit the Clear Linux OS version from this base layer. + +For details on how we leveraged multi-stage Docker builds, see the article +`Minimizing Clear Linux OS container sizes`_. + +#. :command:`clearlinux/os-core` is built once per day. It is a container + containing a minimal Linux userspace. + +#. The target container image uses either :command:`clearlinux/os-core` as a + base layer or another container image :command:`clearlinux/` as a base + layer. + +#. Bundle(s) containing the application are downloaded during the first stage + of the build process using :command:`swupd`. + +#. The final container image is a composition of its base layer and the + specific feature layer, via :command:`FROM clearlinux/:latest + , such as: os-core, httpd, and via :command:`COPY --from=builder / + install_root /`. Using this method, the target container images are kept + up to date without file duplication. For application-centric containers, + `os-core-update` is excluded to improve size optimization. + +Related topics +************** + +* :ref:`docker` +* :ref:`container-image-new` + +.. _images on Docker Hub: https://hub.docker.com/u/clearlinux +.. _GitHub\*: https://github.com/clearlinux/dockerfiles +.. _clearlinux image on Docker Hub: https://hub.docker.com/_/clearlinux +.. _clearlinux microservice dockerfile repo: https://github.com/clearlinux/dockerfiles + +.. _multi-stage build: https://docs.docker.com/develop/develop-images/multistage-build/ + +.. _Minimizing Clear Linux OS container sizes: https://clearlinux.org/blogs-news/minimizing-clear-linux-os-container-sizes + +.. _multi-stage build to reduce image size: https://clearlinux.org/blogs-news/minimizing-clear-linux-os-container-sizes diff --git a/_sources/guides/maintenance/container-image-new.rst.txt b/_sources/guides/maintenance/container-image-new.rst.txt new file mode 100644 index 00000000..73e80b1e --- /dev/null +++ b/_sources/guides/maintenance/container-image-new.rst.txt @@ -0,0 +1,324 @@ +.. _container-image-new: + +Build a new |CL|-based container image +###################################### + +This guide describes how to build a new |CL-ATTR|-based container image. The +official base |CL-ATTR| container image is published on Docker\* Hub and is +updated on a regular basis. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* You must perform these steps on a |CL| system because the + :abbr:`swupd (software updater)` is used to manage bundles in the + container. +* You must install the :file:`containers-basic` bundle on the |CL| system + or Docker will not work. +* You have a basic understanding of Docker. + +Build the base container image +****************************** + +#. Log in and get root privileges. + + .. code-block:: bash + + sudo -s + +#. Verify Docker is installed and running. + + .. code-block:: bash + + docker info + + If Docker is installed and running, the output is similar to + this example: + + .. code-block:: console + + Containers: 0 + Running: 0 + Paused: 0 + Stopped: 0 + Images: 4 + Server Version: 17.05.0-ce + Storage Driver: overlay + Backing Filesystem: extfs + Supports d_type: true + Logging Driver: json-file + Cgroup Driver: cgroupfs + Plugins: + Volume: local + Network: bridge host macvlan null overlay + Swarm: inactive + Runtimes: runc + Default Runtime: runc + Init Binary: docker-init + containerd version: (expected: 9048e5e50717ea4497b757314bad98ea3763c145) + runc version: N/A (expected: 9c2d8d184e5da67c95d601382adf14862e4f2228) + init version: N/A (expected: ) + Kernel Version: 4.12.7-377.native + Operating System: Clear Linux OS for Intel Architecture + OSType: linux + Architecture: x86_64 + CPUs: 4 + Total Memory: 15.62GiB + Name: clr-os + ID: XQHJ:DYEM:3Q4D:DKLM:JOA4:RUSF:GAFR:DLPA:HOJP:W5FF:ULEE:7HZ3 + Docker Root Dir: /var/lib/docker + Debug Mode (client): false + Debug Mode (server): false + Registry: https://index.docker.io/v1/ + Experimental: false + Insecure Registries: + 127.0.0.0/8 + Live Restore Enabled: false + + If Docker is not installed, enter the commands: + + .. code-block:: bash + + swupd bundle-add containers-basic + systemctl start docker + +#. Use :command:`os-install` to download and install the bundles. + + .. code-block:: bash + + swupd os-install --url https://cdn.download.clearlinux.org/update --statedir "$PWD"/swupd-state --no-boot-update -B os-core-update,editors,network-basic base + + + The swupd example uses the following flags: + + * :command:`os-install` tells swupd to download and install. + * :command:`--url` specifies the URL of the bundles repository. + * :command:`--statedir` specifies the state directory where downloaded bundles and any state information are stored. + * :command:`--no-boot-update` tells swupd to skip updating boot files because + boot files are not required for a container. + + For more information on swupd flags, enter the :command:`swupd os-install -h` + command. + + Example output: + + .. code-block:: console + + swupd-client software verify 3.12.2 + Copyright (C) 2012-2017 Intel Corporation + + Verifying version 17870 + Attempting to download version string to memory + Downloading packs... + + Extracting python-basic pack for version 17820 + ...14% + Extracting perl-basic pack for version 17790 + ...28% + Extracting openssh-server pack for version 17660 + ...42% + Extracting editors pack for version 17850 + ...57% + Extracting network-basic pack for version 17650 + ...71% + Extracting os-core pack for version 17870 + ...85% + Extracting os-core-update pack for version 17870 + ...100% + Adding any missing files + ...88% + Inspected 33982 files + 33974 files were missing + 33974 of 33974 missing files were replaced + 0 of 33974 missing files were not replaced + Calling post-update helper scripts. + WARNING: boot files update skipped due to --no-boot-update argument + Fix successful + + .. note:: + + The WARNING message is expected and can be ignored. + +#. Create a tarball and compress it. + + .. code-block:: bash + + tar -C base -cf base.tar . + xz -v -T0 base.tar + +#. Create the Dockerfile to build the image. + + .. code-block:: bash + + cat > Dockerfile << EOF + FROM scratch + MAINTAINER First Last + ADD base.tar.xz / + CMD ["/bin/bash"] + EOF + +#. Build the |CL| container image. + + .. code-block:: bash + + docker build -t my-custom-clear-linux-container . + + Example output: + + .. code-block:: console + + Sending build context to Docker daemon 806.5MB + Step 1/4 : FROM scratch + ---> + Step 2/4 : MAINTAINER First Last + ---> Running in 7238f35abcd0 + ---> ec5064287c60 + Removing intermediate container 7238f35abcd0 + Step 3/4 : ADD base.tar.xz / + ---> 2723b7d20716 + Removing intermediate container 16e3ed0df8da + Step 4/4 : CMD /bin/bash + ---> Running in efa893350647 + ---> 5414c3a12993 + Removing intermediate container efa893350647 + Successfully built 5414c3a12993 + Successfully tagged my-custom-clear-linux-container:latest + +#. List the newly created |CL| container image. + + .. code-block:: bash + + docker images + + Example output: + + .. code-block:: console + + REPOSITORY TAG IMAGE ID CREATED SIZE + my-custom-clear-linux-container latest 5414c3a12993 About a minute ago 616MB + +#. Launch the built |CL| container. + + .. code-block:: bash + + docker run -it my-custom-clear-linux-container + +Manage bundles in a container +***************************** + +You can add and remove bundles from a |CL| container using the +:command:`RUN swupd` command in the Dockerfile. + +Add a bundle +============ + +This example Dockerfile adds the :file:`pxe-server` bundle to an existing |CL| +Docker image: + +.. code-block:: bash + + cat > Dockerfile << EOF + FROM my-customer-clear-linux-container + MAINTAINER First Last + RUN swupd bundle-add pxe-server + CMD ["/bin/bash/bash"] + EOF + +Example output: + +.. code-block:: console + + docker build -t my-clearlinux-with-pxe-server-bundle . + + Sending build context to Docker daemon 806.5MB + Step 1/4 : FROM my-custom-clear-linux-container + ---> 5414c3a12993 + Step 2/4 : MAINTAINER First Last + ---> Running in 19b4411cf4bd + ---> 08d400baffde + Removing intermediate container 19b4411cf4bd + Step 3/4 : RUN swupd bundle-add pxe-server + ---> Running in 3e634d6e0792 + swupd-client bundle adder 3.12.2 + Copyright (C) 2012-2017 Intel Corporation + + Attempting to download version string to memory + Downloading packs... + + Extracting pxe-server pack for version 17820 + . + Installing bundle(s) files... + .............................................................................. + .............................................................................. + .............................................................................. + .............................................................................. + .............................................................................. + .............................................................................. + Calling post-update helper scripts. + WARNING: systemctl not operable, unable to run systemd update triggers + Bundle(s) installation done. + ---> 8ead5f2c0c33 + Removing intermediate container 3e634d6e0792 + Step 4/4 : CMD /bin/bash + ---> Running in 0ceae320279b + ---> dcd9adb40611 + Removing intermediate container 0ceae320279b + Successfully built dcd9adb40611 + Successfully tagged my-clearlinux-with-pxe-server-bundle:latest + +.. note:: + + The WARNING message can be ignored because systemd does not run inside + a container. + +Remove a bundle +=============== + +This example Dockerfile removes the :file:`pxe-server` bundle from an existing +|CL| Docker image: + +.. code-block:: bash + + cat > Dockerfile << EOF + FROM my-clearlinux-with-pxe-server-bundle + MAINTAINER First Last + RUN swupd bundle-remove pxe-server + CMD ["/bin/bash/bash"] + EOF + +Example output: + +.. code-block:: console + + docker build -t my-clearlinux-remove-pxe-server-bundle . + + Sending build context to Docker daemon 806.5MB + Step 1/4 : FROM my-clearlinux-with-pxe-server-bundle + ---> dcd9adb40611 + Step 2/4 : MAINTAINER First Last + ---> Running in 71b60f15003e + ---> 742192751c1a + Removing intermediate container 71b60f15003e + Step 3/4 : RUN swupd bundle-remove pxe-server + ---> Running in ad28a3390ecc + swupd-client bundle remover 3.12.2 + Copyright (C) 2012-2017 Intel Corporation + + Removing bundle: pxe-server + Deleting bundle files... + Total deleted files: 92 + Untracking bundle from system... + Success: Bundle removed + 1 bundle(s) were removed successfully + ---> d6ee7903e14d + Removing intermediate container ad28a3390ecc + Step 4/4 : CMD /bin/bash + ---> Running in 7694989e97de + ---> ec23189ef954 + Removing intermediate container 7694989e97de + Successfully built ec23189ef954 + Successfully tagged my-clearlinux-remove-pxe-server-bundle:latest diff --git a/_sources/guides/maintenance/cpu-performance.rst.txt b/_sources/guides/maintenance/cpu-performance.rst.txt new file mode 100644 index 00000000..828a0f00 --- /dev/null +++ b/_sources/guides/maintenance/cpu-performance.rst.txt @@ -0,0 +1,273 @@ +.. _cpu-performance: + +CPU Power and Performance +######################### + +This guide explains the CPU power and performance mechanisms in |CL-ATTR|. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +Modern x86 :abbr:`CPUs (central processing units)` employ a number of features +to balance performance, energy, and thermal efficiency. + +By default, |CL| prioritizes maximum CPU performance, assuming that +the faster the program finishes execution, the faster the CPU can return to a +low energy idle state. It is important to understand and evaluate the impact +of each feature when troubleshooting or considering changing the defaults. + +.. contents:: + :local: + :depth: 1 + +CPU power saving mechanisms +*************************** + +C-states and P-states are both CPU power saving mechanisms that are entered +under different operating conditions. The tradeoff is a slightly longer time +to exit these states when the CPU is needed. + +.. _c-states-section: + +C-states (idle states) +====================== + +Hardware enters a C-state when the CPU is idle and not executing instructions. +C-states decrease power utilization by reducing clock frequency, +voltages, and features in each state. Although C-states can typically be +limited or disabled in a system's UEFI or BIOS configuration, these settings +are overridden when the `intel_idle driver`_ is in use. + +To view the current ``cpuidle`` driver run this command in a terminal: + +.. code-block:: bash + + cat /sys/devices/system/cpu/cpuidle/current_driver + +For troubleshooting, C-states can be limited with a kernel command line boot +parameter by adding :command:`processor.max_cstate=N intel_idle.max_cstate=N` +or completely disabled with :command:`idle=poll`. + +.. note:: + + * :command:`processor.max_cstate=0` is changed to a valid value by the + kernel: :command:`processor.max_cstate=1`. + + * :command:`intel_idle.max_cstate=0` disables the Intel Idle driver rather + than set it to C-state 0. + +.. _p-states-section: + +P-states (performance states) +============================= + +The CPU can enter a P-state, also known as Intel SpeedStep® technology on +Intel processors or AMD\* Cool'n'Quiet\* technology, while it is active +and executing instructions. P-states reduce power utilization by adjusting CPU +clock frequency and voltages based on CPU demand. P-states can typically be +limited or disabled in a system's firmware (UEFI/BIOS). + +Turbo boost +----------- + +`Intel® Turbo Boost Technology`_, found on some modern Intel CPUs, allows +cores on a processor to temporarily operate at a higher than rated CPU clock +frequency to accommodate demanding workloads if the CPU is under defined power +and thermal thresholds. Intel Turbo Boost Technology is an extension of +P-states, so it can be impacted by limiting C-states or P-states. + +Intel Turbo Boost Technology can be disabled in a system's UEFI/BIOS or in +|CL|: + +.. code-block:: bash + + echo 1 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo + +Linux CPU clock frequency scaling +********************************* + +The ``CPUFreq`` subsystem in Linux allows the OS to control +:ref:`C-states ` and :ref:`P-states ` +via CPU drivers and governors that provide algorithms that define how and when +to enter these states. + +Scaling driver +============== + +Linux uses the `Intel P-state driver`_, :command:`intel_pstate`, for +modern Intel processors from the Sandy Bridge generation or newer. Other +processors may default to the :command:`acpi-cpufreq` driver which reads +values from the systems UEFI or BIOS. + +To view the current CPU frequency scaling driver, run this command in a +terminal: + +.. code-block:: bash + + cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver + +Scaling governor +================ + +|CL| sets the CPU governor to ``performance`` which calls for the CPU to +operate at maximum clock frequency. In other words, P-state P0. While this may +sound wasteful at first, it is important to remember that power utilization +does not increase significantly simply because of a locked clock frequency +without a workload. + +To view the current CPU frequency scaling governor, run this command in a +terminal: + +.. code-block:: bash + + cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor + +Each core will report its own status. Your output should look similar to this +example with four cores: + +.. code-block:: console + + performance + performance + performance + performance + +The list of all governors can be found in the Linux kernel documentation on +`CPUFreq Governors`_. + +.. note:: + + The intel_pstate driver only supports *performance* and *powersave* governors. + +There are 2 ways to change the CPU frequency scaling governor: + +#. Disable |CL| enforcement of certain power and performance settings: + + .. code-block:: bash + + sudo systemctl mask clr-power.timer + +#. Change the governor value in :file:`/sys/devices`. In the example below, + the governor is set to *performance*: + + .. code-block:: bash + + echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor + +Thermal management +****************** + +`thermald`_ is a Linux thermal management daemon used to prevent platforms +from overheating. :command:`thermald` forces a C-state by inserting CPU sleep +cycles and adjusting any available cooling methods. This can be especially +desirable for laptops. + +:command:`thermald` is disabled by default in |CL| and starts automatically +if it detects battery power. Enable :command:`thermald` manually by using +the systemd service by running the command: + +.. code-block:: bash + + sudo systemctl enable --now thermald + +For more information, see the :command:`thermald` man page: + +.. code-block:: bash + + man thermald + +`ThermalMonitor`_ is a GUI application that can visually graph and log +temperatures from :command:`thermald`. To use ThermalMonitor, add the +:command:`desktop-apps-extras` bundle and add your user account to the power +group: + +.. code-block:: bash + + sudo swupd bundle-add desktop-apps-extras + sudo usermod -a -G power + ThermalMonitor + +.. note:: + + After adding a new group, you must log out and log back in for the new group + to take effect. + +Enhanced thermal configuration +=============================== + +Better thermal control and performance can be achieved by providing platform +specific configuration to :command:`thermald`. + +`Linux DPTF Extract Utility`_ is a companion tool to :command:`thermald`, +This tool uses Intel® Dynamic Platform and Thermal Framework (Intel® DPTF) +technology and can convert to the :file:`thermal_conf.xml` configuration format +used by :command:`thermald`. Closed-source projects, like this one, cannot be +packaged as a bundle in |CL|, so you must install it manually: + +#. Make sure your machine's BIOS has DPTF feature and is enabled. It will usually be in the :guilabel:`Advanced` or :guilabel:`Advanced>Power` section of the BIOS. + + .. figure:: /_figures/cpu-perf-guide/dptf_bios.png + + .. note:: + + Intel DPTF requires BIOS support and is typically only available on + laptops. + +#. Generate thermal configuration. :command:`thermald` configuration files + will be generated and saved to :file:`/etc/thermal/` folder. + + .. code-block:: bash + + sudo swupd bundle-add acpica-unix2 # install acpi tools + git clone https://github.com/intel/dptfxtract.git + cd dptfxtract + sudo acpidump > acpi.out + acpixtract -a acpi.out + sudo ./dptfxtract *.dat + +#. Restart :command:`thermald` service to take effect. + + .. code-block:: bash + + sudo systemctl restart thermald.service + +#. Check whether the configuration is in use. + + .. code-block:: bash + + sudo systemctl status thermald.service + +The following output means the configuration has already been applied: + +.. code-block:: console + + thermald[*]: [WARN]Using generated /etc/thermald/thermal-conf.xml.auto + +*Intel® Turbo Boost Technology requires a PC with a processor with Intel Turbo +Boost Technology capability. Intel Turbo Boost Technology performance varies +depending on hardware, software and overall system configuration. Check with +your PC manufacturer on whether your system delivers Intel Turbo Boost Technology. +For more information, see http://www.intel.com/technology/turboboost* + +*Intel, Intel SpeedStep, and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + + +.. _`Intel P-state driver`: https://www.kernel.org/doc/Documentation/cpu-freq/intel-pstate.txt + +.. _`CPUFreq Governors`: https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt + +.. _thermald: https://01.org/linux-thermal-daemon + +.. _`intel_idle driver`: https://github.com/torvalds/linux/blob/master/drivers/idle/intel_idle.c + +.. _`ThermalMonitor`: https://github.com/intel/thermal_daemon/tree/master/tools/thermal_monitor + +.. _`Intel® Turbo Boost Technology`: https://www.intel.com/content/www/us/en/architecture-and-technology/turbo-boost/turbo-boost-technology.html + +.. _`Linux DPTF Extract Utility`: https://github.com/intel/dptfxtract + +.. _`Intel DPTF`: https://software.intel.com/en-us/articles/2-in-1-tablet-mode-game-performance-with-intel-dynamic-platform-and-thermal-framework-intel diff --git a/_sources/guides/maintenance/deploy-at-scale.rst.txt b/_sources/guides/maintenance/deploy-at-scale.rst.txt new file mode 100644 index 00000000..617b265e --- /dev/null +++ b/_sources/guides/maintenance/deploy-at-scale.rst.txt @@ -0,0 +1,255 @@ +.. _deploy-at-scale: + +Deploy at Scale +############### + +This guide describes deployment considerations and strategies when deploying +|CL-ATTR| at scale in your environment. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +In this guide the term *endpoint* refers to a system targeted for |CL| +installation, whether that is a datacenter system or unit deployed in field. + +.. note:: + + This guide is not a replacement or blueprint for designing your own IT + operating environment. + + Implementation details for a scale deployment are beyond the scope of this + guide. + + Your |CL| deployment should complement your existing environment and + available tools. It is assumed core IT dependencies of your environment, + such as your network, are healthy and scaled to suit the deployment. + +Pick a usage and update strategy +******************************** + +Different business scenarios call for different deployment methodologies. +|CL| offers the flexibility to continue consuming the upstream |CL| +distribution or the option to fork away from the |CL| distribution and +act as your own :abbr:`OSV (Operating System Vendor)`. + +Below is an overview of some considerations. + +Create your own Linux distribution (mix) +======================================== + +This approach forks away from the |CL| upstream and has you act as your own +:abbr:`OSV (Operating System Vendor)` by leveraging the :ref:`mixer` process to +create customized images based on |CL|. This is a level of responsibility +that requires having more infrastructure and processes to adopt. In return, +this approach *offers you a high degree of control and customization*. Consider: + +* Development systems that generate bundles and updates should have + sufficient performance for the task and be separate from the swupd update + webservers that serve update content to production machines. + +* swupd update webservers that serve update content to production machines + should be appropriately scaled. Specific implementation details for a scalable, + resilient web server are beyond the scope of this document. + + (See :ref:`mixer` for more information about update servers.) + +Adopt an agile methodology +========================== + +The cloud, and other scaled deployments, are all about flexibility and speed. +It only makes sense that any |CL| deployment strategy should follow suit. + +Manually rebuilding your own bundles or mix for every release is not +sustainable at a large scale. A |CL| deployment pipeline should be agile +enough to validate and produce new versions with speed. Whether or not those +updates actually make their way to your production can be separate +business decision. However this *ability to frequently roll new versions* of +software to your endpoints is an important prerequisite. + +You own the validation and lifecycle of the OS and should treat it like any +other software development lifecycle. Below are some pointers: + +* Thoroughly understand the custom software packages that you will need to + integrate with |CL| and maintain along with their dependencies. + +* Setup a path to production for building |CL| based images. At minimum this + should include: + + * A development clr-on-clr environment to test building packages and + bundles for |CL| systems. + + * A pre-production environment to deploy |CL| versions to before + production + +* Employ a continuous integration and continuous deployment (CI/CD) + philosophy in order to: + + - Automatically pull custom packages as they are updated from their + upstream projects or vendors. + + - Generate |CL| bundles and potentially bootable images with your + customizations, if any. + + - Measure against metrics and indicators which are relevant to your + business (e.g. performance, power, etc) from release to release. + + - Integrate with your organization's governance processes, such as change + control. + +Versioning infrastructure +========================= + +|CL| version numbers are very important as they apply to the whole +infrastructure stack from OS components to libraries and applications. + +Good record keeping is important, so you should keep a detailed registry and +history of previously deployed versions and their contents. + +With a glance at the |CL| version numbers deployed, you should be able to +tell if your Clear systems are patched against a particular security +vulnerability or incorporate a critical new feature. + +Pick an image distribution strategy +*********************************** + +Once you have decided on a usage and update strategy, you should understand +*how* |CL| will be deployed to your endpoints. In a large scale deployment, +interactive installers should be avoided in favor of automated installations +or prebuilt images. + +There are many well-known ways to install an operating system at scale. Each +have their own benefits, and one may lend itself easier in your environment +depending on the resources available to you. + +See the available :ref:`image-types`. + +Below are some common ways to install |CL| to systems at scale: + +Bare metal +========== + +Preboot Execution Environments (PXE) or other out-of-band booting options are +one way to distribute |CL| to physical bare metal systems on a LAN. + +This option works well if your customizations are fairly small in size +and infrastructure can be stateless. + +The |CL| `Downloads`_ page offers a live image that can be deployed as +a PXE boot server if one doesn't already exist in your environment. Also see +documentation on how to :ref:`bare-metal-install-server`. + +Cloud instances or virtual machines +=================================== + +Image templates in the form of cloneable disks are an effective way to +distribute |CL| for virtual machine environments, whether on-premises or +hosted by a Cloud Solution Provider (CSP). + +When used in concert with cloud VM migration features, this can be a good option +for allowing your applications a degree of high availability and workload +mobility; VMs can be restarted on a cluster of hypervisor host or moved between +datacenters transparently. + +The |CL| `Downloads`_ page offers example prebuilt VM images and is readily +available on popular CSPs. Also see documentation on how to +:ref:`virtual-machine-install`. + +Containers +========== + +Containerization platforms allow images to be pulled from a repository and +deployed repeatedly as isolated containers. + +Containers with a |CL| image can be a good option to blueprint and ship +your application, including all its dependencies, as an artifact while +allowing you or your customers to dynamically orchestrate and scale +applications. + +|CL| is capable of running a Docker host, has a container image which can +be pulled from DockerHub, or can be built as a customized container. +For more information visit the `Containers`_ page. + +Considerations with stateless systems +************************************* + +An important |CL| concept is statelessness and partitioning of system data +from user data. This concept can change the way you think about an at scale +deployment. + +Backup strategy +=============== + +A |CL| system and its infrastructure should be considered a commodity and +be easily reproducible. Avoid focusing on backing up the operating system +itself or default values. + +Instead, focus on backing up what's important and unique - the application +and data. In other words, only focus on backing up critical areas like +:file:`/home`, :file:`/etc`, and :file:`/var`. + +Meaningful logging & telemetry +============================== + +Offload logging and telemetry from endpoints to external servers, so it is +persistent and can be accessed on another server when an issue occurs. + +* Remote syslogging in |CL| is available through the + `systemd-journal-remote.service`_ + +* |CL| offers a :ref:`telem-guide`, which can be a powerful tool + for a large deployment to quickly crowdsource issues of interest. Take + advantage of this feature with careful consideration of the target audience + and the kind of data that would be valuable, and expose events + appropriately. + + Like any web server, the telemetry server should be appropriately scaled and + resilient. Specific implementation details for a scalable, resilient web + server are beyond the scope of this document. + +Orchestration and configuration management +========================================== + +In cloud environments, where systems can be ephemeral, being able to +configure and maintain generic instances is valuable. + +|CL| offers an efficient cloud-init style solution, `micro-config-drive`_, +through the *os-cloudguest* bundles which allow you to configure many Day 1 +tasks such as setting hostname, creating users, or placing +SSH keys in an automated way at boot. For more information on +automating configuration during deployment of |CL| endpoints see the +:ref:`ipxe-install` guide. + +A configuration management tool is useful for maintaining consistent system +and application-level configuration. Ansible\* is offered through the +*sysadmin-hostmgmt* bundle as a configuration management and automation +tool. + +Cloud-native applications +========================= + +An Infrastructure OS can design for good behavior, but it is ultimately up +to applications to make agile design choices. Applications deployed +on |CL| should aim to be host-aware but not depend on any specific host to +run. References should be relative and dynamic when possible. + +The application architecture should incorporate an appropriate tolerance for +infrastructure outages. Don't just keep stateless design as a noted feature. +Continuously test its use; Automate its use by redeploying |CL| and +application on new hosts. This naturally minimizes configuration drift, +challenges your monitoring systems, and business continuity plans. + +.. _`Downloads`: https://clearlinux.org/downloads/ +.. _`Containers`: https://clearlinux.org/downloads/containers +.. _`systemd-journal-remote.service`: https://www.freedesktop.org/software/systemd/man/systemd-journal-remote.service.html +.. _`micro-config-drive`: https://github.com/clearlinux/micro-config-drive + +.. |WEB-SERVER-SCALE| replace:: + There are many well-known ways to achieve a scalable and resilient web + server for this purpose, however implementation details are not in the + scope of this document. In general, they should be close to your + endpoints, highly available, and easy to scale with a load balancer when + necessary. diff --git a/_sources/guides/maintenance/developer-workstation.rst.txt b/_sources/guides/maintenance/developer-workstation.rst.txt new file mode 100644 index 00000000..5333fedc --- /dev/null +++ b/_sources/guides/maintenance/developer-workstation.rst.txt @@ -0,0 +1,225 @@ +.. _developer-workstation: + +Developer Workstation +##################### + +This guide helps you find the minimum set of bundles needed to start your +|CL-ATTR| development project. + +Before continuing, review the :ref:`swupd ` guide to learn more +about the swupd tool and how |CL| simplifies software versioning compared to +other Linux\* distributions. + +.. contents:: + :local: + :depth: 1 + +Workstation Setup +***************** + +After installing the minimum set of bundles required to get started, you can +add more bundles relevant to your specific use case. + +To run any process required for |CL| development, you can add the large +bundle :ref:`*os-clr-on-clr* `. However, you may want to deploy a leaner OS with only bundles relevant to your project. + +Use the **Developer Profiles** tabs to start installing *suggested bundles* +based on your role or project. Installing any ``dkms`` bundle gives all the +tools you need to start. Consider these profiles as a starting point. + +.. tip:: + + Click on a bundle to learn how to install it using :command:`swupd`. + +.. tabs:: + + .. tab:: AI/ML Engineer + + .. list-table:: + :widths: 50, 50 + :header-rows: 1 + + * - Function + - Bundle + + * - Build machine learning applications with a full suite of libraries. + - `machine-learning-basic `_ + + * - Build machine learning applications with PyTorch, an optimized tensor library for deep learning. + - `machine-learning-pytorch `_ + + * - Build machine learning applications using Tensorflow, a library for numerical computation using deep neural networks. + - `machine-learning-tensorflow `_ + + * - Web-based, interactive tools for machine learning. + - `machine-learning-web-ui `_ + + * - Machine learning Docker container. + - `machine-learning `_ + + * - Pre-built Python libraries for Data Science. + - `python-extras `_ + + * - API helper for cloud access. + - `cloud-api `_ + + .. tab:: Computer Vision Engineer + + .. list-table:: + :widths: 50, 50 + :header-rows: 1 + + * - Function + - Bundle + + * - Build computer vision applications. + - `computer-vision-basic `_ + + * - Work with deep learning and edge-optimized models. + - `computer-vision-models `_ + + * - API helper for cloud access. + - `cloud-api `_ + + * - Run container applications from Dockerhub in lightweight virtual machines. + - `containers-virt `_ + + * - All content for pkgconfig file opencv.pc. + - `devpkg-opencv `_ + + * - *Refer also to Cloud Orchestration Engineer* + - + + .. tab:: Cloud Orchestration Engineer + + .. list-table:: + :widths: 50, 50 + :header-rows: 1 + + * - Function + - Bundle + + * - Contains Clear Linux\* OS native software for cloud. + - `ethtool `_ + + * - Utilities for controlling TCP/IP networking and traffic control. + - `iproute2 `_ + + * - API helper for cloud access. + - `cloud-api `_ + + * - C++ runtime support. + - `libstdcpp `_ + + * - Load and enumerate PKCS#11 modules. + - `p11-kit `_ + + .. tab:: Kernel Developer + + .. list-table:: + :widths: 50, 50 + :header-rows: 1 + + * - Function + - Bundle + + * - Installs kernel, initrd, kernel config, system map; creates a bootloader entry. + - `kernel-install `_ + + * - Support module for building/loading via Dynamic Kernel Module System (DKMS) in LTS kernel. + - `kernel-lts-dkms `_ + + * - Support module for building/loading via Dynamic Kernel Module System (DKMS) in native kernel. + - `kernel-native-dkms `_ + + * - Support module for building/loading via Dynamic Kernel Module System (DKMS) in AWS kernel. + - `kernel-aws-dkms `_ + + * - Run the Kernel-based Virtual Machine (KVM) with |CL| as a guest under KVM. + - `kernel-kvm `_ + + * - Linux Test Project. + - `ltp `_ + + .. tab:: Maker Developer + + .. list-table:: + :widths: 50, 50 + :header-rows: 1 + + * - Function + - Bundle + + * - Basic tools for makers and experimenters. + - `maker-basic `_ + + * - GIS/Mapping tools for makers. + - `maker-gis `_ + + * - Electronic Design Tool. + - `Fritzing `_ + + * - Open-source electronics prototyping platform. + - `arduino-ide `_ + + .. tab:: System Administrator + + .. list-table:: + :widths: 50, 50 + :header-rows: 1 + + * - Function + - Bundle + + * - Run popular terminal text editors. + - `editors `_ + + * - Run network utilities and modify network settings. + - `network-basic `_ + + * - Run a secure shell (SSH) server for access from remote machines. + - `openssh-server `_ + + * - Run an HTTP server. + - `nginx `_ + + * - Run an application server via HTTP. + - `application-server `_ + + * - Run a SQLite database. + - `sqlite `_ + + * - Bundle to automatically launch the GUI upon boot. + - `desktop-autostart `_ + +swupd search +************ + +We recommend learning about :ref:`swupd `, to learn the +commands to search for and add bundles relevant to your project. + +The guide provides an :ref:`example ` +that shows you how to: + +* Use swupd to search for bundles +* Use swupd to add bundles + +Core Concepts +************* + +We recommend that you understand these core concepts in |CL| *before* +developing your project. + +* :ref:`Software update ` +* :ref:`Mixer ` +* :ref:`Autospec ` + +Related topics +-------------- + +* `Developer Tooling Framework`_ for |CL| +* `Bundle Definition Files`_ + +.. _Bundle Definition Files: https://github.com/clearlinux/clr-bundles + +.. _Developer Tooling Framework: https://github.com/clearlinux/common diff --git a/_sources/guides/maintenance/download-verify-decompress.rst.txt b/_sources/guides/maintenance/download-verify-decompress.rst.txt new file mode 100644 index 00000000..9142bc34 --- /dev/null +++ b/_sources/guides/maintenance/download-verify-decompress.rst.txt @@ -0,0 +1,176 @@ +.. _download-verify-decompress: + +Download, verify, and decompress a |CL-ATTR| image +################################################## + +This guide describes the available types of |CL| images, where to +download them, how to verify their integrity, and how to decompress them. +Follow the steps for your OS. + +.. contents:: + :local: + :depth: 1 + + +.. include:: ../../reference/image-types.rst + :start-after: image-types-content: + :end-before: incl-image-filename-end: + +.. _download-verify-decompress-linux: + +Linux OS steps +************** + +.. _verify-linux: + +Verify the integrity of the |CL| image +====================================== + +Before you use a downloaded |CL| image, verify its integrity. This action +eliminates the small chance of a corrupted image due to download issues. To +support verification, each released |CL| image has a corresponding SHA512 +checksum file designated with the suffix `-SHA512SUMS`. + +#. Download the corresponding SHA512 checksum file of your |CL| `image`_. +#. Open a Terminal. +#. Go to the directory with the downloaded image and checksum files. +#. Verify the integrity of the image and compare it to its original checksum + with the command: + + .. code-block:: bash + + sha512sum -c ./clear-[version number]-[image type].[compression type]-SHA512SUMS + +If the checksum of the downloaded image is different than the original +checksum, a warning is displayed with a message indicating the computed +checksum does **not** match. Otherwise, the name of the image is printed on +the screen followed by `OK`. + +For a more in-depth discussion of image verification including checking the +certificate see :ref:`image-content-validation`. + +.. incl-decompress-image: + +Decompress the |CL| image +========================= + +Released |CL| images are compressed with either GNU zip (*.gz*) or XZ +(*.xz*). The compression type depends on the target platform or +environment. To decompress the image, follow these steps: + +#. Open a Terminal. +#. Go to the directory with the downloaded image. + + To decompress an XZ image, enter: + + .. code-block:: bash + + unxz clear-[version number]-[image type].xz + + To decompress a GZ image, enter: + + .. code-block:: bash + + gunzip clear-[version number]-[image type].gz + +.. incl-decompress-image-end: + +.. _download-verify-decompress-mac: + +macOS\* steps +************* + +.. _verify-mac: + +Verify the integrity of the |CL| image +====================================== + +Before you use a downloaded |CL| image, verify its integrity. This action +eliminates the small chance of a corrupted image due to download issues. To +support verification, each released |CL| image has a corresponding SHA512 +checksum file designated with the suffix `-SHA512SUMS`. + +#. Download the corresponding SHA512 checksum file of your |CL| `image`_. +#. Open a Terminal. +#. Go to the directory with the downloaded image and checksum files. +#. Verify the integrity of the image and compare it to its original checksum + with the command: + + .. code-block:: bash + + shasum -a512 clear-[version number]-[image type].[compression type] | diff clear-[version number]-[image type].[compression type]-SHA512SUMS - + +If the checksum of the downloaded image is different than the original +checksum, the differences will be displayed. Otherwise, an empty output indicates +a match and your downloaded image is good. + +Decompress the |CL| image +========================= + +We compress all released |CL| images by default with either GNU zip +(`.gz`) or xz (`.xz`). The compression type we use depends on the target +platform or environment. To decompress the image, follow these steps: + +#. Open a Terminal. +#. Go to the directory with the downloaded image. +#. Use the :command:`gunzip` command to decompress either compression type. For example: + + .. code-block:: bash + + gunzip clear-[version number]-[image type].xz + gunzip clear-[version number]-[image type].gz + +.. _download-verify-decompress-windows: + +Windows\* OS steps +****************** + +.. _verify-windows: + +Verify the integrity of the |CL| image +====================================== + +Before you use a downloaded |CL| image, verify its integrity. This action +eliminates the small chance of a corrupted image due to download issues. To +support verification, each released |CL| image has a corresponding SHA512 +checksum file designated with the suffix `-SHA512SUMS`. + +#. Download the corresponding SHA512 checksum file of your |CL| `image`_. +#. Start Command Prompt. +#. Go to the directory with the downloaded image and checksum files. +#. Get the SHA512 checksum of the image with the command: + + .. code-block:: bash + + CertUtil -hashfile ./clear-[version number]-[image type].[compression type] SHA512 + +#. Manually compare the output with the original checksum value shown in + the downloaded checksum file and make sure they match. + +Decompress the |CL| image +========================= + +Released |CL| images are compressed with either GNU zip (*.gz*) or XZ +(*.xz*). The compression type depends on the target platform or +environment. To decompress the image, follow these steps: + +#. Download and install `7-Zip`_. +#. Go to the directory with the downloaded image and right-click it. +#. From the pop-up menu, select :guilabel:`7-Zip` and select + :guilabel:`Extract Here` as shown in Figure 1. + + .. figure:: figures/download-verify-decompress-windows-fig-1.png + :scale: 80 % + :alt: 7-Zip extract file + + Figure 1: Windows 7-Zip extract file. + +.. _7-Zip: http://www.7-zip.org/ + +Image types +*********** + +.. include:: ../../reference/image-types.rst + :start-after: incl-image-filename-end: + +.. _image: https://clearlinux.org/downloads diff --git a/_sources/guides/maintenance/enable-systemd-boot-menu.rst.txt b/_sources/guides/maintenance/enable-systemd-boot-menu.rst.txt new file mode 100644 index 00000000..54d5d3a3 --- /dev/null +++ b/_sources/guides/maintenance/enable-systemd-boot-menu.rst.txt @@ -0,0 +1,23 @@ +.. _enable-systemd-boot-menu: + +Enable systemd-boot Menu +######################## + +The default installation of |CL| does not set a timeout value for the systemd-boot +bootloader. At boot time, you will not be presented with the systemd-boot menu. Without +a menu, you cannot interact with systemd-boot such as selecting a different kernel, +editing kernel command line parameters, etc. + +To set a timeout value for the systemd-boot menu, follow these steps: + +#. Boot up |CL|. + +#. Log in. + +#. Set a timeout (for example: 20 seconds). + + .. code-block:: bash + + sudo clr-boot-manager set-timeout 20 + +#. Reboot. diff --git a/_sources/guides/maintenance/enable-user-space.rst.txt b/_sources/guides/maintenance/enable-user-space.rst.txt new file mode 100644 index 00000000..e51fa59e --- /dev/null +++ b/_sources/guides/maintenance/enable-user-space.rst.txt @@ -0,0 +1,103 @@ +.. _enable-user-space: + +Create and enable a new user space +################################## + +This guide provides steps to complete the following basic setup tasks for +a newly installed |CL-ATTR| system: + +.. contents:: + :local: + :depth: 1 + +Create a new user +***************** + +To create a new user and set a password for that user, enter the following +commands as a root user: + +.. code-block:: bash + + useradd + passwd + +Replace the with the name of the user account you want to create +including the password for that user. The :command:`passwd` command prompts +you to enter a new password. Retype the new password for the new user +account just created. + +Add the new user to the *wheel* group +************************************* + +Before logging off as root and logging into your new user account, +enable the :command:`sudo` command for your new . + +To be able to execute all applications with root privileges, add the + to the `wheel group`_. + +#. Add to the wheel group: + + .. code-block:: bash + + usermod -G wheel -a + +#. Log out of root and into the new . + + To log off as root, enter :command:`exit`. + +#. Enter the new and the password created earlier. + + You will now be in the home directory of . + +Install and update the OS software to its current version +********************************************************* + +The |CL| software utility :ref:`swupd ` allows you to perform +system updates while reaping the benefits of upstream development. + +To update your newly installed OS, run: + +.. code-block:: bash + + sudo swupd update + +Add a bundle +************ + +Software applications are installed as bundles using the command +:command:`swupd bundle-add`. Experienced Linux users might compare swupd +to running :command:`apt-get` or :command:`yum install` for package +management. However |CL| manages packages at the level of bundles, which +are integrated stacks of packages. + +For example, the :command:`sysadmin-basic` bundle installs the majority of +applications useful to a system administrator. To install it, enter: + +.. code-block:: bash + + swupd bundle-add sysadmin-basic + +View a full list of bundles and packages installed with the `sysadmin-basic`_ +bundle. You can also view all `bundles`_ for |CL|, active or deprecated. + +Expand your knowledge of :command:`swupd` and check out our developer resources: + +* :ref:`swupd-guide` +* :ref:`developer-workstation` + +Next steps +********** + +Check out our guides and tutorials. + +* :ref:`guides` +* :ref:`tutorials` + +.. _`sysadmin-basic`: + https://github.com/clearlinux/clr-bundles/blob/master/bundles/sysadmin-basic + +.. _`bundles`: + https://github.com/clearlinux/clr-bundles/tree/master/bundles + +.. _`wheel group`: + https://en.wikipedia.org/wiki/Wheel_(Unix_term) diff --git a/_sources/guides/maintenance/fix-broken-install.rst.txt b/_sources/guides/maintenance/fix-broken-install.rst.txt new file mode 100644 index 00000000..0baa6f42 --- /dev/null +++ b/_sources/guides/maintenance/fix-broken-install.rst.txt @@ -0,0 +1,117 @@ +.. _fix-broken-install: + +Fix a broken installation +######################### + +This guide explains how to fix a broken installation of |CL-ATTR| using a live +desktop image on a USB. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +This guide assumes you have installed |CL| on a target system, but the OS +does not boot or function properly. + +The process described in this guide can only verify and fix files that +:ref:`swupd` owns in :file:`/usr` and :file:`/var`. Files outside +of this path, such as :file:`/home/`, :file:`/etc`, etc., cannot be +repaired by this process. + +Prerequisites +************* + +* Download and burn the live desktop image on a USB. + See :ref:`bare-metal-install-desktop` for instructions. + +Boot a live desktop image to fix target system +********************************************** + +#. Boot the |CL| live desktop image. + +#. Select |CL| in the boot menu. + +Mount root partition, verify, and fix +************************************* + +#. Ensure the system is connected to the Internet in order to access the + the |CL| update server. + +#. Open a terminal window. + +#. Find the |CL| root partition by using the :command:`lsblk` command with + these options: ``-o NAME,LABEL,PARTTYPE,PARTLABEL``. + + .. code-block:: bash + + lsblk -o NAME,LABEL,PARTTYPE,PARTLABEL + + Example output: + + .. code-block:: console + :emphasize-lines: 9 + + NAME SIZE LABEL PARTTYPE PARTLABEL + /dev/loop0 643.6M + /dev/sda 14.3G CLR_ISO + ├─/dev/sda1 835M CLR_ISO 0x0 + └─/dev/sda2 100M "CLEAR_EFI" 0xef + /dev/sdb 74.5G + ├─/dev/sdb1 142M boot c12a7328-f81f-11d2-ba4b-00a0c93ec93b EFI + ├─/dev/sdb2 244M swap 0657fd6d-a4ab-43c4-84e5-0933c84b4f4f linux-swap + └─/dev/sdb3 74.2G root 4f68bce3-e8cd-4db1-96e7-fbcaf984b709 / + + In the example above, ``/dev/sdb3/`` is the root partition. + +#. Next, mount the root partition. + + .. code-block:: bash + + sudo mount /dev/sdb3 /mnt + +#. Verify that you mounted the correct root partition by verifying the content + of ``/mnt/usr/lib/os-release`` looks similar to the example below. + + .. code-block:: bash + + cat /mnt/usr/lib/os-release + + Example output: + + .. code-block:: console + + NAME="Clear Linux OS" + VERSION=1 + ID=clear-linux-os + ID_LIKE=clear-linux-os + VERSION_ID=32150 + PRETTY_NAME="Clear Linux OS" + ANSI_COLOR="1;35" + HOME_URL="https://clearlinux.org" + SUPPORT_URL="https://clearlinux.org" + BUG_REPORT_URL="mailto:dev@lists.clearlinux.org" + PRIVACY_POLICY_URL="http://www.intel.com/privacy" + +#. Next, run :command:`swupd repair` to fix any issues on the target system. + + .. code-block:: bash + + sudo swupd repair --picky --path=/mnt --statedir=/mnt/var/lib/swupd + + :ref:`Learn more about how swupd works `. + +#. After the process is complete, unmount the root partition. + + .. code-block:: bash + + sudo umount /mnt + +#. Reboot the system, remove the live desktop USB drive, + and boot into the repaired system. + + .. code-block:: bash + + sudo reboot diff --git a/_sources/guides/maintenance/hostname.rst.txt b/_sources/guides/maintenance/hostname.rst.txt new file mode 100644 index 00000000..ce0c225c --- /dev/null +++ b/_sources/guides/maintenance/hostname.rst.txt @@ -0,0 +1,81 @@ +.. _hostname: + +Modify hostname +############### + +This guide describes how to modify and view the hostname of your |CL-ATTR| +system. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +By default, |CL| installations have a machine generated name, which is a +long string of letters and numbers. The generated name is fine for computers +but is not human-friendly. Administrators and users will often want to rename +their machines with a name that is easier to remember, type, and search +for. Renaming a machine also makes it easier to identify, by including +meaningful data in the name. The following examples show human-friendly machine +names: + +* *regression-test* +* *sally-test-box1* +* *az-bldg2-lab* + +Set your hostname +***************** + +|CL| uses the :command:`hostnamectl` command to display and modify the machine +name. :command:`hostnamectl` is part of the :command:`os-core` bundle, which +provides a basic Linux\* user space and utilities. + +This example sets the hostname to *telemetry-test-2-h15*, to identify a +|CL| telemetry test machine on the second floor at grid location H15. +Make sure to reboot after setting a new hostname. + +.. code-block:: bash + + sudo hostnamectl set-hostname telemetry-test-2-h15 + sudo reboot + +.. note:: + + There are three types of hostname: *static*, *transient*, and *pretty*. + The most common is the static hostname. Static hostnames must be between + two and 63 characters long, must start and end with a letter or number, + and may contain letters (case-insensitive), numbers, dashes, or dots. + + If the static hostname exists, it is used to generate the transient hostname, + which is maintained by the kernel. The transient hostname can be changed + by DHCP or mDNS at runtime. + + The pretty hostname is a free-form UTF8 name used for presentation to the user. + +View your hostname +****************** + +View your current hostname using the following command: + +.. code-block:: bash + + hostnamectl + +You should see output similar to: + +.. code-block:: console + + Static hostname : telemetry-test-2-h15 + Pretty hostname : telemetry-test-2-h15 + Icon name : computer-desktop + Chassis : desktop + Machine ID : 4d0d60207a904ebbab96680a51ac1339 + Boot ID : 98d3514e5a984e8cbbdf46a2f0d6b397 + Operating System : Clear Linux OS + Kernel : Linux 4.18.8-632.native + Architecture : x86-64 + + +**Congratulations!** You successfully modified the hostname of your |CL| system. diff --git a/_sources/guides/maintenance/increase-virtual-disk-size.rst.txt b/_sources/guides/maintenance/increase-virtual-disk-size.rst.txt new file mode 100644 index 00000000..f2f22719 --- /dev/null +++ b/_sources/guides/maintenance/increase-virtual-disk-size.rst.txt @@ -0,0 +1,182 @@ +.. _increase-virtual-disk-size: + +Increase Virtual Disk Size of an Image +###################################### + +|CL-ATTR| pre-built images come in different sizes, ranging from 300 MB to 20 +GB. This guide describes how to increase the disk size of your pre-built +image if you need more capacity. We will use the :ref:`KVM image` as +an example to demonstrate the process of increasing disk size and expanding +the last partition to take up the added space. + +.. contents:: + :local: + :depth: 1 + +Determine disk size and list of partitions +****************************************** + +There are two methods to find the disk size and the list of partitions of +a pre-built |CL| image. + +Method 1: Use :command:`lsblk` on the VM +======================================== + +The first method is to boot up your VM and execute the :command:`lsblk` +command as shown below: + +.. code-block:: bash + + lsblk + +An example output of the :command:`lsblk` command: + +.. code-block:: console + :emphasize-lines: 4,7 + + NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT + fd0 2:0 1 4K 0 disk + sr0 11:0 1 1024M 0 rom + vda 254:0 0 8.6G 0 disk + ├─vda1 254:1 0 510M 0 part + ├─vda2 254:2 0 33M 0 part [SWAP] + └─vda3 254:3 0 8G 0 part / + +An example of this can also be seen in Figure 1. + +Method 2: Look at the image configuration YAML file +=================================================== + +The second method to look at the image configuration YAML file that was +used to produce the image. + +For example, to find the size of the KVM image version number 31880, +follow these steps: + +#. Go to the `releases`_ repository. +#. Drill down into the `31880 > clear > config > image` directory. +#. Download and open the :file:`kvm.yaml` file. +#. Locate the `targetMedia` section. + + The example shows a total disk size of 8.54 GB, 512 MB for the EFI + partition, 32 MB for the swap partition, and 8 GB for the root partition. + + .. code-block:: console + :linenos: + :emphasize-lines: 3,9,13,18 + + targetMedia: + - name: ${bdevice} + size: "8.54G" + type: disk + children: + - name: ${bdevice}1 + fstype: vfat + mountpoint: /boot + size: "512M" + type: part + - name: ${bdevice}2 + fstype: swap + size: "32M" + type: part + - name: ${bdevice}3 + fstype: ext4 + mountpoint: / + size: "8G" + type: part + +Increase virtual disk size +************************** + +Before you can expand the last partition of your image, you must make +space available by increasing the virtual disk size. After that, you +can resize the last partition and finally resize the filesystem. +Follow these steps: + +Increase virtual disk size +========================== + +#. Shut down your VM. +#. Use the process defined by your hypervisor or cloud provider to increase + the virtual disk size of your |CL| VM. +#. Power up your VM. + +Resize the last partition of the virtual disk +============================================= + +#. Log in. +#. Open a terminal window. +#. Add the :command:`storage-utils` bundle to install the + :command:`parted` and :command:`resize2fs` tools. + + .. code-block:: bash + + sudo swupd bundle-add storage-utils + +#. Launch the :command:`parted` tool. + + .. code-block:: bash + + sudo parted + +#. In the `parted` tool, perform these steps: + + a. Press :command:`p` to print the partitions table. + #. If the warning message below is displayed, enter :command:`Fix`. + + .. code-block:: console + + Warning: Not all of the space available to :file:`/dev/sda` appears + to be used, you can fix the GPT to use all of the space (an extra ... + blocks) or continue with the current setting? + + Fix/Ignore? + + #. Enter :command:`resizepart ` where + ** is the number of the partition to modify. + #. Enter the new `End` size. + + .. note:: + + If you want a partition to take up the remaining disk space, then + enter the total size of the disk. When you print the partitions + table with the :command:`p` command, the total disk size is shown + after the :guilabel:`Disk` label. + + An example of this can be seen in Figure 1. + + #. Enter :command:`q` to exit `parted` when you are finished resizing the + partition. + + Figure 1 depicts the described steps to resize the partition of the + virtual disk from 8.5 GB to 30 GB. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/increase-virtual-disk-size/01-increase-virtual-disk-size.png + :scale: 100 % + :alt: Increase root partition size + + Figure 1: Increase root partition size + +Resize the filesystem +===================== + +#. Enter :command:`sudo resize2fs -p /dev/` where + ** is the partition that was changed in the `parted` + tool. + +#. Run :command:`lsblk` to verify that the filesystem size has increased. + + Figure 2 depicts the described steps to resize the filesystem of the virtual + disk from 8.5 GB to 30 GB. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/increase-virtual-disk-size/02-increase-virtual-disk-size.png + :scale: 100 % + :alt: Increase root filesystem with resize2fs + + Figure 2: Increase root filesystem with :command:`resize2fs` + +.. _releases: https://cdn.download.clearlinux.org/releases/ diff --git a/_sources/guides/maintenance/query-upstream.rst.txt b/_sources/guides/maintenance/query-upstream.rst.txt new file mode 100644 index 00000000..65bd614f --- /dev/null +++ b/_sources/guides/maintenance/query-upstream.rst.txt @@ -0,0 +1,226 @@ +.. _query-upstream: + +Query package info from upstream repository +########################################### + +This guide describes how to query package information from the |CL| upstream +repositories. This guide is intended for developers and advanced users. + +.. contents:: + :local: + :depth: 1 + :backlinks: top + +Overview +******** + +In |CL-ATTR|, the :ref:`swupd` tool manages software +dependencies and installs bundles instead of packages. Although a bundle is +a collection of one or more packages, |CL| does not work with packages on +the client side. However, on the upstream/factory side, |CL| does work with +packages using a process called *mixing*. + +Currently, :command:`swupd` does not report which packages are installed, +provide package version information, or return other package details. This +guide describes a method for retrieving package information from the |CL| +upstream repositories using :abbr:`DNF(Dandified Yum)` commands. + +Prerequisites +************* + +This guide assumes you have installed |CL| on your host system. +For detailed instructions on installing |CL| on a bare metal system, visit +the :ref:`bare metal installation guide `. + +Before you install any new packages, update |CL| with the following command: + +.. code-block:: bash + + sudo swupd update + +Configure DNF +************* + +#. Install the DNF bundle with the command: + + .. code-block:: bash + + sudo swupd bundle-add dnf + +#. Create a :file:`dnf.conf` file with the commands: + + .. code-block:: bash + + sudo mkdir -p /etc/dnf + sudo curl -L https://github.com/clearlinux/common/raw/master/conf/dnf.conf --output /etc/dnf/dnf.conf + + +#. Edit the :file:`/etc/dnf/dnf.conf` file and set the **baseurl** variable + for binary and source RPMs as shown in lines 3 and 9 in the following + example. + + .. code-block:: bash + :linenos: + :emphasize-lines: 3,9 + + [clear] + name=Clear + baseurl=https://cdn.download.clearlinux.org/releases/$releasever/clear/x86_64/os/ + enabled=1 + gpgcheck=0 + [clear-source] + name=Clear sources + failovermethod=priority + baseurl=https://cdn.download.clearlinux.org/releases/$releasever/clear/source/SRPMS/ + enabled=1 + gpgcheck=0 + +#. Initialize the RPM database with the command: + + .. code-block:: bash + + sudo rpm --initdb + + +DNF command usage examples +************************** + +.. contents:: :local: + :depth: 1 + :backlinks: top + +List all binary and source RPMs in the current release +====================================================== + +Command: + +.. code-block:: bash + + dnf repoquery --releasever=current + +Sample output: + +.. code-block:: console + + Clear 5.1 MB/s | 13 MB 00:02 + Clear sources 1.8 MB/s | 1.7 MB 00:00 + AVB-AudioModules-0:4.1.0-1.src + AVB-AudioModules-0:4.1.0-1.x86_64 + AVB-AudioModules-data-0:4.1.0-1.x86_64 + AVB-AudioModules-dev-0:4.1.0-1.x86_64 + AVB-AudioModules-lib-0:4.1.0-1.x86_64 + AVB-AudioModules-license-0:4.1.0-1.x86_64 + AVBStreamHandler-0:1.1.0-21.src + AVBStreamHandler-0:1.1.0-21.x86_64 + AVBStreamHandler-abi-0:1.1.0-21.x86_64 + AVBStreamHandler-bin-0:1.1.0-21.x86_64 + AVBStreamHandler-data-0:1.1.0-21.x86_64 + AVBStreamHandler-dev-0:1.1.0-21.x86_64 + AVBStreamHandler-lib-0:1.1.0-21.x86_64 + AVBStreamHandler-license-0:1.1.0-21.x86_64 + ... + + +Show version information for a package in current release +========================================================= + +This example queries version information for the zstd package. + +Command: + +.. code-block:: bash + + dnf repoquery --releasever=current zstd + +Sample output: + +.. code-block:: console + + Last metadata expiration check: 0:02:30 ago on Tue 16 Jul 2019 03:03:34 PM PDT. + zstd-0:1.4.0-46.src + zstd-0:1.4.0-46.x86_64 + + +Show version information for a package in a specific release +============================================================ + +This example queries version information for the zstd package in release +21000. + +Command: + +.. code-block:: bash + + dnf repoquery --releasever=21000 zstd + +Sample output: + +.. code-block:: console + + Clear + 2.7 MB/s | 3.9 MB 00:01 + Clear sources + 628 kB/s | 559 kB 00:00 + zstd-0:1.3.3-20.src + zstd-0:1.3.3-20.x86_64 + +Show only version and release information for a package in a specific release +============================================================================= + +This example queries version and release information for the zstd package in +release 15000. + +Command: + +.. code-block:: bash + + dnf repoquery --releasever=15000 --qf="%{VERSION}\n%{RELEASE}" zstd + +Sample output: + +.. code-block:: console + + Clear + 3.4 MB/s | 3.9 MB 00:01 + Clear sources + 345 kB/s | 528 kB 00:01 + 1.1.4 + 5 + +Show the binary package for a specified binary file +=================================================== + +This example returns the binary package that contains the +:file:`/usr/bin/zip` binary file. + +Command: + +.. code-block:: bash + + dnf repoquery --releasever=current --whatprovides /usr/bin/zip + +Sample output: + +.. code-block:: console + + Last metadata expiration check: 0:04:47 ago on Tue 16 Jul 2019 03:03:34 PM PDT. + zip-bin-0:3.0-23.x86_64 + +Show the source package for a specified binary file +=================================================== + +This example returns the source package that contains the +:file:`/usr/bin/zip` binary file. + +Command: + +.. code-block:: bash + + dnf repoquery --releasever=current --whatprovides /usr/bin/zip --srpm + +Sample output: + +.. code-block:: console + + Last metadata expiration check: 0:05:50 ago on Tue 16 Jul 2019 03:03:34 PM PDT. + zip-0:3.0-23.src diff --git a/_sources/guides/maintenance/resource-limits.rst.txt b/_sources/guides/maintenance/resource-limits.rst.txt new file mode 100644 index 00000000..670aff68 --- /dev/null +++ b/_sources/guides/maintenance/resource-limits.rst.txt @@ -0,0 +1,114 @@ +.. _resource-limits: + +Resource limits +############### + +Linux systems employ limiting or quota mechanisms to provide quality of +service for system resources and contain rogue processes. + +These limits are layered at the system-level and user-level. If these limits +need to be modified, it is useful to understand the different limit +configurations. + +.. contents:: :local: + :depth: 2 + + +System-wide limits +================== + +Some global resource limits are implemented in the Linux kernel and are +controllable with kernel parameters. + +For example, a global limit for the maximum number of open files is set with +the *fs.file-max* parameter. This limit applies to all processes and users an +cannot be exceeded other limit values. + +Checking limit +************** + +You can check a current value with :command:`sysctl -n `. For +example: + +.. code:: bash + + sysctl -n fs.file-max + + +This *fs.file-max* value is set intentionally high on |CL| systems by +default. You can check the maximum value supported by the system with: + +.. code:: + + cat /proc/sys/fs/file-max + + +Overriding limit +**************** + +You can override a value with :command:`sysctl -w `. For +example: + +.. code:: bash + + sudo sysctl -w fs.file-max= + +If needed permanently, the value can be set by creating a +:file:`/etc/sysctl.d/*.conf` file (see :command:`man sysctl.d` for details). +For example: + +.. code:: bash + + sudo mkdir -p /etc/sysctl.d/ + + sudo tee /etc/sysctl.d/fs-file-max.conf > /dev/null <<'EOF' + fs.file-max= + EOF + + + + + +Per-user limits +=============== + +For processes not managed by systemd, resource limits can be set for PAM +logins on a per-user basis with upper and lower limits in the +:file:`/etc/security/limits.conf` file. + +You can set temporary values and check the current values with the +:command:`ulimit` command. For example, to change the soft limit of maximum +number of open file descriptors for the current user: + +.. code:: + + ulimit -S -n + +See :command:`man limits.conf` for details. + + +Service limits +============== + +Resource limits for services started with systemd units do not follow normal +user limits because the process is started in a separate `Linux control group +(cgroup) `_ Linux +cgroups associate related process groups and provide resource accounting. + +Resource limits for individual systemd services can be controlled inside their +unit files or its configuration drop-in directory with the resource Limit +directives. See `process properties section of the systemd.exec man page +`_. + +Resource limits for all systemd services can be controlled with a file in the +:file:`/etc/systemd/system.conf.d/` directory. For example, to have no +restriction on the number of open files: + +.. code:: + + sudo mkdir -p /etc/systemd/system.conf.d/ + + sudo tee /etc/systemd/system.conf.d/50-nfiles.conf > /dev/null <<'EOF' + [Manager] + DefaultLimitNOFILE=infinity + EOF diff --git a/_sources/guides/maintenance/restart.rst.txt b/_sources/guides/maintenance/restart.rst.txt new file mode 100644 index 00000000..ddd4a17c --- /dev/null +++ b/_sources/guides/maintenance/restart.rst.txt @@ -0,0 +1,164 @@ +.. _restart: + +Restart system services after an OS update +########################################## + +This guide describes how to use the :command:`clr-service-restart` tool. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +|CL-ATTR| includes a :command:`clr-service-restart` tool that shows which +system daemons require a restart. + +:command:`clr-service-restart` reads various files in the :file:`procfs` +filesystem provided by the kernel and relies on :command:`systemd` to +determine which services to restart. + + +How it works +************ + +:command:`clr-service-restart` implements a whitelist to identify which +daemons can be restarted. As a system administrator, you can customize the +default |CL| OS whitelist using :command:`allow` or :command:`disallow` options +for restarting system services. When a software update occurs, +:command:`clr-service-restart` consults the whitelist to see if a service daemon +is allowed to be restarted or not. + + +Basic options +************* + +:command:`clr-service-restart` has three basic options: :command:`allow`, +:command:`disallow`, and :command:`default`. + +allow +===== + +The :command:`allow` option identifies a daemon to restart after an OS software +update. The :command:`clr-service-restart` daemon creates a symlink in +:file:`/etc/clr-service-restart` as a record. The example below tells +:command:`clr-service-restart` to restart the *tallow* daemon after an +OS software update. + +.. code-block:: bash + + sudo clr-service-restart allow tallow.service + +disallow +======== + +The :command:`disallow` option tells :command:`clr-service-restart` not to +restart the specified daemon even if the OS defaults permit the daemon to be +restarted. The :command:`clr-service-restart` daemon creates a symlink in +:file:`/etc/clr-service-restart` that points to :file:`/dev/null` as a +record. The example below tells :command:`clr-service-restart` not to +restart the *rngd* daemon after an OS software update. + +.. code-block:: bash + + sudo clr-service-restart disallow rngd + +default +======= + +The :command:`default` option makes :command:`clr-service-restart` revert back +to the OS defaults and delete any symlink in :file:`/etc/clr-service-restart`. +The example below tells :command:`clr-service-restart` to restart *rngd* +automatically again, because *rngd* is whitelisted for automatic service +restarts by default in |CL|. + +.. code-block:: bash + + sudo clr-service-restart default rngd + +Monitor options +*************** + +:command:`clr-service-restart` works in the background and is invoked with +:command:`swupd` automatically. Review the journal output to verify that +services are restarted after an OS software update. + +If you pass both options (:command:`-a` and :command:`-n`) described below, +:command:`clr-service-restart` displays a complete list of system services +that require a restart. Use both options to verify that all desired daemons +are restarted. + + +-n option +========= + +The :command:`-n` option makes :command:`clr-service-restart` perform no restarts. +Instead it displays the services that could potentially be restarted. When used, +:command:`clr-service-restart` outputs a list of messages showing: + +* Which service needs a restart. +* What unit it is. +* Why it needs a restart. +* Which command is required to restart the unit. + +-a option +========= + +The :command:`-a` option makes :command:`clr-service-restart` consider all system +services, not only the ones that are whitelisted. Because the default whitelist +in |CL| is relatively short, you can use this option to restart all impacted +services when you log in on the system. + +Example +******* + +In the example below, :command:`clr-service-restart` is invoked with both the +:command:`-a` and :command:`-n` options, which displays a complete list of system +services that require a restart. + +Command: + +.. code-block:: bash + + sudo clr-service-restart -a -n + +Sample output: + +.. code-block:: console + + upower.service: needs a restart (a library dependency was updated) + /usr/bin/systemctl --no-ask-password try-restart upower.service + NetworkManager.service: needs a restart (a library dependency was + updated) + /usr/bin/systemctl --no-ask-password try-restart NetworkManager.service + .... + +Telemetry +********* + +:command:`clr-service-restart` may cause problems such as a short service +outage when a daemon is being restarted, or if a daemon fails to properly +restart. To minimize issues, :command:`clr-service-restart` creates a +telemetry record and sends it to the optional |CL| telemetry service if both +conditions below are met: + +* If a unit fails to automatically restart after an OS update. +* If that unit resides in the system location :file:`/usr/lib/systemd/system`. + +If you do not install the |CL| telemetrics bundle, the data is discarded. If +you install the telemetrics bundle and you opt to send telemetry, then the +system unit name is sent to the |CL| telemetry service. We evaluate the +report and update the whitelist to remove services that are not safe to +restart. + + +Conclusion +********** + +The |CL| team enjoys coming up with simple and efficient solutions to make +your work easier. We made a GitHub\* project of :command:`clr-service-restart` +and we invite you to look at the code, share your thoughts, and work with us +on improving the project. You can find the project at: + +https://github.com/clearlinux/clr-service-restart diff --git a/_sources/guides/maintenance/validate-signatures.rst.txt b/_sources/guides/maintenance/validate-signatures.rst.txt new file mode 100644 index 00000000..f9d2fd21 --- /dev/null +++ b/_sources/guides/maintenance/validate-signatures.rst.txt @@ -0,0 +1,139 @@ +.. _validate-signatures: + +Validate signatures +################### + + +This guide describes how to validate the contents of a |CL-ATTR| image. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +Validating the contents of an image is a manual process and is the same process +that :ref:`swupd-guide` performs internally. + +|CL| offers a way to validate the content of an image or an update. All +validation of content works by creating and signing a hash. A valid signature +creates a chain of trust. A broken chain of trust, seen as an invalid +signature, means the content is not valid. + + +.. _image-content-validation: + +Image content validation +************************ + +In the steps below, we used the installer image of the latest release +of |CL|. You may use any image of |CL| you choose. + +#. Download the image, the signature of the SHA512 sum of the image, and the + |CL| certificate used for signing the SHA512 sum. + + .. code-block:: console + + # Image + curl -O https://cdn.download.clearlinux.org/current/clear-$(curl https://cdn.download.clearlinux.org/latest)-installer.img.xz + # Signature of SHA512 sum of image + curl -O https://cdn.download.clearlinux.org/current/clear-$(curl https://cdn.download.clearlinux.org/latest)-installer.img.xz-SHA512SUMS.sig + # Certificate + curl -O https://cdn.download.clearlinux.org/releases/$(curl https://cdn.download.clearlinux.org/latest)/clear/ClearLinuxRoot.pem + +#. Generate the SHA256 sum of the |CL| certificate. + + .. code-block:: console + + sha256sum ClearLinuxRoot.pem + +#. Ensure the generated SHA256 sum of the |CL| certificate matches the + following SHA256 sum to verify the integrity of the certificate. + + .. code-block:: console + + 4b0ca67300727477913c331ff124928a98bcf2fb12c011a855f17cd73137a890 ClearLinuxRoot.pem + +#. Generate the SHA512 sum of the image and save it to a file. + + .. code-block:: console + + sha512sum clear-$(curl https://cdn.download.clearlinux.org/latest)-installer.img.xz > sha512sum.out + +#. Ensure the signature of the SHA512 sum of the image was created using the + |CL| certificate. This confirms that the image is trusted and has not + been modified. + + .. code-block:: console + + openssl smime -verify -purpose any -in clear-$(curl https://cdn.download.clearlinux.org/latest)-installer.img.xz-SHA512SUMS.sig -inform der -content sha512sum.out -CAfile ClearLinuxRoot.pem + + .. note:: + + The :command:`-purpose any` option is required when using OpenSSL 1.1. + If you use an earlier version of OpenSSL, omit this option to perform + signature validation. The :command:`openssl version` command may be used + to determine the version of OpenSSL in use. + +#. The output should contain "Verification successful". If the output + contains "bad_signature" anywhere, then the image is not trustworthy. + +Update content validation +************************* + +**swupd** validates all update content automatically before applying the +update content. The process swupd follows internally is illustrated here +with manual steps using the latest |CL| release. There is no need to perform +these steps manually when performing a :command:`swupd update`. + +#. Download the :abbr:`MoM (top-level manifest)`, the signature of the MoM, + and the Swupd certificate used for signing the signature of the MoM. + + .. code-block:: console + + # MoM + curl -O https://cdn.download.clearlinux.org/update/$(curl https://cdn.download.clearlinux.org/latest)/Manifest.MoM + # Signature of MoM + curl -O https://cdn.download.clearlinux.org/update/$(curl https://cdn.download.clearlinux.org/latest)/Manifest.MoM.sig + # Swupd certificate + curl -O https://cdn.download.clearlinux.org/releases/$(curl https://cdn.download.clearlinux.org/latest)/clear/Swupd_Root.pem + +#. Generate the SHA256 sum of the swupd certificate. + + .. code-block:: console + + sha256sum Swupd_Root.pem + +#. Confirm that the generated SHA256 sum of the swupd certificate matches the + SHA256 sum shown below to verify the integrity of the certificate. + + .. code-block:: console + + ff06fc76ec5148040acb4fcb2bc8105cc72f1963b55de0daf3a4ed664c6fe72c Swupd_Root.pem + +#. Confirm that the signature of the MoM was created using the Swupd + certificate. This signature validates the update content is trustworthy and + has not been modified. + + .. code-block:: console + + openssl smime -verify -purpose any -in Manifest.MoM.sig -inform der -content Manifest.MoM -CAfile Swupd_Root.pem + + .. note:: + + The :command:`-purpose any` option is required when using OpenSSL 1.1. + If you use an earlier version of OpenSSL, omit this option to perform + signature validation. The :command:`openssl version` command may be used + to determine the version of OpenSSL in use. + + .. note:: + + The SHA512 sum of the MoM is not generated and then signed. Instead, the + MoM is signed directly because it is small in size compared to an image of + |CL|. + +#. The output should contain "Verification successful". If the output + contains "bad_signature" anywhere, then the MoM cannot be trusted. + Because the MoM contains a list of hashes for bundle manifests, if the MoM + cannot be trusted, then the bundle content cannot be trusted. diff --git a/_sources/guides/network/assign-static-ip.rst.txt b/_sources/guides/network/assign-static-ip.rst.txt new file mode 100644 index 00000000..6c259afe --- /dev/null +++ b/_sources/guides/network/assign-static-ip.rst.txt @@ -0,0 +1,184 @@ +.. _assign-static-ip: + +Assign a static IP address +########################## + + +This guide explains how to assign a static IP address. This may be helpful in +scenarios such as a network with no DHCP server. + +.. contents:: + :local: + :depth: 1 + +Identify which program is managing the interface +************************************************ + +New installations of |CL-ATTR| use NetworkManager as the default network +interface manager for all network connections. + +.. note:: + + * The cloud |CL| images continue to use `systemd-networkd` to manage + network connections. + + * In earlier |CL| versions, `systemd-network` was used to manage Ethernet + interfaces and NetworkManager was used for wireless interfaces. + + +Before defining a configuration for assigning a static IP address, verify +which program is managing the network interface. + +#. Check the output of :command:`nmcli device` to see if NetworkManager is + managing the device. + + .. code-block:: bash + + nmcli device status + + If the STATE column for the device shows *connected* or *disconnected*, the + network configuration is being managed by NetworkManager, then use the + instructions for :ref:`using NetworkManager `. + + If the STATE column for the device shows *unmanaged*, then check if the + device is being managed by systemd-networkd. + + +#. Check the output of :command:`networkctl list` to see if + `systemd-networkd` is managing the device. + + .. code-block:: bash + + networkctl list + + If the SETUP column for the device shows *configured*, the network + configuration is being managed by `systemd-networkd`, then use the + instructions for :ref:`using systemd-networkd `. + + +.. _nm-static-ip: + +Using NetworkManager +******************** + +Network connections managed by NetworkManager are stored as files with the +:file:`.nmconnection` file extension in the +:file:`/etc/NetworkManager/system-connections/` directory. + +A few tools exists to aid to manipulate network connections managed by +NetworkManager: + +* nmcli - a command-line tool + +* nmtui - a text user interface that provides a pseudo graphical menu in the + terminal + +* nm-connection-editor - a graphical user interface + +The method below uses the command line tool nmcli to modify network +connection. + + +#. Identify the existing connection name: + + .. code:: bash + + nmcli connection show + + Sample output: + + .. code:: bash + + NAME UUID TYPE DEVICE + Wired connection 1 00000000-0000-0000-0000-000000000000 802-3-etherneten01 + + If a connection does not exist, create it with the + :command:`nmcli connection add` command. + + +#. Modify the connection to use a static IP address. Replace the variables in + brackets with the appropriate values. Replace *[CONNECTION_NAME]* with the + NAME from the command above. + + .. code:: + + sudo nmcli connection modify "[CONNECTION_NAME]" \ + ipv4.method "manual" \ + ipv4.addresses "[IP_ADDRESS]/[CIDR_NETMASK]" \ + ipv4.gateway "[GATEWAY_IP_ADDRESS]" \ + ipv4.dns "[PRIMARY_DNS_IP],[SECONDARY_DNS_IP]" + + + See the `nmcli developer page `_ for more + configuration options. For advanced configurations, the + :file:`/etc/NetworkManager/system-connections/*.nmconnection`. can be edited + directly. + + +#. Restart the NetworkManager server to reload the DNS servers: + + .. code-block:: bash + + sudo systemctl restart NetworkManager + + +#. Verify your static IP address details have been set: + + .. code-block:: bash + + nmcli + + + +.. _networkd-static-ip: + +Using systemd-networkd +********************** + +Network connections managed by systemd-networkd are stored as files with the +:file:`.network` file extension the :file:`/etc/systemd/network/` directory. + +Files to manipulate network connections managed by systemd-networkd must be +created manually. + +#. Create the :file:`/etc/systemd/network` directory if it does not already exist: + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/network + +#. Create a :file:`.network` file and add the following content. Replace the + variables in brackets with the appropriate values. Replace *[INTERFACE_NAME]* + with LINK from the output of the :command:`networkctl list` command that was + run previously. + + .. code-block:: bash + + sudo $EDITOR /etc/systemd/network/70-static.network + + [Match] + Name=[INTERFACE_NAME] + + [Network] + Address=[IP_ADDRESS]/[CIDR_NETMASK] + Gateway=[GATEWAY_IP_ADDRESS] + DNS=[PRIMARY_DNS_IP] + DNS=[SECONDARY_DNS_IP] + + See the `systemd-network man page + `_ + for more configuration options. + +#. Restart the `systemd-networkd` service: + + .. code-block:: bash + + sudo systemctl restart systemd-networkd + + +#. Verify your static IP address details have been set: + + .. code-block:: bash + + networkctl status + diff --git a/_sources/guides/network/dpdk.rst.txt b/_sources/guides/network/dpdk.rst.txt new file mode 100644 index 00000000..5072c4ad --- /dev/null +++ b/_sources/guides/network/dpdk.rst.txt @@ -0,0 +1,352 @@ +.. _dpdk: + +Use DPDK to send packets between platforms +########################################## + +This guide describes how to send packets between two platforms. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +Figure 1 shows how to send packets between two platforms in a simple +configuration. The example uses the :abbr:`Data Plane Development Kit (DPDK)`, +which is a set of libraries, drivers, sample applications, and tools for fast +packet processing. + +.. figure:: ./figures/pktgen_lw3fd.png + :align: center + :alt: Platform A and B + + Figure 1: Environment for l3fwd DPDK application + +This example uses the following DPDK components: + +* pktgen: Traffic generator. See `pktgen`_ documentation for details. +* l3fwd: Layer 3 forwarding example application. See + `l3fwd`_ documentation for details. + +Prerequisites +************* + +* Two platforms using |CL-ATTR| release `31130`_ or higher. +* Both images must include the :command:`kernel-native` bundle. +* Install the following packages: + + .. code-block:: bash + + sudo swupd bundle-add network-basic-dev dpdk devpkg-dpdk + +* Each platform must have at least one :abbr:`NIC (Network Interface Card)`. + Check the `DPDK project`_ for the list of supported `dpdk.org NICs`_. + +* Two network cables. + +Install dpdk and build l3fwd example (Platform B) +************************************************* + +#. Change to the :file:`l3fwd` example directory. + + .. code-block:: bash + + sudo cd /usr/share/dpdk/examples/l3fwd + +#. Assign :envvar:`RTE_SDK` variable to the makefiles path. + + .. code-block:: bash + + sudo export RTE_SDK=/usr/share/dpdk/ + +#. Assign :envvar:`RTE_TARGET` variable to the location of the gcc\* config + file. + + .. code-block:: bash + + sudo export RTE_TARGET=x86_64-native-linux-gcc + +#. Build the `l3fwd` application and add the configuration header to + the :makevar:`CFLAGS` variable. + + .. code-block:: bash + + sudo make + + +Build pktgen (Platform A) +************************* + +#. Download the `pktgen tar package`_ v3.1.2 or newer. + +#. Decompress packages and move to uncompressed source directory. + +#. Assign :envvar:`RTE_SDK` variable to the path where makefiles are located. + + .. code-block:: bash + + sudo export RTE_SDK=/usr/share/dpdk/ + +#. Assign :envvar:`RTE_TARGET` to the location of the gcc config file. + + .. code-block:: bash + + sudo export RTE_TARGET=x86_64-native-linux-gcc + +#. Build the `pktgen` project and set the :makevar:`CONFIG_RTE_BUILD_SHARED_LIB` variable + to "n". + + .. code-block:: bash + + sudo make CONFIG_RTE_BUILD_SHARED_LIB=n + +Bind NICs to DPDK kernel drivers (Platforms A and B) +**************************************************** + +The `l3fwd` application uses two NICs. The DPDK includes tools for binding +NICs to DPDK modules to run DPDK applications. + +#. Load the DPDK I/O kernel module. + + .. code-block:: bash + + sudo modprobe vfio-pci + +#. Check the NIC status to determine which network cards are not + busy. When another application is using them, the status shows "Active", + and those NICs cannot be bound. + + .. code-block:: bash + + sudo dpdk-devbind --status + +#. Bind two available NICs. The general syntax for binding is: + :command:`dpdk-devbind --bind=vfio-pci `. + A working example is shown below: + + .. code-block:: bash + + sudo dpdk-devbind --bind=vfio-pci 01:00.0 + +#. Check the NIC status to verify that the NICs are bound correctly. If + successful, `drv` displays the value `igb_uio`, which confirms + that the NICs are using the DPDK modules. + + +Set hugepages (Platforms A and B) +********************************* + +|CL| supports `hugepages` for the large memory pool allocation used for +packet buffers. + +#. Set the number of hugepages. + + .. code-block:: bash + + echo 1024 | sudo tee /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages + +#. Allocate pages on NUMA machines. + + .. code-block:: bash + + echo 1024 | sudo tee /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages + echo 1024 | sudo tee /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages + +#. Make memory available for DPDK. + + .. code-block:: bash + + sudo mkdir -p /mnt/huge $ mount -t hugetlbfs nodev /mnt/huge + + For more information, refer to the `DPDK guide`_ System Requirements + section. + + +Set up the physical environment (Platforms A and B) +*************************************************** + +Connect the NICs on Platform A to the NICs on Platform B using the network +cables as shown in figure 2. + +.. figure:: ./figures/pyshical_net.png + + Figure 2: Physical network environment + + +Run l3fwd application (Platform B) +********************************** + +The `l3fwd` application is one of the DPDK examples available when you +install the :file:`dpdk-dev` bundle. `l3fwd` forwards packets from one +NIC to another. For details, refer to the `l3fwd`_ documentation. + +#. Open the l3fwd example directory. + + .. code-block:: bash + + sudo cd /usr/share/dpdk/examples/l3fwd + +#. **This step is very important.** + + #. DPDK needs poll mode drivers to operate. + #. Poll mode drivers are shared objects in :file:`/usr/lib64`. + #. See the full list of supported NICs at `dpdk.org NICs`_. + #. You must know which kernel module each NIC is using and choose a poll + mode driver that corresponds to your NICs. + +#. NIC binding and `pktgen` configuration depends upon network use cases and + available system resources. Use the :command:`-d` flag to set the poll mode + driver. + + The following example assumes that the NICs use the `e1000` network driver + and the `e1000` poll mode driver. The :file:`librte_pmd_e1000.so` is + located in :file:`/usr/lib64` in |CL|. + + .. code-block:: bash + + sudo ./build/l3fwd -c 0x3 -n 2 -d librte_pmd_e1000.so -- -p 0x3 --config="(0,0,0),(1,0,1)" + +#. The `l3fwd` application shows port initialization details at startup. + After port 0 initialization completes, `l3fwd` shows a MAC address and + information for port 1. + + Save the MAC address for configuring the `pktgen` project. + +Run pktgen application (Platform A) +*********************************** + +`pktgen` is a network traffic generator included in the DPDK. + +#. `pktgen` configuration depends upon the network setup and the + available system resources. The following example shows a basic + configuration. + + .. code-block:: bash + + sudo ./app/app/x86_64-native-linux-gcc/pktgen -c 0xf -n 4 -- -p 0xf -P -m "1.0, 2.1" + +#. Enable active colorful output (optional). + + .. code-block:: bash + + Pktgen> theme enable + +#. Use the MAC addresses shown by the `l3fwd` application during initialization. + The command to set the MAC addresses in `pktgen` has the format: + + .. code-block:: bash + + set mac + + Here is a working example: + + .. code-block:: bash + + Pktgen> set mac 0 00:1E:67:CB:E8:C9 + Pktgen> set mac 1 00:1E:67:CB:E8:C9 + +#. Send packets. + + .. code-block:: bash + + Pktgen> start 0-1 + +For more details, see the `pktgen`_ documentation. + +Appendix A: Use pass-through for virtual machines +************************************************* + +This section explains how to set up a virtual environment where virtual +machines control the NICs on the host. + +#. Create a new directory and move to it. + +#. Download or create a :file:`start_qemu.sh` script for running a kvm virtual + machine: + + .. code-block:: bash + + sudo curl -O https://cdn.download.clearlinux.org/image/start_qemu.sh + +#. Download a bare-metal image of |CL| and rename it as :file:`clear.img`. + +#. Look for an Ethernet\* device entry that contains vendor and device ID: + + .. code-block:: bash + + sudo lspci -nn | grep Ethernet + + An example output: + + .. code-block:: console + + 03:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] + + where `03:00.0` is the device entry and `8086:1521` is the `vendor:device + ID`. Record this information, because you need it to unbind the NICs from a + host. + + +#. Unbind the NICs from the host to do pass-through with virtual machines. |CL| + supports this action. The commands take the format: + + .. code-block:: bash + + echo "vendor device_ID" > /sys/bus/pci/drivers/pci-stub/new_id + echo "entry for device" > /sys/bus/pci/drivers/igb/unbind + echo "entry for device" > /sys/bus/pci/drivers/pci-stub/bind + echo "vendor device_ID" > /sys/bus/pci/drivers/pci-stub/remove_id + + Here is a working example: + + .. code-block:: bash + + echo "8086 1521" | sudo tee /sys/bus/pci/drivers/pci-stub/new_id + echo "0000:03:00.0" | sudo tee /sys/bus/pci/drivers/igb/unbind + echo "0000:03:00.0" | sudo tee /sys/bus/pci/drivers/pci-stub/bind + echo "8086 1521" | sudo tee /sys/bus/pci/drivers/pci-stub/remove_id + +#. Assign the unbound NICs to the KVM virtual machine (guest). + Modify the :file:`start_qemu.sh` script in `qemu-system-x86_64` arguments, and + add the lines with the host's NICs information in the format: + + .. code-block:: bash + + -device pci-assign,host="",id=passnic0,addr=03.0 + -device pci-assign,host="",id=passnic1,addr=04.0 + + Here is a working example: + + .. code-block:: bash + + -device pci-assign,host=03:00.0,id=passnic0,addr=03.0 \ + -device pci-assign,host=03:00.3,id=passnic1,addr=04.0 \ + +#. Add more NUMA machines to the virtual machine by adding lines to the + Makefile boot target in the format: + + .. code-block:: bash + + -numa node,mem=,cpus= + + Here is a working example for a virtual machine with 4096 memory and four + CPUs: + + .. code-block:: bash + + -numa node,mem=2048,cpus=0-1 \ + -numa node,mem=2048,cpus=2-3 \ + + .. note:: Each NUMA machine must use the same quantity of memory. + +#. Run the :file:`start_qemu.sh` script. + + +.. _31130: https://cdn.download.clearlinux.org/releases/31130/clear/ +.. _DPDK project: http://dpdk.org +.. _dpdk.org NICs: http://dpdk.org/doc/nics +.. _pktgen tar package: http://dpdk.org/browse/apps/pktgen-dpdk/refs +.. _DPDK guide: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html +.. _l3fwd: http://dpdk.org/doc/guides/sample_app_ug/l3_forward.html +.. _pktgen: http://pktgen-dpdk.readthedocs.io/en/latest/index.html diff --git a/_sources/guides/network/firewall.rst.txt b/_sources/guides/network/firewall.rst.txt new file mode 100644 index 00000000..fe8fd446 --- /dev/null +++ b/_sources/guides/network/firewall.rst.txt @@ -0,0 +1,201 @@ +.. _firewall: + +Firewall +######### + +Firewalls control access to and from systems based on network packet +attributes like IP address, port, payload and more. + +The `Netfilter `_ framework in the Linux kernel +performs packet filtering and provides the means for implementing a software +firewall in Linux. |CL-ATTR| has a couple different firewall front-end options +for managing the Linux firewall. + + +.. contents:: :local: + :depth: 2 + + +Default ruleset +*************** + +|CL| does not impose a firewall policy out of the box. All traffic is allowed +inbound and all traffic is allowed outbound. However, `tallow`_ is installed +by default and may dynamically create a rule temporarily restricting access +from external hosts. + +.. warning:: + + Changing firewall configuration can cause abrupt network disconnection. If + this happens on a remote host, local recovery may be required. + + Be sure to test your firewall configuration before committing it + permanently to ensure your system will remain accessible remotely, if + required. + +Firewall software +***************** + +iptables +======== + +:command:`iptables` is a well-known user-space administration tool for +configuring IPv4 Linux firewall rules. :command:`ip6tables` is the +complimentary tool for configuring IPv6 Linux firewall rules. + +Below is information on using :command:`iptables` on |CL|: + +#. Make sure the *iptables* bundle is installed + + .. code:: bash + + sudo swupd bundle-add iptables + + +#. Define new iptables rules/chains for the running configuration using the + :command:`iptables` command. See :command:`man iptables` for iptables + concepts and configuration options. + + Below is a common restrictive firewall configuration which denies all + incoming connections, unless the connection was initiated by the host. + + .. code:: bash + + # Set default chain policies + sudo iptables -P INPUT DROP + sudo iptables -P FORWARD DROP + sudo iptables -P OUTPUT ACCEPT + + # Accept on localhost loopback device + sudo iptables -A INPUT -i lo -j ACCEPT + sudo iptables -A OUTPUT -o lo -j ACCEPT + + # Allow established sessions to receive traffic + sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT + + +#. Test the running firewall configuration to make sure it behaving as + you expect. + +#. Run the *iptables-save* service to make the running configuration + persistent. This will perform a one-time save of the running configuration + to :file:`/etc/iptables.rules` : + + .. code:: bash + + sudo systemctl start iptables-save + +#. Enable the *iptables-resolve* service the iptables rules to be + automatically applied at boot from the :file:`/etc/iptables.rules` file: + + .. code :: bash + + sudo systemctl enable iptables-restore.service + + +ipset +===== + +`ipset `_ is a framework in the Linux kernel for +storing and efficiently indexing combinations of IP addresses, networks, +(TCP/UDP) port numbers, MAC addresses, and interface names. + +IP sets makes writing network policy rules simpler and processing them against +a large and/or changing sets of hosts more efficient. + +By themselves, IP sets do not enforce network traffic rules but can be used to +extend iptables rules for matching. It is important to note that the ipset +must be defined before a netfilter rule can match against it. + +* Running IP sets can be manipulated with the :command:`ipset` utility. + +* Custom IP sets can be stored in the :file:`/etc/ipset.conf` file + +* IP sets in :file:`/etc/ipset.conf` can be automatically applied at boot by + enabling the *ipset* service with the command :command:`sudo systemctl + enable ipset`. + +See :command:`man ipset` to learn more about using ipsets. + + +firewalld +========= + +`firewalld `_ is based on nftables, the successor to +iptables and parts of the netfilter framework. The description of firewalld +helps highlight some of the differences compared to iptables: + + firewalld provides a dynamically managed firewall with support for + network/firewall zones to define the trust level of network connections or + interfaces. It has support for IPv4, IPv6 firewall settings and for + ethernet bridges and has a separation of runtime and permanent + configuration options. It also supports an interface for services or + applications to add firewall rules directly. + +See :command:`man firewalld` for more information. + +Below is information on using :command:`firewalld` on |CL|: + +#. Install he *firewalld* bundle: + + .. code:: bash + + sudo swupd bundle-add firewalld + + +#. Disable *iptables* and *ipset* services as they conflict with firewalld: + + .. code:: + + sudo systemctl mask iptables-restore ipset + + +#. :command:`firewall-cmd` can be used to configure the running or permanent + firewall configuration. See the `firewalld documentation + `_ to learn more about + firewalld concepts and configuration options. + + Below is a common example to allow HTTPS traffic in public zones: + + .. code:: + + sudo firewall-cmd --permanent --zone=public --add-service=https + + +#. Enable the *firewalld* service the so that the firewalld daemon is + automatically started and rules applied at boot from the + :file:`/etc/firewalld/*` file: + + .. code :: bash + + sudo systemctl enable --now firewalld.service + + +#. Verify that firewalld is running: + + .. code :: bash + + sudo firewall-cmd --state + + + +Troubleshooting +*************** + +When troubleshooting connectivity issues that may be related to firewall +rules. + +* Consider restrictions at the physical network level. + +* For inbound connections, make sure your application is listening on the + network port you're expecting with :command:`lsof` or :command:`netstat`. + +* For outbound connections, make sure the destination host is responding to + the network port you're expecting with :command:`nc`. If the connection is + refused, then there may be a problem with the destination server. + +* If you're using firewalld, check the daemon status with the command: + :command:`systemctl status firewalld`. + + +.. _`tallow`: https://github.com/clearlinux/tallow diff --git a/_sources/guides/network/network-bonding.rst.txt b/_sources/guides/network/network-bonding.rst.txt new file mode 100644 index 00000000..41ba15b9 --- /dev/null +++ b/_sources/guides/network/network-bonding.rst.txt @@ -0,0 +1,130 @@ +.. _network-bonding: + +Combine multiple interfaces with network bonding +################################################ + +This guide describes how to configure systemd to use the :command:`bonding` +driver. + +Network bonding combines multiple network interfaces into a single logical +interface to provide redundancy and bandwidth aggregation. + +|CL-ATTR| includes the Linux `Bonding driver`_ and `Team driver`_ . + +The example demonstrates how to: + +* Bond all four ports of a quad-port NIC in 802.3ad mode. + +* Enable jumbo frames to optimize large data transfers on the local network. + +Your NICs and network switch must support 802.3ad mode and jumbo frames. The +example explains how to configure your NICs for both features. Your switch may +require additional configuration. See your switch documentation for details. + +.. note:: + + You must run all commands in this guide as root. + +#. Log in and get root privileges. + + .. code-block:: console + + sudo -s + +#. Create the :file:`/etc/systemd/network` directory. + + .. code-block:: bash + + mkdir -p /etc/systemd/network + + The :file:`/etc/systemd/network` directory contains configuration files and + network settings for the virtual device and its underlying physical + interfaces. + +#. Configure systemd to create a virtual network device called `bond1`. Use a + text editor to create a file named :file:`30-bond1.netdev`. + + .. code-block:: console + + [NetDev] + Name=bond1 + Kind=bond + + [Bond] + Mode=802.3ad + TransmitHashPolicy=layer3+4 + MIIMonitorSec=1s + LACPTransmitRate=fast + + Refer to the `systemd.netdev`_ manpage for :file:`30-bond1.netdev` file + syntax. This example is based on Example 9 on the manpage. Modify the + example for your configuration. + +#. Configure the slave interfaces. Create a text file named + :file:`30-bond1-enp1s0.network`. Assign the slave interfaces to the virtual + `bond1` device and use the syntax shown in `systemd.network`_. + + .. code-block:: console + + [Match] + Name=enp1s0f* + + [Network] + Bond=bond1 + + [Link] + MTUBytes=9000 + + The example bonds all four ports of a quad-port NIC as a slave of `bond1`. + The example uses a wildcard match because the NIC names are in the range + `enp1s0f0-enp1s0f3`. If your NIC names are not wildcard-compatible, create + a separate :file:`.network` file for each NIC. + + For best results, do not assign addresses or DHCP support to the individual + NICs. + + The `MTUBytes` setting enables jumbo frames of up to 9000 bytes. Your + switch may require additional configuration to support this setting. + +#. Configure the bonded interface in a file named :file:`30-bond1.network`. + + .. code-block:: console + + [Match] + Name=bond1 + + [Network] + BindCarrier=enp1s0f0 enp1s0f1 enp1s0f2 enp1s0f3 + Address=192.168.1.201/24 + + [Link] + MTUBytes=9000 + + `bond1` is a virtual interface with no physical link status. + + `BindCarrier` indicates that the `bond1` link status is determined by the + status of the listed slave devices. + + `Address` contains an IP address that you assign to the logical interface. + DHCP bonded interfaces are complex and outside the scope of this example. + + `MTUBytes` must be set to 9000 on all slave interfaces and on the bonded + interface for successful jumbo frames operation. If `MTUBytes` is not the + same on all interfaces, then the lowest value is used. + +#. Apply the new network configuration with the command: + + .. code-block:: bash + + systemctl restart systemd-networkd + + The `MTUBytes` settings do not take effect until you reboot or manually + apply the settings with a utility such as ifconfig. + +.. _Bonding driver: https://www.kernel.org/doc/Documentation/networking/bonding.txt + +.. _Team driver: https://www.kernel.org/doc/Documentation/networking/team.txt + +.. _systemd.netdev: https://www.freedesktop.org/software/systemd/man/systemd.netdev.html + +.. _systemd.network: https://www.freedesktop.org/software/systemd/man/systemd.network.html diff --git a/_sources/guides/network/openssh-server.rst.txt b/_sources/guides/network/openssh-server.rst.txt new file mode 100644 index 00000000..664ff6e3 --- /dev/null +++ b/_sources/guides/network/openssh-server.rst.txt @@ -0,0 +1,159 @@ +.. _openssh-server: + +Enable and configure SSH service +################################ + +This guide describes how to set up the SSH service. + +.. contents:: + :local: + :depth: 1 + +Overview +******** + +The :command:`openssh-server` bundle provides the OpenSSH package that +enables an SSH service in |CL-ATTR|. Remote users require an SSH service to be +able to use an encrypted login shell. The SSH daemon has all of its configuration built in and no template configuration file is present on the file system. + +|CL| enables the `sshd.socket` unit, which listens on port 22 by default +and starts the OpenSSH service as required. The first time OpenSSH starts, it +generates the server SSH keys needed for the service. + +Prerequisites +************* + +Ensure the :command:`openssh-server` bundle is installed. + +To list all bundles on your host, enter: + +.. code-block:: bash + + sudo swupd bundle-list + +To add the :command:`openssh-server` bundle, enter: + +.. code-block:: bash + + sudo swupd bundle-add openssh-server + +Change default port +******************* + +Perform the following steps to change the default listening port for the +OpenSSH service. + +#. Open the :file:`sshd.socket` file: + + .. code-block:: bash + + sudo systemctl edit sshd.socket + +#. Add the `[Socket]` section and `ListenStream` option to the sshd.socket + file as shown below. The first `ListenStream` entry removes the |CL| + default listen port value. The second `ListenStream` entry sets the new + default listen port value. In this example, we set the new default port + to 4200: + + .. code-block:: console + + [Socket] + ListenStream= + ListenStream=4200 + + + Make sure to include a new line after the last line of text in the :file:`sshd.socket` file. + +#. Verify your changes: + + .. code-block:: bash + + cat /etc/systemd/system/sshd.socket.d/override.conf + + The following output is displayed: + + .. code-block:: console + + [Socket] + ListenStream= + ListenStream=4200 + +#. Reload the systemd daemon configurations: + + .. code-block:: bash + + sudo systemctl daemon-reload + +#. Restart the sshd.socket unit: + + .. code-block:: bash + + sudo systemctl restart sshd.socket + +#. Confirm the sshd.socket unit is listening on your new port: + + .. code-block:: bash + + sudo systemctl status sshd.socket + + .. note:: + + Output should show :guilabel:`Active:` as `active(listening)`. + +Enable SFTP +*********** + +|CL| *disables* the :abbr:`SFTP (SSH File Transfer Protocol)` subsystem by +default due to security considerations. To enable the SFTP subsystem, you can +configure the :file:`/etc/ssh/sshd_config` file. + +#. Create the following file, if it does not already exist: + :file:`/etc/ssh/sshd_config` + +#. Add the the SFTP subsystem in :file:`/etc/ssh/sshd_config`: + + .. code-block:: console + + subsystem sftp /usr/libexec/sftp-server + + +Congratulations! The SFTP subsystem is enabled. You do not need to restart the sshd service. + +Enable root login +***************** + +To enable root login via SSH, perform the following steps: + + +#. Create the following file, if it does not already exist: + :file:`/etc/ssh/sshd_config` + +#. Set the configuration variable in :file:`/etc/ssh/sshd_config`: + + .. code-block:: console + + PermitRootLogin yes + +You have now enabled root login on your system. You do not need to restart the sshd service. + +Enable X11-forwarding +********************* + +X11 forwarding allows you to securely run graphical applications (that is, X +clients) over the SSH connection. This enables remote GUI apps without the need +for full VNC or remote desktop setup. To enable X11-forwarding via SSH, +perform the following steps: + +#. Create the following file, if it does not already exist: + :file:`/etc/ssh/sshd_config` + +#. Set the following configuration variables. + + .. code-block:: bash + + AllowTcpForwarding yes + X11UseLocalhost yes + X11DisplayOffset 10 + X11Forwarding yes + +You have now enabled X11-forwarding! You do not need to restart the sshd service. diff --git a/_sources/guides/network/proxy.rst.txt b/_sources/guides/network/proxy.rst.txt new file mode 100644 index 00000000..b4c1f448 --- /dev/null +++ b/_sources/guides/network/proxy.rst.txt @@ -0,0 +1,116 @@ +.. _proxy: + +Proxy Configuration +################### + +When working behind a corporate proxy server, one typically has to configure +proxy settings for applications to reach the Internet. |CL-ATTR| has +implemented an :ref:`autoproxy` feature to try and eliminate manual +configurations as much as possible. However, there are still some applications +that cannot take full advantage of the :ref:`autoproxy` feature due to their +own ways of configuring proxy settings. This guide shows you how to configure +proxy settings for some of the known applications manually. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* You have installed |CL| on your host system. + + For detailed instructions on installing |CL| on a bare metal system, visit + the :ref:`bare metal installation guide `. + +General proxy settings for many applications +============================================ + +#. First, apply these general proxy settings which should work for many + applications. If they do not work for a specific application, such as the + ones listed below, apply application-specific proxy settings as needed. + + Proxy settings: + + .. code-block:: none + + export http_proxy=http:// + export https_proxy=http:// + export ftp_proxy=http:// + export socks_proxy=http:// + export no_proxy=",localhost" + export HTTP_PROXY=$http_proxy + export HTTPS_PROXY=$https_proxy + export FTP_PROXY=$ftp_proxy + export SOCKS_PROXY=$socks_proxy + export NO_PROXY=$no_proxy + + * *User-specific*, put them in :file:`$HOME/.bashrc`. + + * *For all users*, put them in :file:`/etc/profile.d/proxy.conf`. + +#. For the proxies to take effect, either :command:`source` the file manually + or log out and log back in. + +Docker\* +======== + +Please refer the official Docker links on how to configure proxies: + +* `Docker client`_ +* `Docker daemon`_ + +git over SSH +============ + +Add the following to your :file:`~/.ssh/config` file: + +.. code-block:: none + + host github.com + port 22 + user git + ProxyCommand connect-proxy -S %h %p + +.. note:: + + Though :command:`netcat` is included with |CL|, it is not the BSD version, + which is the one usually used to enable git over SSH. + +autospec/mock +============= + +:ref:`autospec` uses mock to do builds. Configure mock's proxy settings with +these steps: + +#. Override the general mock configuration file with a custom one, otherwise + your settings will get overwritten each time autospec is updated. + + .. code-block:: bash + + sudo mkdir -p /etc/mock + sudo cp ~/clearlinux/projects/common/conf/clear.cfg /etc/mock/clear-custom.cfg + +#. :command:`sudoedit` :file:`/etc/mock/clear-custom.cfg` and add the highlighted + lines. + + .. code-block:: none + :emphasize-lines: 3-5 + + ... + config_opts['use_bootstrap_container'] = False + config_opts['http_proxy'] = ':' + config_opts['https_proxy'] = ':' + config_opts['no_proxy'] = ',192.168.0.0/16,localhost,127.0.0.0/8' + +Kubernetes +========== + +See :ref:`Setting proxy servers for Kubernetes `. + +.. _Docker client: + https://docs.docker.com/network/proxy/#configure-the-docker-client + +.. _Docker daemon: + https://docs.docker.com/config/daemon/systemd/#httphttps-proxy + diff --git a/_sources/guides/network/time.rst.txt b/_sources/guides/network/time.rst.txt new file mode 100644 index 00000000..e30e3d8b --- /dev/null +++ b/_sources/guides/network/time.rst.txt @@ -0,0 +1,59 @@ +.. _time: + +Set system time +############### + +This guide describes how to reset the time in your |CL-ATTR| system when +the default :abbr:`NTP (Network Time Protocol)` servers cannot be reached. + +|CL| uses the `systemd-timesyncd.service` daemon to synchronize time. + + +#. Install the :command:`sysadmin-basic` bundle. + + .. code-block:: bash + + sudo swupd bundle-add sysadmin-basic + +#. Set your time zone. This example uses Los Angeles. + + .. code-block:: bash + + timedatectl set-timezone America/Los_Angeles + + .. note:: + + To see a list of time zones, use the command: + :command:`timedatectl list-timezones | grep ` + +#. Create a :file:`/etc/systemd/` directory. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/ + +#. Create a new file named :file:`/etc/systemd/timesyncd.conf` and enter the + following text. + + .. code-block:: console + + [Time] + NTP= + FallbackNTP= + +#. Enable the `systemd-timesyncd` service. + + .. code-block:: bash + + sudo timedatectl set-ntp true + +.. note:: + + To check the service status, use the :command:`timedatectl status` command. + + To restart the `timesyncd` daemon, enter :command:`systemctl restart + systemd-timesyncd` into your terminal emulator. + +**Congratulations!** You successfully set up the time in your |CL| system. + + diff --git a/_sources/guides/network/vnc.rst.txt b/_sources/guides/network/vnc.rst.txt new file mode 100644 index 00000000..c7c11ba9 --- /dev/null +++ b/_sources/guides/network/vnc.rst.txt @@ -0,0 +1,780 @@ +.. _vnc: + +Remote-desktop to a host using VNC +################################## + +This guide describes how to use :abbr:`VNC (Virtual Network Computing)` to +connect to a remote |CL-ATTR| host. + +VNC is a client-server GUI-based tool that allows you to connect via +remote-desktop to your |CL| host. + +.. contents:: + :local: + :depth: 1 + +Install the VNC server and misc. components on your host +******************************************************** + +To configure VNC to work on your |CL| host, install these bundles: + +* :command:`desktop-autostart`: Installs :abbr:`GDM (Gnome Desktop Manager)`, sets + it to start automatically on boot, and installs TigerVNC Viewer. +* :command:`vnc-server`: Installs the TigerVNC server. + +Follow these steps: + +#. Open a terminal window. + +#. Install the |CL| bundles. + + .. code-block:: bash + + sudo swupd bundle-add desktop-autostart vnc-server + +#. Reboot your |CL| host. + +Configure a VNC-server-start method on your host +************************************************ + +There are three methods you can use to configure and start the VNC server on +your |CL| host: + +.. list-table:: Table 1: VNC-server-start Configuration Methods + :widths: 10,20,20,20 + :header-rows: 1 + + * - Attribute + - Method 1: Manually start a VNC session + - Method 2: Automatically start a VNC session via a systemd service script + - Method 3: Create multi-user logins with authentication through GDM + * - Description + - This is the traditional method where you SSH into the |CL| host, manually + start a VNC session to get a display ID, and connect to it by + supplying the display ID. + - The system administrator sets up a systemd service script for you with + a pre-assigned display ID. You make a VNC connection and supply + your pre-assigned display ID. + - The system administrator configures GDM to accept connection requests. + When you make a VNC connection to the |CL| host, you see + the GDM login screen and authenticate as if you are local. + * - Who configures VNC settings? + - You + - System administrator + - System administrator + * - Who starts VNC session? + - You + - Set to start automatically on boot by system administrator + - Set to start automatically on boot by system administrator + * - Who ends VNC session? + - You + - You + - System administrator can disable VNC service altogether + * - Requires VNC password to authenticate? + - Yes + - Yes + - No. Use |CL| account username and password through GDM + + +Although all three methods can coexist on the same |CL| host, we recommend +you pick a method that suits your needs. + +For simplicity, the rest of this guide refers to these methods as +Method 1, Method 2, and Method 3. + +Method 1: Manually start a VNC session +====================================== + +You (and each user) must perform these steps to initialize your VNC settings. + +#. Log in. + +#. Open a terminal window. + +#. Start VNC with the :command:`vncserver` command. Since this is your + first time starting VNC, it adds default configuration files and asks you + to set a VNC password. + + .. code-block:: bash + + vncserver + + Example output: + + .. code-block:: console + + You will require a password to access your desktops. + + Password: + Verify: + Would you like to enter a view-only password (y/n)? n + xauth: file /home/vnc-user-a/.Xauthority does not exist + + New 'clr-linux:2 (vnc-user-a)' desktop is clr-linux:2 + + Creating default startup script /home/vnc-user-a/.vnc/xstartup + Creating default config /home/vnc-user-a/.vnc/config + Starting applications specified in /home/vnc-user-a/.vnc/xstartup + Log file is /home/vnc-user-a/.vnc/clr-linux:2.log + + Upon completion, you can find the default configuration files and the + password file hidden in the :file:`.vnc` directory in your home directory. + + A VNC session starts and shows a unique display ID, which is the + number following the hostname and the colon ":". In the above example, the + display ID is 2. In a later step, you will supply the display ID to + your VNC viewer app for connection. + +#. Kill the active VNC session for the time being with the + :command:`vncserver -kill :[display ID]` command. Substitute [display ID] + with your active VNC session display ID. For example: + + .. code-block:: bash + + vncserver -kill :2 + + .. note:: + + If you do not recall the active session display ID, use the + :command:`vncserver -list` command to find it. + +#. Optional configurations: + + * To customize settings such as screen size, security type, etc., + modify the :file:`$HOME/.vnc/config` file. + * To customize the applications to run at startup, modify the + :file:`$HOME/.vnc/xstartup` file. + +Method 2: Automatically start a VNC session via a systemd service script +======================================================================== + +To configure VNC for this method, you must have root privileges. You will +set up a systemd service file for all intended VNC users with their own +preassigned unique display ID. + +#. Log in. + +#. Make sure the user accounts already exist. Use the following command to + list all users. + + .. code-block:: bash + + sudo cut -d: -f1 /etc/passwd + +#. Create the path :file:`/etc/systemd/system`. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/system + +#. Create a systemd service script file :file:`vncserver@:[X].service`, + where [X] is the display ID, for each user in :file:`/etc/systemd/system` + Each user must be assigned a unique display ID. Be sure the correct + username is entered in the :guilabel:`User` field. The example below shows user + vnc-user-b who is assigned the display ID 5. + + .. code-block:: bash + + sudo tee -a /etc/systemd/system/vncserver@:5.service << EOF + + [Unit] + Description=VNC Remote Desktop Service for "vnc-user-b" with display ID "5" + After=syslog.target network.target + + [Service] + Type=simple + User=vnc-user-b + PAMName=login + PIDFile=/home/%u/.vnc/%H%i.pid + ExecStartPre=/bin/sh -c '/usr/bin/vncserver -kill %i > /dev/null 2>&1 || :' + ExecStart=/usr/bin/vncserver %i -geometry 2000x1200 -alwaysshared -fg + ExecStop=/usr/bin/vncserver -kill %i + + [Install] + WantedBy=multi-user.target + + EOF + +#. Have each user log into their account and set a VNC password with + the :command:`vncpasswd` command before proceeding to the next step. + +#. Start the VNC service script and set it to start automatically on + boot for each user. Replace the [X] with the display ID. + + .. code-block:: bash + + sudo systemctl daemon-reload + sudo systemctl enable vncserver@:[X].service + sudo systemctl start vncserver@:[X].service + +#. After starting the services, verify they are running. + + .. code-block:: bash + + systemctl | grep vnc + + The example below shows 2 VNC sessions that were successfully started for + users vnc-user-b with display ID 5 and vnc-user-c with display ID 6. + + .. code-block:: console + + systemctl | grep vnc + + vncserver@:5.services loaded active running VNC Remote Desktop Service for "vnc-user-b" with display ID "5" + vncserver@:6.services loaded active running VNC Remote Desktop Service for "vnc-user-c" with display ID "6" + system-vncserver.slice loaded active active system-vncserver.slice + +Method 3: Multi-user logins with authentication through GDM +=========================================================== + +For this method, VNC is configured as a systemd service that listens on port +5900 and GDM is configured to accept access requests from VNC. When you +make a VNC connection to your |CL| host, you are presented with the GDM login +screen and you authenticate as if you are local. You must have root privileges +to perform this configuration. + +#. Log in. + +#. Create the path :file:`/etc/systemd/system`. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/system + +#. Create a systemd socket file :file:`xvnc.socket` with the following content: + + .. code-block:: bash + + sudo tee -a /etc/systemd/system/xvnc.socket << EOF + + [Unit] + Description=XVNC Server on port 5900 + + [Socket] + ListenStream=5900 + Accept=yes + + [Install] + WantedBy=sockets.target + + EOF + +#. Create a systemd service file :file:`xvnc@.service` with the following content: + + .. code-block:: bash + + sudo tee -a /etc/systemd/system/xvnc@.service << EOF + + [Unit] + Description=Daemon for each XVNC connection + + [Service] + ExecStart=-/usr/bin/Xvnc -inetd -query localhost -geometry 2000x1200 -once -SecurityTypes=None + User=nobody + StandardInput=socket + StandardError=syslog + + EOF + +#. Create the path :file:`/etc/gdm`. + + .. code-block:: bash + + sudo mkdir -p /etc/gdm + + +#. Create a GDM :file:`custom.conf` file with the following content: + + .. code-block:: bash + + sudo tee -a /etc/gdm/custom.conf << EOF + + [xdmcp] + Enable=true + Port=177 + + EOF + +#. Start the VNC socket script and set it to start automatically on boot. + + .. code-block:: bash + + sudo systemctl daemon-reload + sudo systemctl enable xvnc.socket + sudo systemctl start xvnc.socket + +#. After starting the socket, verify it is running. + + .. code-block:: bash + + systemctl | grep vnc + + The example below shows the xvnc.socket is running. + + .. code-block:: console + + systemctl | grep vnc + + xvnc.socket loaded active listening XVNC Server on port 5900 + system-xvnc.slice loaded active active system-xvnc.slice + +See the vncserver Man page for additional information. + +Install a VNC viewer app and an SSH client on your client system +**************************************************************** + +You need a VNC viewer app on your client system to connect to your |CL| host. +An SSH client is only needed if you chose to use Method 1 or you plan to +encrypt your VNC traffic, which is discussed later in this guide. + +Perform the steps below to add these apps to your client system. + +Install a VNC viewer app +======================== + +On |CL|: + +.. code-block:: bash + + sudo swupd bundle-add desktop-autostart + +On Ubuntu\*, Mint\*: + +.. code-block:: bash + + sudo apt-get install xtightvncviewer + +On Fedora\*: + +.. code-block:: bash + + sudo dnf install tigervnc + +On Windows\*: + +* Install `RealVNC for Windows`_ + +On macOS\*: + +* Install `RealVNC for macOS`_ + +Install an SSH client +===================== + +* On most Linux distros (|CL|, Ubuntu, Mint, Fedora, etc.) and macOS, + SSH is built-in so you don't need to install it. +* On Windows, you can install `Putty`_. + +Establish a VNC connection to your host +*************************************** + +Depending on the VNC-server-configuration method chosen, use the appropriate VNC +connection: + +* If you chose Method 1, you must take a few extra steps by using SSH to connect + to your |CL| host and then manually launching VNC. + +* If you chose Method 2, get your preassigned VNC display ID from your system + administrator first and then proceed to the :ref:`connect-to-vnc-session` + section below. + +* If you chose Method 3, proceed to the :ref:`connect-to-vnc-session` below. + + +SSH into your host and launch VNC +================================= + +#. SSH into your |CL| host + + a. On Linux distros and macOS: + + .. code-block:: bash + + ssh [username]@[clear-linux-host-ip-address] + + #. On Windows: + + i. Launch Putty. + #. Under the :guilabel:`Category` section, select :guilabel:`Session`. + See Figure 1. + #. Enter the IP address of your |CL| host in the + :guilabel:`Host Name (or IP address)` field. + #. Set the :guilabel:`Connection type` option to :guilabel:`SSH`. + #. Click the :guilabel:`Open` button. + + .. figure:: ../../_figures/vnc/vnc-1.png + :scale: 90 % + :alt: Putty - configure SSH session settings + + Figure 1: Putty - configure SSH session settings + +#. Log in with your |CL| username and password. Do not use your VNC password. +#. Start a VNC session. + + .. code-block:: bash + + vncserver + + Example output: + + .. code-block:: console + + New 'clr-linux:3 (vnc-user-c)' desktop is clr-linux:3 + + Starting applications specified in /home/vnc-user-c/.vnc/xstartup + Log file is /home/vnc-user-c/.vnc/clr-linux:3.log + +#. Take note of the generated display ID because you will input it into + the VNC viewer app to establish the connection. The above example shows + the display ID is 3. + + .. note:: + + VNC automatically picks a unique display ID unless you specify one. + To specify a display ID, enter a unique number that is not already + in use after the colon. For example: + + .. code-block:: bash + + vncserver :8 + +#. You can now end the SSH connection by logging out. This does + not terminate your active VNC session. + +.. _connect-to-vnc-session: + +Connect to your VNC session +=========================== + +For Method 1 and Method 2, you must connect to a specific active session +or display ID using one of two options: + +* Use a fully-qualified VNC port number, which consists of the default VNC + server port (5900) plus the display ID +* Use the display ID + +For example, if the display ID is 3, it can be specified as 5903 or just +as 3. For Method 3, VNC does not expect a display ID. Use 5900. For simplicity, +the instructions below use the fully-qualified VNC port number. + +**On Linux distros:** + +#. Open a terminal window and enter: + + .. code-block:: bash + + vncviewer [clear-linux-host-ip-address]:[fully-qualified VNC port number] + +#. Enter your credentials. + + * For Method 1 and Method 2, enter your VNC password. No username is + required. + * For Method 3, enter your |CL| account username and password through + GDM. + + .. note:: + + With Method 3, you cannot remotely log into your |CL| host through + VNC if you are logged in locally and vice versa. + +**On Windows and macOS using RealVNC app:** + +#. Start the RealVNC viewer app. See Figure 2. +#. Enter the IP address of the |CL| host and the fully-qualified + VNC port number. + + The following screenshot shows connecting to |CL| host + 192.168.25.54 with a fully-qualified VNC port number 5902. + + .. figure:: ../../_figures/vnc/vnc-2.png + :scale: 90 % + :alt: RealVNC Viewer + + Figure 2: RealVNC Viewer + +#. Press the :kbd:`Enter` key. + +#. Enter your credentials. + + * For Method 1 and Method 2, enter your VNC password. No username is + required. + * For Method 3, enter your |CL| account username and password through + GDM. + + .. note:: + + With Method 3, you cannot remotely log into your |CL| host through + VNC if you are logged in locally and vice versa. + +Optional: Configure RealVNC Image Quality +----------------------------------------- + +To increase the RealVNC viewer image quality, manually change the :guilabel:`ColorLevel` +value. Follow these steps: + +#. Right-click a connection node and select :guilabel:`Properties...`. + See Figure 3. + + .. figure:: ../../_figures/vnc/vnc-3.png + :scale: 90 % + :alt: RealVNC Viewer - change connection node properties + + Figure 3: RealVNC Viewer - change connection node properties + +#. Select the :guilabel:`Expert` tab. See Figure 4. + +#. Select the :guilabel:`ColorLevel` setting and change it to your + preferred setting. + + .. figure:: ../../_figures/vnc/vnc-4.png + :scale: 90 % + :alt: RealVNC Viewer - change ColorLevel + + Figure 4: RealVNC Viewer - change :guilabel:`ColorLevel` + +Terminate a VNC connection to your host +*************************************** + +For Method 1 and Method 2, once started, a VNC session remains active +on your |CL| host even if you close your VNC viewer app. If you want to +truly terminate an active VNC session, follow these steps: + +#. SSH into your |CL| host. +#. Open a terminal window. +#. Find the active VNC session display ID with the command + :command:`vncserver -list`. + + .. code-block:: bash + + vncserver -list + +#. Terminate it with the :command:`vncserver -kill` command followed by a + colon and the display ID. + + .. code-block:: bash + + vncserver -kill :[display ID] + +#. For Method 3, only the system administrator can stop and disable the + VNC service by using these commands: + + .. code-block:: bash + + sudo systemctl stop xvnc.socket + sudo systemctl disable xnvc.socket + + +Encrypt VNC traffic through an SSH tunnel +***************************************** + +By default, VNC traffic is not encrypted. Figure 5 shows an example warning +from RealVNC Viewer. + +.. figure:: ../../_figures/vnc/vnc-6.png + :scale: 90 % + :alt: RealVNC Viewer - Connection not encrypted warning + + Figure 5: RealVNC Viewer - Connection not encrypted warning + +To add security, VNC traffic can be routed through an SSH tunnel. This is +accomplished by following these steps: + +#. Configure the VNC server to only accept connection from localhost by + adding the :command:`-localhost` option. +#. Set up an SSH tunnel between your client system and your |CL| host. + Your client system will forward traffic from the localhost (the client) + destined for a specified fully-qualified VNC port number (on the client) + to your |CL| host with the same port number. +#. The VNC viewer app on your client system will now connect to localhost, + instead of the IP address of your |CL| host. + +Configure VNC to only accept connection from localhost +====================================================== + +For Method 1: + +#. Edit the :file:`config` file located in :file:`$HOME/.vnc` and uncomment + the `# localhost` line. It should look like this: + + .. code-block:: console + + ## Supported server options to pass to vncserver upon invocation can be listed + ## in this file. See the following manpages for more: vncserver(1) + Xvnc(1). + ## Several common ones are shown below. Uncomment and modify to your liking. + ## + # securitytypes=vncauth,tlsvnc + # desktop=sandbox + # geometry=2000x1200 + localhost + # alwaysshared + +#. If an active session exists, kill it, and then restart it. + +For Method 2: + +#. Edit the systemd service script :file:`vncserver@:[X].service` located in + :file:`/etc/systemd/system` and add :command:`-localhost` to the `ExecStart` + line. The example below uses vncserver@:5.service: + + .. code-block:: console + + [Unit] + Description=VNC Remote Desktop Service for "vnc-user-b" with display ID "5" + After=syslog.target network.target + + [Service] + Type=simple + User=vnc-user-b + PAMName=login + PIDFile=/home/%u/.vnc/%H%i.pid + ExecStartPre=/bin/sh -c '/usr/bin/vncserver -kill %i > /dev/null 2>&1 || :' + ExecStart=/usr/bin/vncserver %i -geometry 2000x1200 -localhost -alwaysshared -fg + ExecStop=/usr/bin/vncserver -kill %i + + [Install] + WantedBy=multi-user.target + +#. Restart the service script: + + .. code-block:: bash + + sudo systemctl daemon-reload + sudo systemctl restart vncserver@:5.service + +For Method 3: + +#. No change is needed to the :file:`xvnc@service` script. + + After you have restarted your VNC session, you can verify that it only + accepts connections from localhost by using the :command:`netstat` + command like this: + + .. code-block:: bash + + netstat -plant + + .. note:: + + Add the |CL| :command:`network-basic` bundle to get the :command:`netstat` + command. + +Figure 6 shows two VNC sessions (5901 and 5905) accepting connections from +any host as specified by the `0.0.0.0`'s. This is before the +:command:`-localhost` option was used. + +.. figure:: ../../_figures/vnc/vnc-7.png + :scale: 100 % + :alt: VNC session accepting connection from any host + + Figure 6: VNC sessions (5901 and 5905) accepting connections from any host + +Figure 7 shows two VNC sessions (5901 and 5905) only accepting connections from +localhost as specified by `127.0.0.1`'s. This is after the +:command:`-localhost` option was used. + +.. figure:: ../../_figures/vnc/vnc-8.png + :scale: 100 % + :alt: VNC session only accepting connection from localhost + + Figure 7: VNC sessions (5901 and 5905) only accepting connections from localhost + +Set up an SSH tunnel from your client system to your |CL| host +============================================================== + +**On Linux distros and macOS:** + +#. Open terminal window and enter: + + .. code-block:: bash + + ssh -L [client port number]:localhost:[fully-qualified VNC port number] \ + -N -f -l [username] [clear-linux-host-ip-address] + +#. Enter your |CL| account password (not your VNC password). + + .. note:: + + * `-L` specifies that [client port number] on the localhost (on the + client side) is forwarded to [fully-qualified VNC port number] + (on the server side). + * Replace `[client port number]` with an available client port number + (for example: 1234). For simplicity, you can make the + `[client port number]` the same as the `[fully-qualified VNC port number]`. + * Replace `[fully-qualified VNC port number]` with 5900 (default VNC + port) plus the display ID. For example, if the display ID is 2, + the fully-qualified VNC port number is is 5902. + * `-N` tells SSH to only forward ports and not execute a remote + command. + * `-f` tells SSH to go into the background before command execution. + * `-l` specifies the username to log in as. + +**On Windows:** + +#. Launch Putty. +#. Specify the |CL| VNC host to connect to. + + a. Under the :guilabel:`Category` section, select :guilabel:`Session`. + See Figure 1. + #. Enter the IP address of your |CL| host in the + :guilabel:`Host Name (or IP address)` field. + #. Set the :guilabel:`Connection type` option to :guilabel:`SSH`. + +#. Configure the SSH tunnel. See Figure 8 for an example. + + a. Under the :guilabel:`Category` section, go to + :guilabel:`Connection` > :guilabel:`SSH` > :guilabel:`Tunnels`. + + #. In the :guilabel:`Source port` field, enter an available client + port number (for example: 1234). For simplicity, you can make the + `Source port` the same as the fully-qualified VNC port number. + + #. In the :guilabel:`Destination` field, enter + `localhost:` plus the fully-qualified VNC port number. + + #. Click the :guilabel:`Add` button. + + .. figure:: ../../_figures/vnc/vnc-9.png + :scale: 100 % + :alt: Putty - configure SSH tunnel + + Figure 8: Putty - configure SSH tunnel + +#. Click the :guilabel:`Open` button. +#. Enter your |CL| account password (not your VNC password). + +Connect to a VNC session through an SSH tunnel +============================================== + +After you have set up an SSH tunnel, follow these instructions to connect to +your VNC session. + +**On Linux distros:** + +#. Open terminal window and enter: + + .. code-block:: bash + + vncviewer localhost:[client port number] + +**On Windows and macOS using `RealVNC`:** + +#. Start the RealVNC viewer app. +#. Enter `localhost` and the fully-qualified VNC port number. See Figure 9 + for an example. + + .. figure:: ../../_figures/vnc/vnc-10.png + :scale: 100 % + :alt: RealVNC viewer app connecting to localhost:1234 + + Figure 9: RealVNC viewer app connecting to `localhost:1234` + + .. note:: + + RealVNC will still warn that the connection is not encrypted even + though its traffic is going through the SSH tunnel. You can ignore + this warning. + +.. _RealVNC for Windows: https://www.realvnc.com/en/connect/download/viewer/windows/ +.. _RealVNC for macOS: https://www.realvnc.com/en/connect/download/viewer/macos/ +.. _Putty: https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html diff --git a/_sources/guides/network/wifi.rst.txt b/_sources/guides/network/wifi.rst.txt new file mode 100644 index 00000000..bb8dd525 --- /dev/null +++ b/_sources/guides/network/wifi.rst.txt @@ -0,0 +1,293 @@ +.. _wi-fi: + +Configure Wi-Fi +############### + +We recommend using `NetworkManager +`_ to +manage wireless network connections. If you choose to connect to Wi-Fi while +using the :ref:`live installer ` image, your Wi-Fi +settings will be added to your system during the installation process. + +NetworkManager provides three simple methods for configuring Wi-Fi: Desktop, +CLI, and TUI. NetworkManager uses :command:`wpa_supplicant`, which can also be +used on its own for a more lightweight installation. + +.. contents:: + :local: + :depth: 1 + +Using Network Manager +********************* + +Desktop GUI (Graphical User Interface) +====================================== + +1. Click anywhere on the icons at the right side of the top of the screen to + bring up a menu and click on :guilabel:`Wi-Fi Not Connected` then choose + :guilabel:`Select Network`. + + .. figure:: /_figures/wifi/wifi-1.1.png + +2. Scroll through the list and select the network you'd like to connect to and + click :guilabel:`Connect`. + + .. figure:: /_figures/wifi/wifi-2.png + +3. Enter the password and click :guilabel:`Connect`. + + .. figure:: /_figures/wifi/wifi-3.png + +4. The Wi-Fi icon should now show the signal strength of the connection. + + .. figure:: /_figures/wifi/wifi-4.png + +5. If you are installing using the live image, resume the + :ref:`installation process ` now. Your Wi-Fi + configuration will automatically be included in the install. + + .. figure:: /_figures/wifi/wifi-5.png + +CLI (Command Line Interface) +============================ + +#. List the available Wi-Fi networks + + .. code-block:: bash + + nmcli device wifi list + + .. code-block:: console + + IN-USE SSID MODE CHAN RATE SIGNAL BARS SECURITY + 1st Network Infra 1 54 Mbit/s 65 ▂▄▆_ -- + 2nd Network Infra 1 130 Mbit/s 52 ▂▄__ -- + 3rd Network Infra 10 195 Mbit/s 29 ▂___ WPA2 + +#. Join the network. + + .. code-block:: bash + + nmcli device wifi connect $SSID password $password + + .. code-block:: console + + Device 'wlp1s0' successfully activated with 'f2501e67-27a3-4cf2-a8d9-cce3d029b788'. + +.. note:: + + To avoid having the Wi-Fi password stored in bash history, consider using the TUI. + +.. _wifi-nm-tui: + +TUI (Text-based User Interface) +=============================== + +#. Launch the NetworkManager Text User Interface + + .. code-block:: bash + + nmtui + +#. Select :guilabel:`Activate a connection` and hit :kbd:`return`. + + .. figure:: /_figures/wifi/nmtui_1.png + +#. Use the arrow keys to select your network and then select + :guilabel:`Activate` and hit :kbd:`return`. + + .. figure:: /_figures/wifi/nmtui_2.png + +#. Enter your password and hit :kbd:`return` to select :guilabel:`OK`. + + .. figure:: /_figures/wifi/nmtui_3.png + +#. Select :guilabel:`Back` and hit :kbd:`return`. + + .. figure:: /_figures/wifi/nmtui_4.png + +#. Select :guilabel:`Quit` and hit :kbd:`return` to exit. + + .. figure:: /_figures/wifi/nmtui_5.png + + +Using wpa_supplicant +******************** + +wpa_suppliant can be used directly, without NetworkManager, to associate a +wireless adapter with an access point. After association is established, an IP +address needs to be assigned or obtained. + + +Associate with a wireless access point +====================================== + +#. Make sure NetworkManager is stopped and disabled by masking the service. + + .. code-block:: bash + + sudo systemctl stop NetworkManager.service + sudo systemctl mask NetworkManager.service + + +#. Stop the wpa_supplicant.service, which may have been started by NetworkManager. + + .. code-block:: bash + + sudo systemctl stop wpa_supplicant.service + +#. Create a ``wpa_supplicant`` configuration directory. + + .. code-block:: bash + + sudo mkdir -p /etc/wpa_supplicant + +#. Determine your wireless interface name. + + .. code-block:: bash + + iw dev + + Use the name following "Interface" on the first line (eg. wlp1s0) + + .. code-block:: console + :emphasize-lines: 1 + + Interface wlp1s0 + ifindex 3 + wdev 0x1 + addr 00:xx:xx:38:34:7a + type managed + txpower 0.00 dBm + + Set the $INTERFACE_NAME environment variable to take advantage of copying + and pasting commands. + + .. code-block:: bash + + export INTERFACE_NAME=wlp1s0 + +#. Create a minimal configuration file called + :file:`/etc/wpa_supplicant/wpa_supplicant-$INTERFACE_NAME.conf` + and add the following: + + .. code-block:: bash + + sudo tee /etc/wpa_supplicant/wpa_supplicant-$INTERFACE_NAME.conf > /dev/null <<'EOF' + ctrl_interface_group=wheel + ctrl_interface=/run/wpa_supplicant + update_config=1 + EOF + +#. Start the wpa_supplicant service to complete the configuration process. + + .. code-block:: bash + + sudo systemctl start wpa_supplicant@$INTERFACE_NAME.service + +#. Use :command:`wpa_cli` (interactive mode) to scan for available networks. + In this example, our network is named *Network1*. + + .. code-block:: bash + :emphasize-lines: 1,2,5,7 + + sudo wpa_cli + > scan + OK + <3>CTRL-EVENT-SCAN-STARTED + <3>CTRL-EVENT-SCAN-RESULTS + > scan_results + bssid / frequency / signal level / flags / ssid + 00:xx:xx:73:7b:46 5180 -55 [WPA2-PSK-CCMP][ESS] Network1 + 00:xx:xx:83:fa:70 5240 -76 [WPA2-EAP-CCMP][ESS] Network2 + 00:xx:xx:4f:e9:2c 2412 -67 [WPA2-PSK-CCMP][ESS][P2P] Printer + 00:xx:xx:af:fe:3e 5765 -79 [WPA2-PSK-CCMP][ESS] Network3 + 00:xx:xx:e9:eb:29 2412 -76 [WPA2-PSK-CCMP][ESS] Network4 + 00:xx:xx:26:4a:b9 2412 -79 [WPA2-PSK-CCMP][ESS][P2P] Printer2 + 00:xx:xx:b9:0d:d4 2462 -79 [WPA2-PSK-CCMP][ESS] Network5 + +#. Set up your network connection replacing *Network1* with your wireless + SSID name and *Network1Password* with the password for your network. + + .. code-block:: bash + :emphasize-lines: 1,3,5,7 + + > add_network + 0 + > set_network 0 ssid "Network1" + OK + > set_network 0 psk "Network1Password" + OK + > enable_network 0 + OK + <3>CTRL-EVENT-SCAN-STARTED + <3>CTRL-EVENT-SCAN-RESULTS + <3>SME: Trying to authenticate with 00:xx:xx:5d:d9:26 (SSID='Network1' freq=5180 MHz) + <3>Trying to associate with 00:xx:xx:5d:d9:26 (SSID='Network1' freq=5180 MHz) + <3>Associated with 00:xx:xx:5d:d9:26 + <3>CTRL-EVENT-SUBNET-STATUS-UPDATE status=0 + <3>WPA: Key negotiation completed with 00:xx:xx:5d:d9:26 [PTK=CCMP GTK=CCMP] + <3>CTRL-EVENT-CONNECTED - Connection to 00:xx:xx:5d:d9:26 completed [id=0 id_str=] + +#. Save the configuration and quit out of :command:`wpa_cli`. + + .. code-block:: bash + + > save_config + OK + > quit + +.. note:: + + The network password is saved as plaintext in + :file:`/etc/wpa_supplicant/wpa_supplicant-$INTERFACE_NAME.conf`. Use + `wpa_passphrase + `_ + for a more secure method. + +Assign an IP address +==================== + +After the wireless adapter has been associated with wireless access point, an +IP address needs to be assigned for access to the network. + +The example below uses ``systemd-networkd`` to request an IP address from the +access point via DHCP. Another network manager can be used if preferred. If +there is a static IP address you'd like to assign, see the +:ref:`assign-static-ip` documentation. + +#. Create the :file:`/etc/systemd/network` directory + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/network + +#. Create a :file:`/etc/systemd/network/25-wireless-$INTERFACE_NAME.network` file + with a Match and Network section. + + .. code-block:: bash + + printf "[Match]\nName=$INTERFACE_NAME\n\n[Network]\nDHCP=ipv4" | sudo tee /etc/systemd/network/25-wireless-$INTERFACE_NAME.network + +#. Restart the ``systemd-networkd.service``. + + .. code-block:: bash + + sudo systemctl restart systemd-networkd.service + + +#. Enable the ``systemd-networkd`` and ``wpa_supplicant`` services to start automatically + on future boots. + + .. code-block:: bash + + sudo systemctl enable --now systemd-networkd.service + sudo systemctl enable --now wpa_supplicant@$INTERFACE_NAME.service + + +Other resources +*************** + +* NetworkManager CLI `documentation `_. +* Additional CLI `examples `_. +* wpa_supplicant `advanced usage documentation `_ diff --git a/_sources/index.rst.txt b/_sources/index.rst.txt new file mode 100644 index 00000000..05e93e1c --- /dev/null +++ b/_sources/index.rst.txt @@ -0,0 +1,72 @@ +.. _clear-linux: + +|CL-PRJ| Documentation +############################################# + +.. container:: multicolumns + + .. container:: column verticalcard + + .. rst-class:: colh2 + + Highlights + + :ref:`autospec` + **autospec** is a tool to assist in the automated creation and + maintenance of RPM packaging in Clear Linux OS. + + :ref:`docker` + Clear Linux OS supports multiple containerization platforms, + including a Docker solution. + + :ref:`developer-workstation` + Developer Workstation helps you find the Bundles you need to start + your Clear Linux OS development project. + + .. container:: column verticalcard + + .. rst-class:: colh2 + + Quicklinks + + :ref:`get-started` + Get up and running fast with Clear Linux\* OS. Use these step-by-step + instructions to guide you through the installation of Clear Linux OS + from bare metal to live image. + + :ref:`bare-metal-install-desktop` + These instructions guide you through the installation of Clear Linux + OS on bare metal using a bootable USB drive. + + :ref:`architect-lifecycle` + This guide provides DevOps with a model to architect the life-cycle + of a Clear Linux OS derivative that integrates custom software and + content using distinct workflows. + + :ref:`bundles` + Useful bundle commands for working with bundles on the Clear Linux OS. + + .. container:: column featurecard + + **Community** + + | `Clear Linux Forum `_ + | `irc.libera.chat IRC: #clearlinux `_ + + +.. container:: video + + .. raw:: html + + + +.. toctree:: + :hidden: + + get-started/index + about + guides/index + tutorials/index + reference/index + FAQ/index + collaboration/collaboration diff --git a/_sources/reference/bundles/bundles.rst.txt b/_sources/reference/bundles/bundles.rst.txt new file mode 100644 index 00000000..7572e4a6 --- /dev/null +++ b/_sources/reference/bundles/bundles.rst.txt @@ -0,0 +1,18 @@ +.. _bundles: + +Available bundles +################# + +This document provides a current list of available bundles. View the `clr-bundles repo`_ on GitHub\*, or select the bundle +:file:`Name` for more details. + +To learn more about how |CL-ATTR| uses bundles for software deployment, visit +:ref:`bundles-guide`. + +Bundle list +=========== + +.. raw:: html + :file: bundles.html.txt + +.. _clr-bundles repo: https://github.com/clearlinux/clr-bundles/tree/master/bundles diff --git a/_sources/reference/compatible-hardware.rst.txt b/_sources/reference/compatible-hardware.rst.txt new file mode 100644 index 00000000..9acca6d4 --- /dev/null +++ b/_sources/reference/compatible-hardware.rst.txt @@ -0,0 +1,114 @@ +.. _compatible-hardware: + +Compatible Hardware +################### + +This document describes hardware that has been tested and confirmed as +compatible with |CL-ATTR|. This list is not comprehensive and will continue to +grow. + +.. list-table:: **Table 1. Compatible Hardware** + :widths: 20, 20 + :header-rows: 1 + + * - Processor + - Platform + + * - Intel® Core™ i5-6260U processor + - + + * - Intel® Core™ i5-6560U processor + - Dell XPS\* 13 9350 + + * - Intel® Celeron® J3455 processor + - NUC6CAYS + + * - Intel® Core™ i5-4250U processor + - + + * - Intel® Core™ i7-5557U processor + - + + * - Intel® Core™ i9-7900X X-series processor + - Gigabyte\* X299 + + * - Intel® Core™ i3-4130 processor + - Lenovo Thinkserver\* TS140 + + * - Intel® Core™ i7-7567U processor + - NUC7i7BNH + + * - Intel® Core™ i7-8809G processor + - NUC8i7HVK + + * - Intel® Core™ i5-7260U processor + - NUC7i5BNH + + * - Intel® Core™ i7-8650U processor + - NUC7i7DNKE + + * - Intel® Core™ i5-7300U processor + - NUC7i5DNHE + + * - Intel® Xeon® Gold 6138 processor + - + + * - Intel® Xeon® E5-2699A v4 processor + - Dell PowerEdge\* R630 + + * - Intel® Xeon® E5-2620 v3 processor + - + + * - Intel® Core™ i5-6600 processor + - Gigabyte\* Z170X-UD5 + + * - Intel® Core™ i5-4250U processor + - D54250WYK + + * - Intel® Xeon® E5-2699 v3 processor + - S2600WT2 + + * - Intel Atom™ J3455 processor + - NUC6CAYB + + * - Intel® Xeon® Bronze 3104 processor + - 0W23H8 + + * - Intel Atom™ C2750 processor + - SuperMicro\* A1SAi + + * - Intel Atom™ E3825 processor + - CircuitCo MinnowBoard MAX\* + + * - Intel® Core™ i7-8700 processor + - Gigabyte\* H370 WIFI + + * - Intel® Core™ i7-3667U processor + - Lenovo ThinkPad\* X1 Carbon laptop + + * - Intel® Core™ i5-4210U processor + - Dell XPS\* 13 laptop + + * - Intel® Celeron® J3455 processor + - NUC6CAYB + + * - Intel® Core™ i7-4790 processor + - Gigabyte\* desktop + + * - Intel® Core™ i5-6260U processor + - NUC6I6SYH + + * - Intel® Core™ i7-5557U processor + - NUC5I7RYH + + * - Intel® Core™ i7-4700MQ processor + - Lenovo ThinkPad\* T540p + + * - Intel® Core™ i7-5557U processor + - NUC5I7RYB + + * - Intel® Core™ i5-6260U processor + - NUC6I5SYH + +*Intel, Celeron, Xeon, Intel Atom, and Intel Core are trademarks of Intel +Corporation or its subsidiaries.* \ No newline at end of file diff --git a/_sources/reference/image-types.rst.txt b/_sources/reference/image-types.rst.txt new file mode 100644 index 00000000..6d661f09 --- /dev/null +++ b/_sources/reference/image-types.rst.txt @@ -0,0 +1,80 @@ +.. _image-types: + +|CL-ATTR| image types +######################### + +.. _image-types-content: + +|CL-ATTR| offers many types of `images`_ for different platforms and environments. + +.. _incl-image-filename: + +The naming convention of a |CL| image filename is: + +.. code-block:: console + + clear-[version number]-[image type].[compression type] + +* The *[version number]* field specifies the version number. + +* The *[image type]* field specifies the type of image and its corresponding + file format. + +* The *[compression type]* field specifies the compression type. Two types of + compression methods are used: GNU\* zip (*.gz*) and XZ (*.xz*). + +.. _incl-image-filename-end: + +Table 1 lists the currently available images that are platform independent. +Table 2 lists the currently available images that are platform specific. + +.. list-table:: Table 1: Types of platform-independent |CL| images + :widths: 15, 85 + :header-rows: 1 + + * - Image Type + - Description + + * - live-desktop.img or live-desktop.iso + - Image for booting to GNOME\* desktop to preview or install the OS. + + * - live-server.img or live-server.iso + - Image for booting to server command prompt to preview or install the OS. + +.. list-table:: Table 2: Types of platform-specific |CL| images + :widths: 15, 85 + :header-rows: 1 + + * - Image Type + - Description + + * - aws.img + - Image suitable for use with Amazon\* AWS\*. + + * - cloudguest.img + - Image with generic cloud guest virtual machine (VM) requirements + installed. + + * - gce.tar + - Image with the Google Compute Engine (GCE) specific kernel. + + * - azure-hyperv.vhd + - Image for Microsoft* Azure and Hyper-V generation 1 VMs. Includes + :ref:`optimized kernel ` for Hyper-V. + + * - kvm.img + - Image for booting in a simple VM with start_qemu.sh. Includes + :ref:`optimized kernel ` for KVM. + + * - kvm-legacy.img + - Image for booting in a simple VM using legacy BIOS, if using + start_qemu.sh make sure to remove -bios parameter. + + * - pxe.tar + - Image suitable for use with PXE server. + + * - vmware.vmdk + - Virtual Machine Disk for VMware\* platforms including Player, + Workstation, and ESXi. + +.. _images: https://clearlinux.org/downloads diff --git a/_sources/reference/index.rst.txt b/_sources/reference/index.rst.txt new file mode 100644 index 00000000..a498527b --- /dev/null +++ b/_sources/reference/index.rst.txt @@ -0,0 +1,17 @@ +.. _reference: + +Reference +######### + +This section provides additional information on the |CL| project and +features. + +.. toctree:: + :maxdepth: 1 + + compatible-hardware + bundles/bundles + system-requirements + image-types + man-pages + tutorial-ratings diff --git a/_sources/reference/man-pages.rst.txt b/_sources/reference/man-pages.rst.txt new file mode 100644 index 00000000..02484d48 --- /dev/null +++ b/_sources/reference/man-pages.rst.txt @@ -0,0 +1,95 @@ +.. _man-pages: + +Man pages +######### + +These pages are generated from `Clear Linux\* tool +repositories `__. Updated |today|. + + +`clr-man-pages `__ +=============================================================== + +.. toctree:: + :maxdepth: 1 + + manpages/stateless.7.rst + manpages/os-core.7.rst + manpages/os-format.7.rst + +`clr-power-tweaks `__ +===================================================================== + +.. toctree:: + :maxdepth: 1 + + manpages/clr-power-tweaks.conf.5.rst + manpages/clr_power.1.rst + +`clrtrust `__ +===================================================== + +.. toctree:: + :maxdepth: 1 + + manpages/clrtrust.1.rst + +`micro-config-drive `__ +========================================================================= + +.. toctree:: + :maxdepth: 1 + + manpages/ucd.1.rst + manpages/cloud-config.5.rst + manpages/ucd-data-fetch.1.rst + +`mixer-tools `__ +=========================================================== + +.. toctree:: + :maxdepth: 1 + + manpages/mixer.bundle.1.rst + manpages/mixer.build.1.rst + manpages/mixer.add-rpms.1.rst + manpages/mixer.1.rst + manpages/mixer.repo.1.rst + manpages/mixer.versions.1.rst + manpages/mixer.init.1.rst + manpages/mixer.config.1.rst + +`swupd-client `__ +============================================================= + +.. toctree:: + :maxdepth: 1 + + manpages/swupd-update.service.4.rst + manpages/update-triggers.target.4.rst + manpages/swupd.1.rst + manpages/swupd-update.timer.4.rst + +`tallow `__ +================================================= + +.. toctree:: + :maxdepth: 1 + + manpages/tallow.1.rst + manpages/tallow.conf.5.rst + manpages/tallow.patterns.5.rst + +`telemetrics-client `__ +========================================================================= + +.. toctree:: + :maxdepth: 1 + + manpages/telemetrics.conf.5.rst + manpages/telem-record-gen.1.rst + manpages/telempostd.1.rst + manpages/telemprobd.1.rst + manpages/telemctl.1.rst + manpages/telemetry.3.rst + diff --git a/_sources/reference/manpages/cloud-config.5.rst.txt b/_sources/reference/manpages/cloud-config.5.rst.txt new file mode 100644 index 00000000..0c239e22 --- /dev/null +++ b/_sources/reference/manpages/cloud-config.5.rst.txt @@ -0,0 +1,273 @@ +cloud-config +============ + +`cloud-config(5) `__ – User data format used by `ucd(1) `__ + +SYNOPSIS +-------- + +``user-data`` + +``#cloud-config`` + +cloud-config data is provided by the cloud infrastructure to a cloud +instance. This data is parsed by ``cloud-init(1)`` which then configures +the cloud instance based on the information in the cloud-config. + +DESCRIPTION +----------- + +The user-data as provided can be of several formats. ``cloud-init(1)`` +supports the ``cloud-config`` format, but also supports generic shell +scripts. If the user-data starts with ``#!``, it is assumed that the +user-data is a generic shell script, and ``cloud-init`` will attempt to +execute the data as a script. If the user-data starts with the +``#cloud-config`` stanza, it is assumed the user-data is in the +``cloud-config`` format, described below. + +The ``cloud-config`` format is used to structure configuration data +provided to the cloud instance. This data is structured in the YAML +format (``http://yaml.org``). Note that ``cloud-init``\ (1) uses the +libyaml library, which supports YAML version 1.1. + +In the YAML content, the root node (the document) contains a sequence of +directives. These directives inform cloud-init that the content of the +directive are processed by a directive-specific handler, or “module”. + +cloud-init includes many modules, each expecting a specifically +constructed content. This document lists the correct format and +organization of each data section for the supported modules, describes +their effect, and states various parameters for each option. + +Table of implemented directives. Column 3 (“Ref”) and column 4 +(“CoreOS”) list whether the reference specification or the CoreOS +implementation support these directives. Note that implementation +details may differ, due to the underlying differences in Operating +Systems. + +:: + + Directive |Description |Ref. |CoreOS. + --------------------|-------------------------------------|-----|------- + groups |Configure user account groups |yes |no + package_upgrade |Perform a software update |yes |no + packages |Install additional software |yes |no + runcmd |Execute system commands |yes |no + service |Perform system service configuration |no |no + ssh_authorized_keys |Provision SSH public keys |yes |yes + users |Configure user accounts |yes |yes + write_files |Write content to arbitrary files |yes |yes + hostname |Define the system's hostname |yes |yes + envar |Set environment variables |no |no + bootcmd |Execute system commands on first boot|no |no + wait_for_network |Halt execution until network is up |no |no + +OPTIONS +------- + +For each of the directives listed in the table above, zero or more +options may be provided. The list below documents the implemented +options per directive. The “Type” field values are: + +TYPES +~~~~~ + +:: + + Type |Description + -----------|------------------------------------------------------------------- + boolean |Either true, false, yes, no, 0 or 1 (upper case and initial capitol + |letter versions treated identical) + string |A generic string encoded value. Can be multiline or YAML encoded + |content + integer |A string encoded decimal integer value + octal |A string encoded octal integer value + hex |A string encoded hexadecimal integer value + [] |if a type is listed with [] appended, it means there may be zero or + |more values, lists of values or any arbitrary depth of these nested + * |Indicates this value isn't a separate key, but directly the + |associated value of the root node. This is applicable for directives + |that only have a single configuration parameter, + |e.g. "package_upgrade" + +groups +~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + name |string[] |no |Create a new user account group and + | | |place listed user account names in + | | |that group + +package_upgrade +~~~~~~~~~~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + * |boolean |yes |Enables or disables whether software + | | |update is performed + +This option implies the ``wait_for_network`` option. + +packages +~~~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + name |string[] |no |Enables installation of software bundles + +This option implies the ``wait_for_network`` option. + +runcmd +~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + * |string[] |no |Executes a command, if the item is a list, + | | |the list will be converted to a string + | | |and executed as a command line. + +hostname +~~~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + * |string |yes |Defines the system's hostname + +service +~~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + start |string[] |no |Start units + stop |string[] |no |Stop units + restart |string[] |no |Restart units + enable |string[] |no |Enable units to start automatically + disable |string[] |no |Disable units from starting automatically + reload |string[] |no |Reload service units + isolate |string[] |no |Change target to a new unit + mask |string[] |no |Prevent units from starting + unmask |string[] |no |Remove unit start prevention mask + +ssh_authorized_keys +~~~~~~~~~~~~~~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + * |string[] |no |Specifies an SSH public key value, as + | | |string. This will be added to the default + | | |user account's SSH configuration + +users +~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + * |[] |no |This directive expects a list of user + | | |definitions. Each definition can + | | |additionally specify the following + | | |sub-options: + name |string |yes |A user account name to be created + gecos |string |no |A real user name, can be left empty + homedir |string |no |A home directory path. Default under "/home" + primary-group|string|no |A group name. + groups |string |no |Comma-separated list of group names or + | | |single group name. Specifies additional + | | |groups to put this user account into + lock-passwd|boolean |no |Lock the user account + inactive |boolean |no |Mark the user account as inactive + passwd |string |no |Password hash for the user account + no-create-home|boolean|no |Omits creating a home directory + no-user-group|boolean|no |Omits group creating for the account + no-log-init|boolean |no |Omits this account from lastlog/faillog + expiredate|string |no |A date at which to expire the password + ssh-authorized-keys|string[]|no |Add SSH public keys to ssh configuration + sudo |string[] |no |Add sudoers lines for this account, the account + | | |name is automatically prepended + system |boolean |no |Make the account a system account + +write_files +~~~~~~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + * |[] |no |This directive expects a list of write_files + | | |definitions. Each definition itself contains the + | | |following options: + content |string |yes |The content to write to a file + path |string |yes |The full path and filename of the file to + | | |be written out + owner |string |no |Username and optionally group name, separated + | | |by ":" or ".". Defaults to "root.root" + permissions|octal |no |Octal value describing the file permissions + | | |default value is influenced according to + | | |`umask` + +envar +~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + * |string |yes |Add environment variables to /etc/profile.d/cloud-init.sh + | | |and to current cloud-init process + +bootcmd +~~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + * |string[] |no |Similar to runcmd but bootcmd will run only on first boot + +wait_for_network +~~~~~~~~~~~~~~~~ + +:: + + Option |Type |Required |Function + ----------|---------|------------|----------------------------------- + * |boolean |no |Enable to force waiting for a functional network. + +This option makes the program wait until an active network is detected. +To prevent waiting, ensure that this option is disabled explicitly +disabled, before other options are used. This may be needed when using +the ``packages`` or ``package_upgrade`` options, which imply that this +option is used. + +COPYRIGHT +--------- + +- Copyright (C) 2017 Intel Corporation, License: CC-BY-SA-3.0 + +SEE ALSO +-------- + +``cloud-init``\ (1) + +NOTES +----- + +Creative Commons Attribution-ShareAlike 3.0 Unported + +- http://creativecommons.org/licenses/by-sa/3.0/ diff --git a/_sources/reference/manpages/clr-power-tweaks.conf.5.rst.txt b/_sources/reference/manpages/clr-power-tweaks.conf.5.rst.txt new file mode 100644 index 00000000..8305ed20 --- /dev/null +++ b/_sources/reference/manpages/clr-power-tweaks.conf.5.rst.txt @@ -0,0 +1,93 @@ +clr-power-tweaks.conf +===================== + +:Manual section: 5 + +SYNOPSIS +-------- + +``clr-power-tweaks.conf`` is an optional configuration file for the +`clr_power(1) `__ utility. + +``/etc/clr-power-tweaks.conf`` + +DESCRIPTION +----------- + +``clr-power-tweaks.conf`` is an optional configuration file for the +`clr_power(1) `__ utility. The configuration file contains kernel and device +parameter values used by the `clr_power(1) `__ utility. + +Configuration parameter values placed in ``clr-power-tweaks.conf`` will +supersede the built-in defaults configured by `clr_power(1) `__. New +parameters can be added to the configuration to supplement the built-in +values. Built-in parameters can be changed by defining them in +``clr-power-tweaks.conf`` with a different value. Parameters can be +excluded from being modified by `clr_power(1) `__ by adding them in +``clr-power-tweaks.conf`` without a value. + +FILE FORMAT +----------- + +``clr-power-tweaks.conf`` uses a key-value pairs to define parameter +values. + +Each line contains one parameter-value pair, delimited by a space +character. + +Parameters are paths from the proc(5) filesystem. Parameters paths can +include glob(n) wildcards to match multiple paths. + +Acceptable values are determined by the parameter being set. Any +parameter listed in the configuration file without a value will be +excluded by `clr_power(1) `__ and the parameter will inherit the kernel’s +built-in default parameter value. + +EXAMPLES +-------- + +1. ``clr-power-tweaks.conf`` entry to override the default value in + `clr_power(1) `__ for CPU governor with **powersave**. All other values + built-in to `clr_power(1) `__ would still get applied. + + Note the globbing used with ``*`` to target all CPU cores. + + :: + + /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor powersave + +2. ``clr-power-tweaks.conf`` entry to exclude a parameter causing it to + use the kernel’s built-in default value (60). All other values + built-in to `clr_power(1) `__ would still get applied. + + Note the omission of a value. + + :: + + /proc/sys/vm/swappiness + +3. ``clr-power-tweaks.conf`` entry to add two new parameters to increase + UDP buffers All other values built-in to `clr_power(1) `__ would still get + applied. + + :: + + /proc/sys/net/ipv4/udp_rmem_min 8192 + /proc/sys/net/ipv4/udp_wmem_min 8192 + +BUGS +---- + +The default `clr-power-tweaks.conf(5) `__ aims to have reasonable defaults +for achieving power and performance efficiency. Default values may not +be optimal for all systems and environments. + +See GitHub Issues: https://github.com/clearlinux/clr-power-tweaks/issues + +SEE ALSO +-------- + +**clr_power.conf(5)**, **proc(5)**, **sysctl(2)** + +For parameters understood by the kernel, please see +https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html diff --git a/_sources/reference/manpages/clr_power.1.rst.txt b/_sources/reference/manpages/clr_power.1.rst.txt new file mode 100644 index 00000000..bf5cf435 --- /dev/null +++ b/_sources/reference/manpages/clr_power.1.rst.txt @@ -0,0 +1,73 @@ +clr_power +========= + +:Manual section: 1 + +SYNOPSIS +-------- + +``clr_power`` is a utility to adjust power and performance settings in +the operating system. It is configurable and can enforce power and +performance policies. + +**clr_power** [**-d**\ \|\ **–debug**] + +DESCRIPTION +----------- + +``clr_power`` adjusts power and performance in the operating system by +setting runtime kernel and device parameters in the proc(5) filesystem. +``clr_power`` operates with built-in default values. Values can be added +or changed in user-defined `clr-power-tweaks.conf(5) `__ file. + +``clr_power`` applies settings and immediately exits after it is +started. It can act as a daemon that enforces settings with the included +systemd files. + +The program handles server and desktop-like systems differently and will +set values based on whether a system is a server platform or a desktop +platform. Most systems will be assumed a server, and if it can’t be +determined what type of platform the system is, treated as a server +platform. By default, systems with CPU’s that identify as ``Core(TM)``, +``Celeron`` and ``Pentium`` are treated as desktop systems, and systems +with CPU’s that identify as ``Xeon`` are treated as server systems. + +Tunings applied will vary per system. To see what tunings are applied, +run the program with the ``--debug`` option. + +OPTIONS +------- + +``-d``, ``--debug`` Display debug/verbose output including built-in +values. + +``-S``, ``--server`` Treat the system as a server, and apply server +specific tunings. + +``-D``, ``--desktop`` Treat the system as a desktop, and apply desktop +specific tunings + +FILES +----- + +- *clr-power.service* systemd service unit that executes **clr_power**. + +- *clr-power.timer* systemd timer that periodically executes the + *clr-power.service* to enforce settings. + +- *clr-power-rfkill.service* systemd service that stops bluetooth + devices at boot to prevent power drain. + +- */etc/clr-power-tweaks.conf* Optional user-defined configuration file + to override or att values. See `clr-power-tweaks.conf(5) `__ for more + information. + +BUGS +---- + +See GitHub Issues: https://github.com/clearlinux/clr-power-tweaks/issues + +SEE ALSO +-------- + +`clr-power-tweaks.conf(5) `__, **proc(5)** diff --git a/_sources/reference/manpages/clrtrust.1.rst.txt b/_sources/reference/manpages/clrtrust.1.rst.txt new file mode 100644 index 00000000..41c2ea49 --- /dev/null +++ b/_sources/reference/manpages/clrtrust.1.rst.txt @@ -0,0 +1,185 @@ +clrtrust +======== + +:Manual section: 1 + +SYNOPSIS +-------- + +**clrtrust** is a tool for generating and managing a centralized trusted +certificate store. + +``clrtrust [-v|--verbose] [-h|--help] [-c|--internal-rehash] [options]`` + +DESCRIPTION +----------- + +A trust store contains a set of X.509 certificates which the operating +system and applications should consider trustworthy. + +The ``clrtrust`` tool provides a frontend for centralized trust store +management. It allows for adding (trusting) and removing (distrusting) +certificate authorities (CAs). It also provides maintenance commands for +viewing and re-generating the trust store. + +Certificates can be provided by the operating system for out-of-box +functionality. Certificates can also be provided and modified by +privileged users. + +It is up to each application to make use of the trust store generated by +``clrtrust``. + +OPTIONS +------- + +:: + + Usage: clrtrust [-v|--verbose] [-h|--help] [-c|--internal-rehash] [options] + + -v | --verbose Shows more details about execution + -c | --internal-rehash Forces use of internal implementation of c_rehash + -h | --help Prints this message + + Commands + generate generates the trust store + list list CAs + add add trust to a CA + remove remove trust to a CA + restore restore trust to previously removed CA + check sanity/consistency check of the trust store + + clrtrust --help to get help on specific command. + +**Commands that modify the trust store require root privileges.** + +- ``clrtrust generate [-f|--force]`` + + The ``generate`` command has no arguments and generates a unified + trust store composed of system-provided and user-provided + certificates, if any. The optional ``--force`` parameter will + forcibly generate the trust store, even if it results in an empty + store. See the FILES section for paths used for trust store + generation. + +- ``clrtrust list`` + + The ``list`` command has no arguments and outputs a list of trusted + certificates with the following fields: + + ``id`` uniquely identifies the certificate. It can be used as input + to other ``clrtrust`` commands such as ``remove`` or ``restore``. + + ``File`` contains the file path of the certificate in the trust + store. + + ``Authority`` shows the name of the organization that issued the + certificate. This field is extracted from the certificate file. + + ``Expires`` shows the expiration date of the certificate. This field + is extracted from the certificate file. + +- ``clrtust add [ ...] [-f|--force]`` + + The ``add`` command takes one or more certificates as required + argument(s). The certificate is identified by a file path. The + certificate file(s) must be PEM-encoded with only one certificate per + file. The optional ``--force`` parameter will forcibly add the + certificate to the trust store, even if it is not a root CA. + + Adding a root CA to the trust store allows applications using the + trust store to trust the root CA certificate, trust certificate + chains issued by the authority, verify the authenticity of peer’s + certificate, and establish a connection. + +- ``clrtrust remove [ ...]`` + + The ``remove`` command takes one or more certificates as required + argument(s). The certificate is identified by a file path or ``id``. + The argument can be an ``id`` of the certificate (see the ``list`` + command) or the file path of the certificate. + + Removing a root CA from the trust store distrusts the certificate for + applications using the trust store. Certificate chains issued by the + authority will no longer be trusted, authenticity of the peer’s + certificate will no longer be verified, and a connection will not be + established. + +- ``clrtrust check`` + + The ``check`` command has no arguments and validate the consistency + of a previously generated unified trust store. + +EXAMPLES +-------- + +View the list of trusted CAs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``clrtrust list`` + +The command above outputs a list of trusted certificates in the format +below: + +:: + + id: FA:B7:EE:36:97:26:62:FB:2D:B0:2A:F6:BF:03:FD:E8:7C:4B:2F:9B + File: /var/cache/ca-certs/anchors/certSIGN_ROOT_CA.crt + Authority: /C=RO/O=certSIGN/OU=certSIGN ROOT CA + Expires: Jul 4 17:20:04 2031 GMT + +The certificate can be further inspected using the ``openssl x509`` +command. For example: + +:: + + openssl x509 -in /var/cache/ca-certs/anchors/certSIGN_ROOT_CA.crt -noout -text + +Add (trust) a root CA +~~~~~~~~~~~~~~~~~~~~~ + +``clrtrust add ~/PrivateCA.pem`` + +The command above will add a root CA certificate located in the +``~/PrivateCA.pem`` file. If the certificate file is not in the PEM +format, use ``openssl x509`` command to convert to PEM first. For +example: + +:: + + openssl x509 -in PrivateCA.cer -inform der -out PrivateCA.pem -outform pem + +Remove (distrust) a root CA +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +``clrtrust remove ~/PrivateCA.pem`` + +The command above will remove a root CA certificate located in the +``~/PrivateCA.pem`` file from the trust store and distrust it. + +FILES +----- + +*/var/cache/ca-certs* + +Generated directory of certificates and verification keys. Do not modify +contents outside of ``clrtrust``. + +*/usr/share/ca-certs/* + +Operating-system provided certificates and keys. Do not modify contents +outside of ``clrtrust``. + +*/etc/ca-certs/* + +Generated directory of user-supplied certificates and verification keys. +Do not modify contents outside of ``clrtrust``. + +BUGS +---- + +See GitHub Issues: https://github.com/clearlinux/clrtrust/issues + +SEE ALSO +-------- + +**openssl(1)** diff --git a/_sources/reference/manpages/mixer.1.rst.txt b/_sources/reference/manpages/mixer.1.rst.txt new file mode 100644 index 00000000..d6ca50eb --- /dev/null +++ b/_sources/reference/manpages/mixer.1.rst.txt @@ -0,0 +1,149 @@ +===== +mixer +===== + +------------------------ +OS software update mixer +------------------------ + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``mixer [subcommand] `` + + +DESCRIPTION +=========== + +`mixer(1) `__ is a software update generator that uses input RPMs and bundle +definition files to create update metadata consumable by `swupd(1) `__. + +``mixer`` operates in a mix workspace using configuration files existing in the +workspace. The tool allows users to use RPMs from an upstream URL (default +download.clearlinux.org), provide their own RPMs, and use RPMs from other remote +repositories to build updates. Users can also define their own local bundle +definitions as well as use the default bundle definitions from the upstream URL. + +``mixer`` provides commands to manage local RPMs and bundle definitions and +configure remote RPM repositories. + +The output of ``mixer`` is a set of manifests readable by ``swupd`` as well as +all the OS content ``swupd`` needs to perform its update operations. The OS +content includes all the files in an update as well as zero- and delta-packs for +improved update performance. The content that ``mixer`` produces is tied to a +specific format so that ``swupd`` is guaranteed to understand it if the client +is using the right version of ``swupd``. See `swupd(1) `__ and `os-format(7) `__ +for more details. + + +OPTIONS +======= + +The following options are applicable to most subcommands, and can be +used to modify the core behavior and resources that ``mixer`` uses. + +- ``-h, --help`` + + Display general help information. If put after a subcommand, it will + display help specific to that subcommand. + +- ``--check`` + + Check all external dependencies needed by mixer. + +- ``-v, --version`` + + Displays the version information of the ``mixer`` program and exit. + +- ``--offline`` + + Skip caching upstream bundles and work entirely with local bundles. + Do not reach out over network to perform operations. + + +SUBCOMMANDS +=========== + +``add-rpms`` + + Add RPMs from the configured LOCAL_RPM_DIR to the local dnf repository. + See `mixer.add-rpms(1) `__ for more details. + +``build`` + + Build various pieces of OS content, including all metadata needed by + ``swupd`` to perform updates. ``mixer build`` is a complicated tool in + itself. See `mixer.build(1) `__ for more details. + +``bundle`` + + Perform various configuration actions on local and upstream bundles. The + user can add or remove bundles from their mix, create new bundle definitions, + or validate local bundle definition files. See `mixer.bundle(1) `__ for more details. + +``config`` + + Perform configuration related actions, including configuration file + validation and conversion from deprecated formats. See `mixer.config(1) `__ + for more details. + +``help`` + + Print help text for any ``mixer`` subcommand. + +``init`` + + Initialize ``mixer`` configuration and workspace. See `mixer.init(1) `__ for + more details. + +``repo`` + + Add, list, remove, or edit RPM repositories to be used by mixer. This + subcommand allows users to configure which remote or local repositories + mixer should use to look for RPMs. See `mixer.repo(1) `__ for more + information. + +``versions`` + + Manage mix and upstream versions. By itself the command will print the + current version of mix and upstream, and also report the latest version of + upstream available. Also allows the user to update mix and upstream + versions. See `mixer.versions(1) `__ for more details. + + +FILES +===== + +`/builder.conf` + + The mixer configuration file. + +`/.yum-mix.conf` + + The default location for the DNF configuration file. + + +EXIT STATUS +=========== + +On success, 0 is returned. A non-zero return code indicates a failure. + +SEE ALSO +-------- + +* `mixer.add-rpms(1) `__ +* `mixer.build(1) `__ +* `mixer.bundle(1) `__ +* `mixer.config(1) `__ +* `mixer.init(1) `__ +* `mixer.repo(1) `__ +* `mixer.versions(1) `__ +* `swupd(1) `__ +* `os-format(7) `__ +* https://github.com/clearlinux/mixer-tools +* https://github.com/clearlinux/swupd-client +* https://clearlinux.org/documentation/ diff --git a/_sources/reference/manpages/mixer.add-rpms.1.rst.txt b/_sources/reference/manpages/mixer.add-rpms.1.rst.txt new file mode 100644 index 00000000..56fd3ae4 --- /dev/null +++ b/_sources/reference/manpages/mixer.add-rpms.1.rst.txt @@ -0,0 +1,50 @@ +============== +mixer.add-rpms +============== + +--------------------- +Add local RPMs to mix +--------------------- + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``mixer add-rpms [flags]`` + + +DESCRIPTION +=========== + +Adds RPMs from the `LOCAL_RPM_DIR` (configured in the `builder.conf`) to the +local RPM repository to be used in creating a mix. + + +OPTIONS +======= + +In addition to the globally recognized ``mixer`` flags (see `mixer(1) `__ for +more details), the following options are recognized. + +- ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses the + default `builder.conf` in the mixer workspace if this option is not provided. + +- ``-h, --help`` + + Display ``add-rpm`` help information and exit. + + +EXIT STATUS +=========== + +On success, 0 is returned. A non-zero return code indicates a failure. + +SEE ALSO +-------- + +* `mixer(1) `__ diff --git a/_sources/reference/manpages/mixer.build.1.rst.txt b/_sources/reference/manpages/mixer.build.1.rst.txt new file mode 100644 index 00000000..4d3a3f38 --- /dev/null +++ b/_sources/reference/manpages/mixer.build.1.rst.txt @@ -0,0 +1,298 @@ +=========== +mixer.build +=========== + +----------------------------------- +Build varioius pieces of OS content +----------------------------------- + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``mixer build [command] [flags]`` + + +DESCRIPTION +=========== + +Builds various pieces of OS content and output update metadata to +`/update/www//`. This metadata can be published +directly to a web-server and consumed by client update systems via the +`swupd(1) `__ update tool. All steps can be run at once using ``build all`` or +the steps can be run individually using the subcommands. + + +OPTIONS +======= + +In addition to the globally recognized ``mixer`` flags (see `mixer(1) `__ for +more details), the following options are recognized across all ``build`` +subcommands. + +- ``--bundle-workers`` + + Number of parallel workers when building bundles, passing 0 or omitting this + flag defaults the number of workers to the number of CPUs on the system. + +- ``--delta-workers`` + + Number of parallel workers when creating deltas, passing 0 or omitting this + flag defaults the number of workers to the number of CPUs on the system. + +- ``--fullfile-workers`` + + Number of parallel workers when creating fullfiles, passing 0 or omitting this + flag defaults the number of workers to the number of CPUs on the system. + +- ``--skip-format-check`` + + Skip check for compatible upstream format when building on top of a new + upstream. + +- ``-h, --help`` + + Display ``build`` help information and exit. + + +SUBCOMMANDS +=========== + +``all`` + + Build all content for the mix. Implicitly runs ``build bundles`` followed by + ``build update``. In addition to the global options ``mixer build all`` + takes the following options. + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``--format {number}`` + + Supply the format number to use for the build. + + - ``-h, --help`` + + Display ``build all`` help information and exit. + + - ``--increment`` + + Automatically increment the mix version post build. + + - ``--min-version {version}`` + + Supply minimum version for ``mixer`` to use old content from. This option + tells ``mixer`` to regenerate all mix content starting from a certain + version. ``mixer`` will not use any OS content from a version older than + the min-version passed here. + + - ``--no-signing`` + + Do not generate a certificate and do not sign the Manifest.MoM + + - ``--prefix {path}`` + + Supply the `path` to the file system where the ``swupd`` binaries live. + +``bundles`` + + Build the bundles for your mix. This is done by extracting dependency + information and file lists for each package in each bundle definition for the + mix. In addition to the global options ``mixer build bundles`` takes the + following options. + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``-h, --help`` + + Display ``build bundles`` help information and exit. + + - ``--no-signing`` + + Do not generate a certificate and do not sign the Manifest.MoM + +``delta-packs`` + + Build packs to optimize ``swupd update``\s between versions. When a + ``swupd`` client updates a bundle it looks for a pack file from its current + version to the new version. If available ``swupd`` will download and apply + the pack content to the file system. Delta packs contain binary diff files + that describe changes between updates whenever possible and full files only + when necessary. Because of this delta packs are a significant performance + optimization for client updates. Because the client can fall back to full + files if a pack is not available, delta packs are not necessary for a + functional update. In addition to the global options ``mixer build + delta-packs`` takes the following options. + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``--from {version}`` + + Generate packs from the specified `version`. + + - ``-h, --help`` + + Display ``build delta-packs`` help information and exit. + + - ``--previous-versions {number}`` + + Generate packs for `number` of previous versions. + + - ``--report`` + + Report reason each file in the `to` manifest was packed in the delta pack + or not. + + - ``--to {version}`` + + Generate packs targeting a specific `to` `version`. + +``delta-manifests`` + + Build manifest deltas to optimize ``swupd update``\s between versions. When a + ``swupd`` client update runs, it will first try to get a delta manifest file + if it exists and apply that on the bundle manifest file for the version + installed on their system (if it exists). This can save a large amount of + content being downloaded in the case of few files changing in a manifest. + Because the client can fall back to the full manifest file if a delta is not + available, delta manifests are not necessary for a functional update. In + addition to the global options ``mixer build delta-manifests`` takes the + following options. + + - ``--from {version}`` + + Generate packs from the specified `version`. + + - ``-h, --help`` + + Display ``build delta-manifests`` help information and exit. + + - ``--previous-versions {number}`` + + Generate packs for `number` of previous versions. + + - ``--to {version}`` + + Generate packs targeting a specific `to` `version`. + +``image`` + + Build an image from the mix content. In addition to the global options + ``mixer build image`` takes the following options. + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``--format {number}`` + + Supply the format `number` used for the mix. + + - ``-h, --help`` + + Display ``build image`` help information and exit. + + - ``--template {path}`` + + Provide the `path` to the image template file to use. + +``update`` + + Build the update content for the mix. This command builds the actual update + metadata (manifests) and content (full files and zero packs) necessary for + ``swupd`` to perform updates on client systems. ``update`` relies on the + output of ``build bundles`` as the input for this step and expects the + output of ``build bundles`` to exist in the + `/update/image/` directory. In addition to the + global options ``mixer build update`` takes the following options. + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``--format {number}`` + + Supply the format `number` used for the mix. + + - ``-h, --help`` + + Display ``build update`` help information and exit. + + - ``--increment`` + + Automatically increment the mix version post build. + + - ``--min-version {version}`` + + Supply minimum version for ``mixer`` to use old content from. This option + tells ``mixer`` to regenerate all mix content starting from a certain + version. ``mixer`` will not use any OS content from a version older than + the min-version passed here. + + - ``--no-signing`` + + Do not generate a certificate and do not sign the Manifest.MoM + + - ``--prefix {path}`` + + Supply the `path` to the file system where the ``swupd`` binaries live. + +``validate`` + + Compare two versions to validate that manifest file changes align with corresponding + package changes. Inconsistencies between manifest entries and package contents are + reported as errors. When no errors occur, package update statistics are displayed. + + - ``--from {version}`` + + Compare manifests ``from`` a specific version + + - ``--to {version}`` + + Compare manifests ``to`` a specific version + + - ``--from-repo-url {repo}={URL}`` + + Overrides the baseurl value for the provided repo in the DNF config file for the ``from`` version + + - ``--to-repo-url {repo}={URL}`` + + Overrides the baseurl value for the provided repo in the DNF config file for the ``to`` version + + - ``--table-width {width}`` + + Max width of package statistics table, defaults to terminal width and disabled by negative numbers + + - ``-h, --help`` + + Display ``build validate`` help information and exit. + + +EXIT STATUS +=========== + +On success, 0 is returned. A non-zero return code indicates a failure. + +SEE ALSO +-------- + +* `mixer(1) `__ +* `swupd(1) `__ diff --git a/_sources/reference/manpages/mixer.bundle.1.rst.txt b/_sources/reference/manpages/mixer.bundle.1.rst.txt new file mode 100644 index 00000000..3c321722 --- /dev/null +++ b/_sources/reference/manpages/mixer.bundle.1.rst.txt @@ -0,0 +1,207 @@ +============ +mixer.bundle +============ + +------------------------------------------------ +Perform various configuration actions on bundles +------------------------------------------------ + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``mixer bundle [command] [flags]`` + + +DESCRIPTION +=========== + +Performs various configuration actions on upstream and local bundle definition +files. List and validate mix bundles. Validate local bundle definition files. + + +SUBCOMMANDS +=========== + +``add {bundle} [{bundle}...] [flags]`` + + Add local or upstream bundles to your mix by modifying the mix bundles list + at `/mixbundles`. In addition to the global options ``mixer + bundle add`` takes the following options. + + - ``--all-local`` + + Add all local bundles defined in `/local-bundles` to the + mix. This command takes precedence over bundle list. + + - ``--all-upstream`` + + Add all upstream bundles (cached in `/upstream-bundles` + to the mix. This command takes precedence over bundle list. + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``--git`` + + Automatically apply a new git commit after modifying the mix bundles list + or bundle definitions. This is useful if your local bundles are kept under + version control. + + - ``-h, --help`` + + Display ``bundle add`` help information and exit. + +``create`` + + Create new bundles or copy existing bundles. + This command will locate the bundle by first looking in local-bundles, + and then in upstream-bundles. If the bundle is only found upstream, + the bundle file will be copied to your local-bundles directory. If the bundle is + not found anywhere, a blank template will be created with the correct name. + + Passing '--add' will also add the bundle(s) to your mix. Please note that + bundles are added after all bundles are created, and thus will not be added + if any errors are encountered earlier on. + + In addition to the global options ``mixer bundle create`` takes the following + options. + + - ``--add`` + + Add the bundle(s) to your mix after editing. + + - ``--local`` + + Skip upstream check and create empty local bundle(s). + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``--git`` + + Automatically apply a new git commit after modifying the mix bundles list + or bundle definitions. This is useful if your local bundles are kept under + version control. + + - ``-h, --help`` + + Display ``bundle create`` help information and exit. + +``list [mix|local|upstream] [flags]`` + + List the bundles in the mix, the available local bundles, or the available + upstream bundles. In addition to the global options ``mixer bundle list`` + takes the following options. + + - ``mix`` + + List the bundles in the mix. + + - ``local`` + + List available locally-defined bundles. + + - ``upstream`` + + List available upstream bundles. + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``-h, --help`` + + Display ``bundle list`` help information and exit. + + - ``--tree`` + + Pretty-print the bundle list as a tree showing include information. + +``remove`` + + Removes bundles from your mix by modifying the mix bundle list (stored in + the `/mixbundles` file). The mix bundle list is parsed, the + bundles are removed, and the resultant list is written back out in sorted + order. If bundles do not exist in the mix, they are skipped. In addition to + the global options ``mixer bundle remove`` takes the following options. + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``--git`` + + Automatically apply a new git commit after modifying the mix bundles list + or bundle definitions. This is useful if your local bundles are kept under + version control. + + - ``-h, --help`` + + Display ``bundle remove`` help information and exit. + + - ``--local`` + + Also remove the bundle file from local-bundles. This action is + irrevocable. + + - ``--mix={bool}`` + + Remove bundle from the mix bundle list. This defaults to true. + +``validate`` + + Checks bundle definition files for validity. Only local bundle files are + checked; upstream bundles are trusted as valid. Valid bundles yield no + output. Any invalid bundles will yield a non-zero return code. + + Basic validation includes checking syntax and structure, and that the bundle + has a valid name. Commands like ``mixer bundle add`` run basic validation + automatically. + + In addition to the global options ``mixer bundle remove`` takes the + following options. + + - ``--all-local`` + + Run validation against all local bundles. + + - ``-c, --config {path}`` + + Optionally tell ``mixer`` to use the configuration file at `path`. Uses + the default `builder.conf` in the mixer workspace if this option is not + provided. + + - ``-h, --help`` + + Display ``bundle validate`` help information and exit. + + - ``--strict`` + + Perform strict validation to additionally check that the bundle header + fields are parse-able and non-empty, and that the header 'Title' is itself + valid and matches the bundle filename. + + +EXIT STATUS +=========== + +On success, 0 is returned. A non-zero return code indicates a failure. + +SEE ALSO +-------- + +* `mixer(1) `__ diff --git a/_sources/reference/manpages/mixer.config.1.rst.txt b/_sources/reference/manpages/mixer.config.1.rst.txt new file mode 100644 index 00000000..2333ac7f --- /dev/null +++ b/_sources/reference/manpages/mixer.config.1.rst.txt @@ -0,0 +1,79 @@ +============ +mixer.config +============ + +------------------------------------------ +Perform configuration manipulation actions +------------------------------------------ + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``mixer config [command]`` + + +DESCRIPTION +=========== + +Perform various configuration manipulation actions on the mixer configuration +files. + + +OPTIONS +======= + +In addition to the globally recognized ``mixer`` flags (see `mixer(1) `__ for +more details), the following options are recognized. + +- ``-h, --help`` + + Display ``config`` help information and exit. + + +SUBCOMMANDS +=========== + +``convert`` + + Convert an old config file to the new TOML format. The command will generate + a backup file of the old config and will replace it with the converted one. + Environment variables will not be expanded and the values will not be + validated. In addition to the global options ``mixer config convert`` takes + the following options. + + - ``-c, --config {path}`` + + The `path` to the configuration file to convert. + + - ``-h, --help`` + + Display ``config convert`` help and exit. + +``validate`` + + Parse a builder config file and display its properties. Properties + containing environment variables will be expanded. In addition to the + global options ``mixer config validate`` takes the following options. + + - ``-c, --config {path}`` + + The `path` to the configuration file to validate. + + - ``-h, --help`` + + Display ``config validate`` help and exit. + + +EXIT STATUS +=========== + +On success, 0 is returned. A non-zero return code indicates a failure. + +SEE ALSO +-------- + +* `mixer(1) `__ diff --git a/_sources/reference/manpages/mixer.init.1.rst.txt b/_sources/reference/manpages/mixer.init.1.rst.txt new file mode 100644 index 00000000..f9932f0e --- /dev/null +++ b/_sources/reference/manpages/mixer.init.1.rst.txt @@ -0,0 +1,89 @@ +========== +mixer.init +========== + +------------------------------ +Initialize mixer and workspace +------------------------------ + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``mixer init [flags]`` + + +DESCRIPTION +=========== + +Initialize the mixer configuration and mixer workspace by writing default +configuration files if necessary and building the proper directory structure. + +OPTIONS +======= + +In addition to the globally recognized ``mixer`` flags (see `mixer(1) `__ for +more details), the following options are recognized. + +- ``--all-local`` + + Initialize mix with all local bundles automatically included. + +- ``--all-upstream`` + + Initialize the mix with all upstream bundles automatically included. + +- ``--clear-version {version}`` + + Upstream version used to compose the mix. It must be either an integer or 'latest' (default "latest") + +- ``--config {path}`` + + Supply a specific configuration file at `path` to use for mixing. + +- ``--format {format}`` + + Supply the format version for the mix. + +- ``--git`` + + Track mixer's internal work directory with git. + +- ``-h, --help`` + + Display ``init`` help information and exit. + +- ``--local-rpms`` + + Create and configure local RPM directories. + +- ``--mix-version {version}`` + + Supply the mix version to build (default is 10) + +- ``--no-default-bundles`` + + Skip adding default bundles to the mix + +- ``--upstream-url {url}`` + + Supply an upstream URL to use for mixing (default is + "https://download.clearlinux.org") + +- ``--upstream-version {version}`` + + Alias to `--clear-version` (default is "latest") + + +EXIT STATUS +=========== + +On success, 0 is returned. A non-zero return code indicates a failure. + +SEE ALSO +-------- + +* `mixer(1) `__ diff --git a/_sources/reference/manpages/mixer.repo.1.rst.txt b/_sources/reference/manpages/mixer.repo.1.rst.txt new file mode 100644 index 00000000..2c505843 --- /dev/null +++ b/_sources/reference/manpages/mixer.repo.1.rst.txt @@ -0,0 +1,85 @@ +========== +mixer.repo +========== + +--------------------------------------------------------------- +Perform various configuration actions on mixer RPM repositories +--------------------------------------------------------------- + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``mixer repo [command]`` + + +DESCRIPTION +=========== + +Perform various configuration actions on mixer RPM repositories. These RPM +repositories are used as content sources for mixer to build updates from. + + +OPTIONS +======= + +In addition to the globally recognized ``mixer`` flags (see `mixer(1) `__ for +more details), the following options are recognized. + +- ``-c, --config {path}`` + + The `path` to the configuration file to use. + +- ``-h, --help`` + + Display subcommand help information and exit. + + +SUBCOMMANDS +=========== + +``add {name} {url}`` + + Add the repo named `name` at the `url` url. In addition to the global + options ``mixer repo add`` takes the following options. + + - ``--priority`` + + Repo priority between 1 and 99, where 1 is highest. This flag defaults to 1. + +``init`` + + Initialize the DNF configuration file with the default `Clear` repository + enabled. + +``list`` + + List all RPM repositories configured in the DNF configuration file used by + mixer. + +``remove {name}`` + + Remove the repo `name` from the DNF configuration file used by mixer. + +``set-priority {name} {priority}`` + + Sets the priority for repo `name` to the provided `priority`. + +``set-url {name} {url}`` + + Sets the URL for repo `name` to the provided `url`. If `name` does not exist + the repo will be added to the configuration. + + +EXIT STATUS +=========== + +On success, 0 is returned. A non-zero return code indicates a failure. + +SEE ALSO +-------- + +* `mixer(1) `__ diff --git a/_sources/reference/manpages/mixer.versions.1.rst.txt b/_sources/reference/manpages/mixer.versions.1.rst.txt new file mode 100644 index 00000000..e0375ad8 --- /dev/null +++ b/_sources/reference/manpages/mixer.versions.1.rst.txt @@ -0,0 +1,92 @@ +============== +mixer.versions +============== + +-------------------------------- +Manage mix and upstream versions +-------------------------------- + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``mixer versions`` + +``mixer versions [command]`` + + +DESCRIPTION +=========== + +Manage mix and upstream versions. By itself the command will print the current +version of mix and upstream, and also report on the latest version of upstream +available. This command also allows the user to increment the mix version and +optionally the upstream version. + + +OPTIONS +======= + +In addition to the globally recognized ``mixer`` flags (see `mixer(1) `__ for +more details), the following options are recognized. + +- ``-h, --help`` + + Display subcommand help information and exit. + + +SUBCOMMANDS +=========== + +``update`` + + Increment the mix version to generate a new release. By default the mix + version is incremented by 10, following Clear Linux conventions to leave + room for intermediate versions if necessary. The increment can be configured + with the `--increment` flag. The ``update`` command also allows the user to + update the upstream version. + + In addition to the global options ``mixer versions update`` takes the + following options. + + - ``--clear-version {version}`` + + Alias to `--upstream-version` + + - ``-c, --config {path}`` + + Supply the `path` to the configuration file to use. + + - ``--increment {number}`` + + Amount to increment the current mix version (default is 10). + + - ``--mix-version {version}`` + + Set a specific mix `version`. + + - ``--upstream-version {version}`` + + Set a specific next upstream version (either version number or the default + "latest" string). + + - ``--skip-format-check`` + + Skip check for compatible upstream format when changing upstream version. + + This command will not update to an upstream version of a different format + ("format bumps"). At the moment this needs to be handled manually. + + +EXIT STATUS +=========== + +On success, 0 is returned. A non-zero return code indicates a failure. + +SEE ALSO +-------- + +* `mixer(1) `__ diff --git a/_sources/reference/manpages/os-core.7.rst.txt b/_sources/reference/manpages/os-core.7.rst.txt new file mode 100644 index 00000000..b71daa52 --- /dev/null +++ b/_sources/reference/manpages/os-core.7.rst.txt @@ -0,0 +1,37 @@ +======= +os-core +======= + +---------------------------------------------------- +Base files for Clear Linux OS for Intel Architecture +---------------------------------------------------- + +:Copyright: \(C\) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 7 + + +SYNOPSIS +======== + +* ``init`` +* ``root filesystem`` +* ``shell`` + + +DESCRIPTION +=========== + +os-core provides a minimal base for the Clear Linux OS for Intel +Architecture. It contains required utilities for running the init +program systemd, letting users log in and run bash shell commands. The +core also contains pieces of packages that are required to run any +other package in the distribution. + +This is the only required bundle for Clear Linux OS and can not be +removed as all other bundles depend upon it. + + +SEE ALSO +======== + +``_ diff --git a/_sources/reference/manpages/os-format.7.rst.txt b/_sources/reference/manpages/os-format.7.rst.txt new file mode 100644 index 00000000..1cb7b36b --- /dev/null +++ b/_sources/reference/manpages/os-format.7.rst.txt @@ -0,0 +1,55 @@ +========= +OS Format +========= + +-------------------------------------- +A summary of OS formats in Clear Linux +-------------------------------------- + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 7 + + +SYNOPSIS +======== + +``/usr/share/defaults/swupd/format`` + +DESCRIPTION +=========== + +A format defines a range of OS versions that have compatible update metadata and +content. An update client can update a system from the oldest version in the +format to the latest version in the format without worrying about compatibility +issues in the update content for the version it is updating to. + +A format bump occurs when the update metadata or content is changed in such a +way that will cause client updates to break. In this case the format number must +be incremented so clients will not attempt to update to the new versions in the +new format without crossing the format boundary. Update clients update only to +the latest build in their format. Once that update is complete the update client +may then update forward again because the last version in the current format has +identical content to the first version in the new format, including the new +update client needed to understand the new format. + +Because the update system in Clear Linux (``swupd``) has auto-update turned on +by default most users will never be aware of their system changing to a new +format. Those users who have disabled auto-update may occasionally see ``swupd`` +perform two updates in a row when they only invoked ``swupd update`` once. This +is because ``swupd`` detects when it crossed a format boundary and immediately +re-executes an update to carry it to the latest version in the new format. This +re-execution will actually invoke the new version of ``swupd`` that was +delivered in the first update. + +Format bumps are simply a way for Clear Linux to introduce breaking changes to +the OS without breaking user's update stream or workflow. + + +SEE ALSO +======== + +* `mixer(1) `__ +* `swupd(1) `__ +* https://clearlinux.org/documentation/ +* https://github.com/clearlinux/swupd-client/ +* https://github.com/clearlinux/mixer-tools/ diff --git a/_sources/reference/manpages/stateless.7.rst.txt b/_sources/reference/manpages/stateless.7.rst.txt new file mode 100644 index 00000000..ff3cadf1 --- /dev/null +++ b/_sources/reference/manpages/stateless.7.rst.txt @@ -0,0 +1,309 @@ +========= +stateless +========= + +--------------------------------------------------------------------------- +A guide to stateless configuration in Clear Linux OS for Intel Architecture +--------------------------------------------------------------------------- + +:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 7 + + +SYNOPSIS +======== + +``/etc/`` + +``/usr/share/defaults/`` + +``/usr/share/defaults/etc/`` + +``/var/`` + +``/var/cache/`` + +``/usr/local/`` + +``/usr/src/`` + +``/usr/lib/kernel/`` + +``/usr/lib/modules/`` + + +DESCRIPTION +=========== + +The Clear Linux OS for Intel Architecture has a unique way of +providing customization and configuration to system administrators and +users. This man page aims to provide both an explanation of what this +method is and how users of Clear Linux can use it and benefit from it. + +The goal of "stateless" is to provide a system OS that functions +without user configuration. A system should not require editing of +configuration files by the end user before it is functional, nor should +it place lengthy and confusing configuration files automatically in +user-maintained file system areas (``/etc/``) by default. And +additionally, any configuration placed in user-maintained configuration +should be removable without breaking functionality. + +This is achieved by several methods, each of which implements a part +of the stateless goal. + + +* Removal of configuration files + +The first step taken to achieve stateless configuration is to embed +proper default configuration values in the software. Any missing +critical configuration value should have a built-in default value. + +* Providing of default configuration files outside of ``/etc/`` + +Software is adjusted to use a distribution provided default +configuration file in ``/usr/share/defaults``. If no configuration +file exists in ``/etc/`` for the software, the software must use the +distribution default configuration file. + +* Allowing the end user to provide configuration in ``/etc/`` + +If the user provides a properly formatted configuration file in +the ``/etc/`` filesystem area (or, wherever it is relevant for the +software), the software is instructed to use this configuration +file instead of any other. + + +Consequences for the system administrator (user) +------------------------------------------------ + +The user should create configuration files as needed and avoid +modifying distribution provided defaults. The filesystem folders and +all content under ``/etc/`` and ``/var/`` may be modified as needed, but +the content under ``/usr/``, ``/lib/``, ``/lib64/``, ``/bin/``, ``/sbin/`` should +never be modified, and will be overwritten by `swupd(1) `__ as needed. + +Some default configuration structure and data is automatically created +under ``/etc/`` and ``/var/``. The user may remove these file system +structures entirely - a reboot of the OS should properly restore the +system to its factory default. This may also provide the user with +a way to repair a defective system configuration. + +The user should, if user configuration of a service is needed, +attempt to place the configuration file in the ``/etc/`` structure as +the service requests. Often, template files for the configuration +format can be found under the ``/usr/share/defaults/`` file structure, +and these files can be copied to the ``/etc/`` file structure. + +To modify system service configuration (``systemd``\(1) service units), +the user should not touch or modify unit files under the ``/usr/`` +file structure directly, as changes in those files will be lost after +a system software update with `swupd(1) `__. + +A list of package specific hints and best practices is listed below. In +many cases, the man pages for the respective packages also provides +detailed information as to how to configure the software. Please +consult the relevant manual pages for the software to find information +on the specific syntax and options for each software. + + +Where can I install system-wide files then? +------------------------------------------- + +`swupd(1) `__ has a list of exempted locations where the system +administrator can place files that will not get overwritten or removed +at all. The default whitelisted directories are: + + ``/usr/lib/modules`` + ``/usr/lib/kernel`` + ``/usr/local`` + ``/usr/src`` + +Using these locations for your own software is highly recommended. Not +only do these locations provide a standard FHS compliant way of adding +local software, they are sufficiently separated from OS software that +maintaining them will be much more simple. + + +ldconfig +-------- + + ``ldconfig``\(8) + +The default paths that the linker searches includes only ``/usr/lib64`` +and paths below that. This explicitly omits ``/usr/local/``. If you +compile libraries manually and install them in other paths, you may +need to configure the ``ld.so``\(8) linker to find these before you run +``ldconfig``. For example: + + ``echo "/usr/local/lib" | sudo tee -a /etc/ld.so.conf`` + + ``sudo ldconfig`` + + +systemd +------- + + ``systemd``\(1) + +Unit files can be created under ``/etc/systemd/system`` as needed and +function normally. To override unit file options, the simplest method +is to have ``systemctl``\(1) copy it for you by invoking it as: + + ``systemctl edit --full foo.service`` + +This creates an exact copy of the default unit file and invokes the +editor for the user, allowing the user to override any part of the unit. + +Unit files can be started as normal with ``systemctl start ``. + +To enable services to start at boot time, use ``systemctl enable ``. + + +sshd +---- + + ``sshd``\(8) + ``sshd_config``\(5) + +The SSH daemon has all of its configuration built in and no template +configuration file is present on the file system. The man page for +``sshd_config``\(5) explains the format, and it suffices to put only a +single option in the file + + ``/etc/ssh/sshd_config`` + +For example, to enable X11 forwarding through sshd all one has to do is +add one line containing ``X11Forwarding yes``. Other often used options +include ``PermitRootLogin yes`` to allow root ssh login access, and the +following 3 lines to disable password authentication entirely: + + ``ChallengeResponseAuthentication no`` + + ``PasswordAuthentication no`` + + ``UsePAM no`` + +To modify the listening port of sshd, one needs to determine whether +``sshd.socket`` or ``sshd.service`` is enabled first, since the methods +for changing the port number depend on whether ``sshd``\(8) is controlling +the port number, or whether ``systemd``\(1) is: + + ``systemctl is-enabled sshd.socket`` + +If enabled, the ``sshd.socket`` unit should be edited to modify the port: + + ``systemctl edit --full sshd.socket`` + +And, the user should modify the port number at ``ListenStream=`` to the +desired new port number. + +If ``sshd.service`` is enabled, the user should create, and edit a new +``/etc/ssh/sshd_config`` file: + + ``mkdir -p /etc/ssh/`` + ``vi /etc/ssh/sshd_config`` + +And add a line in that file that reads: + + ``Port 10022`` + +to, for instance, change the port number sshd.service will listen on +to port 10022. + +Root login over SSH is disabled by default and should remain disabled +for most systemd. However, in some cases this is acceptable and it can +be easily enabled by adding the following line to ``/etc/ssh/sshd_config`` +that reads: + + ``PermitRootLogin yes`` + + +nginx +----- + +Nginx ships by default in a non-functional configuration. However, +an example configuration file is present that can be used to enable +a simple server. To use this template configuration, create: + + ``mkdir -p /etc/nginx/conf.d`` + +And then copy configuration templates over to this folder: + + ``cp /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf`` + ``cp /usr/share/nginx/conf/server.conf.example /etc/nginx/conf.d/server.conf`` + +Edit the file to assure options such as SSL and PHP are enabled in +the preferred method. In the default configuration, PHP is enabled +to run listening to ``/run/php-fpm.sock``. The template file has PHP +by default disabled, but the listed example lines can be uncommented +to make the nginx service process php documents. + + +php-fpm +------- + + ``php-fpm``\(8) + +Php's default configuration file doesn't allow us to provide an +alternative as it is programmed to only read the builtin file. If you +wish to have php-fpm use a different configuration, you must pass it a +startup option to tell it where it is. This can be done by ``systemctl +edit --full php-fpm.service``. That command copies the default php-fpm +service unit to ``/etc/systemd/system/`` and allows the user to override +any option. It spawns an editor with the copy. + +Then, the user should change the line: + + ``ExecStart=/usr/sbin/php-fpm --nodaemonize`` + +to: + + ``ExecStart=/usr/sbin/php-fpm --nodaemonize --fpm-config /etc/php-fpm.conf`` + +The template php-fpm.conf can be found at ``/usr/share/defaults/php/php-fpm.conf``. +One should copy this to a place in ``/etc/``: + + ``cp /usr/share/defaults/php/php-fpm.conf /etc/php-fpm.conf`` + +Then, the user should edit ``/etc/php-fpm.conf`` and assure that +configuration options are all properly set as needed. + +Care must be taken using the default ``pool`` configuration. If needed, +the user should also create ``/etc/php-fpm.d/`` and include pool +configuration files from either ``/usr/share/defaults/php/php-fpm.d/`` or +copy them and modify them as needed as well, as well as adjust the +``include`` configuration option in ``php-fpm.conf`` to point to a new +location for pool configuration files. + +Network interface management +---------------------------- + +Clear Linux has switched the network interface management model to be fully +managed by NetworkManager when installed on real hardware. Previously, Ethernet +interfaces were managed by systemd-networkd whereas Wi-Fi and others were +managed by NetworkManager. Clear Linux cloud images continue to use +systemd-networkd. + +When updating from previous versions of Clear Linux, the installation will be +reconfigured to continue with systemd-networkd managing the Ethernet interfaces. +Since the connectivity could be lost during the procedure, physical access to the +system is required. To switch to NetworkManager for all the interfaces, the +user should disable and stop systemd-networkd: + + ``sudo systemctl disable systemd-networkd`` + ``sudo systemctl stop systemd-networkd`` + +Then, remove the file ``/etc/NetworkManager/conf.d/systemd-networkd-unmanaged.conf`` + +Finally, restart NetworkManager + + ``sudo systemctl restart NetworkManager`` + +SEE ALSO +======== + +* `swupd(1) `__ +* ``systemd``\(1) +* https://clearlinux.org/documentation/ +* https://clearlinux.org/features/stateless +* https://github.com/clearlinux/swupd-client/ diff --git a/_sources/reference/manpages/swupd-update.service.4.rst.txt b/_sources/reference/manpages/swupd-update.service.4.rst.txt new file mode 100644 index 00000000..ea04b98b --- /dev/null +++ b/_sources/reference/manpages/swupd-update.service.4.rst.txt @@ -0,0 +1,46 @@ +==================== +swupd-update.service +==================== + +--------------------- +Performs an OS update +--------------------- + +:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 4 + + +SYNOPSIS +======== + +``swupd-update.service`` + +``/usr/lib/systemd/system/swupd-update.service`` + + +DESCRIPTION +=========== + +Instructs the ``systemd``\(1) system daemon how to start and control the +``swupd`` service to perform an update. + +When this unit runs, the output will be sent to the systemd journal, and +it can be inspected with the ``journalctl``\(1) command. + +The user can disable all background updates (either started through +timers or not) by executing the following command: + +``systemctl mask swupd-update.service`` + + +ENVIRONMENT +=========== + +This unit is a ``systemd``\(1) unit file. + + +SEE ALSO +======== + +`swupd(1) `__, `swupd-update.timer(4) `__ + diff --git a/_sources/reference/manpages/swupd-update.timer.4.rst.txt b/_sources/reference/manpages/swupd-update.timer.4.rst.txt new file mode 100644 index 00000000..a7b22517 --- /dev/null +++ b/_sources/reference/manpages/swupd-update.timer.4.rst.txt @@ -0,0 +1,40 @@ +================== +swupd-update.timer +================== + +---------------------------- +Schedules periodical updates +---------------------------- + +:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 4 + + +SYNOPSIS +======== + +``swupd-update.timer`` + +``/usr/lib/systemd/system/swupd-update.timer`` + + +DESCRIPTION +=========== + +Instructs the ``systemd``\(1) system daemon when to periodically start a +system software update. The update itself may not execute if the +`swupd-update.service(4) `__ is disabled. See that manual page for +information on how to disable that unit. + + +ENVIRONMENT +=========== + +This unit is a ``systemd``\(1) unit file. + + +SEE ALSO +======== + +`swupd(1) `__, `swupd-update.service(4) `__ + diff --git a/_sources/reference/manpages/swupd.1.rst.txt b/_sources/reference/manpages/swupd.1.rst.txt new file mode 100644 index 00000000..75de4b05 --- /dev/null +++ b/_sources/reference/manpages/swupd.1.rst.txt @@ -0,0 +1,771 @@ +===== +swupd +===== + +-------------------------- +OS software update program +-------------------------- + +:Copyright: \(C) 2020 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``swupd [subcommand] `` + + +DESCRIPTION +=========== + +`swupd(1) `__ is an OS-level software update program that applies updates +to system software. + +The updates are fetched from a central software update server. If a +valid update is found on the server, it can be downloaded and applied. + +The ``swupd`` tool can also install and remove bundles, check for +updates without applying them, perform system-level diagnose of +the system software, and install an OS. + +A *version url* server provides version information. This server +notifies the program of available updates. + +A *content url* server (can be the same as *version url* server) +provides the file and metadata content for all versions. The content url +server provides metadata in the form of manifests. These Manifest files +list and describe file contents, symlinks, directories. Additionally, +the actual content is provided to clients in the form of archive files. + +``swupd`` consumes update artifacts generated by ``mixer`` in the specific +format the installed version of ``swupd`` understands. For more information +about how these artifacts are generated see `mixer(1) `__ and `os-format(7) `__. + +The ``swupd`` tool can also manage 3rd-party content, this allows users to +install, remove, and update bundles from 3rd-party repositories. + +OPTIONS +======= + +The following options are applicable to most subcommands, and can be +used to modify the core behavior and resources that swupd uses. + +-h, --help Display general help information. If put after a subcommand, it + will display help specific to that subcommand. + +-v, --version Displays the version information of the swupd program, and exit. + It also displays compile options and copyright information. + +-p , --path= Optionally set the top-level directory for the + swupd-managed system. This can be used to point to a chroot installation + of the OS or a custom mount. If not specified this will default to ``/``. + +-u , --url= Specify an RFC-3986 encoded url. The url will be used to + download version information and file content downloads. + +-P , --port= Specify the port number of the server to connect to. + Applies to both version and file content url server connections. + +-c , --contenturl= Specify an RFC-3986 encoded url. The url will be + used for file content downloads only. + +-v , --versionurl= Specify an RFC-3986 encoded url. The url will be + used to download version information. + +-F , --format= Specify the format suffix for + version file downloads. Is usually one of ``1``, ``2``, ``3``, etc. or + ``staging``. Software update formats may change regularly and normally + you should consult the swupd server data for the appropriate latest + version available. If that version is not supported by your version of + ``swupd``, you should subtract ``1`` from the number and try again until + it succeeds. + +-S , --statedir= Specify an alternate path for swupd cache and data directory. + Normally ``swupd`` uses ``/var/lib/swupd``. + +-K , --cachedir= Specify an alternate swupd cache directory. + Normally ``swupd`` uses ``/var/lib/swupd``. + +-Z , --datadir= Specify an alternate swupd data directory. + Normally ``swupd`` uses ``/var/lib/swupd``. + +-C , --certpath= Specify alternate path to swupd certificate store (pem file). + Default is /usr/share/clear/update-ca/Swupd_Root.pem + +-W , --max-parallel-downloads= Set the maximum number of parallel downloads. + +-r , --max-retries= Maximum number of retries for download failures. + +-d , --retry-delay= Initial delay in seconds between download retries, this will + be doubled for each retry until the download succeeds or the maximum + number of retries has been reached. + +-n, --nosigcheck Do not attempt to enforce certificate or signature checking. + +-n, --nosigcheck-latest Do not attempt to enforce signature checking when + retrieving the latest version number. + +-I, --ignore-time Ignore system/certificate time when validating signature. + +-t, --time Show verbose time output for swupd operations. + +-N, --no-scripts Do not run the post-update scripts and boot update tool. + +-b, --no-boot-update Do not update the boot files using clr-boot-manager. + +-j, --json-output Prints the swupd output as a machine readable JSON stream. + +-y, --yes Assume yes as answer to all prompts and run non-interactively. + +--allow-insecure-http For security reasons, swupd only allows system updates + using secure https connections by default. This option forces swupd + to allow updates over insecure http connections. + + ``Warning``: although it is not recommended, if an http server is + set up as the upstream server, the `allow_insecure_http=true` option will + need to be setup in the swupd configuration file for the autoupdate + command to continue to work. + +--quiet Sets `swupd` to print a minimal and more stable output that is easier + to parse for its commands. Only the most relevant information and errors + are printed out. + Output displayed when using this option is rarely going to change, so + this is a good option to use when writing scripts that use `swupd`. + +--verbose Enable verbosity for commands. + +--debug Print extra information to help debugging problems. + +--no-progress Don't print progress report on commands that informs the + percentage left in current operation. + +--wait-for-scripts Wait for the post-update scripts to complete. + +--assume= Sets an automatic response to all prompts and run + non-interactively. + + +SUBCOMMANDS +=========== + +info +---- + + Shows the current OS version and the URLs used for updates. + +autoupdate +---------- + + Enables or disables automatic updates, or reports current + status. Enabling updates does not cause an immediate update - + use ``swupd update`` to force one if desired. + +--enable Enable autoupdates +--disable Disable autoupdates + +check-update +------------ + + Checks whether an update is available and prints out the information + if so. Does not download update content. + +update +------ + + Performs a system software update. + + The program will contact the version server at the version url, and + check to see if a system software update is available. If an update + is available, the update content will be downloaded from the content + url and stored in the `/var/lib/swupd` state path. Once all content + is downloaded and verified, the update is applied to the system. + + In case any problem arises during a software update, the program + attempts to correct the issue, possibly by performing a ``swupd repair`` + operation, which corrects broken or missing files and other issues. + + After the update is applied, the system performs an array of + post-update actions. These actions are triggered through `systemd(1)` + and reside in the `update-triggers.target(4) `__ system target. + +-V , --version= Update to a specific version, also accepts 'latest' (default). + +-s, --status Do not perform an update, instead display whether an update is + available on the version url server, and what version number is + available. This is the same as running ``swupd check-update``. + +-k, --keepcache Do not delete the swupd state directory content after + updating the system. + +--download Do not perform an update, instead download all resources needed + to perform the update, and exit. + +--update-search-file-index Update the index used by search-file to speed up + searches. Don't enable this if you have download or space restrictions. + +--3rd-party If update is successfull, also update content from 3rd-party + repositories. + + +bundle-add +-------------------- + + Installs new software bundles. Any bundle name listed after ``bundle-add`` + will be installed in the system. A list of all existing bundles can be + displayed with the ``bundle-list --all`` command. + + The names can also be aliases that are not actual bundles names but instead + are names in an alias configuration file. See ``swupd-alias``\(7) + +--skip-optional Do not install optional bundles (`also-add` flag in + Manifests). + A bundle may include other bundles that will also get installed + when installing the bundle that includes them. This included bundles + can be either optional, or mandatory. Optional bundles can be skipped + at install time by using this option. + +--skip-diskspace-check Skip checking for available disk space before installing + a bundle. + By default, swupd attempts to determine if there is enough free + disk space to add the passed in bundle before attempting to install. + The current implementation will check free space in ``/usr/`` by default, + or it will check the passed in --path option with ``/usr/`` appended. + +bundle-remove +----------------------- + + Removes software bundles. Any bundle name listed after ``bundle-remove`` + will be removed from the system. If the bundle is required by another + bundle(s) on the system, a tree will be displayed to indicate which bundles + are blocking removal. + +-x, --force Removes a bundle along with all the bundles that depend on it. + + ``Warning``: This operation is dangerous and must be used with care since + it can remove many unexpected bundles. + +-R, --recursive Removes a bundle and its dependencies recursively, except for + bundle os-core. + + ``Warning``: This operation is dangerous and must be used with care since + it can remove many unexpected bundles. + +--orphans Removes all orphaned bundles. Orphan bundles are those that are no + longer required by any of the tracked bundles. + + ``Warning``: This operation is dangerous and must be used with care since + it can remove many unexpected bundles. + +bundle-list +----------- + + List all installed software bundles in the local system. Available bundles + can be listed with the ``--all`` option. + +-a, --all Lists all available software bundles, either installed or not, that + are available. + +-D , --has-dep= Displays a list of all bundles which include + the passed BUNDLE as a dependency. Combine with ``--all`` to report all + bundles including those not installed on the system. Combine with + ``--verbose`` to show a tree of those bundles. + +--status Show the installation status of the listed bundles. Bundles + installation status can be; "explicitly installed", meaning that they + were specifically requested to be installed by the user, or they can be + "implicitly installed", meaning they were installed as a dependency of + another explicitly installed bundle. + +--deps= Lists all bundle dependencies of the passed BUNDLE, + including recursively included bundles. + +--orphans List orphaned bundles. Orphan bundles are those that are installed + but no longer required by any tracked bundle. + +bundle-info +----------- + + Display detailed information about a bundle. + +-V , --version= Show the bundle info for the specified + version, it also accepts 'latest'.i It defaults to the current version + if no version is specified. + +--dependencies Show the bundle's direct and indirect dependencies as well as if + they are optional or mandatory dependencies. Direct dependencies are + those that are specifically included by the bundle in question, while + indirect dependencies are those that are included by the bundles that + are a direct dependency of the bundle in question. + +--files Show the files directly included in this bundle, in other words it + shows the files included in the bundle's manifest. If this option is used + along with the ``--dependencies`` option, all files installed by the + bundle are listed, including those files installed by the dependencies + of the bundle. + +search +------ + + Swupd search functionality is provided by the swupd-search binary, available + on os-core-search bundle. + + For more information run: + + ``$ swupd search --help`` + +search-file +-------------------- + + Search for matching paths in manifest data. The specified `{string}` + is matched in any part of the path listed in manifests, and all + matches are printed, including the name of the bundle in which the + match was found. + + If manifest data is not present in the state folder, it is + downloaded from the `content url`. + + Because this search consults all manifests, it normally requires to + download all manifests for bundles that are not installed, and may + result in the download of several mega bytes of manifest data. + +-V , --version= Search for a match of the given file in the + specified version version. + +-l, --library Restrict search to designated dynamic shared library paths. + +-B, --binary Restrict search to designated program binary paths. + +-T , --top= Only display the top specified number of + results for each bundle. + +-m, --csv Output the search results in a machine readable CSV format. + +-i, --init Just perform the collection and download of all required manifest + resources needed to perform the search, then exit. + +-o , --order= Sort the output in one of two ways: + + - Use 'alpha' to order alphabetically (default) + - Use 'size' to order by bundle size (smaller to larger) + +diagnose +-------- + + Perform system software installation verification. The program will + obtain all the manifests needed from version url and content url to + establish whether the system software is correctly installed and not + overwritten, modified, missing or otherwise incorrect (permissions, etc.). + + After obtaining the proper resources, all files that are under + control of the software update program are verified according to the + manifest data + +-V , --version= Diagnose against the specified manifest VERSION. + +-x, --force Attempt to proceed even if non-critical errors found. + +-q, --quick Omit checking hash values. Instead only looks for missing files + and directories and/or symlinks. + +-B , --bundles= Forces swupd to only diagnose the (comma separated) list of bundles provided. + + Examples: + + - ``--bundles xterm,vim`` + + Diagnoses only bundles `xterm` and `vim`. + +-Y, --picky Also list files which should not exist. Only files listed in the + manifests should exist. By default swupd only looks for these + files at ``/usr``, this path can be changed using ``--picky-tree``. + Some paths at ``/usr`` are skipped by default: + ``/usr/lib/modules``, ``/usr/lib/kernel``, ``/usr/local`` + and ``/usr/src``. These paths can be changed using + ``--picky-whitelist``. + +-X , --picky-tree= Changes the path where ``--picky`` and + ``--extra-files-only`` looks for extra files. To be specified as + absolute PATH. + The default path is ``/usr``. + +-w , --picky-whitelist= Any path matching the POSIX extended regular expression regex is ignored by ``--picky``. The given expression is always + wrapped in ``^(`` and ``)$`` and thus has to match the entire path. + Matched directories get skipped completely. + + The default is to ignore ``/usr/lib/kernel``, + ``/usr/lib/modules``, ``/usr/src`` and ``/usr/local``. + + Examples: + + - ``/var|/etc/machine-id`` + + Ignores ``/var`` or ``/etc/machine-id``, regardless of + whether they are directories or something else. In the + usual case that ``/var`` is a directory, also everything + inside it is ignored because the directory gets skipped + while scanning the directory tree. + + - empty string or ``^$`` + + Matches nothing, because `paths` are never empty. + +--extra-files-only Like ``--picky``, but it only looks for extra files. + It omits checking hash values, and for missing files, directories and/or + symlinks. + +--file Forces swupd to only diagnose the specified file or directory + (recursively). + +repair +------ + + Correct any issues found. This will overwrite incorrect file content, + add missing files and do additional corrections, permissions, etc. + +-V , --version= Repair against the specified manifest + version. + +-x, --force Attempt to proceed even if non-critical errors found. + +-q, --quick Omit repairing corrupt files. Instead only add missing files + and directories and/or symlinks. + +-B , --bundles= Forces swupd to only repair the (comma separated) list + of bundles provided. + + Examples: + + - ``--bundles xterm,vim`` + + Repairs only bundles `xterm` and `vim`. + +-Y, --picky Also removes files which should not exist. Only files listed + in the manifests should exist. By default swupd only looks for these + files at ``/usr``, this path can be changed using ``--picky-tree``. + Some paths at ``/usr`` are skipped by default: + ``/usr/lib/modules``, ``/usr/lib/kernel``, ``/usr/local`` + and ``/usr/src``. These paths can be changed using + ``--picky-whitelist``. + +-X , --picky-tree= Changes the path where ``--picky`` and + ``--extra-files-only`` looks for extra files. To be specified as + absolute PATH. The default path is ``/usr``. + +-w , --picky-whitelist= Any path matching the POSIX extended regular + expression regex is ignored by ``--picky``. The given expression is + always wrapped in ``^(`` and ``)$`` and thus has to match the entire + path. Matched directories get skipped completely. + + The default is to ignore ``/usr/lib/kernel``, + ``/usr/lib/modules``, ``/usr/src`` and ``/usr/local``. + + Examples: + + - ``/var|/etc/machine-id`` + + Ignores ``/var`` or ``/etc/machine-id``, regardless of + whether they are directories or something else. In the + usual case that ``/var`` is a directory, also everything + inside it is ignored because the directory gets skipped + while scanning the directory tree. + + - empty string or ``^$`` + + Matches nothing, because paths are never empty. + +--extra-files-only Like ``--picky``, but it only removes extra files. It omits + repairing corrupt files, and adding missing files, directories and/or + symlinks. + +--file Forces swupd to only repair the specified file or directory + (recursively). + +os-install +---------- + + Perform system software installation in the specified location. Install + all files into `{path}` as specified by the ``swupd os-install {path}`` + option. Useful to generate a new system root. The only bundle that will + be installed by default is ``os-core`` unless more bundles are specified + with the ``--bundles`` option. + +-V , --version= Install the specified version of the OS. + +-x, --force Attempt to proceed even if non-critical errors found. + +-B , --bundles= Include the (comma separated) list of + bundles with the base OS install. + + Examples: + + - ``--bundles xterm,vim`` + + Installs bundles `xterm` and `vim`, along with `os-core` + (installed by default). + +-s , --statedir-cache= After checking for content in the + `statedir`, check the `statedir-cache` before downloading it over the + network. + +--download Do not perform an install, instead download all resources + needed to perform the install, and exit. + +--skip-optional Do not install optional bundles (`also-add` flag in + Manifests). + A bundle may include other bundles that will also get installed + when installing the bundle that includes them. This included bundles + can be either optional, or mandatory. Optional bundles can be skipped + at install time by using this option. + +mirror +------ + + Configure a `mirror URL` for swupd to use instead of the defaults on the + system or compiled into the swupd binary. + +-s , --set= Set the `content` and `version URLs` to URL by adding + configuration files to ``/etc/swupd/mirror_contenturl`` and + ``/etc/swupd/mirror_versionurl`` + +-U, --unset Remove the `content` and `version URL` configuration by removing + ``/etc/swupd`` + +clean +----- + + Removes files cached by swupd. + + Note that removing these files may cause swupd to perform slower the next time + it is used since it may need to download some files from the update server + again. + +--all | Removes all the content including recent metadata. + +--dry-run Just prints files that would be removed. + +hashdump +-------- + + Calculates and print the Manifest hash for a specific file on disk. + +-n, --no-xattrs Ignore extended attributes when calculating hash. + +-p , --path= Specify the PATH to use for operations. This can be + used to point to a chroot installation of the OS or a custom mount. + +3rd-party +--------- + + Manages 3rd-party repositories and content installed from them. A 3rd-party + repository enables the distribution of user produced content. + + The following subcommands are available to manage `3rd-party repositories`: + + - ``add`` + + Adds a 3rd-party repository. + + - ``force`` + + Attempt to proceed with the removal of the repo even if non-critical + errors found. + + - ``remove`` + + Removes a 3rd-party repository along with all the content installed + from it from the system. + + - ``force`` + + Attempt to proceed with the removal of the repo even if non-critical + errors found. + + - ``list`` + + Lists the 3rd-party repositories available to the system. These + repositories must have been previously added using ``swupd 3rd-party add``. + + Most of the swupd subcommands used for managing `upstream` content are + supported to manage `3rd-party` content along with most of their options. + To use these subcommands for 3rd-party content, it is necessary to use the + ``3rd-party`` subcommand followed by the desired operation to be performed. + + This is the syntax for 3rd-party operations to manage content: + + ``$ swupd 3rd-party [option(s)]`` + + Example: + + - ``swupd 3rd-party bundle-add my_bundle`` + + Looks for the 3rd-party bundle `my_bundle` among all the available + 3rd-party repositories, and installs it in the system as long as + it is found in one, and only one, repository. If the bundle exists + in more than one 3rd-party repository, users are required to specify + the repository to install it from by using the ``--repo`` option. + + There is no need to specify the 3rd-party repository if the bundle + name is unique among 3rd-party repositories, even if a bundle with + the same name exists in the upstream update server. Bundles from + 3rd-party repositories are installed in a different location so they + don't clash with upstream bundles. + + - ``swupd 3rd-party update --repo my_repo`` + + Performs a software update for content installed from the 3rd-party + repository `my_repo`. If no repository is specified, content from + all 3rd-party repositories is updated. + + All 3rd-party content is installed in the following location: + ``/opt/3rd-party//`` + + The following subcommands are available to manage `3rd-party content`: + + - ``update`` + + Update to latest version of a 3rd-party repository. + For information about the options for this command please refer to + the ``swupd update`` section. + + - ``bundle-add`` + + Installs a bundle from a 3rd-party repository. + For information about the options for this command please refer to + the ``swupd bundle-add`` section. + + - ``bundle-remove`` + + Remove a bundle from a 3rd-party repository. + For information about the options for this command please refer to + the ``swupd bundle-remove`` section. + + - ``bundle-list`` + + List bundles from a 3rd-party repository. + For information about the options for this command please refer to + the ``swupd bundle-list`` section. + + - ``bundle-info`` + + Display information about a bundle in a 3rd-party repository. + For information about the options for this command please refer to + the ``swupd bundle-info`` section. + + - ``diagnose`` + + Verify content from a 3rd-party repository. + For information about the options for this command please refer to + the ``swupd diagnose`` section. + + - ``repair`` + + Repair local issues relative to a 3rd-party repository. + For information about the options for this command please refer to + the ``swupd repair`` section. + + - ``check-update`` + + Check if a new version of a 3rd-party repository is available. + For information about the options for this command please refer to + the ``swupd check-update`` section. + + - ``clean`` + + Clean cached files of a 3rd-party repository. + For information about the options for this command please refer to + the ``swupd clean`` section. + + +FILES +===== + +/usr/share/defaults/swupd + + Sometimes a set of flags is always used for one, or many swupd commands. The + ``swupd configuration file`` provides a convenient way of persistently define + these flags so they don't need to be specified every time a command is run. + + The configuration file is an INI type of file that consists of sections, each led + by a [section] header, followed by key/value entries separated by a '=' character. + Note that there should be no whitespace between key=value. The configuration + file may include comments, prefixed by either the '#' or the ';' characters. + + There can be one section for each swupd command (e.g. [bundle-add], [update], etc.) + and one for global options (e.g. [GLOBAL]). Global options can be specified in the + either in the GLOBAL section, in a command section, or in both. Global options + specified in the command section have higher precedence than those specified in the + GLOBAL section, so it is possible to define a GLOBAL option that will apply to all + swupd command except for that one overwritten in the command section. + + A sample swupd configuration file can be found at this location (this file should not + be modified): + `/usr/share/defaults/swupd` + + To use it, copy it to `/etc/swupd` where swupd reads the configuration from. + + +EXIT STATUS +=========== + +On success, ``0`` is returned. A ``non-zero`` return code signals a failure. + +If the subcommand ``check-update`` was specified, the program returns +``0`` if an update is available, ``1`` if no update available, and a +return value higher than ``1`` signals a failure. + +If the subcommand was ``autoupdate`` without options, then the program +returns ``0`` if automatic updating is enabled. + +If the subcommand was ``diagnose``, then the program returns ``0`` if the system +is consistent at the end of the process or ``1`` if there are invalid/missing +files in the system. + +The non-zero return codes for other operations are listed here: + +**2** A required bundle was removed or was attempted to be removed + +| **3** The specified bundle is invalid +| **4** Unable to download or read MoM manifest +| **5** Unable to delete a file +| **6** Unable to rename a directory +| **7** Unable to create a file +| **8** Unable to recursively load included manifests +| **9** Unable to obtain lock on state directory +| **10** Unable to rename a file +| **11** Unable to initialize curl agent +| **12** Initialization error +| **13** Bundle not tracked on system +| **14** Unable to load manifest into memory +| **15** Invalid command-line option +| **16** Unable to connect to update server +| **17** File download issue +| **18** Unable to untar a file +| **19** Unable to create required directory +| **20** Unable to determine current version of the OS +| **21** Unable to initialize signature verification +| **22** System time is off by a large margin +| **23** Pack download issue +| **24** Unable to verify server SSL certificate +| **25** There is not enough disk space left (or it cannot be determined) +| **26** The required path was not found in any manifest +| **27** Unexpected condition found +| **28** Unable to execute another program in a subprocess +| **29** Unable to list the content of a directory +| **30** An error occurred computing the hash of a file +| **31** Unable to get current system time +| **32** Unable to write a file +| **34** swupd ran out of memory +| **35** Unable to fix/replace/delete one or more files +| **36** Unable to execute binary, is either missing or invalid +| **37** Invalid 3rd-party repository (not found) +| **38** File is missing or invalid + + +SEE ALSO +======== + +| `swupd-update.service(4) `__, `swupd-update.timer(4) `__, `update-triggers.target(4) `__, `mixer(1) `__, `os-format(7) `__ + +| Official repository https://github.com/clearlinux/swupd-client/ +| Official documentation https://clearlinux.org/documentation/ diff --git a/_sources/reference/manpages/tallow.1.rst.txt b/_sources/reference/manpages/tallow.1.rst.txt new file mode 100644 index 00000000..d04c6b80 --- /dev/null +++ b/_sources/reference/manpages/tallow.1.rst.txt @@ -0,0 +1,63 @@ +tallow +====== + +Reduce log clutter due to ssh login attempts. + +SYNOPSIS +-------- + +``/usr/sbin/tallow`` + +DESCRIPTION +----------- + +``tallow`` is a daemon that watches the systemd journal for messages +from the ``sshd`` service. It parses the messages and looks for +attempted random logins such as failed logins to the root account and +failed logins to invalid user accounts, and various other obviously +malicious login attempts that try things as forcing old protocols, or +weak key systems. + +If such logins were detected, the offending IP address is stored in a +list. Items from this list are regularly purged, but if the amount of +times that a specific IP address is seen exceeds a threshold, an +ipset(1) entry is inserted in the ``tallow`` or ``tallow6`` ipset, and +further packets from that ip address will be blocked by an +``iptables(1)`` or ``ip6tables(1)`` rule that tallow creates at startup. +Additionally, certain types of login failure will trigger a short term +ban of further packets from the offending IP address immediately. + +The system administrator needs to assure that the tallow and tallow6 +ipsets are left alone and that the inserted iptables rules are properly +matching on packets. + +Care should be taken to assure that legitimate users are not blocked +inadvertently. You may wish to list any valid IP address with the +whitelist option in `tallow.conf(5) `__. Multiple addresses can be +whitelisted. + +OPTIONS +------- + +The ``tallow`` daemon itself has no runtime configuration. All +configuration is done through the `tallow.conf(5) `__ config file. + +SIGNALS +------- + +The ``USR1`` signal causes ``tallow`` to print out it’s internal +tracking table of IP addresses. This requires that tallow is compiled +with the ``-DDEBUG=1`` symbol passed to the compiler. + +SEE ALSO +-------- + +systemd-journald(1), iptables(1), ipset(1), `tallow.conf(5) `__, +`tallow.patterns(5) `__ + +BUGS +---- + +``tallow`` is ``NOT A SECURITY SOLUTION``, nor does it protect against +random password logins. A attacker may still be able to logon to your +systems if you allow password logins. diff --git a/_sources/reference/manpages/tallow.conf.5.rst.txt b/_sources/reference/manpages/tallow.conf.5.rst.txt new file mode 100644 index 00000000..7f8b5b0b --- /dev/null +++ b/_sources/reference/manpages/tallow.conf.5.rst.txt @@ -0,0 +1,92 @@ +tallow.conf +=========== + +The tallow configuration file + +NAME +---- + +tallow.conf - Tallow daemon configuration file + +SYNOPSIS +-------- + +``/etc/tallow.conf`` + +DESCRIPTION +----------- + +This file is read on startup by the `tallow(1) `__ daemon, and can be used to +provide options to the tallow daemon. If not present, tallow will +operate with built-in defaults. + +OPTIONS +------- + +``fwcmd_path``\ =\ ```` Specifies the location of the ipset(1) +firewall-cmd(1) programs. By default, tallow will look in “/usr/sbin” +for them. + +``ipt_path``\ =\ ```` Specifies the location of the ipset(1) +program and iptables(1) or ip6tables(1) programs. By default, tallow +will look in “/usr/sbin” for them. + +``expires``\ =\ ```` The number of seconds that IP addresses are +blocked for. Note that due to the implementation, IP addresses may be +blocked for much longer than this period. If IP addresses are seen, but +not blocked within this period, they are also removed from the watch +list. Defaults to 3600s. + +``whitelist``\ =\ ```` Specify an IP address or +``pattern`` that should never be blocked. Multiple IP addresses can be +included by repeating the ``whitelist`` option several times. By +default, 127.0.0.1, 192.168., and 10. are whitelisted. If you create a +manual whitelist, you must include these entries if you want to continue +them to be whitelisted as well, otherwise they will be omitted from the +whitelist. + +If the last character of the listed ip adress is a ``.`` or a ``:``, +then the matching is only performed on the leftmost characters of an IP +address against the whitelist entry. For instance, if you whitelist +``10.`` then all IP addresses in the ``10/8`` subnet mask will match +this whitelist entry and never be blocked. + +``ipv6``\ =\ ``<0|1>`` Enable or disable ipv6 (ip6tables) support. Ipv6 +is disabled automatically on systems that do not appear to have ipv6 +support and enabled when ipv6 is present. Use this option to explicitly +disable ipv6 support if your system does not have ipv6 or is missing +ip6tables. Even with ipv6 disabled, tallow will track and log ipv6 +addresses. + +``nocreate``\ =\ ``<0|1>`` Disable the creation of firewall rules and +ipset sets. By default, tallow will create new firewall-cmd(1) or +iptables(1) and ip6tables(1) rules when needed automatically. If set to +``1``, `tallow(1) `__ will not create any new firewall DROP rules or +ipset sets that are needed work. You should create them manually before +tallow starts up and remove them afterwards using the sets of commands +below. + +Use the following commands if you’re using iptables(1): + +:: + + ipset create tallow hash:ip family inet timeout 3600 + iptables -t filter -I INPUT 1 -m set --match-set tallow src -j DROP + + ipset create tallow6 hash:ip family inet6 timeout 3600 + ip6tables -t filter -I INPUT 1 -m set --match-set tallow6 src -j DROP + +Use the following commands if you’re using firewalld(1): + +:: + + firewall-cmd --permanent --new-ipset=tallow --type=hash:ip --family=inet --option=timeout=3600 + firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 1 -m set --match-set tallow src -j DROP + + firewall-cmd --permanent --new-ipset=tallow6 --type=hash:ip --family=inet6 --option=timeout=3600 + firewall-cmd --permanent --direct --add-rule ipv6 filter INPUT 1 -m set --match-set tallow6 src -j DROP + +SEE ALSO +-------- + +`tallow(1) `__, `tallow.patterns(5) `__ diff --git a/_sources/reference/manpages/tallow.patterns.5.rst.txt b/_sources/reference/manpages/tallow.patterns.5.rst.txt new file mode 100644 index 00000000..f0f68724 --- /dev/null +++ b/_sources/reference/manpages/tallow.patterns.5.rst.txt @@ -0,0 +1,136 @@ +tallow.patterns +=============== + +Tallow pattern matching configuration files. + +SYNOPSIS +-------- + +`tallow(1) `__ uses regular expressions to match journal entries and extract +an IP address from them. JSON files are used to configure the patterns +and banning thresholds used by `tallow(1) `__. + +``/etc/tallow/*.json`` ``/usr/share/tallow/*.json`` + +DESCRIPTION +----------- + +`tallow(1) `__ uses regular expressions to match journal entries and extract +an IP address from them. JSON files are used to configure the patterns +and banning thresholds used by `tallow(1) `__. This adds the ability to +extend the patterns `tallow(1) `__ will recognize. Many JSON files can exist +for logical grouping. The `tallow(1) `__ daemon will read all JSON files in +the configuration directories at startup. + +`tallow(1) `__ operates with default pattern definitions +in\ ``/usr/share/tallow/*.json``. Users can add more patterns with their +own JSON files under ``/etc/tallow``. The default JSON files can be +overridden by creating the same file under ``/etc/tallow``. + +FILE FORMAT +----------- + +Pattern configuration files use the JavaScript Object Notation (JSON) +format. + +The JSON must be two levels deep and all properties are required. The +root object is an array containing objects with a ``filter`` key and an +``items`` key. + +- ``filter`` is a string that defines a field for filtering the journal + file. This helps make sure patterns are only matched to a subset of + journal entries. See systemd.journal-fields(7) for valid journal + fields. + +- ``items`` is an array of objects that contains three elements: + ``ban``, ``score``, and ``pattern``. + + - ``ban`` is an integer that defines the number of seconds to ban + originating IP for. If this value is > 0, the IP address get + banned immediately when a journal entry matches ``pattern``. + + - ``score`` is a double that defines a value to add to the + accumulated “score” of an originating IP address each time a + journal entry matches the ``pattern``. If the combined score is > + 1.0, tallow bans the originating IP for the default time of 1 + hour. The ``ban`` element value above is not used for bans made + due to ``score``. + + - ``pattern`` is a string that defines a Perl Compatible Regular + Expressions (PCRE) to match against the filtered journal entries. + The PCRE should extract exactly one substring: the originating IP + address for `tallow(1) `__. See systemd.journal-fields(7) for valid + journal fields. + +EXAMPLES +-------- + +1. The JSON below is a snippet from one of the default pattern + configuration files for blocking certain failed ``sshd`` connections. + + The first pattern will ban an IP address after it fails to login 6 + times causing it to reach a total score > 1.0. + + The second pattern will ban an IP address for 10 seconds every time a + login is attempted with an invalid user. Additionally, it will ban + the IP address for 1 hour if it attempts to login with an invalid + user 6 times causing it to reach a total score > 1.0. + + See the ``/usr/share/tallow/sshd.json`` file for more ``sshd`` + examples. + + :: + + [ + { + "filter": "SYSLOG_IDENTIFIER=sshd", + "items": [ + { + "ban": 0, + "score": 0.2, + "pattern": "MESSAGE=Failed .* for .* from ([0-9a-z:.]+) port \\d+ ssh2" + }, + { + "ban": 10, + "score": 0.2, + "pattern": "MESSAGE=Invalid user .* from ([0-9a-z:.]+) port \\d+" + } + ] + } + ] + +2. The JSON below defines a pattern for blocking connections based on + error logs from ``nginx-mainline`` if placed in a + ``/etc/tallow/nginx-mainline.json`` file. + + The pattern will ban an IP address for 15 seconds every time it + attempts to access a script that does not exist. Additionally, it + will ban the IP address for 1 hour if it attempts to access invalid + scripts 4 times causing it to reach a total score > 1.0. + + :: + + [ + { + "filter": "SYSLOG_IDENTIFIER=nginx-mainline", + "items": [ + { + "ban": 15, + "score": 0.3, + "pattern": ".Primary script unknown. while reading response header from upstream, client: ([0-9a-z:.]+)," + } + ] + } + ] + +SEE ALSO +-------- + +`tallow(1) `__, `tallow.conf(5) `__ + +BUGS +---- + +``tallow`` is ``NOT A SECURITY SOLUTION``, nor does it protect against +random password logins. An attacker may still be able to logon to your +systems if you allow password logins. diff --git a/_sources/reference/manpages/telem-record-gen.1.rst.txt b/_sources/reference/manpages/telem-record-gen.1.rst.txt new file mode 100644 index 00000000..702cc6e5 --- /dev/null +++ b/_sources/reference/manpages/telem-record-gen.1.rst.txt @@ -0,0 +1,81 @@ +================ +telem-record-gen +================ + +------------------------------ +Create custom telemetry record +------------------------------ + +:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``telem-record-gen`` + + +DESCRIPTION +=========== + +Generate and attempt to send a telemetry record to `telemprobd(1) `__. This +program can be used to create a telemetry record from a shell script +or from another program using e.g. ``system()``. The program connects +to a local `telemprobd(1) `__ daemon to spool the record for delivery to +the actual collection service. + +The payload can be provided as standard input, as a file with the +``--payload-file`` option, or as a string argument with the ``--payload`` +option. If neither the ``--payload`` or ``--payload-file`` is provided, +the payload is assumed to be on . + +Each record is expected to have a class identifier, a record version +number, and a severity level. A class identifier is required to be +provided. + + +OPTIONS +======= + + * ``-h``, ``--help``: + Show help options. + + * ``-V``, ``--version``: + Print the program version. + + * ``-f``, ``--config-file`` : + Path to configuration file (not implemented yet). + + * ``-s``, ``--severity``: + Severity level (1-4) - (default 1). + + * ``-c``, ``--class``: + Classification ``level_1``/``level_2``/``level_3``. + + * ``-p``, ``--payload`` : + Record body (max size = 8k). + + * ``-P``, ``--payload-file`` : + File to read payload from. + + * ``-R``, ``--record-version`` : + Version number for format of payload (default 1). + + * ``-e``, ``--event-id`` : + Event id to use in the record. If not provided a randomly generated id will be assigned to record. + + + +RETURN VALUES +============= + +0 on success. A non-zero exit code indicates a failure occurred. + + +SEE ALSO +======== + +* `telemprobd(1) `__ +* https://github.com/clearlinux/telemetrics-client +* https://clearlinux.org/documentation/ diff --git a/_sources/reference/manpages/telemctl.1.rst.txt b/_sources/reference/manpages/telemctl.1.rst.txt new file mode 100644 index 00000000..0a003fc5 --- /dev/null +++ b/_sources/reference/manpages/telemctl.1.rst.txt @@ -0,0 +1,58 @@ +======== +telemctl +======== + +------------------------------------- +Telemetry service administration tool +------------------------------------- + +:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``telemctl`` + +``/etc/telemetrics/opt-in`` + + +DESCRIPTION +=========== + +Control actions for telemetry services. The command can be used to start, +restart, or stop `telemprobd(1) `__ and `telempostd(1) `__, or to opt-in or opt-out of telemetry delivery of records to a central telemetry service. + + +OPTIONS +======= + + * ``start``|``stop``|``restart``: + Starts, stops or restarts all running telemetry services. + + * ``opt-in``: + Opts in to telemetry and the opt-in file ``/etc/telemetrics/opt-in`` + is created. Note: this is a one time required operation before + telemetry can be used the first time. + + * ``opt-out``: + Opts out of telemetry, and stops telemetry services. The opt-in file + ``/etc/telemetrics/opt-in`` is deleted. + + * ``is-active``: + Checks if telemetry client daemons are active (telemprobd and telempostd). + + +RETURN VALUES +============= + +0 on success. A non-zero exit code indicates a failure occurred. + + +SEE ALSO +======== + +* `telemprobd(1) `__ +* https://github.com/clearlinux/telemetrics-client +* https://clearlinux.org/documentation/ diff --git a/_sources/reference/manpages/telemetrics.conf.5.rst.txt b/_sources/reference/manpages/telemetrics.conf.5.rst.txt new file mode 100644 index 00000000..1d242ef8 --- /dev/null +++ b/_sources/reference/manpages/telemetrics.conf.5.rst.txt @@ -0,0 +1,109 @@ +================ +telemetrics.conf +================ + +------------------------------------ +Telemetry service configuration file +------------------------------------ + +:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 5 + + +SYNOPSIS +======== + +``/etc/telemetrics/telemetrics.conf`` + +``/usr/share/defaults/telemetrics/telemetrics.conf`` + + +DESCRIPTION +=========== + +This file contains configuration parameters for the `telemprobd(1) `__ and `telempostd(1) `__ telemetry service daemons. The daemon reads this file at startup if it exists. + + +SYNTAX +====== + +The configuration file contains ``key=value`` pairs, formatted as plain +text, one option per line. Comments can be added by preceding them with the +``#`` character. All configuration options should be in a section marked +with ``[settings]``. + + +OPTIONS +======= + +- ``server=`` + + Server URL including protocol designator. + +- ``socket_path=`` + + Path to the socket that `telemprobd` will listen on. + +- ``cainfo=`` + + Certificate file to use for validation of SSL endpoint. + +- ``tidheader=
`` + + Telemetry ID in header format, usually ``tidheader=X-Telemetry-TID:`` + post header used to group records in ingestion service, which may be + ingesting for more than one set of clients. Can be set to any string. + +- ``record_expiry=`` + + Record expiry time in minutes. + +- ``spool_dir=`` + + Local spool directory used to store records being processed. + +- ``spool_max_size=`` + + maximum size of the spool directory in kB. A value of ``-1`` indicates + no limit. The block size of the files in this directory is considered, + and not the actual size of the record itself. + +- ``spool_process_time=`` + + Time in seconds for processing spool. Valid range: 120..300. Values + outside this range are clamped. + +- ``rate_limit_enabled=`` + + Enable rate limiting. If this is set to false then all rate-limiting + disabled. It is possible to disable each rate-limit individually below. + +- ``record_burst_limit=`` + + Rate limiting record burst limit. Valid Range: 0..``INT_MAX``, -1 = disabled. + +- ``record_window_length=`` + + Rate limiting record window length in minutes. Valid Range: 0..59. + +- ``byte_burst_limit=`` + + Rate limiting byte burst limit. Valid Range: 0..`INT_MAX`, -1 = disabled. + +- ``byte_window_length=`` + + Rate limiting byte window length in minutes. Valid Range: 0..59. + +- ``rate_limit_strategy=`` + + Rate limit strategy - what to do with record if rate-limiting prevents + delivery over network. Valid stategies: ``spool``, ``drop``. + + +SEE ALSO +======== + +* `telemprobd(1) `__ +* `telempostd(1) `__ +* https://github.com/clearlinux/telemetrics-client +* https://clearlinux.org/documentation/ diff --git a/_sources/reference/manpages/telemetry.3.rst.txt b/_sources/reference/manpages/telemetry.3.rst.txt new file mode 100644 index 00000000..096bf9d7 --- /dev/null +++ b/_sources/reference/manpages/telemetry.3.rst.txt @@ -0,0 +1,71 @@ +========= +telemetry +========= + +---------------------------------------------- +C programming interface for telemetrics-client +---------------------------------------------- + +:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 3 + + +SYNOPSIS +======== + +``#include "telemetry.h"`` + +``struct telem_ref { struct telem_record *record; };`` + +``int tm_create_record(struct telem_ref **t_ref, uint32_t severity, char *classification, uint32_t payload_version)`` + +``int tm_set_payload(struct telem_ref *t_ref, char *payload)`` + +``int tm_send_record(struct telem_ref *t_ref)`` + +``void tm_free_record(struct telem_ref *t_ref)`` + +``int tm_set_config_file(const char *c_file)`` + +``int tm_is_opted_in(void)`` + + +DESCRIPTION +=========== + +The functions in the telemetry library facilitate the delivery of +telemetry data to the `telemprobd(1) `__ service. + +The function ``tm_create_record()`` initializes a telemetry record and +sets the severity and classification of that record, as well as the +payload version number. The memory needed to store the telemetry record +is allocated and should be freed with ``tm_free_record()`` when no longer +needed. + +The function ``tm_set_payload()`` attaches the provided telemetry record +data to the telemetry record. The current maximum payload size is 8192b. + +The function ``tm_send_record()`` delivers the record to the local +`telemprobd(1) `__ service. + +The function ``tm_set_config_file()`` can be used to provide an alternate +configuration path to the telemetry library. + +``tm_is_opted_in`` is a utility provided to check if the one time opt-in +has been performed. + +RETURN VALUES +============= + +All these functions return ``0`` on success, or a non-zero return value +if an error occurred. The function ``tm_free_record()`` does not return +any value. ``tm_is_opted_in`` returns ``1`` when telemetry is opted-in +otherwise ``0``. + + +SEE ALSO +======== + +* `telemprobd(1) `__ +* https://github.com/clearlinux/telemetrics-client +* https://clearlinux.org/documentation/ diff --git a/_sources/reference/manpages/telempostd.1.rst.txt b/_sources/reference/manpages/telempostd.1.rst.txt new file mode 100644 index 00000000..64b7d4c7 --- /dev/null +++ b/_sources/reference/manpages/telempostd.1.rst.txt @@ -0,0 +1,71 @@ +========== +telempostd +========== + +------------------------ +Telemetry client service +------------------------ + +:Copyright: \(C) 2018 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``telempostd`` \ + +``/etc/telemetrics/telemetrics.conf`` + +``/usr/share/defaults/telemetrics/telemetrics.conf`` + +``/etc/telemetrics/opt-in-static-machine-id`` + + +DESCRIPTION +=========== + +The ``telempostd`` program delivers locally generated telemetry records to a remote +telemetry service. Telemetry data can be in any format, and is relayed as-is. + + +OPTIONS +======= + + * ``-f``, ``--config_file`` \[\\]: + Configuration file. This overrides the other parameters. + + * ``-h``, ``--help``: + Display this help message. + + * ``-V``, ``--version``: + Print the program version. + + +FILES +===== + +* ``/usr/share/defaults/telemetrics/telemetrics.conf`` + + If no custom configuration file is found, ``telempostd`` uses the + settings in this file. + +* ``/etc/telemetrics/telemetrics.conf`` + + Custom configuration file that ``telempostd`` reads. See `telemetrics.conf(5) `__. + + +EXIT STATUS +=========== + +0 when no errors occurred. A non-zero exit status indicates a failure occurred. + + +SEE ALSO +======== + +* `telemetry(3) `__ +* `telemetrics.conf(5) `__ +* https://github.com/clearlinux/telemetrics-client +* https://clearlinux.org/documentation/ + diff --git a/_sources/reference/manpages/telemprobd.1.rst.txt b/_sources/reference/manpages/telemprobd.1.rst.txt new file mode 100644 index 00000000..7bb2657c --- /dev/null +++ b/_sources/reference/manpages/telemprobd.1.rst.txt @@ -0,0 +1,78 @@ +========== +telemprobd +========== + +------------------------ +Telemetry probes service +------------------------ + +:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 1 + + +SYNOPSIS +======== + +``telemprobd`` \ + +``/etc/telemetrics/telemetrics.conf`` + +``/usr/share/defaults/telemetrics/telemetrics.conf`` + +``/etc/telemetrics/opt-in-static-machine-id`` + + +DESCRIPTION +=========== + +The ``telemprobd`` program handles communication between telemetry client and telemetry +probes. + + +OPTIONS +======= + + * ``-f``, ``--config_file`` \[\\]: + Configuration file. This overides the other parameters. + + * ``-h``, ``--help``: + Display this help message. + + * ``-V``, ``--version``: + Print the program version. + + +FILES +===== + +* ``/usr/share/defaults/telemetrics/telemetrics.conf`` + + If no custom configuration file is found, ``telemprobd`` uses the + settings in this file. + +* ``/etc/telemetrics/telemetrics.conf`` + + Custom configuration file that ``telemprobd`` reads. See `telemetrics.conf(5) `__. + + +* ``/etc/telemetrics/opt-in-static-machine-id`` + + If this file exists, the first line of this file will be sent to + the server as the machine ID. Otherwise, the machine ID will be + random and changed regularly. + + +EXIT STATUS +=========== + +0 when no errors occurred. A non-zero exit status indicates a failure occurred. + + +SEE ALSO +======== + +* `telemetry(3) `__ +* `telemetrics.conf(5) `__ +* https://github.com/clearlinux/telemetrics-client +* https://clearlinux.org/documentation/ + diff --git a/_sources/reference/manpages/ucd-data-fetch.1.rst.txt b/_sources/reference/manpages/ucd-data-fetch.1.rst.txt new file mode 100644 index 00000000..fdaba950 --- /dev/null +++ b/_sources/reference/manpages/ucd-data-fetch.1.rst.txt @@ -0,0 +1,53 @@ +ucd-data-fetch +============== + +`ucd-data-fetch(1) `__ – Fetch user-data and ssh keys from cloud services + +SYNOPSIS +-------- + +``/usr/bin/ucd-data-fetch `` + +DESCRIPTION +----------- + +``ucd-data-fetch`` is a helper agent program that fetches openstack +user-data from the http://169.254.*/ service, as well as the SSH pubkey +provided to the cloud instance. After fetching the two pieces of data, +ucd-data-fetch combines them into a valid ``#cloud-config`` user-data +text block and passes the output to `ucd(1) `__ for +processing/execution. + +The user-data file is currently only fetched for aws instances. + +OPTIONS +------- + +The only parameter supported is the name of the cloud service provider, +and can be ``aws``, ``oci`` or ``tencent``. It is required, and if not +passed as argument 1, will cause the program to terminate with an error +message. + +EXIT STATUS +----------- + +On success, 0 is returned, a non-zero failure code otherwise. The exit +code returned may be the exit code of the subsequent ``ucd`` program +execution. + +COPYRIGHT +--------- + +- Copyright (C) 2017-2019 Intel Corporation, License: CC-BY-SA-3.0 + +SEE ALSO +-------- + +`ucd(1) `__ + +NOTES +----- + +Creative Commons Attribution-ShareAlike 3.0 Unported + +- http://creativecommons.org/licenses/by-sa/3.0/ diff --git a/_sources/reference/manpages/ucd.1.rst.txt b/_sources/reference/manpages/ucd.1.rst.txt new file mode 100644 index 00000000..1acc07a5 --- /dev/null +++ b/_sources/reference/manpages/ucd.1.rst.txt @@ -0,0 +1,164 @@ +ucd +=== + +micro-config-drive(1) – Configure a cloud instance using config-drive +data + +SYNOPSIS +-------- + +``/usr/bin/ucd`` + +``ucd`` [OPTIONS…] + +DESCRIPTION +----------- + +``ucd`` runs at boot time with the purpose of configuring a cloud +instance for use of the end user. + +ucd can perform any of the following (optional) tasks: + +- create a default account with sudo privileges +- lock root and default account, the only way to login is using ssh + keys (security feature) +- maintain the cloud instance in a right state (resize/fix partitions + and filesystem) +- get and process userdata and metadata from an attached config-drive + block device + +Userdata formats supported: + +- ``cloud-config``: begins with ``#cloud-config`` and is used to + execute certain tasks in a human friendly format +- ``shell-script``: begins with ``#!`` and is used to execute a shell + script + +Metadata formats supported: + +- ``openstack`` + +OPENSTACK DATASOURCES +--------------------- + +Datasources are used to retrive instance-specific data in order to +configure a new cloud instance. Openstack attaches two datasources to +the cloud instance when it boots. + +- ``config-drive``: is a disk formatted with vfat or iso9660 and have a + label of config-2 +- ``metadata service``: instances can access to it at + http://169.254.169.254 ``[NOT SUPPORTED]`` + +more information about openstack datasources can be found in: + +- http://docs.openstack.org/user-guide/cli_config_drive.html +- http://docs.openstack.org/admin-guide/compute-networking-nova.html#metadata-service + +WORKFLOW +-------- + +:: + + `|` Parse options + `|` Init openstack datasource + `|->` if config-drive is found then define it as datasource otherwise exit + `|` Start openstack datasource + `|->` Get userdata and metadata from datasource + `|` Run async tasks + `|->` resize disk + `|->` resize filesystem + `|` If a datasource was found + `|->` if VM first boot was detected + `| |->` Run async tasks + `| | |->` lock root account + `| | |->` Create sudoers file for default user account + `| |->` Create default user account + `|->` Process openstack metadata + `|->` Process openstack userdata + `|` Join async tasks + `|` Finish openstack datasource (free resources) + +OPTIONS +------- + +The following options are understood: + +- ``-h``, ``--help``: + + Prints a help message. + +- ``-u`` FILE, ``--user-data-file`` FILE: + + Path to a cloud-config user data file&. If omitted, ``ucd`` will + attempt to fetch user-data from the openstack link-local connected + data service URL. + +- ``--openstack-metadata-file`` FILE: + + Path to an openstack metadata file. more information: + http://docs.openstack.org/user-guide/cli_config_drive.html#openstack-metadata-format + +- ``--openstack-config-drive`` PATH: + + Path to openstack config drive (iso9660 or vfat filesystem), it will + be used to process metadata and userdata. more information: + http://docs.openstack.org/user-guide/cli_config_drive.html#configuration-drive-contents + +- ``--user-data``: + + Get and process user data from data sources. + +- ``--user-data-once``: + + Only on first boot get and process user data from data sources. Note + this option is ignored if ``--user-data`` is enabled. + +- ``--metadata``: + + Get and process metadata from data sources. + +- ``--fix-disk``: + + Fix disk and filesystem if it is needed. + +- ``-v``, ``--version``: + + Prints version information. + +- ``--first-boot-setup``: + + Setup the instance in its first boot. When first boot of the instance + is detected, ``ucd`` will perform the following tasks: create a + default user account, create sudoers file for default user account, + lock the root account. + +EXIT STATUS +----------- + +On success, 0 is returned, a non-zero failure code otherwise. + +COPYRIGHT +--------- + +- Copyright (C) 2017 Intel Corporation, License: CC-BY-SA-3.0 + +SEE ALSO +-------- + +This project is a limited-functionality implementation of the cloud-init +specification. The full documentation of the generic implementation is +available online and can be referenced here: + +``https://cloudinit.readthedocs.org/en/latest/`` + +The cloud-config format that `ucd(1) `__ supports is documented in +`cloud-config(5) `__. This lists the options that are supported, their +structure and function. + +NOTES +----- + +Creative Commons Attribution-ShareAlike 3.0 Unported + +- http://creativecommons.org/licenses/by-sa/3.0/ diff --git a/_sources/reference/manpages/update-triggers.target.4.rst.txt b/_sources/reference/manpages/update-triggers.target.4.rst.txt new file mode 100644 index 00000000..86f17481 --- /dev/null +++ b/_sources/reference/manpages/update-triggers.target.4.rst.txt @@ -0,0 +1,55 @@ +====================== +update-triggers.target +====================== + +---------------------------------------- +Provides post OS software-update actions +---------------------------------------- + +:Copyright: \(C) 2017 Intel Corporation, CC-BY-SA-3.0 +:Manual section: 4 + + +SYNOPSIS +======== + +``update-triggers.target`` + +``/usr/lib/systemd/system/update-triggers.target`` + + +DESCRIPTION +=========== + +Instructs the ``systemd``\(1) system daemon what units needed to be +started as part of the software update process. After the main software +update content is applied to the OS, the `swupd(1) `__ program starts +this *systemd* target. + +The main function of this target is to allow optional corrections and +adjustments to be made after installation that can not easily be made +through static file installation methods. This usually applies to things +like cached lists, dynamic configuration files and similar volatile +databases that are regularly altered and may contain user data. + +One example is the dynamic loader cache which needs to be updated when +ever there are modifications to the system installed dynamic libraries. + +The user is not intended to execute these scripts and plugin units +directly. If however so this is desired, one can execute the following +command to re-execute all the actions: + +``systemctl start update-triggers.target`` + + +ENVIRONMENT +=========== + +This unit is a ``systemd``\(1) target file. + + +SEE ALSO +======== + +`swupd(1) `__ + diff --git a/_sources/reference/system-requirements.rst.txt b/_sources/reference/system-requirements.rst.txt new file mode 100644 index 00000000..75be00d2 --- /dev/null +++ b/_sources/reference/system-requirements.rst.txt @@ -0,0 +1,95 @@ +.. _system-requirements: + +Recommended minimum system requirements +####################################### + +|CL-ATTR| can run on most modern hardware and is capable of running with +modest hardware resources. Out of the box, |CL| can run on a single CPU core, +1 GB RAM, and minimum of disk space of: + +* 4 GB for the *live server* +* 20 GB for the *live desktop* + +.. caution:: + + Advanced users who wish to install on a disk using less than the recommended + space requirements may use the flag ``--skip-validation-size``. Use of this + flag may cause the installation to fail due to inadequate disk space. + + +For use cases requiring minimal resources, |CL| :ref:`about ` can +be used to create a highly customized installation that can even run on a +system with a 128MB of memory and 600MB of disk space, for example. + + +Installer requirements +********************** + +The *live desktop* installer requires at least 1 GB of RAM because more +resources are required to run in live mode than after |CL| is installed onto +persistent storage. + +For hardware with less resources, use the *live server* installer because it +has a smaller memory footprint. + +See https://clearlinux.org/downloads for more download options. + + +System requirements +******************* + +|CL| requires an x86 64-bit processor which supports Intel® Streaming SIMD +Extensions 4.2 (Intel® SSE 4.2). + +For information on the boot loader, see the `clr-boot-manager readme`_ . + +The |CL| installer performs a system compatibility check upon booting. To +manually verify system compatibility with |CL|, run the :ref:`compatibility +check tool` or go to http://ark.intel.com and check for +these features: + +* Instruction Set: + + - 64-bit + +* Instruction Set Extensions: + + - Supplemental Streaming SIMD Extension 3 (SSSE3) + - Intel® Streaming SIMD Extensions 4.1 (Intel® SSE 4.1) + - Intel® Streaming SIMD Extensions 4.2 (Intel® SSE 4.2) + - Carry-less Multiplication (PCLMUL) + +The following processor families have been verified to run |CL|: + +* Intel® Core™ processor family (2nd generation or greater) +* Intel® Xeon® E3-xxxx processor +* Intel® Xeon® E5-xxxx processor +* Intel® Xeon® E7-xxxx processor +* Intel Atom® processor C Series +* Intel Atom® processor E Series + + +Recommended configurations +************************** + +For general |CL| desktop use the recommended minimum requirements include: + +========= =============================== +Component Configuration +--------- ------------------------------- +Processor Compatible x86 64-bit processor +--------- ------------------------------- +Memory 4GB RAM +--------- ------------------------------- +Disk 20 GB +--------- ------------------------------- +Graphics Device with openGL support (e.g. Intel HD/UHD Graphics) +--------- ------------------------------- +Network Active Internet connection +========= =============================== + + +*Intel, Intel Core, Xeon, Intel Atom, and the Intel logo are trademarks of +Intel Corporation or its subsidiaries.* + +.. _clr-boot-manager readme: https://github.com/clearlinux/clr-boot-manager \ No newline at end of file diff --git a/_sources/reference/tutorial-ratings.rst.txt b/_sources/reference/tutorial-ratings.rst.txt new file mode 100644 index 00000000..b538d1be --- /dev/null +++ b/_sources/reference/tutorial-ratings.rst.txt @@ -0,0 +1,52 @@ +.. _tutorial-ratings: + +Tutorial difficulty ratings +########################### + +Tutorial difficulty ratings provide a simple way to start using and developing with |CL-ATTR|. If you're new to the distro, we suggest starting with ``Easy`` tutorials and working towards the more ``Difficult``. Ratings not only expose learning paths but also provide a starting point from which to advance or improve use cases, so be sure to :ref:`share your insights `. Three main metrics help us to determine how to rate a tutorial: + +.. contents:: + :local: + :depth: 1 + +The sum total of these metrics, the rating shown in Figure 1, represents the +ability to successfully complete a tutorial based on skill level, balanced against the risk of failure. + +.. figure:: /_figures/reference/tutorial-ratings-01.svg + :scale: 100% + :alt: Tutorial difficulty ratings + + Figure 1: Tutorial difficulty ratings + +Time and complexity +******************* + +Are there about 8 or more *complex* steps? *Complex steps* are those that: + +* Require more than one action +* Require external reading/review +* Include explanation or context +* Give alternative(s) + +This metric factors in the cognitive load and its impact on a user. + +User experience level +********************* + +Our tutorials primarily target two types of Linux users. + +**Experienced** A Linux\* user who is familiar with common topics like userspace, networking, sudo privileges, services, and more. + +**Advanced** A Linux user who is beyond Experienced and is familiar with most sysadmin and programming topics. + +This metric establishes a starting point for skills, based on user +experience. + +Impact of failure +****************** + +The impact of failure calculates the risk of failing to complete a tutorial as a result of entering incorrect data or configuration, or failing to follow the steps in the given order. We estimate the potential state of a system, given these failure scenarios and their severity. This metric also factors in the ability to troubleshoot and recover when faced with errors. Therefore, the final impact incorporates the previous two metrics while it helps to predict an appropriate difficulty rating. + +* Will impact of errors be inconsequential? ``Easy`` +* Will impact of errors cause inconvenience (but system still works)? ``Moderate`` +* Will impact of errors cause system failure (difficult to recover)? ``Difficult`` diff --git a/_sources/tutorials/apache-hadoop.rst.txt b/_sources/tutorials/apache-hadoop.rst.txt new file mode 100644 index 00000000..265305f5 --- /dev/null +++ b/_sources/tutorials/apache-hadoop.rst.txt @@ -0,0 +1,257 @@ +.. _hadoop: + +Apache\* Hadoop\* +################# + +This tutorial explains the process of installing, configuring, and +running Apache Hadoop on |CL-ATTR|. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +For this tutorial, you will install Hadoop in a single machine +running both the master and slave daemons. + +The Apache Hadoop software library is a framework for distributed processing +of large data sets across clusters of computers using simple programming +models. It is designed to scale up from single servers to thousands of +machines, with each machine offering local computation and storage. + +Prerequisites +************* + +* :ref:`bare-metal-install-desktop` + +* In |CL|, run :command:`swupd update` + + +Install Apache Hadoop +********************* + +Apache Hadoop is included in the :command:`big-data-basic` bundle. To install +the framework, enter the following command: + +.. code-block:: bash + + sudo swupd bundle-add big-data-basic + +Configure Apache Hadoop +*********************** + +#. To create the configuration directory, enter the following command: + + .. code-block:: bash + + sudo mkdir /etc/hadoop + +#. Copy the defaults from :file:`/usr/share/defaults/hadoop` to + :file:`/etc/hadoop` with the following command: + + .. code-block:: bash + + $ sudo cp /usr/share/defaults/hadoop/* /etc/hadoop + + .. note:: Since |CL| is a stateless system, never modify the + files under the :file:`/usr/share/defaults` directory. The software + updater will overwrite those files. + + Once all the configuration files are in :file:`/etc/hadoop`, edit them to + fit your needs. The `NameNode` server is the master server that manages the + namespace of the files' system and regulates the clients' access to files. + The first file to be edited, :file:`/etc/hadoop/core-site.xml`, informs the + Hadoop daemon where `NameNode` is running. In this tutorial, `NameNode` runs + in the `localhost`. + +#. Open the :file:`/etc/hadoop/core-site.xml` file using any editor and modify + the file as follows: + + .. code-block:: xml + + + + + + fs.default.name + hdfs://localhost:9000 + + + +#. Edit the :file:`/etc/hadoop/hdfs-site.xml` file. This file configures the + :abbr:`HDFS (Hadoop Distributed File System)` daemons. This configuration + includes the list of permitted and excluded data nodes and the size of + those blocks. For this example, set the number of block replication to 1 + from the default of 3 as follows: + + .. code-block:: xml + :emphasize-lines: 6 + + + + + + dfs.replication + 1 + + + dfs.permission + false + + + +#. Edit the :file:`/etc/hadoop/mapred-site.xml` file. This file configures + all daemons related to `MapReduce`: `JobTracker` and `TaskTrackers`. With + `MapReduce`, Hadoop can process big amounts of data in multiple systems. In + our example, we set :abbr:`YARN (Yet Another Resource Manager)` as our + runtime framework for executing `MapReduce` jobs as follows: + + .. code-block:: xml + :emphasize-lines: 5,6 + + + + + + mapreduce.framework.name + yarn + + + +#. Edit the :file:`/etc/hadoop/yarn-site.xml` file. This file configures all + daemons related to `YARN`: `ResourceManager` and `NodeManager`. In our + example, we implement the `mapreduce_shuffle` service, which is the + default as follows: + + .. code-block:: xml + :emphasize-lines: 4,5,8,9 + + + + + yarn.nodemanager.aux-services + mapreduce_shuffle + + + yarn.nodemanager.auxservices.mapreduce.shuffle.class + org.apache.hadoop.mapred.ShuffleHandler + + + +Configure your SSH key +********************** + +#. Create a SSH key. If you already have one, skip this step. + + .. code-block:: bash + + sudo ssh-keygen -t rsa + + +#. Copy the key to your authorized keys. + + .. code-block:: bash + + sudo cat /root/.ssh/id_rsa.pub | sudo tee -a /root/.ssh/authorized_keys + +#. Log into the localhost. If no password prompt appears, you are ready to + run the Hadoop daemons. + + .. code-block:: bash + + sudo ssh localhost + +Run the Hadoop daemons +********************** + +With all the configuration files properly edited, you are ready to start the +daemons. + +When you format the `NameNode` server, it formats the metadata related to +data nodes. Thus, all the information on the data nodes is lost and the nodes +can be reused for new data. + +#. Format the `NameNode` server with the following command: + + .. code-block:: bash + + sudo hdfs namenode -format + +#. Start the DFS in `NameNode` and `DataNodes` with the following command: + + .. code-block:: bash + + sudo start-dfs.sh + + The console output should be similar to: + + .. code-block:: console + + Starting namenodes on [localhost] + The authenticity of host 'localhost (::1)' can't be established. + ECDSA key fingerprint is + SHA256:97e+7TnomsS9W7GjFPjzY75HGBp+f1y6sA+ZFcOPIPU. + Are you sure you want to continue connecting (yes/no)? + +#. Enter `yes` to continue. + +#. Start the `YARN` daemons `ResourceManager` and `NodeManager` with the + following command: + + .. code-block:: bash + + sudo start-yarn.sh + +#. Ensure everything is running as expected with the following command: + + .. code-block:: bash + + sudo jps + + The console output should be similar to: + + .. code-block:: console + + 22674 DataNode + 26228 Jps + 22533 NameNode + 23046 ResourceManager + 22854 SecondaryNameNode + 23150 NodeManager + +Run the MapReduce wordcount example +*********************************** + +#. Create the input directory. + + .. code-block:: bash + + sudo hdfs dfs -mkdir -p /user/root/input + +#. Copy a file from the local file system to the HDFS. + + .. code-block:: bash + + sudo hdfs dfs -copyFromLocal local-file /user/root/input + +#. Run the `wordcount` example. + + .. code-block:: bash + + sudo hadoop jar /usr/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.0.jar wordcount input output + +#. Read the output file "part-r-00000". This file contains the number of times + each word appears in the file. + + .. code-block:: bash + + sudo hdfs dfs -cat /user/root/output/part-r-00000 + +**Congratulations!** + +You have successfully installed and setup a single node Hadoop cluster. +Additionally, you ran a simple wordcount example. + +Your single node Hadoop cluster is up and running! \ No newline at end of file diff --git a/_sources/tutorials/archive/kata_migration.rst.txt b/_sources/tutorials/archive/kata_migration.rst.txt new file mode 100644 index 00000000..3ccc4142 --- /dev/null +++ b/_sources/tutorials/archive/kata_migration.rst.txt @@ -0,0 +1,101 @@ +.. _kata_migration: + +Migrate Clear Containers to Kata Containers\* +############################################# + +This tutorial explains how to migrate from Clear Containers to Kata Containers. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +`Clear Containers`_ and `Kata Containers`_ can co-exist in the same system. +Both can be installed through the :command:`containers-virt` bundle. However, +we recommend that you migrate to Kata Containers because Clear Containers is no +longer `maintained`_ and will soon be deprecated on |CL-ATTR|. + +Prerequisites +************* + +* Clear Containers is on a Docker\* system. +* Kata Containers is installed. See :ref:`kata`. + + +Stop Clear Containers instances +******************************* + +As an unprivileged user, stop all running instances of Clear Containers: + +.. code-block:: bash + + for container in $(sudo docker ps -q); do sudo docker stop $container; done + + +Manually migrate customized configuration files +*********************************************** + +If you have made changes to your `Clear Containers configuration`_, review +those changes and decide whether to manually apply those changes to your +`Kata Containers configuration`_. + +Make any required changes before continuing this process. + +.. note:: + + You do not need to manually remove any Clear Containers packages. + + +Enable Kata Containers as default +********************************* + +#. Configure Docker to use the Kata Containers runtime by default. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/system/docker.service.d/ + cat <=4.7). +As such, it is not part of the Linux kernel, cannot be distributed by |CL|, +and has to be built as an out-of-tree kernel module. + +.. note:: + + It is recommended to use the :ref:`LTS kernel ` if you + have to use this driver. + +#. See if your device is supported and download the **Linux\* STA 64-bit + driver** from + `Broadcom's download website + `_ + or another trusted source. + + +#. Extract the downloaded archive into a separate folder. For example: + + .. code-block:: bash + + mkdir ./broadcom-wl/ + tar xvf ./hybrid-v35_64-nodebug-pcoem-6_30_223_271.tar.gz -C broadcom-wl/ + cd ./broadcom-wl/ + +#. Create a patches folder in the source tree and copy any necessary patches + to it. You will have to research which specific set of patches are required + for the running kernel version. The `gentoo repository for broadcom-sta + `_ + is a good place to start looking for up-to-date patches. + + .. code-block:: bash + + mkdir ./patches/ + +#. :ref:`Install the DKMS bundle ` for your + kernel. DKMS provides the framework to automatically rebuild the wl driver + against new kernels versions from |CL| updates. + +#. In the extracted driver directory, create a :file:`dkms.conf` file based + the contents below to provide DKMS information about how to build and + install the kernel module. This example uses version *6.30.223.271*. + + .. code-block:: bash + + cat <<'EOF' >> dkms.conf + PACKAGE_NAME=broadcom-wl + PACKAGE_VERSION=6.30.223.271 + MAKE="make KBASE=/lib/modules/${kernelver}" + CLEAN="make KBASE=/lib/modules/${kernelver} clean" + BUILT_MODULE_NAME=wl + DEST_MODULE_LOCATION=/kernel/drivers/net/wireless + AUTOINSTALL=yes + EOF + +#. Add the filename of any patches previously added to the :file:`patches` + folder to the :file:`dkms.conf` file so that DKMS applies them to the + driver source before building. Below are example patch names to show the + format used in :file:`dkms.conf`. + + .. code-block:: bash + + echo "PATCH[0]="first.patch" >> dkms.conf + echo "PATCH[1]="second.patch" >> dkms.conf + echo "PATCH[2]="third.patch" >> dkms.conf + +#. Copy the directory to the dkms tree. This example uses version + *6.30.223.271*. + + .. code-block:: bash + + sudo cp -Rv . /usr/src/broadcom-wl-6.30.223.271 + +#. Run the :command:`dkms` commands to add the broadcom-wl module to the dkms + tree, build it, and install it. This example uses version *6.30.223.271*. + + .. code-block:: bash + + sudo dkms add -m broadcom-wl -v 6.30.223.271 + sudo dkms install -m broadcom-wl -v 6.30.223.271 + +#. Blacklist all other variations of Broadcom drivers from loading to prevent + conflicts and problems. + + .. code-block:: bash + + sudo mkdir -p /etc/modprobe.d/ + + sudo tee /etc/modprobe.d/broadcom.conf > /dev/null <<'EOF' + blacklist b43 + blacklist b43legacy + blacklist ssb + blacklist bcm43xx + blacklist brcm80211 + blacklist brcmfmac + blacklist brcmsmac + blacklist bcma + EOF + +#. Reboot the system and check that the module is loaded and working. If not, + try manually updating dependencies and loading the module. + + .. code-block:: bash + + depmod -a + modprobe wl + + +Firmware +******** + +In addition to device drivers, devices require firmware that gets loaded onto +the device directly. + +Firmware for Broadcom devices are not fully open-source and not always +licensed for redistribution. |CL| kernel bundles :ref:`include the +linux-firmware bundle ` which contains the firmware +binaries that are able to be redistributed. If your device's firmware is part +of the linux-firmware repository, nothing else is needed. This is usually +the case for devices supported by the *brcmfmac* and *brcmsmac* drivers. + +In other cases, firmware may need to be obtained or extracted manually from a +trusted source because it is not licensed for distribution. This is usually +the case for devices supported by the *b43* and *b43legacy* drivers. Obtaining +these firmware is out of scope for this document, however there is information +about solutions to this problem available on the `Linux wireless wiki +`_. + +On |CL| systems, firmware should be placed in :file:`/etc/firmware`. See the +:ref:`firmware` documentation for more information on loading custom +firmware. + + +Troubleshooting +*************** + +- See which drivers are currently loaded with the :command:`lsmod` and + :command:`modinfo` commands. + +- If your device is not showing up or having intermittent issues, ensure the + card is not blocked by the kernel with the :command:`rfkill` command. + +- Try blacklisting all the other variations of drivers not intended to be + used. In some cases, the wrong device driver will be loaded causing + problems. + +- If an external firmware image is required, it may be trying to load from a + different path than expected. Check the output of :command:`sudo dmesg | + grep -i firmware` for firmware loading issues. + +*The Bluetooth® word mark and logos are registered trademarks owned by Bluetooth SIG, Inc. and any use of such marks by Intel Corporation is under license.* \ No newline at end of file diff --git a/_sources/tutorials/docker.rst.txt b/_sources/tutorials/docker.rst.txt new file mode 100644 index 00000000..2fb391b5 --- /dev/null +++ b/_sources/tutorials/docker.rst.txt @@ -0,0 +1,242 @@ +.. _docker: + +Docker\* +######### + +|CL-ATTR| supports multiple containerization platforms, including a Docker +solution. |CL| has many unique features including a minimal default +installation, which makes it compelling to use as a host for container +workloads, management, and orchestration. + +This tutorial covers: + +.. contents:: :local: + :depth: 1 + +.. note:: + + This tutorial focuses on the installation of the Docker ecosystem. + If you want to use |CL| as a Docker container image, refer to the + official |CL| container image + `published on Docker* Hub `_ + and our guide to :ref:`container-image-new`. + +Prerequisites +************* + +This tutorial assumes you have installed |CL| on your host system. +For detailed instructions on installing |CL| on a bare metal system, follow +the :ref:`bare metal installation instructions`. + +Before you install any new packages, update |CL| with the following command: + +.. code-block:: bash + + sudo swupd update + +Additionally, you should have: + +* A basic understanding of Linux\* and Docker. + +* |CL| environment that has transparent network access to the Internet. + If you are behind a HTTP proxy server, in a corporate setting for example, + please refer to the `Docker proxy instructions`_ . + +Install the containers-basic bundle +*********************************** + +Software in |CL| is offered in the form of :ref:`bundles` to provide a +complete function. The *containers-basic* provides all the required software +packages to run Docker images as containers. + +#. First, install the *containers-basic* bundle by running this + :command:`swupd` command: + + .. code-block:: bash + + sudo swupd bundle-add containers-basic + +#. Start the Docker daemon through systemd manager by running this command: + + .. code-block:: bash + + sudo systemctl start docker + + If you want Docker to start automatically on boot, enable the + systemd service by running this command: + + .. code-block:: bash + + sudo systemctl enable docker + +#. Finally, verify :command:`docker` has been installed by running this + command and checking the version output for both *client* and *server*: + + .. code-block:: bash + + sudo docker version + +Congratulations! At this point, you have a working installation of Docker +on |CL|. You are ready to start using container images on your system. + +Integration with Kata Containers\* (optional) +********************************************* + +`Kata Containers`_, is an open source project aiming to increase security +of containers by using a hardware-backed virtual machine container runtime +rather than software namespace containers that are provided by the standard +Docker *runc* runtime. + +|CL| provides easy integration of the *kata-runtime* with Docker. +More information on installing and using the *kata-runtime* may be found at :ref:`kata`. + + +.. note:: + + The remaining sections of this tutorial are standard to Docker setup + and configuration. If you are familiar with Docker basics, you do not + need to continue reading. The following sections are provided here for + sake of completeness. + +.. _additional-docker-configuration: + +Additional Docker configuration +******************************* + +Perform additional Docker daemon configuration via a configuration file +typically located at :file:`/etc/docker/daemon.json`. |CL| features a +:ref:`stateless` system so the configuration file :file:`daemon.json` does +*NOT* exist by default. + +#. Create the :file:`daemon.json` by running this command: + + .. code-block:: bash + + sudo touch /etc/docker/daemon.json + + .. note:: + + Refer to the `Docker documentation on daemon configuration`_ for the + full list of available configuration options and examples. + +#. For production systems, we follow Docker's recommendation to use the + `OverlayFS storage driver`_ `overlay2`, shown below: + + .. code-block:: json + + { + "storage-driver": "overlay2" + } + + .. note:: + + A testing version is found in `Docker Device Mapper storage driver`_. + If using this storage driver, a warning message may appear: "usage of + loopback devices is strongly discouraged for production use". + +#. Save and close :file:`daemon.json`. + +#. Once you've made any required changes, be sure to restart the + Docker daemon through systemd manager by running this command: + + .. code-block:: bash + + sudo systemctl restart docker + +Pulling and running an image from Docker Hub\* +********************************************** + +`Docker Hub`_ is a publicly available container image repository which +comes pre-configured with Docker. In the example below we will pull and run +an the official Docker image for nginx\*, an open source reverse proxy server. + +#. First, pull a container image from Docker Hub using the + :command:`docker pull` command. Download the latest nginx\* Docker + container image by running this command: + + .. code-block:: bash + + sudo docker pull nginx + +#. Create and launch a new container using the :command:`docker run` + command. Launch a nginx container by running this command: + + .. code-block:: bash + + sudo docker run --name test-nginx -d -p 8080:80 nginx + + .. note:: + + Below is an explanation of switches used in the command above. For + detailed :command:`docker run` switches and syntax, refer to the + `Docker Documentation`_ . + + * The *--name* switch lets you provide a friendly name to + target the container for future operations + + * The *-d* switch launches the container in the background + + * The *-p* switch allows the container's HTTP port (80) to be + accessible from the |CL| host on port 8080 + +#. You can access the Welcome to Nginx! splash page running in the container + by browsing to \http://127.0.0.1:8080 or by running this :command:`curl` + command from your |CL| machine: + + .. code-block:: bash + + curl 127.0.0.1:8080 + +#. Finally, stop and delete the nginx container by running the + :command:`docker stop` and :command:`docker rm` commands. + + .. code-block:: bash + + sudo docker stop test-nginx + sudo docker rm test-nginx + +Congratulations! At this point, you have successfully pulled a nginx +container image from `Docker Hub`_ and have run an example container. + +Creating a Docker swarm cluster +******************************* + +Clusters of Docker hosts are referred to as *swarms*. + +The process in this tutorial can be repeated to install Docker on multiple +|CL| hosts with the intent to form a Docker swarm cluster. + +The `Docker documentation on swarm key concepts`_ and +`Docker documentation on creating a swarm`_ can be referenced +for further instructions on setting up a swarm. + +Related topics +************** + +* `Docker Home`_ +* `Docker Documentation`_ +* `Docker Hub`_ +* `Kata Containers`_ + + +.. _Docker proxy instructions: https://docs.docker.com/config/daemon/systemd/#httphttps-proxy + +.. _Docker documentation on daemon configuration: https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file + +.. _Kata Containers: https://katacontainers.io/ + +.. _Docker Home: https://www.docker.com/ + +.. _Docker Documentation: https://docs.docker.com/ + +.. _Docker Hub: https://hub.docker.com/ + +.. _Docker documentation on swarm key concepts: https://docs.docker.com/engine/swarm/key-concepts/ + +.. _Docker documentation on creating a swarm: https://docs.docker.com/engine/swarm/swarm-tutorial/create-swarm/ + +.. _Configure direct lvm mode for production: https://docs.docker.com/storage/storagedriver/device-mapper-driver/ + +.. _OverlayFS storage driver: https://docs.docker.com/storage/storagedriver/overlayfs-driver/ + +.. _Docker Device Mapper storage driver: https://docs.docker.com/storage/storagedriver/device-mapper-driver/ diff --git a/_sources/tutorials/flatpak.rst.txt b/_sources/tutorials/flatpak.rst.txt new file mode 100644 index 00000000..5a5b43a9 --- /dev/null +++ b/_sources/tutorials/flatpak.rst.txt @@ -0,0 +1,135 @@ +.. _flatpak-tutorial: + +Flatpak\* +######### + +This tutorial shows how to install a `Flatpak`_ app on |CL| using GNOME\* Software +and the command line. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +`Flatpak`_ is a framework for building and distributing desktop apps on +Linux\*. Flatpak allows you to build a single app and install it on different +distributions of Linux. Flatpak apps are available through `Flathub`_ or the +`Clear Linux Store`_. + +Prerequisites +************* + +* |CL| installed on host system + + Refer to :ref:`Get started ` for installation instructions. + +* `desktop-autostart` bundle installed + + Flatpak is included via `desktop`, which is included in the + `desktop-autostart` bundle. The Flathub repository is pre-configured when + the `desktop-autostart` bundle is installed. + + Install the `desktop-autostart` bundle with the following command: + + .. code-block:: bash + + sudo swupd bundle-add desktop-autostart + +Install a Flatpak app with GNOME Software +***************************************** + +|CL| desktop comes with GNOME Software installed. Flatpak apps can be +installed from within GNOME Software. + +#. Launch GNOME Software from your desktop. + +#. Search for the Flatpak app that you want to install, as shown in Figure 1. + + .. figure:: /_figures/flatpak/flatpak-01.png + :scale: 50% + :alt: Searching for Filezilla app in GNOME Software + + Figure 1: Searching for Filezilla\* app in GNOME Software + +#. When you find the app you want to install, click it to view application + details. + +#. On the app detail page, click the :guilabel:`Install` button, as shown in + Figure 2. + + .. figure:: /_figures/flatpak/flatpak-02.png + :scale: 50% + :alt: Filezilla Flatpak detail page in GNOME Software + + Figure 2: Filezilla Flatpak detail page in GNOME Software + +#. After installation is complete, the new application will be in your + GNOME applications list, as shown in Figure 3. + + .. figure:: /_figures/flatpak/flatpak-03.png + :scale: 50% + :alt: Newly installed Filezilla application + + Figure 3: Newly installed Filezilla application + +#. Click the application icon to launch the application. + +Install a Flatpak with the command line +*************************************** + +Both Flathub and the Clear Linux Store provide the command line instructions +for installing a Flatpak. Figure 4 shows the command line instructions to +install Filezilla\* from the Clear Linux Store: + +.. figure:: /_figures/flatpak/flatpak-04.png + :scale: 50% + :alt: Command line instructions to install Filezilla from the Clear Linux Store + + Figure 4: Command line instructions to install Filezilla from the Clear Linux Store + +In this example, we install Filezilla. + +#. Open a terminal and enter the install command for the desired app: + + .. code-block:: bash + + flatpak install flathub org.filezillaproject.Filezilla + + You may be prompted to select which repository to use: + + .. code-block:: bash + + Looking for matches… + Remote ‘flathub’ found in multiple installations: + + 1) system + 2) user + + Which do you want to use (0 to abort)? [0-2]: 2 + + org.filezillaproject.Filezilla permissions: + ipc network ssh-auth wayland x11 + dri file access [1] dbus access [2] + + [1] host, xdg-run/dconf, ~/.config/dconf:ro + [2] ca.desrt.dconf, org.freedesktop.Notifications, org.freedesktop.PowerManagement, + org.gnome.SessionManager + + + ID Arch Branch Remote Download + 1. [✓] org.filezillaproject.Filezilla x86_64 stable flathub 11.5 MB / 11.5 MB + 2. [✓] org.filezillaproject.Filezilla.Locale x86_64 stable flathub 4.6 kB / 3.8 MB + + Installation complete. + +#. After installation, run the application with the following command: + + .. code-block:: bash + + flatpak run org.filezillaproject.Filezilla + +.. _Flatpak: https://flatpak.org +.. _Flathub: https://flathub.org +.. _Clear Linux Store: https://clearlinux.org/software diff --git a/_sources/tutorials/fmv.rst.txt b/_sources/tutorials/fmv.rst.txt new file mode 100644 index 00000000..402ac04e --- /dev/null +++ b/_sources/tutorials/fmv.rst.txt @@ -0,0 +1,286 @@ +.. _fmv: + +Function Multi-Versioning +######################### + +In this tutorial, we will use :abbr:`FMV (Function Multi-Versioning)` on +general code and on :abbr:`FFT (Fast Fourier Transform)` library code (FFTW). +Upon completing the tutorial, you will be able to use this technology on your +code and use the libraries to deploy architecture-based optimizations to your +application code. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +CPU architectures often gain interesting new instructions as they evolve but +application developers find it difficult to take advantage of those +instructions. The reluctance to lose backward-compatibility is one of the +main roadblocks slowing developers from using advancements in newer computing +architectures. FMV, which first appeared in `GCC`_ 4.8, is a way to have +multiple implementations of a function, each using different +architecture-specialized instruction-set extensions. GCC 6 introduces +changes to FMV to make it even easier to bring architecture-based +optimizations to the application code. + + +Install and configure a |CL| host on bare metal +*********************************************** + +First, follow our guide to :ref:`bare-metal-install-desktop`. Once the bare +metal installation and initial configuration are complete, add the +:command:`desktop-dev` bundle to the system. :command:`desktop-dev` contains +the necessary development tools like GCC and Perl\*. + +#. To install the bundles, run the following command in the :file:`$HOME` + directory: + + .. code-block:: bash + + sudo swupd bundle-add desktop-dev + +Detect loop vectorization candidates +************************************ + +Now, we need to detect the loop vectorization candidates to be cloned for +multiple platforms with FMV. As an example, we will use the following +simple C code: + +.. code-block:: c + :linenos: + + #include + #include + #include + #define MAX 1000000 + + int a[256], b[256], c[256]; + + void foo(){ + int i,x; + for (x=0; x log + +#. To generate the patch files, execute: + + .. code-block:: bash + + perl ./make-fmv-patch/make-fmv-patch.pl log . + +#. The :file:`make-fmv-patch.pl` script takes two arguments: `` + and ``. Replace `` and `` with the + proper values and execute: + + .. code-block:: bash + + perl make-fmv-patch.pl + + The command generates the following :file:`example.c.patch` patch: + + .. code-block:: console + + --- ./example.c 2017-09-27 16:05:42.279505430 +0000 + +++ ./example.c~ 2017-09-27 16:19:11.691544026 +0000 + @@ -5,6 +5,7 @@ + + int a[256], b[256], c[256]; + + +__attribute__((target_clones("avx2","arch=atom","default"))) + void foo(){ + int i,x; + for (x=0; x + #include + #include + #define MAX 1000000 + + int a[256], b[256], c[256]; + + __attribute__((target_clones("avx2","arch=atom","default"))) + void foo(){ + int i,x; + for (x=0; x y) ? x : y; } + + +__attribute__((target_clones("avx2","arch=atom","default"))) + static double aerror(C *a, C *b, int n) + { + if (n > 0) { + @@ -111,6 +112,7 @@ + } + + /* make array hermitian */ + +__attribute__((target_clones("avx2","arch=atom","default"))) + void mkhermitian(C *A, int rank, const bench_iodim *dim, int stride) + { + if (rank == 0) + @@ -148,6 +150,7 @@ + } + + /* C = A + B */ + +__attribute__((target_clones("avx2","arch=atom","default"))) + void aadd(C *c, C *a, C *b, int n) + { + int i; + @@ -159,6 +162,7 @@ + } + + /* C = A - B */ + +__attribute__((target_clones("avx2","arch=atom","default"))) + void asub(C *c, C *a, C *b, int n) + { + int i; + @@ -170,6 +174,7 @@ + } + + /* B = rotate left A (complex) */ + +__attribute__((target_clones("avx2","arch=atom","default"))) + void arol(C *b, C *a, int n, int nb, int na) + { + int i, ib, ia; + @@ -192,6 +197,7 @@ + } + } + +With these patches, we can select where to apply the FMV technology, which +makes it even easier to bring architecture-based optimizations to +application code. + +**Congratulations!** + +You have successfully installed an FMV development environment on |CL|. +Furthermore, you used cutting edge compiler technology to improve the +performance of your application based on Intel® architecture and +profiling of the specific execution of your application. + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _GCC: https://gcc.gnu.org +.. _make-fmv-patch: https://github.com/clearlinux/make-fmv-patch diff --git a/_sources/tutorials/hpc.rst.txt b/_sources/tutorials/hpc.rst.txt new file mode 100644 index 00000000..a97eed66 --- /dev/null +++ b/_sources/tutorials/hpc.rst.txt @@ -0,0 +1,736 @@ +.. _hpc: + +HPC Cluster +########### + +This tutorial demonstrates how to set a simple :abbr:`HPC (High +Performance Computing)` cluster using `Slurm`_, `MUNGE`_, and +`pdsh`_. For this tutorial, this cluster consists of a controller node +and four worker nodes, as shown in Figure 1. For the sake of simplicity, +each node resides on a separate host and their hostnames are hpc-controller, +hpc-worker1, hpc-worker2, hpc-worker3, and hpc-worker4. + +.. rst-class:: dropshadow + +.. figure:: ../_figures/hpc/hpc-01.png + :alt: Simple HPC cluster + + Figure 1: Simple HPC cluster + +The configuration is intentionally kept simple, notably avoiding setting +up cgroups and accounting. These and many more additional configuration +options can be added later. + +.. note:: + + This tutorial assumes you start with a new installation of |CL| for all + nodes. + +Prerequisites +************* + +* Knowledge and experience with HPC +* Familiarity with Slurm, MUNGE, and pdsh +* All nodes have synchronized clocks (typically by NTP) + +.. contents:: + :local: + :depth: 1 + +Set up controller node +********************** + +In this step, install the cluster tools, configure and enable the MUNGE service, +and enable the Slurm controller service. + +#. Install |CL| on the controller node, add a user with adminstrator + privilege, and set its hostname to `hpc-controller`. + +#. Boot it up and log in. + +#. Update |CL| to the latest version. + + .. code-block:: bash + + sudo swupd update + +#. Set the date and time to synchronize with an NTP server. + + .. code-block:: bash + + sudo timedatectl set-ntp true + +#. Install the `cluster-tools` bundle. + + .. code-block:: bash + + sudo swupd bundle-add cluster-tools + +#. Create a MUNGE key and start the MUNGE service. + + a. Create the MUNGE key. + + .. code-block:: bash + + sudo mkdir /etc/munge + dd if=/dev/urandom bs=1 count=1024 | sudo tee -a /etc/munge/munge.key + + #. Set the ownership to `munge` and set the correct access permissions. + + .. code-block:: bash + + sudo chown munge: /etc/munge/munge.key + sudo chmod 400 /etc/munge/munge.key + + #. Start the MUNGE service and set it to start automatically on boot. + + .. code-block:: bash + + sudo systemctl enable munge --now + +#. Test MUNGE. + + a. Create a MUNGE credential. + + .. code-block:: bash + + munge -n + + Example output: + + .. code-block:: console + + MUNGE:AwQFAAC8QZHhL/+Fqhalhi+ZJBD5LavtMa8RMles1aPq7yuIZq3LtMmrB7KQZcQjG0qkFmoIIvixaCACFe1stLmF4VIg4Bg/7tilxteXHS940cuZ/TxpIuqC6fUH8zLgUZUPwJ4=: + + + #. Validate a MUNGE credential. + + .. code-block:: bash + + munge -n | unmunge | grep STATUS + + Example output: + + .. code-block:: console + + STATUS: Success (0) + +#. Start the Slurm controller service and enable it to start automatically + on boot. + + .. code-block:: bash + + sudo systemctl enable slurmctld --now + +Set up worker nodes +******************* + +For each worker node, perform these steps: + +#. Install |CL| on the worker node, add a user with adminstrator privilege, + and set its hostname to `hpc-worker` plus its number, i.e. hpc-worker1, + hpc-worker2, etc. + + Ensure the username is the same as the one on the controller node. This + is needed to simplify password-less-SSH-access setup, which is needed for + pdsh, in the next section. + +#. Boot it up and log in. + +#. Update |CL| to the latest version. + + .. code-block:: bash + + sudo swupd update + +#. Set the date and time to synchronize with an NTP server. + + .. code-block:: bash + + sudo timedatectl set-ntp true + +#. Install the `cluster-tools` bundle. + + .. code-block:: bash + + sudo swupd bundle-add cluster-tools + +Set up password-less SSH access and pdsh on all nodes +***************************************************** + +To efficiently manage a cluster, it is useful to have a tool +that allows issuing the same command to multiple nodes at once. +And that tool is :abbr:`pdsh (parallel distributed shell)`, which is included +with the `cluster-tools` bundle. pdsh is built with Slurm support, so it can +access hosts as defined in the Slurm partitions. pdsh relies on password-less +SSH access in order for it to work properly. There are two ways to set up +pasword-less SSH authentication: key-based or host-based. In this case, +the latter approach will be used. The controller authenticates a user and +all worker nodes will trust that authentication and not ask the user to +enter a password again. + +#. Configure the controller node. + + a. Log into the controller node. + + #. Configure the SSH service for host-based authentication. + + .. code-block:: bash + + sudo tee -a /etc/ssh/ssh_config << EOF + HostbasedAuthentication yes + EnableSSHKeysign yes + EOF + + #. Restart the SSH service. + + .. code-block:: bash + + sudo systemctl restart sshd + +#. Configure each worker node. + + a. Configure SSH service for host-based authentication. + + .. code-block:: bash + + sudo tee -a /etc/ssh/sshd_config << EOF + HostbasedAuthentication yes + IgnoreRhosts no + UseDNS yes + EOF + + #. Create the :file:`/etc/hosts.equiv` file and add the controller's + :abbr:`FQDN (fully qualified domain name)`. This tells the worker + node to accept connection from the controller. + + For example: + + .. code-block:: console + + hpc-controller.my-domain.com + + #. Set its permission to root access only. + + .. code-block:: bash + + sudo chmod 600 /etc/hosts.equiv + + #. Add the controller's FQDN to :file:`/root/.shosts`. This allows + host-based authentication for the root account so that + actions requiring sudo privileges can be performed. + + .. code-block:: bash + + sudo cp -v /etc/hosts.equiv /root/.shosts + + #. Using the controller's FQDN in :file:`/etc/hosts.equiv`, scan for its + RSA public key and copy it to :file:`/etc/ssh/ssh_known_hosts`. + Verify the scanned RSA public key matches the controller's + :file:`/etc/ssh/ssh_rsa_key.pub` file. + + .. code-block:: bash + + sudo ssh-keyscan -t rsa -f /etc/hosts.equiv > ~/ssh_known_hosts + sudo cp -v ~/ssh_known_hosts /etc/ssh + rm ~/ssh_known_hosts + + #. Restart the SSH service. + + .. code-block:: bash + + sudo systemctl restart sshd + +#. On the controller node, SSH into each worker node without having to enter + a password. At the first-time connection to each host, you'll be asked to + add the unknown host to the :file:`$HOME/.ssh/known_hosts` file. Accept + the request. This is will make future SSH connections to each host be + non-interactive. + + .. code-block:: bash + + ssh + + .. note:: + + Setting up host-based authentication on + :abbr:`CSP (Cloud Service Provider)` environments such as Microsoft Azure + and Amazon AWS may require some tweaking on the worker nodes' SSH + configurations due to the CSP's virtual network setup. In general, + cloud VMs have a public and private DNS name. When SSH'ing from the + controller to a worker node, the SSH client may send the controller's + private DNS name, usually something with "internal" in the name, + as the `chost` instead of its public FQDN as expected in worker node's + :file:`/etc/hosts.equiv`, :file:`/root/.shosts`, and + :file:`/etc/ssh/ssh_known_hosts` files. If the above configurations + do not work, meaning you're asked to enter a password when + SSH'ing from the controller to a worker node, on a cloud VM, here are + some suggestions for debugging the problem: + + #. On the controller, try to identify the chost data sent by the SSH + client using :command:`ssh -vvv `. Look for `chost` + in the debug log. If the chost value is different than the controller's + FQDN listed in worker node's :file:`/etc/hosts.equiv`, + :file:`/root/.shosts`, and :file:`/etc/ssh/ssh_known_hosts` files, + then that is likely the cause of the problem. In some cases, chost + data may not be shown. If so, it's safe to assume that the SSH client + is using the controller's private DNS name as the chost. Proceed to + steps 2 and 3 below to fix the problem. + + #. Get the controller's private DNS name either by the above step or by + getting it from your system administrator. + + #. On the worker node, make these changes: + + #. Change the controller's FQDN in :file:`/etc/hosts.equiv`, + :file:`/root/.shosts`, and :file:`/etc/ssh/ssh_known_hosts` + to its private DNS name. + + #. Restart the SSH service on the worker node. + + #. Retest the connection from the controller node to the worker node. + If that still doesn't work, try the SSH directive + `HostbasedUsesNameFromPacketOnly yes` which tell the SSH service + to accept the supplied host name as is and not try to resolve it. + Also, set the directive `UseDNS` to `no` to disable host name lookup. +#. Verify you can issue a simple command over SSH without typing a password. + + a. Issue the :command:`hostname` command. + + .. code-block:: bash + + ssh hostname + + #. Issue the :command:`hostname` command with :command:`sudo`. + + .. code-block:: bash + + ssh sudo hostname + + In both cases, you should get a response with the worker node's hostname. + If the `sudo` version requires additional permission, grant the user + `NOPASSWD` privilege. For example: + + #. Edit the sudoer file. + + .. code-block:: bash + + sudo visudo + + #. Add the following: + + .. code-block:: console + + ALL=(ALL) NOPASSWD: ALL + +Create :file:`slurm.conf` configuration file +******************************************** + +On the controller, create a new :file:`slurm.conf` configuration file +that contains general settings, each node's hardware resource information, +grouping of nodes into different partitions, and scheduling settings for +each partition. This file will be copied to all worker nodes in the cluster. + +#. Create a base :file:`slurm.conf` configuration file. + + .. code-block:: bash + + sudo mkdir -p /etc/slurm + sudo cp -v /usr/share/defaults/slurm/slurm.conf /etc/slurm + +#. Add the controller information. + + a. :command:`sudoedit` the :file:`slurm.conf` file. Set the `ControlMachine` + value to the controller's resolvable hostname. + + For example: + + .. code-block:: console + + ControlMachine=hpc-controller + + .. note:: + + Assuming the controller's FQDN is resolvable, specifying the + controller's IP address with the `ControlAddr` key is optional. + However, it maybe helpful to add it. + + #. Save and exit. + +#. Add the worker nodes information. + + a. Create a file containing a list of the worker nodes. + + .. code-block:: bash + + cat > worker-nodes-list << EOF + hpc-worker1 + hpc-worker2 + hpc-worker3 + hpc-worker4 + EOF + + #. Using pdsh, get the hardware configuration of each node. + + .. code-block:: bash + + pdsh -w ^worker-nodes-list slurmd -C + + Example output: + + .. code-block:: console + + hpc-worker4: NodeName=hpc-worker4 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=1915 + hpc-worker4: UpTime=0-01:23:28 + hpc-worker3: NodeName=hpc-worker3 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=1663 + hpc-worker3: UpTime=0-01:33:41 + hpc-worker2: NodeName=hpc-worker2 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=721 + hpc-worker2: UpTime=0-01:34:56 + hpc-worker1: NodeName=hpc-worker1 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=721 + hpc-worker1: UpTime=0-01:39:21 + + #. :command:`sudoedit` the :file:`slurm.conf` file. Append each worker node + information, but without the `UpTime`, under the `COMPUTE NODES` section. + + .. tip:: + + It is strongly recommended to set the `RealMemory` value for each + worker node slightly below, say 90%, what was reported by + :command:`slurmd -C` + in case some memory gets use by some processes, which would + cause Slurm to make a node not available due to its memory + resource falling below the stated value in the configuration file. + + Here's an example with four worker nodes: + + .. code-block:: console + + # + # COMPUTE NODES (mode detailed values reported by "slurmd -C" on each node) + NodeName=hpc-worker1 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=648 + NodeName=hpc-worker2 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=648 + NodeName=hpc-worker3 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=1497 + NodeName=hpc-worker4 CPUs=1 Boards=1 SocketsPerBoard=1 CoresPerSocket=1 ThreadsPerCore=1 RealMemory=1723 + + #. Create partitions. + + A Slurm partition is basically the grouping of worker nodes. + Give each partition a name and decide which worker node(s) belong to + it. + + For example: + + .. code-block:: console + + PartitionName=workers Nodes=hpc-worker1, hpc-worker2, hpc-worker3, hpc-worker4 Default=YES MaxTime=INFINITE State=UP + PartitionName=debug Nodes=hpc-worker1, hpc-worker3 MaxTime=INFINITE State=UP + + #. Save and exit. + + #. Set the ownership of the :file:`slurm.conf` file to `slurm`. + + .. code-block:: bash + + sudo chown slurm: /etc/slurm/slurm.conf + +#. On the controller node, restart the Slurm controller service. + + .. code-block:: bash + + sudo systemctl restart slurmctld + +#. Verify the Slurm controller service restarted without any issues before + proceeding. + + .. code-block:: bash + + sudo systemctl status slurmctld + + + +Copy MUNGE key and :file:`slurm.conf` to all worker nodes +********************************************************* + +On the controller node, using pdsh, in conjunction with the list of +defined nodes in the :file:`slurm.conf`, copy it and the MUNGE key to +all worker nodes. + +#. On the controller node, copy the MUNGE key to all worker nodes and start the + MUNGE service. + + a. Create the :file:`/etc/munge/` directory on each node. + + .. code-block:: bash + + sudo pdsh -P workers mkdir /etc/munge + + #. Copy the MUNGE key over. + + .. code-block:: bash + + sudo pdcp -P workers /etc/munge/munge.key /etc/munge + + #. Set the ownership of the :file:`munge.key` file to `munge`. + + .. code-block:: bash + + sudo pdsh -P workers chown munge: /etc/munge/munge.key + + #. Start the MUNGE service and set it to start automatically on boot. + + .. code-block:: bash + + sudo pdsh -P workers systemctl enable munge --now + + #. Verify the MUNGE service is running. + + .. code-block:: bash + + sudo pdsh -P workers "systemctl status munge | grep Active" + + Example output: + + .. code-block:: console + + hpc-worker3: Active: active (running) since Wed 2020-04-15 19:47:58 UTC; 55s ago + hpc-worker4: Active: active (running) since Wed 2020-04-15 19:47:58 UTC; 55s ago + hpc-worker2: Active: active (running) since Wed 2020-04-15 19:47:59 UTC; 54s ago + hpc-worker1: Active: active (running) since Wed 2020-04-15 19:47:59 UTC; 54s ago + +#. On the controller node, copy the :file:`slurm.conf` file to all + worker nodes and start the slurmd service on them. + + a. Create the :file:`/etc/slurm/` directory on each worker node. + + .. code-block:: bash + + sudo pdsh -P workers mkdir /etc/slurm + + #. Copy the :file:`slurm.conf` file over. + + .. code-block:: bash + + sudo pdcp -P workers /etc/slurm/slurm.conf /etc/slurm + + #. Set the ownership of the :file:`slurm.conf` file to `slurm`. + + .. code-block:: bash + + sudo pdsh -P workers chown slurm: /etc/slurm/slurm.conf + + #. Start the Slurm service and set it automatically start on boot. + + .. code-block:: bash + + sudo pdsh -P workers systemctl enable slurmd --now + + #. Verify the slurmd service is running. + + .. code-block:: bash + + sudo pdsh -P workers systemctl status slurmd | grep Active + + Example output: + + .. code-block:: console + + hpc-worker3: Active: active (running) since Wed 2020-04-15 19:39:22 UTC; 1min 17s ago + hpc-worker4: Active: active (running) since Wed 2020-04-15 19:39:22 UTC; 1min 17s ago + hpc-worker2: Active: active (running) since Wed 2020-04-15 19:39:23 UTC; 1min 17s ago + hpc-worker1: Active: active (running) since Wed 2020-04-15 19:39:23 UTC; 1min 17s ago + +Verify controller can run jobs on all nodes +******************************************* + +#. Check the state of the worker nodes. + + .. code-block:: bash + + sinfo + + Example output: + + .. code-block:: console + + PARTITION AVAIL TIMELIMIT NODES STATE NODELIST + workers* up infinite 4 idle hpc-worker[1-4] + debug up infinite 2 idle hpc-worker[1,3] + + .. tip:: + + If the nodes are in a "down" state, put them in the "idle" state. + + For example: + + .. code-block:: bash + + sudo scontrol update nodename=hpc-worker[1-4] state=idle reason="" + + Additional `Slurm troubleshooting tips`_. + +#. And finally, verify Slurm can run jobs on all 4 worker nodes by issuing + a simple :command:`hostname` command. + + .. code-block:: bash + + srun -N4 -p workers hostname + + Example output: + + .. code-block:: console + + hpc-worker4 + hpc-worker3 + hpc-worker1 + hpc-worker2 + +Create and run example scripts +****************************** + +Example 1: Return the hostname of each worker and output to :file:`show-hostnames.out` +====================================================================================== + +#. On the controller node, create the Slurm :file:`show-hostnames.sh` script. + + .. code-block:: bash + + cat > show-hostnames.sh << EOF + #!/bin/bash + # + #SBATCH --job-name=show-hostnames + #SBATCH --output=show-hostnames.out + # + #SBATCH --ntasks=4 + #SBATCH --time=10:00 + #SBATCH --mem-per-cpu=100 + #SBATCH --ntasks-per-node=1 + + srun hostname + EOF + +#. Execute the script. + + .. code-block:: bash + + sbatch show-hostnames.sh + + The result will appear on the first node of the partition used. As no + partition was explicitly specified, it would be the default partition. + +#. View the result. + + .. code-block:: bash + + pdsh -w hpc-worker1 "cat show-hostnames.out" + + Example output: + + .. code-block:: console + + hpc-worker1: hpc-worker3 + hpc-worker1: hpc-worker4 + hpc-worker1: hpc-worker1 + hpc-worker1: hpc-worker2 + +Example 2: An MPI "Hello, World!" program +========================================= + +#. On the controller node, create the :file:`mpi-helloworld.c` program. + + .. code-block:: bash + + cat > mpi-helloworld.c << EOF + #include + #include + #include + + int main(int argc, char** argv) + { + // Init the MPI environment + MPI_Init(NULL, NULL); + + // Get the number of processes + int world_size; + MPI_Comm_size(MPI_COMM_WORLD, &world_size); + + // Get the rank of the process + int world_rank; + MPI_Comm_rank(MPI_COMM_WORLD, &world_rank); + + // Get the name of the processor + char processor_name[MPI_MAX_PROCESSOR_NAME]; + int name_len; + MPI_Get_processor_name(processor_name, &name_len); + + // Print a hello world message + printf("Hello, World! from from processor %s, rank %d out of %d processors\n", processor_name, world_rank, world_size); + + // Finalize the MPI environment + MPI_Finalize(); + } + EOF + +#. Add the `c-basic` and `devpkg-openmpi` bundles, which are needed to compile + it. + + .. code-block:: bash + + sudo swupd bundle-add c-basic devpkg-openmpi + +#. Compile it. + + .. code-block:: bash + + mpicc -o mpi-helloworld mpi-helloworld.c + +#. Copy the binary to all worker nodes. + + .. code-block:: bash + + pdcp -P workers ./mpi-helloworld $HOME + +#. Create a Slurm batch script to run it. + + .. code-block:: bash + + cat > mpi-helloworld.sh << EOF + #!/bin/sh + #SBATCH -o mpi-helloworld.out + #SBATCH --nodes=4 + #SBATCH --ntasks-per-node=1 + + srun ./mpi-helloworld + EOF + +#. Run the batch script. + + .. code-block:: bash + + sbatch mpi-helloworld.sh + +#. View the results on first worker node in the partition. + + .. code-block:: bash + + pdsh -w hpc-worker1 "cat mpi-helloworld.out" + + Example output: + + .. code-block:: console + + Hello, World! from from processor hpc-worker3, rank 2 out of 4 processors + Hello, World! from from processor hpc-worker4, rank 3 out of 4 processors + Hello, World! from from processor hpc-worker1, rank 0 out of 4 processors + Hello, World! from from processor hpc-worker2, rank 1 out of 4 processors + +.. _Slurm: + https://en.wikipedia.org/wiki/Slurm_Workload_Manager + +.. _MUNGE: + https://dun.github.io/munge/ + +.. _pdsh: + https://linux.die.net/man/1/pdsh + +.. _Slurm troubleshooting tips: + https://slurm.schedmd.com/troubleshoot.html diff --git a/_sources/tutorials/index.rst.txt b/_sources/tutorials/index.rst.txt new file mode 100644 index 00000000..6f15318e --- /dev/null +++ b/_sources/tutorials/index.rst.txt @@ -0,0 +1,75 @@ +.. _tutorials: + +Tutorials +######### + +Explore our tutorials to discover what you can do with |CL|, conveniently +sorted by difficulty level! Learn about :ref:`how we evaluate tutorials `. + +.. container:: multicolumns + + .. container:: column narrow + + .. rst-class:: colh3 + + Easy + + - :ref:`docker` + - :ref:`flatpak-tutorial` + - :ref:`kata` + - :ref:`lamp-server-install` + - :ref:`proxy` + - :ref:`redis` + - :ref:`smb-desktop` + - :ref:`smb-server` + - :ref:`yubikey-u2f` + + .. container:: column narrow + + .. rst-class:: colh3 + + Moderate + + - :ref:`hadoop` + - :ref:`broadcom` + - :ref:`fmv` + - :ref:`hpc` + - :ref:`kubernetes-bp` + - :ref:`mirror-upstream-server` + - :ref:`nvidia-cuda` + - :ref:`php` + - :ref:`vmware-workstation` + - :ref:`wp-install` + + .. container:: column narrow + + .. rst-class:: colh3 + + Difficult + + - :ref:`kubernetes` + - :ref:`nvidia` + - :ref:`openfaas` + - :ref:`multi-boot` + - :ref:`machine-learning` + - :ref:`zfs` + + +.. toctree:: + :hidden: + :maxdepth: 1 + :glob: + + * + multi-boot/* + +Archive +******* + +Older tutorials that may still be relevant to some users. + +.. toctree:: + :maxdepth: 1 + :glob: + + archive/* diff --git a/_sources/tutorials/kata.rst.txt b/_sources/tutorials/kata.rst.txt new file mode 100644 index 00000000..e8102fa5 --- /dev/null +++ b/_sources/tutorials/kata.rst.txt @@ -0,0 +1,134 @@ + .. _kata: + +Kata Containers\* +################# + +This tutorial describes how to install, configure, and run `Kata Containers`_ +on |CL-ATTR|. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +Kata Containers is an open source project developing a lightweight +implementation of :abbr:`VMs (Virtual Machines)` that offer the speed of +containers and the security of VMs. + +Prerequisites +************* + +This tutorial assumes you have installed |CL| on your host system. +For detailed instructions on installing |CL| on a bare metal system, follow +the :ref:`bare metal installation tutorial`. + +If you have Clear Containers installed on your |CL| system, then follow the +:ref:`migrate Clear Containers to Kata Containers tutorial`. + +Update |CL| with the following command: + +.. code-block:: bash + + sudo swupd update + +Install Kata Containers +*********************** + +Kata Containers is included in the :file:`containers-virt` bundle. +To install the framework: + +#. Install the containers-virt bundle: + + .. code-block:: bash + + sudo swupd bundle-add containers-virt + +#. Reload and restart the Docker\* systemd service. + + .. code-block:: bash + + sudo systemctl daemon-reload + sudo systemctl restart docker + +Run Kata Containers +******************* + +To use kata as the runtime for an individual container, add +:command:`--runtime=kata-runtime` to the :command:`docker run` command. For +example: + +.. code-block:: bash + + sudo docker run --runtime=kata-runtime -ti busybox sh + + +To use kata as the default runtime for all Docker containers: + +#. Set the default runtime for the Docker daemon: + + .. note:: + + The method below uses a systemd drop-in configuration to add a + command-line (CLI) parameter to the Docker daemon for setting the + *default-runtime*. Alternatively, the default runtime can be set in the + `Docker daemon configuration file + `_. + The Docker daemon will not start if the *default-runtime* configuration + in set multiple locations. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/system/docker.service.d/ + + cat <`_. + +For more background information see: + +* `What is Kubernetes?`_ +* `What is a Container Network Interface (CNI)?`_ + +* `What is a Container Runtime Interface (CRI)?`_ + + * `What is CRI+O?`_ + + * `What is containerd?`_ + + * `What is Docker?`_ + +* `What is Kata Containers\*?`_ + +Prerequisites +************* + +This tutorial assumes you have already installed |CL|. For detailed +instructions on installing |CL| on a bare metal system, follow the :ref:`bare +metal installation tutorial`. + +#. Review and make sure the `requirements for kubeadm + `_ + are satisfied for the host system. + +#. Before you continue, update your |CL| installation with the following + command: + + .. code-block:: bash + + sudo swupd update + + Learn about the benefits of having an up-to-date system for cloud + orchestration on the :ref:`swupd-guide` page. + +#. Kubernetes, a set of supported :abbr:`CRI (Container Runtime Interface)` + runtimes, :abbr:`CNI (Container Network Interface)` and `cloud-native-setup + scripts`_ are included in the `cloud-native-basic`_ bundle. Install the + cloud-native-basic bundle to get these components: + + .. code-block:: bash + + sudo swupd bundle-add cloud-native-basic + + +Set up Kubernetes automatically +******************************* + +|CL| provides `cloud-native-setup scripts`_ to automate system setup and +Kubernetes cluster initialization which allows you to get a cluster up and +running quickly. + +.. note:: + + By default, the scripts will update |CL| to the latest version, set up the + system as a Kubernetes master-node with **canal for container networking** + and **crio for container runtime**, and taint the master node to allow + workloads to run on it. Kata is installed as an optional alternative + runtime. The script can be configured to use other CNI's and CRI's by + following the directions on the `README + `_. + + See `What is a Container Network Interface (CNI)?`_ and `What is a + Container Runtime Interface (CRI)?`_ for more information. + +.. important:: + + If network proxy settings are required for Internet connectivity, configure + them now because the scripts will propagate proxy configuration based on + the running configuration. It is especially important to set the + :command:`no_proxy` variable appropriately for Kubernetes. + + The script will also modify the :file:`/etc/environment` and + :file:`/etc/profile.d/proxy.sh` files, if they exist, with the proxy + environment variables in the running shell when the script is executed. + + See the `Setting proxy servers for Kubernetes`_ section for details. + +#. Run the :file:`system-setup.sh` script to configure the |CL| system + settings. + + .. code-block:: bash + + sudo /usr/share/clr-k8s-examples/setup_system.sh + +#. Stop docker and containerd to avoid conflicting CRIs being detected. The + scripts use CRIO for the CRI. + + .. code-block:: bash + + sudo systemctl stop docker + sudo systemctl stop containerd + + +#. Install git as it's a dependency of the :file:`create_stack.sh`. + + .. code-block:: bash + + sudo swupd bundle-add git + + +#. Run the :file:`create_stack.sh` script to initialize the Kubernetes node + and setup a container network plugin. + + .. code-block:: bash + + sudo /usr/share/clr-k8s-examples/create_stack.sh minimal + +#. Follow the output on the screen and continue onto the section on `using + your cluster <#use-your-cluster>`_. + + +Uninstalling +============ + +#. If you need to delete the Kubernetes cluster or want to start from scratch + run the :file:`reset_stack.sh` script. + + .. warning:: + + This will stop components in the stack including Kubernetes, all CNI and + CRIs **and will delete** all containers and networks. + + .. code-block:: bash + + sudo /usr/share/clr-k8s-examples/reset_stack.sh + + +Set up Kubernetes manually +************************** + +Configure host system +===================== + +This tutorial uses the basic default Kubernetes configuration to get started. +You can customize your Kubernetes configuration according to your specific +deployment and security needs. + +The Kubernetes administration tool, :command:`kubeadm`, performs some +"`preflight checks`_" when initializing and starting a cluster. The steps +below are necessary to ensure those preflight checks pass successfully. + + +#. Enable IP forwarding: + + - Create the file :file:`/etc/sysctl.d/60-k8s.conf` to set the + :command:`net.ipv4.ip_forward` parameter + + .. code-block:: bash + + sudo mkdir -p /etc/sysctl.d/ + + sudo tee /etc/sysctl.d/99-kubernetes-cri.conf > /dev/null </dev/null + sudo swapoff -a + + .. note:: + + Kubernetes is designed to work without swap. Performance degradation of other workloads can occur + with swap disabled on systems with constrained memory resources. + +#. Add the the system's hostname to the :file:`/etc/hosts` file. Kubernetes + will read this file to locate the master host. + + .. code-block:: bash + + echo "127.0.0.1 localhost `hostname`" | sudo tee --append /etc/hosts + + +#. Enable the kubelet agent service to start at boot automatically: + + .. code-block:: bash + + sudo systemctl enable kubelet.service + + +.. important:: + + If network proxy settings are required for Internet connectivity, configure + them now because the scripts will propagate proxy configuration based on + the running configuration. It is especially important to set the + :command:`no_proxy` variable for Kubernetes. See the `Setting proxy servers + for Kubernetes`_ section for details. + + +Initialize the master node +************************** + +In Kubernetes, a master node is part of the `Kubernetes Control Plane +`_. + +Initializing a new Kubernetes cluster involves crafting a :command:`kubeadm +init` command. Adding parameters to this command can control the fundamental +operating components of the cluster. This means it is important to understand +and choose network and runtime options before running a :command:`kubeadm +init` command. + + +Choose a pod network add-on +=========================== + +See `What is a Container Network Interface (CNI)?`_ for information on what +pod network add-ons and CNIs. + +It is important to decide which CNI will be used early because some pod +network add-ons require configuration during cluster initialization. Check +whether or not your add-on requires special flags when you initialize the +master control plane. + +If your chosen network add-on requires appending to the :command:`kubeadm +init` command, make note of it before continuing. For example, if you choose +the *flannel* pod network add-on, then in later steps you must add the +following to the :command:`kubeadm init` command: + +.. code-block:: console + + --pod-network-cidr 10.244.0.0/16 + +.. important:: + + The version of CNI plugins installed needs to be compatible with the + version of Kubernetes that is installed otherwise the cluster may fail. + Check the Kubernetes version with :command:`kubeadm version -o short` and + refer to the documentation of the CNI plugins to obtain a compatible + version. + +Choose a container runtime +========================== + +See `What is a Container Runtime Interface (CRI)?`_ for more information on +what a CRI is. + +|CL| supports Kubernetes with the various runtimes +below with or without `Kata Containers`_: + +* `CRI+O`_ +* `containerd`_ +* `Docker`_ + +The container runtime that you choose will dictate the steps necessary to +initialize the master cluster with :command:`kubeadm init`. + +CRI+O +----- + +For information on CRI+O as a Kubernetes CRI, see `What is +CRI+O?`_. To use CRI+O as the Kubernetes CRI: + +#. Start the CRI-O service and enable it to run at boot automatically: + + .. code-block:: bash + + sudo systemctl enable --now crio.service + + When the crio service starts for the first time, it will create a + configuration file for crio at :file:`/etc/crio/crio.conf`. + +#. Run the kubeadm command to initialize the master node with the + :command:`--cri-socket` parameter: + + .. important:: + + You may need to add additional parameters to the command below, + depending the pod network addon in use. + + In this example, the :command:`--pod-network-cidr 10.244.0.0/16` + parameter is to use *flannel* as the pod networking. See `Choose a pod + network add-on`_ for more information. + + .. code-block:: bash + + sudo kubeadm init \ + --cri-socket=unix:///run/crio/crio.sock \ + --pod-network-cidr 10.244.0.0/16 + + +#. (Optional) By default, CRI+O will use runc as the default + runtime. CRI+O can optionally provide Kata Containers as a runtime. See + the `Add the Kata runtime to Kubernetes`_ section for details. + + With CRI+O, the `Kata Containers`_ can be set as the runtime with a + per-pod *RuntimeClass* annotation. + + .. note:: + + If you are using CRI-O + Kata Containers as the runtime and choose the + *flannel* for pod networking (see `Choose a pod network add-on`_), the + :file:`/etc/crio/crio.conf` file needs to include the value below. On + |CL| this is done automatically. + + .. code-block:: console + + [crio.runtime] + manage_network_ns_lifecycle = true + + +#. Once the cluster initialization is complete, continue reading about how to + `Use your cluster`_. + + +containerd +---------- + +For information on containerd as as Kubernetes CRI, see `What is +containerd?`_. To use containerd as the Kubernetes CRI: + +#. Start the containerd service and enable it to run at boot automatically: + + .. code-block:: bash + + sudo systemctl enable --now containerd.service + + +#. Configure kubelet to use containerd. and reload the service. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/system/kubelet.service.d/ + + cat << EOF | sudo tee /etc/systemd/system/kubelet.service.d/0-containerd.conf + [Service] + Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock" + EOF + +#. Configure kubelet to use systemd as the cgroup driver. and reload the + service. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/system/kubelet.service.d/ + + cat << EOF | sudo tee /etc/systemd/system/kubelet.service.d/10-cgroup-driver.conf + [Service] + Environment="KUBELET_EXTRA_ARGS=--cgroup-driver=systemd" + EOF + +#. Reload the systemd manager configuration. + + .. code:: bash + + sudo systemctl daemon-reload + +#. Run the kubeadm command to initialize the master node with the + :command:`--cri-socket` parameter: + + .. important:: + + You may need to add additional parameters to the command below, + depending the pod network addon in use. + + In this example, the :command:`--pod-network-cidr 10.244.0.0/16` + parameter is to use *flannel* as the pod networking. See `Choose a pod + network add-on`_ for more information. + + .. code-block:: bash + + sudo kubeadm init \ + --cri-socket=/run/containerd/containerd.sock + --pod-network-cidr 10.244.0.0/16 + + +#. (Optional) By default, containerd will use runc as the default + runtime. containerd can optionally provide Kata Containers as a runtime. + See the `Add the Kata runtime to Kubernetes`_ section for details. + + With containerd, the `Kata Containers`_ can be set as the runtime with a + per-pod *RuntimeClass* annotation. + +#. Once the cluster initialization is complete, continue reading about how to + `Use your cluster`_. + + +Docker +------ + +For information on Docker, see `What is Docker?`_. To use Docker as the +Kubernetes container runtime: + +#. Make sure Docker is installed: + + .. code:: bash + + sudo swupd bundle-add containers-basic + +#. Start the Docker service and enable it to start automatically at boot: + + .. code:: + + sudo systemctl enable --now docker.service + +#. Configure kubelet to use the |CL| directory for cni-plugins and reload the + service. + + .. code-block:: bash + + sudo mkdir -p /etc/systemd/system/kubelet.service.d/ + + cat << EOF | sudo tee /etc/systemd/system/kubelet.service.d/0-cni.conf + [Service] + Environment="KUBELET_EXTRA_ARGS=--cni-bin-dir=/usr/libexec/cni" + EOF + + .. code:: bash + + sudo systemctl daemon-reload + + +#. Run the kubeadm command to initialize the master node: + + .. important:: + + You may need to add additional parameters to the command below, + depending the pod network addon in use. + + In this example, the :command:`--pod-network-cidr 10.244.0.0/16` + parameter is to use *flannel* as the pod networking. See `Choose a pod + network add-on`_ for more information. + + .. code:: bash + + sudo kubeadm init \ + --pod-network-cidr 10.244.0.0/16 + + +#. Once the cluster initialization is complete, continue reading about how to + `Use your cluster`_. + + + +Add the Kata runtime to Kubernetes +----------------------------------- + +For information on Kata as a container runtime, see `What is Kata Containers\*?`_. +Using Kata Containers is optional. + +You can use *kata-deploy* to install all the necessary parts of Kata +Containers after you have a Kubernetes cluster running with one of the CRI's +using the default runc runtime. Follow the steps in the Kubernetes quick start +section of the `kata-containers GitHub README +`_ +to install Kata. + + + +Use your cluster +**************** + +Once your master control plane is successfully initialized, follow the +instructions presented about how to use your cluster and its *IP*, *token*, +and *hash* values are displayed. It is important that you record this +information because it is required to join additional nodes to the cluster. + +A successful initialization looks like this: + +.. code-block:: console + + Your Kubernetes control-plane has initialized successfully! + + To start using your cluster, you need to run the following as a regular user: + + mkdir -p $HOME/.kube + sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config + sudo chown $(id -u):$(id -g) $HOME/.kube/config + + ... + + You can now join any number of machines by running the following on each node + as root: + + kubeadm join : --token --discovery-token-ca-cert-hash sha256: + + +With the first node of the cluster setup, you can continue expanding the +cluster with additional nodes and start deploying containerized applications. +For further information on using Kubernetes, see `Related topics`_. + +.. note:: + + By default, the master node does not run any pods for security reasons. To + setup a single-node cluster and allow the master node to also run pods, the + master node will need to be untained. See the Kubernetes documentation on + `control plane node isolation + `_. + + +Troubleshooting +*************** + +Package configuration customization +=================================== + +|CL| is a stateless system that looks for user-defined package configuration +files in the :file:`/etc/` directory to be used as default. If +user-defined files are not found, |CL| uses the distribution-provided +configuration files for each package. + +If you customize any of the default package configuration files, you **must** +store the customized files in the :file:`/etc/` directory. If you edit any of +the distribution-provided default files, your changes will be lost in the next +system update as the default files will be overwritten with the updated files. + +Learn more about :ref:`stateless` in |CL|. + + +Logs +==== + +* Check the kubelet service logs :command:`sudo journalctl -u kubelet` + + +Setting proxy servers for Kubernetes +==================================== + +If you receive any of the messages below, check outbound Internet access. You +may be behind a proxy server. + + * Images cannot be pulled. + * Connection refused error. + * Connection timed-out or Access Refused errors. + * The warnings when :command:`kubeadm init` is run. + + .. code-block:: console + + [WARNING HTTPProxy]: Connection to "https://" uses proxy "". If that is not intended, adjust your proxy settings + [WARNING HTTPProxyCIDR]: connection to "10.96.0.0/12" uses proxy "". This may lead to malfunctional cluster setup. Make sure that Pod and Services IP ranges specified correctly as exceptions in proxy configuration + [WARNING HTTPProxyCIDR]: connection to "10.244.0.0/16" uses proxy "". This may lead to malfunctional cluster setup. Make sure that Pod and Services IP ranges specified correctly as exceptions in proxy configuration + + +If you use an outbound proxy server, you must configure proxy settings +appropriately for all components in the stack including :command:`kubectl` and +container runtime services. + +Configure the :ref:`proxy settings `, using the standard *HTTP_PROXY*, +*HTTPS_PROXY*, and *NO_PROXY* environment variables. The *NO_PROXY* values are +especially important for Kubernetes to ensure private IP traffic does not try +to go out the proxy. + +#. Set your environment proxy variables. Ensure that your local IP address is + **explicitly included** in the environment variable *NO_PROXY*. Setting + *localhost* is not sufficient! + + .. code-block:: bash + + export http_proxy=http://proxy.example.com:80 + export https_proxy=http://proxy.example.com:443 + export no_proxy=.svc,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,`hostname`,localhost + + .. important:: + + :command:`kubeadm` commands specifically use these shell variables for proxy + configuration. Ensure they are set your running terminal before running + :command:`kubeadm` commands. + +#. Run the following command to add systemd drop-in configurations for each + service to include proxy settings: + + .. code-block:: bash + + services=(kubelet docker crio containerd) + for s in "${services[@]}"; do + sudo mkdir -p "/etc/systemd/system/${s}.service.d/" + cat << EOF | sudo tee "/etc/systemd/system/${s}.service.d/proxy.conf" + [Service] + Environment="HTTP_PROXY=${http_proxy}" + Environment="HTTPS_PROXY=${https_proxy}" + Environment="SOCKS_PROXY=${socks_proxy}" + Environment="NO_PROXY=${no_proxy}" + EOF + done + + +#. Reload the systemd manager configuration. + + .. code-block:: bash + + sudo systemctl daemon-reload + + +If you had a previously failed initialization due to a proxy issue, restart +the process with the :command:`kubeadm reset` command. + + +DNS issues +========== + +* not found in message. + + Your DNS server may not be appropriately configured. Try adding an entry + to the :file:`/etc/hosts` file with your host's IP and Name. + + Use the commands :command:`hostname` and :command:`hostname -I` to + retrieve them. + + For example: + + .. code:: bash + + 10.200.50.20 myhost + + +* coredns pods are stuck in container creating state and logs show entries + similar to one of the following: + + .. code:: console + + Warning FailedCreatePodSandBox 5m7s kubelet, kata3 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get network JSON for pod sandbox k8s_coredns->-5gpj2_kube-system_): cannot convert version ["" "0.1.0" "0.2.0"] to 0.4.0 + + In this case the :file:`/etc/cni/net.d/10-flannel.conf` or another CNI file + is using an incompatible version. Delete the file and restart the stack. + + + .. code:: console + + Warning FailedCreatePodSandBox 117s (x197 over 45m) kubelet, kata3 (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_coredns->-npsm5_kube-system_: error getting ClusterInformation: Get https://[10.96.0.1]:443/apis/crd.projectcalico.org/v1/clusterinformations/default: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kubernetes") + + In this case, there may be multiple CNI configuration files in the + :file:`/etc/cni/net.d` folder. Delete all the files in this directory and + restart the stack. + + .. code:: console + + Warning FailedScheduling 55s (x3 over 2m12s) default-scheduler 0/1 + nodes are available: 1 node(s) had taints that the pod didn't tolerate. + + In this case, there may be multiple CNI configuration files in the + :file:`/etc/cni/net.d` folder. Delete all the files in this directory, apply + a CNI plugin, and restart the stack. + +Reference +********* + +What is Kubernetes? +=================== + +Kubernetes (K8s) is an open source system for automating deployment, scaling, +and management of containerized applications. It groups containers that make +up an application into logical units for easy management and discovery. + +Kubernetes supports using a variety of `container runtimes +`_. + +What is a Container Network Interface (CNI)? +============================================ + +In Kubernetes, a `pod +`_ is a group of one +or more containers and is the smallest deployable unit of computing in a +Kubernetes cluster. Pods have shared storage/network internally but +communication between pods requires additional configuration. If you want your +pods to be able to communicate with each other you must choose and install a +`pod network add-on`_. + +Some pod network add-ons enable advanced functionality with physical networks +or cloud provider networks. + +What is a Container Runtime Interface (CRI)? +============================================ + +Container runtimes are the underlying fabric that pod workloads execute inside +of. Different container runtimes offer different balances between features, +performance, and security. + +Kubernetes allows integration various container runtimes via a container +runtime interface (CRI). + +What is CRI+O? +-------------- + +`CRI+O `_ is a lightweight alternative to using Docker as +the runtime for kubernetes. It allows Kubernetes to use any OCI-compliant +runtime as the container runtime for running pods, such as runc and +Kata Containers as the container runtimes. + +CRI+O allows setting a different runtime per-pod. + +What is containerd? +------------------- + +`containerd `_ is the runtime that the Docker engine +is built on top of. + +Kubernetes can use containerd directly instead of going through the Docker +engine for increased robustness and performance. See the `blog post on +kubernetes containerd integration +`_ +for more details. + +containerd allows setting a different runtime per-pod. + +What is Docker? +--------------- + +`Docker `_ is an engine for running software packaged +as functionally complete units, called containers, using the same operating +system kernel. + +The default built-in runtime provided by Kubernetes is using the system Docker +installation via Dockershim and as a result is one of the simplest to use. One +limitation of using Dockershim is that all pods on the Kubernetes node will +inherit and use the default runtime that Docker is set to use. To be able to +specify a container runtime per-Kerbernetes service, use CRI+O or containerd. + +What is Kata Containers\*? +========================== + +`Kata Containers`_ is an alternative OCI compatible runtime that secures +container workloads in a lightweight virtual machine. It provides stronger +workloads isolation using hardware virtualization technology as a second layer +of defense for untrusted workloads or multi-tenant scenarios. + +The Kata Containers (kata-runtime) adheres to :abbr:`OCI (Open Container +Initiative*)` guidelines and works seamlessly with Kubernetes through Docker, +containerd, or CRI+O. + + +Related topics +============== + +* `Understanding basic Kubernetes architecture`_ + +* Installing a `pod network add-on`_ + +* `Joining your nodes`_ + +* `Deploying an application to your cluster`_ + +* See our document on :ref:`Kubernetes best practices ` + + + +.. _Kubernetes container orchestration system: https://kubernetes.io/ + +.. _Kata Containers: https://katacontainers.io/ + +.. _cloud-native-basic: https://github.com/clearlinux/clr-bundles/blob/master/bundles/cloud-native-basic + +.. _preflight checks: https://kubernetes.io/docs/reference/setup-tools/kubeadm/implementation-details/#preflight-checks + +.. _Understanding basic Kubernetes architecture: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-3 + +.. _Deploying an application to your cluster: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-2 + +.. _pod network add-on: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network + +.. _Joining your nodes: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#join-nodes + +.. _cloud-native-setup scripts: https://github.com/clearlinux/cloud-native-setup/tree/master/clr-k8s-examples + +.. _control-plane node: https://kubernetes.io/docs/concepts/#kubernetes-control-plane + +.. _RuntimeClass handler: https://kubernetes.io/docs/concepts/containers/runtime-class/ diff --git a/_sources/tutorials/lamp-server.rst.txt b/_sources/tutorials/lamp-server.rst.txt new file mode 100644 index 00000000..b83c65dc --- /dev/null +++ b/_sources/tutorials/lamp-server.rst.txt @@ -0,0 +1,517 @@ +.. _lamp-server-install: + +LAMP Web Server +############### + +This tutorial provides instructions on how to set up a +:abbr:`LAMP (Linux\*, Apache\*, MySQL\*, PHP)` web server on |CL-ATTR| and how +to use phpMyAdmin\* to manage an associated database. Note that this +tutorial installs MariaDB\*, which is a drop-in replacement for MySQL\*. + +In order to create a web server using |CL| as the host OS, your host system +must be running |CL|. This tutorial assumes you have installed |CL| on your +host system. For detailed instructions on installing |CL| on a bare metal +system, visit :ref:`bare-metal-install-desktop`. + +This tutorial covers: + +.. contents:: :local: + :depth: 1 + +Install Apache +************** + +Apache is an open source HTTP web server application that can run on several +operating systems, including |CL|. Go to the `Apache HTTP Server Project`_ +for more information. + +Install the httpd bundle +======================== + +The **httpd** bundle contains the packages needed to install the +Apache software bundle on |CL|. + +.. note:: + + Before you install new packages, update the |CL| with the following + console command: + + .. code-block:: bash + + sudo swupd update + +#. To install the bundle, enter the following command: + + .. code-block:: bash + + sudo swupd bundle-add httpd + +#. Start the Apache service and set it start automatically on boot, + enter the following commands: + + .. code-block:: bash + + sudo systemctl enable --now httpd.service + +#. To verify that the Apache server application is running, open a web + browser and navigate to: \http://localhost. + + If the service is running, a confirmation message will appear, similar to the + message shown in figure 1. + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/web-server-install-1.png + :alt: This web server is operational from host. + :scale: 50% + + `Figure 1: Confirmation that the Apache service is running.` + + .. note:: + + The :file:`index.html` file is located in the :file:`/var/www/html` directory + of your host system. You will copy this file into a new location after you + modify the configuration in the next step. + +Change the default configuration and data directory +*************************************************** + +|CL| is designed to be a :ref:`stateless` operating system which means that you +must create an optional configuration file to override the default values. +The default location of the Apache configuration file, :file:`httpd.conf`, +is located in the :file:`/usr/share/defaults/httpd` directory. |CL| can +override this directory as part of the stateless paradigm. This default +:file:`.conf` file includes the following directives that allow for additional +locations of configuration definitions: + +.. code-block:: console + + # Virtual hosts + IncludeOptional /usr/share/defaults/httpd/conf.d/*.conf + IncludeOptional /usr/share/defaults/httpd/conf.modules.d/*.conf + IncludeOptional /etc/httpd/conf.d/*.conf + IncludeOptional /etc/httpd/conf.modules.d/*.conf + +In this section you will define your own httpd.conf file to override the +default values, and define a custom DocumentRoot for your web server. + +#. Create the directory structure for :file:`/etc/httpd/conf.d`. + + .. code-block:: bash + + sudo mkdir -p /etc/httpd/conf.d + +#. Create and open the :file:`httpd.conf` file in your new :file:`/etc/httpd/conf.d` + directory. + + .. code-block:: bash + + sudo nano /etc/httpd/conf.d/httpd.conf + +#. Add the ``DocumentRoot`` variable to :file:`httpd.conf`. Copy the content + listed below into the new :file:`/etc/httpd/conf.d/httpd.conf` file. + + .. code-block:: console + + # + # Set a new location for DocumentRoot + # + DocumentRoot "/var/www/tutorial" + + # + # Relax access to content within /var/www/tutorial for this example + # + + AllowOverride none + Require all granted + + +#. Create a new ``DocumentRoot`` directory structure and copy the + :file:`index.html` file from :file:`/var/www/html` directory to + :file:`/var/www/tutorial`. + + .. code-block:: bash + + sudo mkdir –p /var/www/tutorial + cd /var/www/tutorial + sudo cp /var/www/html/index.html . + +#. To ensure a successful setup, edit the new :file:`index.html` file with an + obvious change. + + .. code-block:: bash + + sudo nano index.html + + For example, we changed the default message + + "It works!" + + to + + "It works from its new location!" + +#. Restart ``httpd.service``. + + .. code-block:: bash + + sudo systemctl restart httpd.service + +#. Go to \http://localhost to view the new screen. You should see your updated + default message from step 5. + +#. Change the configuration back to the default :file:`/var/www/html` + location. To do this, edit the :file:`/etc/httpd/conf.d/httpd.conf` file + again and replace any instance of /var/www/tutorial with /var/www/html. + + .. code-block:: bash + + sudo nano /etc/httpd/conf.d/httpd.conf + +#. Restart ``httpd.service``. + + .. code-block:: bash + + sudo systemctl restart httpd.service + +#. Go to \http://localhost and verify that you can see the default screen + again. + +#. Optionally, remove the /var/www/tutorial directory you previously created. + + .. code-block:: bash + + sudo rm /var/www/tutorial/index.html + sudo rmdir /var/www/tutorial + +Install PHP +*********** + +An Apache installation allows you to display static web pages. Enabling PHP +allows you to generate and display dynamic web pages. To add this +functionality to your web server, install PHP on your system. + +#. To get the php components, enter the following command: + + .. code-block:: bash + + sudo swupd bundle-add php-basic + +#. To enable PHP and set it to start automatically on boot, enter the + following commands: + + .. code-block:: bash + + sudo systemctl enable --now php-fpm.service + sudo systemctl restart httpd.service + + After restarting the Apache service, test your PHP installation. + +#. Create and open a file named :file:`phpinfo.php` in the :file:`/var/www/html/` + directory using a text editor. + + .. code-block:: bash + + sudo nano /var/www/html/phpinfo.php + +#. Add the following line to the file: + + .. code-block:: php + + + +#. Go to \http://localhost/phpinfo.php. + +#. Verify that the PHP information screen appears, similar to figure 2: + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/web-server-install-2.png + :alt: PHP information screen + :width: 600 + + `Figure 2: The PHP information screen.` + +If the PHP information screen is displayed, you have successfully installed +the PHP components and are now ready to add your database application to +complete your LAMP server implementation. + +Install MariaDB +*************** + +Install MariaDB to store content. MariaDB is a drop-in replacement for MySQL +and is available in the database-basic |CL| bundle. + +#. To install the database-basic bundle, enter the following command: + + .. code-block:: bash + + sudo swupd bundle-add mariadb + +#. To start MariaDB after it is installed and set it to start automatically on + boot, enter the following commands: + + .. code-block:: bash + + sudo systemctl enable --now mariadb + +#. To check the status of MariaDB, enter the following command: + + .. code-block:: bash + + sudo systemctl status mariadb + + Press :kbd:`Ctrl` + :kbd:`c` or :kbd:`q` to exit. + +Security hardening +================== + +With the MariaDB service running, we can perform some basic security +hardening. + +#. To add a basic layer of security, enter the following command: + + .. code-block:: bash + + sudo mysql_secure_installation + +#. Respond to the questions that appear in the script below. + + .. note:: + + Our suggested responses follow each question. + + .. code-block:: none + + Enter current password for root (enter for none): + + In order to secure MariaDB, we need the current password for the root + user. For a newly installed MariaDB without a set root password, the + password is blank. Thus, press enter to continue. + + .. code-block:: none + + OK, successfully used password, moving on... + + Set root password? [Y/n] + + .. _set-password: + + Set the root password to prevent unauthorized MariaDB root user logins. + To set a root password, type 'y'. + + .. code-block:: none + + New password: + + Type the desired password for the root user. + + .. code-block:: none + + Re-enter new password: + + Re-type the desired password for the root user. + + .. code-block:: none + + Password updated successfully! + Reloading privilege tables.. + ... Success! + + Remove anonymous users? [Y/n] + + By default, a MariaDB installation includes an anonymous user that allows + anyone to log in to MariaDB without a user account. This anonymous user + is intended only for testing and for a smoother installation. To remove + the anonymous user and make your database more secure, type 'y'. + + .. code-block:: none + + ... Success! + Disallow root login remotely? [Y/n] + + Normally, root should only be allowed to connect from the 'localhost'. This + ensures that someone cannot guess the root password from the network. To + block any remote root login, type 'y'. + + .. code-block:: none + + ... Success! + Remove test database and access to it? [Y/n] + + By default, MariaDB includes a database named 'test' which anyone can access. + This database is also intended only for testing and should be removed. To + remove the test database, type 'y'. + + .. code-block:: none + + - Dropping test database... + ... Success! + - Removing privileges on test database... + ... Success! + Reload privilege tables now? [Y/n] + + Reloading the privilege tables ensures all changes made so far take + effect immediately. To reload the privilege tables, type 'y'. + + .. code-block:: none + + ... Success! + + Cleaning up... + + All done! If you've completed all of the above steps, your MariaDB + installation should now be secure. + + Thanks for using MariaDB! + +The MariaDB installation is complete, and we can now install phpMyAdmin to +manage the databases. + +Install phpMyAdmin +****************** + +The web-based tool phpMyAdmin is a straightforward way to manage MySQL or +MariaDB databases. Visit the `phpMyAdmin`_ website for the complete +discussion regarding phpMyAdmin, its documentation, the latest downloads, +and other useful information. + +In this tutorial, we use the latest English version of phpMyAdmin. + +#. Download the :file:`phpMyAdmin--english.tar.gz` file to your + :file:`~/Downloads` directory. Here, refers to the current + version available at https://www.phpmyadmin.net/downloads. + + .. note:: + + This example downloads and uses version 4.6.4. + +#. Once the file has been successfully downloaded and verified, decompress + the file and directories into the Apache web server document root + directory. Use the following commands: + + .. code-block:: bash + + cd /var/www/html + sudo tar –xzvf ~/Downloads/phpMyAdmin-4.6.4-english.tar.gz + +#. To keep things simple, rename the newly created + :file:`phpMyAdmin-4.6.4-english` directory to :file:`phpMyAdmin` with the + following command: + + .. code-block:: bash + + sudo mv phpMyAdmin-4.6.4-english phpMyAdmin + +Use phpMyAdmin to manage a database +*********************************** + +You can use the phpMyAdmin web-based tool to manage your databases. Follow the +steps below for setting up a database called "WordPress". + +#. Verify that a successful installation of all LAMP server components by + going to \http://localhost/phpMyAdmin. See figure 3. + +#. Log in with your root userid and the password you set up when you ran the + :ref:`mysql_secure_installation command `. Enter your + credentials and select :guilabel:`Go` to log in: + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/web-server-install-3.png + :alt: phpMyAdmin login page + :width: 600 + + `Figure 3: The phpMyAdmin login page.` + +#. Verify a successful login by confirming that the main phpMyAdmin page + displays, as shown in figure 4: + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/web-server-install-4.png + :alt: phpMyAdmin dashboard + :width: 600 + + `Figure 4: The phpMyAdmin dashboard.` + +#. Set up a database by selecting the :guilabel:`Databases` tab, as shown in + figure 5. + +#. Enter `WordPress` in the text field below the :guilabel:`Create database` + label. + +#. Select the :guilabel:`utf8_unicode_ci` option from the + :guilabel:`Collation` drop-down menu beside the text field. + +#. Click :guilabel:`Create`. + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/web-server-install-5.png + :alt: Databases tab + :width: 600 + + `Figure 5: The Databases tab.` + +#. Set up user permissions by selecting the :guilabel:`WordPress` database + located in the left panel. See figure 6. + +#. Select the :guilabel:`Privileges` tab. Figure 6 shows its contents. + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/web-server-install-6.png + :alt: Privileges tab + :width: 600 + + `Figure 6: The Privileges tab.` + +#. Click :guilabel:`Add user account` located at the bottom of the + :guilabel:`Privileges` tab. The `Add user account` page appears, as shown + in figure 7. + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/web-server-install-7.png + :alt: User accounts tab + :width: 600 + + `Figure 7: The User accounts tab.` + +#. Enter the following information in the corresponding fields that appear + in figure 7 above: + + * User name: wordpressuser + + * Password: wp-example + + * Re-type: wp-example + +#. In the `Database for user account` section, select + :guilabel:`Grant all privileges on database “WordPress”.` + +#. At the bottom of the page, click :guilabel:`Go`. + +If successful, you should see the screen shown in figure 8: + +.. rst-class:: dropshadow + +.. figure:: ../_figures/wordpress/web-server-install-8.png + :alt: User added successfully + :width: 600 + + `Figure 8: The user wordpressuser is successfully added.` + +**Congratulations!** + +You have now created a fully functional LAMP server along with a +WordPress\*-ready database using |CL|. + +Next steps +********** + +Next, add the WordPress components needed to host a WordPress website with :ref:`wp-install`. + +.. _Apache HTTP Server Project: https://httpd.apache.org/ +.. _phpMyAdmin: https://www.phpmyadmin.net/ diff --git a/_sources/tutorials/mirror-upstream-server.rst.txt b/_sources/tutorials/mirror-upstream-server.rst.txt new file mode 100644 index 00000000..126183f4 --- /dev/null +++ b/_sources/tutorials/mirror-upstream-server.rst.txt @@ -0,0 +1,244 @@ +.. _mirror-upstream-server: + +Mirror Upstream |CL| Update Server +################################## + +For organizations that want to use the |CL| upstream updates, but want the +benefits of a local mirror, this tutorial shows how to set up one and +configure your |CL| clients to use it. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* The recommended disk space for the mirror server should have at least 100GB + of disk space as each complete update content is approximately 45GB. + +Install up |CL| server to host updates +************************************** + +#. Follow the :ref:`bare-metal-install-server` guide to install |CL| server. + Add a user with `Administrator` privilege. + +#. After installation is complete, boot it up. + +#. Add the `wget` bundle. This will be used to clone the upstream |CL| server. + + .. code-block:: bash + + sudo swupd bundle-add wget + +Clone the |CL| update content +***************************** + +|CL| periodically releases a "minversion", which is a complete update. +Then, subsequent releases are small updates until the next minversion. + +Download a minversion to start your mirror. + +#. Determine a proper minversion by looking at a `Manifest.MoM`_ file + for a particular release of |CL|. + + For example, if you look at + https://cdn.download.clearlinux.org/update/33010/Manifest.MoM, + you will see that the minversion is 32900. So clone this version as the + starting point. + + .. code-block:: console + :emphasize-lines: 4 + + MANIFEST 30 + version: 33010 + previous: 33000 + minversion: 32900 + filecount: 1131 + timestamp: 1588358889 + contentsize: 0 + +#. Make a directory to store the cache. + + .. code-block:: bash + + mkdir ~/mirror-download-clearlinux-org && cd $_ + +#. Recursively download the :file:`update/0` folder. + + .. code-block:: bash + + wget --no-verbose \ + --no-parent --recursive \ + --no-host-directories -erobots=off \ + --reject "index.html" https://cdn.download.clearlinux.org/update/0/ + +#. Recursively download the :file:`update/version` folder. + + .. code-block:: bash + + wget --no-verbose \ + --no-parent --recursive \ + --no-host-directories -erobots=off \ + --reject "index.html" https://cdn.download.clearlinux.org/update/version/ + +#. Now, recursively download the determined minversion, which for this example + is 32900. + + .. code-block:: bash + + wget --no-verbose \ + --no-parent --recursive \ + --no-host-directories -erobots=off \ + --reject "index.html" https://cdn.download.clearlinux.org/update/32900/ + + .. note:: + + A minversion is pretty big, which is approximately 45GB. Depending on your + proximity to the upstream server and your connection speed to the Internet, + it may take up to a couple of days or more to complete the download. So + be patient. + +#. Download later versions, up to the latest, if you like. + +Setup a web server to host the mirrored content +*********************************************** + +By design, the |CL| swupd client communicates with the update server using +HTTPS for security reasons. However, it can use HTTP by adding the +:command:`--allow-insecure-http` flag, if needed. Setting an HTTPS is a lot +more involved. For this tutorial, we'll just use an HTTP server for +demonstration purpose. + +#. Install the `nginx` bundle. + + .. code-block:: bash + + sudo swupd bundle-add nginx + +#. Configure the web server. + + a. Create a symbolic link to the mirrored update content directory. + + .. code-block:: bash + + sudo mkdir -p /var/www && cd $_ + sudo ln -sf $HOME/mirror-download-clearlinux-org mirror-download-clearlinux-org + + #. Set up nginx configuration files. + + .. code-block:: bash + + sudo mkdir -p /etc/nginx/conf.d + sudo cp /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf + + #. Grant $USER permission to run the web server. + + .. code-block:: bash + + sudo tee -a /etc/nginx/nginx.conf << EOF + user $USER; + EOF + + #. Configure the web server. + + .. code-block:: bash + + sudo tee -a /etc/nginx/conf.d/mirror-download-clearlinux-org.conf << EOF + server { + listen 80; + listen [::]:80; + server_name localhost; + location / { + root /var/www/mirror-download-clearlinux-org; + autoindex on; + } + } + EOF + + #. Set nginx to start automatically on boot and then start it. + + .. code-block:: bash + + sudo systemctl enable nginx --now + +Test your mirror +**************** + +Now, try out your mirror by installing |CL| and adding bundles from it. + +#. Download either the live desktop or live server installer ISO of the + `same version` as the mirrored version, which is 32900 for this tutorial. + Go to `https://cdn.download.clearlinux.org/releases//clear`. + +#. Burn the ISO to a thumb drive. See :ref:`bootable-usb`. + +#. Boot it up and start the installer. Depending on which version of + |CL| you want to install, follow one of these guides: + + * *Desktop* version: :ref:`bare-metal-install-desktop` + * *Server* version: :ref:`bare-metal-install-server` + + In the :guilabel:`Advanced options` tab of the installer, select + :guilabel:`Swupd Mirror`. See Figure 1. + + .. rst-class:: dropshadow + + .. figure:: ../_figures/mirror-upstream-server/mirror-upstream-server-01.png + :scale: 100% + :alt: Advanced options > Swupd Mirror + + `Figure 1: Advanced options > Swupd Mirror` + + In the :guilabel:`Mirror URL` field, set it to the IP address of your mirror. It should be something like this: http:///update. + And check the option :guilabel:`Allow installation over insecure connections (http://)`. See Figure 2. + + .. rst-class:: dropshadow + + .. figure:: ../_figures/mirror-upstream-server/mirror-upstream-server-02.png + :scale: 100% + :alt: Advanced options > Mirror URL setting + + `Figure 2: Advanced options > Mirror URL setting` + +#. After installation completes, boot up, and log in. + +#. Verify that the swupd client is pointing to your mirror. + + .. code-block:: bash + + sudo swupd info + + Example output: + + .. code-block:: console + + Warning: This is an insecure connection + The --allow-insecure-http flag was used, be aware that this poses a threat the system + + Distribution: Clear Linux OS + Installed version: 32900 + Version URL: https://192.168.1.100/update + Content URL: https://192.168.1.100/update + +#. Try listing available bundles on your mirror. + + .. code-block:: bash + + sudo swupd bundle-list -a + +#. Add a bundle. + + .. code-block:: bash + + sudo swupd bundle-add + +Keep your mirror in sync with upstream +************************************** + +Be sure to keep your mirror in sync with upstream so that your clients have the +latest and greatest software and security updates. You can do that continuing +to clone the newer upstream releases. + +.. _Manifest.MoM: + https://docs.01.org/clearlinux/latest/reference/manpages/swupd.1.html diff --git a/_sources/tutorials/multi-boot/dual-boot-linux.rst.txt b/_sources/tutorials/multi-boot/dual-boot-linux.rst.txt new file mode 100644 index 00000000..293c071d --- /dev/null +++ b/_sources/tutorials/multi-boot/dual-boot-linux.rst.txt @@ -0,0 +1,422 @@ +.. _dual-boot-linux: + +Dual-boot |CL-ATTR| with Any GRUB-based Linux\* Distro +########################################################## + +In this tutorial, we show how to install |CL| alongside any GRUB-based +Linux\* distro. To do so, we resize the existing Linux root partition to +make room to install |CL|. Then we show 3 methods to dual-boot |CL| +with an existing Linux distro. + +Although we use Ubuntu\* 19.04 Desktop as the example here, +these instructions also work for other distros such as Mint\*, Kubuntu\*, +Fedora\*, CentOS\*, among others. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* Ubuntu 19.04 Desktop is already installed. +* There is no unallocated space available, but there is free space + on the Ubuntu root partition. +* Follow the Preliminary steps that follow below. + +.. include:: ../../get-started/bare-metal-install-desktop.rst + :start-after: preliminary-steps-install-desktop: + :end-before: install-on-target-start: + +Install |CL| with Advanced Installation +*************************************** + +#. Shut down the Ubuntu OS before proceeding. + +#. Boot up the |CL| live desktop image, and click the penguin + icon, at left, to launch the installer. + +#. Complete the :guilabel:`Required Options` until you reach + :guilabel:`Select Installation Media`. See Figure 1. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-01.png + :scale: 100% + :alt: Required options + + Figure 1: Required options + +#. Click :guilabel:`Select Installation Media`. + +#. Select the “Advanced Installation” option. See Figure 2. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-02.png + :scale: 100% + :alt: Advanced Installation + + Figure 2: Advanced Installation + +#. Click :guilabel:`Partition Media` to start the GParted tool. + +#. Shrink the Ubuntu root partition to free up some space for |CL|. + + a. Select the Ubuntu root partition (in this example: /dev/sda2). + Right-click it and select “Resize/Move”. See Figure 3. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-03.png + :scale: 100% + :alt: Ubuntu root partition + + Figure 3: Ubuntu root partition + + #. In the :guilabel:`New size (MiB)` field, reduce the size of the root + partition, leaving at least 22GB free. This allows enough space to + create a swap partition (250MB) and root partition (approx. 21GB) for + |CL|. + + .. note:: + + The resulting free space appears in the “Free space following + (MiB)”. Click the “Resize/Move” button. See Figure 4. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-04.png + :scale: 100% + :alt: Resize Ubuntu root + + Figure 4: Resize Ubuntu root + + #. Click the green checkmark button to proceed. See Figure 5. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-05.png + :scale: 100% + :alt: New unallocated space + + Figure 5: New unallocated space + +#. Share the existing EFI system partition by designating as such. + + a. Right-click the :guilabel:`EFI System Partition` (e.g., /dev/sda1) + and select :guilabel:`Name Partition`. + + #. Set the name to “CLR_BOOT”. See Figure 6 and Figure 7. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-06.png + :scale: 100% + :alt: Name CLR_BOOT partition + + Figure 6: Name CLR_BOOT partition + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-07.png + :scale: 100% + :alt: Resulting CLR_BOOT main screen + + Figure 7: Resulting CLR_BOOT main screen + +#. Create a swap partition for |CL|. + + a. Right-click the “unallocated” partition and select :guilabel:`New` + to add a new partition. + + #. Enter “250” in the :guilabel:`New Size (MiB)` field to create a + 250MB swap space. + + #. Enter “CLR_SWAP” in the :guilabel:`Partition name` field and select + “linux-swap” as the “File system” type. + + #. Click the “Add” button. See Figure 8. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-08.png + :scale: 100% + :alt: Create CLR_SWAP partition + + Figure 8: Create CLR_SWAP partition + +#. Create |CL| root partition. + + a. Right-click “unallocated” partition again and select :guilabel:`New` + to add a new partition. + + #. Create a partition that is at least 21GB, enter “CLR_ROOT” in the + :guilabel:`Partition name` field, and select a :guilabel:`File system` + type of your choice. + + #. Click the “Add” button. See Figure 9. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-09.png + :scale: 100% + :alt: Create CLR_ROOT partition + + Figure 9: Create CLR_ROOT partition + +#. Click the green checkmark button to create the newly-defined partitions. + See Figure 10. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-10.png + :scale: 100% + :alt: Partitions to be created + + Figure 10: Partitions to be created + +#. Close the GParted window, and the |CL| installer will reappear with + the newly-defined partitions to use. See Figure 11. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-11.png + :scale: 100% + :alt: |CL| installer partitions defined + + Figure 11: |CL| installer partitions defined + +#. Complete the remaining steps of :guilabel:`Required Options` to + to install |CL|. Complete any :guilabel:`Advanced Options` as desired. + +Boot |CL| Using One of Three Methods +************************************ + +Although we installed |CL| last, Ubuntu is still the default boot OS. +There are three methods to boot |CL|: + +#. Make systemd-boot, the boot loader that |CL| uses, the default + boot loader to boot |CL| and also chain-boot GRUB; therefore, boot + Ubuntu. See `boot-clr-method-1`_. + +#. Use GRUB to chain-boot systemd-boot, therefore boot |CL|. + See `boot-clr-method-2`_. + +#. Use your BIOS “Boot Menu” to select and boot |CL|. + Refer to your system's manual on how to bring up the "Boot Menu". + +.. _boot-clr-method-1: + +Method 1: Use systemd-boot to Boot |CL| and also Chain-boot GRUB +================================================================ + +systemd-boot is the bootloader used by |CL|. Because |CL| was installed +after a GRUB-based distro, GRUB is still the default bootloader. +In this method, we make systemd-boot the default bootloader instead and +also provide a path to chain-boot GRUB. + +#. Boot up the |CL| installer image. + +#. Open a terminal window. + +#. Identify the EFI system partition, Ubuntu root partition, and |CL| root + partition. + + .. code-block:: bash + + sudo fdisk -l + + Example output: + + .. code-block:: console + + clrlinux@clr-live~ $ sudo fdisk -l + ... + + Disk /dev/sda: 335.4 GiB, 360080695296 bytes, 703282608 sectors + Disk model: INTEL SSDSCKKF36 + Units: sectors of 1 * 512 = 512 bytes + Sector size (logical/physical): 512 bytes / 512 bytes + I/O size (minimum/optimal): 512 bytes / 1048576 bytes + Disklabel type: gpt + Disk identifier: D5CB69E9-2C27-4A16-9552-3CD6BFA5DA77 + + Device Start End Sectors Size Type + /dev/sda1 2048 1050623 1048576 512M EFI System + /dev/sda2 1050624 498481151 497430528 237.2G Linux filesystem + /dev/sda3 498481152 498993151 512000 250M Linux swap + /dev/sda4 498993152 703281151 204288000 97.4G Linux root (x86-64) + ... + + The above example output contains these partitions: + + * ``/dev/sda1`` is the EFI system partition originally created by Ubuntu + and shared with |CL| + * ``/dev/sda2`` is the Ubuntu root partition + * ``/dev/sda3`` is the swap partition for |CL| + * ``/dev/sda4`` is the |CL| root partition + + The remaining steps will work with these partitions. + +#. Mount these partitions. + + .. code-block:: bash + + sudo mkdir /mnt/clearlinux + sudo mount /dev/sda4 /mnt/clearlinux/ + sudo mount /dev/sda1 /mnt/clearlinux/boot + +#. Make systemd-boot the default bootloader. + + .. code-block:: bash + + sudo bootctl install --esp-path=/mnt/clearlinux/boot + +#. Add a timeout to systemd-boot so that it will present the menu of + bootable OSes and give you time to select the one you want to boot. + + .. code-block:: bash + + sudo clr-boot-manager set-timeout 20 --path=/mnt/clearlinux + +#. Add a system-boot boot entry for GRUB. + + .. code-block:: bash + + sudo tee -a /mnt/clearlinux/boot/loader/entries/grub.conf << EOF + title GRUB menu + efi /EFI/ubuntu/grubx64.efi + EOF + +#. Umount all partitions. + + .. code-block:: bash + + sudo umount /mnt/clearlinux/boot /mnt/clearlinux + +#. Reboot. + + .. code-block:: bash + + sudo reboot + +#. Remove the |CL| installer USB thumb drive. + +#. You should be presented with the :command:`systemd-boot` menu. + See Figure 12. + + .. figure:: ../../_figures/multi-boot/dual-boot-linux-12.png + :scale: 100% + :alt: systemd-boot menu showing GRUB + + Figure 12: systemd-boot menu showing GRUB + +.. _boot-clr-method-2: + +Method 2: Use GRUB to Boot |CL| +=============================== + +In this method, we keep GRUB as the default bootloader, but configure it +to chain-boot systemd-boot, thus allowing us to boot |CL|. Again, we're using +Ubuntu as our working example. + +#. Boot up Ubuntu. + +#. Open a terminal window. + +#. Set a timeout value for the GRUB menu so it will be visible at boot time and + allow you select one which OS to boot. + + a. sudoedit :file:`/etc/default/grub` + + .. code-block:: bash + + sudoedit /etc/default/grub + + #. Set the ``GRUB_TIMEOUT`` variable to a desired value. + +#. Create a menu entry for systemd-boot bootloader. + + a. Identify the UUID for EFI system partition that systemd-boot resides on. + The example below shows the UUID for the EFI system on /dev/sda1 is + "A5A0-337D". + + .. code-block:: bash + + sudo blkid + + Example output: + + .. code-block:: console + :emphasize-lines: 1 + + /dev/sda1: UUID="A5A0-337D" TYPE="vfat" PARTLABEL="CLR_BOOT" PARTUUID="ee664fec-1ade-40d0-9ce4-c08805003c8d" + /dev/sda2: UUID="219969c5-1106-4e9f-b6f5-8188d7d94b8b" TYPE="ext4" PARTUUID="00bdf0dc-264d-493c-bf23-a85105011175" + /dev/sda3: UUID="65221b07-33cb-40b7-9812-ea484c7606c9" TYPE="swap" PARTLABEL="CLR_SWAP" PARTUUID="5a7926b4-bd11-4bad-b932-cc31c0a75d27" + /dev/sda4: UUID="0bedd545-58a7-4f34-b22a-c50bb4a1c2f5" TYPE="ext4" PARTLABEL="CLR_ROOT" PARTUUID="2d60cd03-5739-4cd5-adcc-51a107cde388" + + #. sudoedit :file:`/etc/grub.d/40_custom` and add a menu entry + for |CL| using UUID from the previous step (for example): + + .. code-block:: console + :linenos: + :emphasize-lines: 7-10 + + #!/bin/sh + exec tail -n +3 $0 + # This file provides an easy way to add custom menu entries. Simply type the + # menu entries you want to add after this comment. Be careful not to change + # the 'exec tail' line above. + + menuentry 'Clear Linux OS' { + search --fs-uuid --no-floppy --set=root A5A0-337D + chainloader (${root})/EFI/org.clearlinux/bootloaderx64.efi + } + + #. Update GRUB. + + .. code-block:: bash + + sudo update-grub + +#. Reboot. + +#. At the GRUB boot menu, select :guilabel:`Clear Linux OS` to boot |CL|. + +#. Log in. + +#. Open a terminal window. + +#. By default, any future calls to :command:`clr-boot-manager`, such as + after a kernel update by :command:`swupd` or setting the timeout + value for systemd-boot, will modify the UEFI boot order + which will result in making systemd-boot the first boot + entry and you won't be able to boot Ubuntu any longer. And in order + to boot Ubuntu first, you must change the UEFI boot order back. + To prevent :command:`clr-boot-manager` from touching the UEFI boot + order, which is especially important in a dual-boot setup, + follow these steps: + + .. code-block:: bash + + sudo mkdir -p /etc/kernel + sudo tee -a /etc/kernel/update_efi_vars << EOF + false + EOF + +.. tip:: + + The default installation of |CL| does not set a timeout value for + systemd-boot. Thus, you will not see the systemd-boot menu and the + default kernel will boot right away. + To set a timeout value (for example: 25 seconds), enter: + + .. code-block:: bash + + sudo clr-boot-manager set-timeout 25 + +.. _download the live desktop image: https://clearlinux.org/downloads + +.. _Downloads: https://clearlinux.org/downloads + diff --git a/_sources/tutorials/multi-boot/dual-boot-win.rst.txt b/_sources/tutorials/multi-boot/dual-boot-win.rst.txt new file mode 100644 index 00000000..9ab75e58 --- /dev/null +++ b/_sources/tutorials/multi-boot/dual-boot-win.rst.txt @@ -0,0 +1,397 @@ +.. _dual-boot-win: + +Dual-boot |CL-ATTR| and Windows\* 10 OS +######################################## + +This guide shows how to install |CL-ATTR| adjacent to an existing Windows\* +10 :abbr:`OS (operating system)`. To add |CL| to an existing Windows installation, follow a method below. + +In this tutorial you'll :ref:`bare-metal-install-desktop` as +an additional partition. Alternatively, you may also +:ref:`bare-metal-install-server`. + +.. contents:: + :local: + :depth: 1 + +Method 1: Shrink Windows partition and install |CL| +*************************************************** +For this method, we shrink the Windows 10 OS partition to make space for |CL|. + +#. Boot up the Windows 10 OS. + +#. Launch the :file:`Disk Management` utility found under + :guilabel:`Create and format hard disk partitions`. + +#. Right-click the primary Windows partition and select :guilabel:`Shrink Volume...`. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-win-01.png + :scale: 85% + :alt: Disk Management > Shrink Volume + + Figure 1: Disk Management > Shrink Volume. + +#. Shrink the size of the partition by at least the following amount: + + * For *Desktop* version, allow at least 21GB. + + * For *Server* version, allow at least 4GB. + + +#. We shrink C by about 21GB, as shown in Figure 2. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-win-02.png + :scale: 85% + :alt: Shrink C + + Figure 2: Shrink C. + +#. Shutdown the Windows 10 OS. + +#. Follow one of these guides to install |CL|: + + * *Desktop* version: :ref:`bare-metal-install-desktop` + * *Server* version: :ref:`bare-metal-install-server` + + a. In the :guilabel:`Required options` tab, choose :guilabel:`Select + Installation Media`. + + #. Within that menu, select :guilabel:`Safe Installation`. + + #. Go through remaining steps to complete the installation. + + #. Reboot. + +#. During the BIOS POST stage, press :kbd:`F10`, or the proper F-key for your + system, to launch the :guilabel:`Boot Menu`. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-win-03.png + :scale: 85% + :alt: Boot menu + + Figure 3: Boot menu + +#. In the :guilabel:`Boot Menu`, use the arrow to select the + :guilabel:`OS bootloader` as boot device (highlighted). + + Some BIOSes do not support listing multiple partitions. In this case, + it will only show one bootable partition. + + .. tip:: + + If you don't want to use the BIOS boot menu each time to select an OS, + follow :ref:`Advanced: Use systemd-boot to boot Windows 10 OS and + |CL| `. + +Method 2: Add another hard disk to your system where you installed Windows +************************************************************************** + +#. Shutdown your system. + +#. Open your system and attach another hard drive. + +#. Power up your system. + +#. Follow one of these guides to install |CL|: + + * *Desktop* version: :ref:`bare-metal-install-desktop` + * *Server* version: :ref:`bare-metal-install-server` + + a. In the :guilabel:`Required options` tab, choose :guilabel:`Select + Installation Media`. + + #. Within that menu, select :guilabel:`Destructive Installation`, and + select the new hard drive from the device list. + + .. warning:: + + Make sure you don’t select the drive with your Windows 10 OS. + + #. Go through remaining steps to complete the installation. + + #. Reboot. + +#. During the BIOS POST stage, press :kbd:`F10`, or the proper F-key for your + system, to launch the :guilabel:`Boot Menu`. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-win-03.png + :scale: 85% + :alt: Boot menu + + Figure 4: Boot menu + +#. In the :guilabel:`Boot Menu`, use the arrow to select the + :guilabel:`Linux bootloader` as boot device. + + Some BIOSes do not support listing multiple partitions. In this case, + it will only show one bootable partition. + +.. tip:: + + If you don't want to use the BIOS boot menu each time to select an OS, + follow the next section. + +.. _advanced-systemd-boot: + +Advanced: Use systemd-boot to boot Windows 10 OS and |CL| +********************************************************* + +If you prefer not to use your BIOS to load the :guilabel:`Boot Menu` and select an OS to boot, you can make :command:`systemd-boot` the default bootloader and add Windows 10 OS to the boot list. This option is also a workaround for BIOSes that don’t support booting more than one partition. + +#. Boot up a |CL| live image from a USB thumb drive. + +#. Open a terminal window and enter: + + .. code-block:: bash + + lsblk -po NAME,SIZE,TYPE,FSTYPE,PARTLABEL + + Example output: + + .. code-block:: console + :emphasize-lines: 6,8,9,11 + + clrlinux@clr-live~ $ lsblk -po NAME,SIZE,TYPE,FSTYPE,PARTLABEL + NAME SIZE TYPE FSTYPE PARTLABEL + /dev/loop0 2.3G loop squashfs + /dev/sda 335.4G disk + ├─/dev/sda1 450M part ntfs Basic data partition + ├─/dev/sda2 100M part vfat EFI system partition + ├─/dev/sda3 16M part Microsoft reserved partition + ├─/dev/sda4 286G part ntfs Basic data partition + ├─/dev/sda5 143M part vfat EFI + ├─/dev/sda6 244M part swap linux-swap + └─/dev/sda7 48.5G part ext4 / + sdb 7G disk iso9660 + ├─/dev/sdb1 2.7G part iso9660 + └─/dev/sdb2 100M part vfat + + The example output shows: + + * /dev/sda2 is the EFI system partition created by Windows 10 OS + * /dev/sda4 is the primary Windows partition + * /dev/sda5 is the EFI system partition created by |CL| + * /dev/sda7 is the |CL| root partition + + .. note:: + + To help narrow down a partition even more, you add the ``PARTTYPE`` + option to :command:`lsblk` and cross-reference against the + `partition type GUIDs wiki`_. + +#. Create mount points. + + .. code-block:: bash + + sudo mkdir /mnt/windows-efi + + sudo mkdir /mnt/clearlinux + +#. Mount the EFI system partition for Windows 10 OS. + + .. code-block:: bash + + sudo mount /dev/sda2 /mnt/windows-efi + +#. Mount the |CL| root partition and its EFI system partition. + + .. code-block:: bash + + sudo mount /dev/sda7 /mnt/clearlinux + + sudo mount /dev/sda5 /mnt/clearlinux/boot + +#. Copy Windows 10 OS bootloader, and other data needed to boot it, to the + |CL| EFI system partition. + + .. code-block:: bash + + sudo cp -r /mnt/windows-efi/EFI/Microsoft/ /mnt/clearlinux/boot/EFI/ + +#. Make :command:`systemd-boot` the default bootloader and add Windows 10 + OS Boot Manager. + + .. code-block:: bash + + sudo bootctl install --esp-path=/mnt/clearlinux/boot + +#. Add a timeout value to the :command:`systemd-boot`. This allows enough + time for you to select your preferred OS from the menu. + + .. code-block:: bash + + sudo clr-boot-manager set-timeout 20 --path=/mnt/clearlinux + +#. Umount all partitions. + + .. code-block:: bash + + sudo umount /mnt/windows-efi /mnt/clearlinux/boot /mnt/clearlinux + +#. Reboot + + .. code-block:: bash + + sudo reboot + +#. Remove the |CL| installer USB thumb drive. + +#. You should be presented with the :command:`systemd-boot` menu, as shown + below. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-win-04.png + :scale: 85% + :alt: systemd-boot menu + + Figure 5: systemd-boot menu + + +Alternative: Install Windows 10 OS After |CL| +********************************************* + +The following alternative guide shows how to install Windows 10 OS adjacent +to an existing |CL| installation. + +Prerequisites +************* + +* |CL| is already installed. +* There is unallocated disk space available. If the entire disk has been + allocated to |CL|, then shrink the root partition to make space for + Windows 10 OS. + +Install Windows 10 OS +===================== + +#. Start the Windows installer and follow the prompts. + +#. At the :guilabel:`What type of installation do you want?` screen, choose + :guilabel:`Custom: Install Windows only (advanced)`. See Figure 6. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-win-06.png + :scale: 85% + :alt: Windows - Choose installation type + + Figure 6: Windows - Choose installation type + +#. Select :guilabel:`Unallocated Space` and create a new partition of + the desired size. For this example, we will use the entire unallocated + space. See Figure 7. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-win-07.png + :scale: 85% + :alt: Windows - Create new partition + + Figure 7: Windows - Create new partition + + .. note:: + + Normally, Windows creates its own 100MB EFI partition if none exists. + In our case, where an EFI partition was created by |CL|, Windows will + use the previously-created partition. + +#. Select the newly-created partition and follow the remaining prompts to + complete the Windows installation. See Figure 8. + + .. rst-class:: dropshadow + + .. figure:: ../../_figures/multi-boot/dual-boot-win-08.png + :scale: 85% + :alt: Windows - Install on newly-created partition + + Figure 8: Windows - Install on newly-created partition + +#. Finish the Windows out-of-box-experience process. + +#. Restore `systemd-boot` - the bootloader |CL| uses - and add Windows to + its boot menu. + + At this point, you cannot boot |CL| because Windows became the default + bootloader after it was installed. Follow these steps to restore + `systemd-boot`. + + a. Boot up a |CL| live image from a USB thumb drive. + + #. Open a terminal window. + + #. Find the location of the EFI partition. In this example, it is + ``/dev/sda1``. + + .. code-block:: bash + + lsblk -po NAME,SIZE,TYPE,FSTYPE,PARTLABEL + + Example output: + + .. code-block:: console + :emphasize-lines: 5 + + clrlinux@clr-live~ $ lsblk -po NAME,SIZE,TYPE,FSTYPE,PARTLABEL + NAME SIZE TYPE FSTYPE PARTLABEL + /dev/loop0 2.3G loop squashfs + /dev/sda 100G disk + ├─/dev/sda1 150M part vfat CLR_BOOT + ├─/dev/sda2 250M part swap CLR_SWAP + ├─/dev/sda3 8G part ext4 CLR_ROOT + ├─/dev/sda4 16M part Microsoft reserved partition + ├─/dev/sda5 91.6G part ntfs Basic data partition + sdb 7G disk iso9660 + ├─/dev/sdb1 2.7G part iso9660 + └─/dev/sdb2 100M part vfat + + .. note:: + + To help narrow down a partition even more, you add the + ``PARTTYPE`` option to :command:`lsblk` and + cross-reference against the `partition type GUIDs wiki`_. + + #. Make a mount point for |CL|. + + .. code-block:: bash + + sudo mkdir /mnt/clearlinux + + #. Mount the root and EFI partitions. + + .. code-block:: bash + + sudo mount /dev/sda3 /mnt/clearlinux + sudo mount /dev/sda1 /mnt/clearlinux/boot + + #. Re-install systemd-boot to make it the default bootloader. + + .. code-block:: bash + + sudo bootctl install --esp-path=/mnt/clearlinux/boot + + #. Add a timeout (for example: 25 seconds) to systemd-boot so that it + will present the menu of bootable OSes and give you time to select + the one you want to boot. + + .. code-block:: bash + + sudo clr-boot-manager set-timeout 25 --path=/mnt/clearlinux + sudo clr-boot-manager update --path=/mnt/clearlinux + + #. Unmount the root and EFI partitions. + + .. code-block:: bash + + sudo umount /mnt/clearlinux/boot + sudo umount /mnt/clearlinux + +.. _partition type GUIDs wiki: https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs diff --git a/_sources/tutorials/multi-boot/multi-boot.rst.txt b/_sources/tutorials/multi-boot/multi-boot.rst.txt new file mode 100644 index 00000000..5f4d2ce7 --- /dev/null +++ b/_sources/tutorials/multi-boot/multi-boot.rst.txt @@ -0,0 +1,20 @@ +.. _multi-boot: + +Multi-boot |CL-ATTR| with other operating systems +################################################# + +| The |CL-PRJ| offers the ability to dual-boot |CL| with Windows\* 10 OS. +| More dual-boot and multi-boot use cases will be introduced in the future. + +.. warning:: + + Always back up critical data before installing an additional OS on a target system. Some data loss may occur. + +Detailed procedures +******************* + +.. toctree:: + :maxdepth: 1 + + dual-boot-win + dual-boot-linux diff --git a/_sources/tutorials/nvidia-cuda.rst.txt b/_sources/tutorials/nvidia-cuda.rst.txt new file mode 100644 index 00000000..a57a9e33 --- /dev/null +++ b/_sources/tutorials/nvidia-cuda.rst.txt @@ -0,0 +1,275 @@ +.. _nvidia-cuda: + +NVIDIA\* CUDA Toolkit +##################### + +NVIDIA is a manufacturer of graphics processing units (GPU), also known as +graphics cards. + +CUDA is a parallel computing platform and application programming interface +model created by NVIDIA. It allows software developers and software engineers +to use a CUDA-enabled graphics processing unit + +These instructions show how to install the CUDA Toolkit on |CL| after the +:ref:`proprietary NVIDIA drivers ` have been installed. + +.. note:: + + Software installed outside of :ref:`swupd ` is not updated with |CL| + updates and must be updated and maintained manually. + + + +.. contents:: :local: + :depth: 2 + + + +Prerequisites +************* + +* A |CL| system with a `CUDA-Enabled NVIDIA device `_ +* The :ref:`proprietary NVIDIA drivers ` have been installed. + + + + +Compatibility +************* + +Check compatibility of NVIDIA components +======================================== + +To install the appropriate NVIDIA CUDA Toolkit version, it is important to +understand the compute capability and compatible driver versions of your +NVIDIA hardware. + +Information about NVIDIA compute capability, driver, and toolkit compatibility +can be found at: https://developer.nvidia.com/cuda-gpus and +https://docs.nvidia.com/deploy/cuda-compatibility/ + + + +Check GCC compatibility +======================= + +.. note:: + + This is only required for the development or compilation of CUDA + applications. It is not required to run pre-built applications that have a + dependency on CUDA. + +From the NVIDIA documentation: + + The CUDA development environment relies on tight integration with the host + development environment, including the host compiler and C runtime + libraries, and is therefore only supported on distribution versions that + have been qualified for this CUDA Toolkit release. + +Refer to the `NVIDIA documentation on CUDA system requirements +`_ +for the latest kernel and compiler compatibility. + +For example, CUDA 10.2 on a system with the latest Linux kernel requires GCC8, +which is older than the default GCC version for |CL|. + +Install the compatible version of GCC, if required: + +#. Install the :ref:`bundle ` with the appropriate GCC version. + + .. code:: bash + + sudo swupd bundle-add c-extras-gcc8 + +#. Create the directory :file:`/usr/local/cuda/bin`: + + .. code:: bash + + sudo mkdir -p /usr/local/cuda/bin + + +#. Add symlinks to the older GCC version in the + :file:`/usr/local/cuda/bin` directory. This will cause the older version of + GCC to be used when :file:`/usr/local/cuda/bin` is in the $PATH environment + variable. + + .. code:: bash + + sudo ln -s /usr/bin/gcc-8 /usr/local/cuda/bin/gcc + sudo ln -s /usr/bin/g++-8 /usr/local/cuda/bin/g++ + + +Downloading and Installation +**************************** + + +Download the NVIDIA CUDA Toolkit +================================ + +#. Go to the `NVIDIA CUDA downloads website`_ to get the latest CUDA Toolkit. + If an older version of the CUDA Toolkit is required, go to the `CUDA + Toolkit Archive `_. + + Choose the following settings and click *Download*. + + - Operating System: *Linux* + - Architecture: *x86_64* + - Distribution: *any* + - Version: *any* + - Installer Type: *runfile(local)* + + + +#. Open a terminal and navigate to where the + :file:`cuda__linux.run` file was saved. In this + example, it was saved in the Downloads folder. + + .. code-block:: bash + + cd ~/Downloads/ + +#. Make the :file:`cuda__linux.run` file executable: + + .. code-block:: bash + + chmod +x cuda__linux.run + + + +Install the NVIDIA CUDA Toolkit +=============================== + +The NVIDIA CUDA installer will be directed to install files under +:file:`/opt/cuda` as much as possible to keep its contents isolated from the +rest of the |CL| files under :file:`/usr`. + +The CUDA installer automatically creates a symbolic link that allows the CUDA +Toolkit to be accessed from :file:`/usr/local/cuda` regardless of where it was +installed. + + +#. Configure the dynamic linker to look for and cache shared libraries under + :file:`/opt/cuda/lib64` where the NVIDIA installer will place libraries. + + .. code-block:: bash + + sudo mkdir -p /etc/ld.so.conf.d + echo "include /etc/ld.so.conf.d/*.conf" | sudo tee --append /etc/ld.so.conf + + + The CUDA installer will automatically create a file + :file:`/etc/ld.so.conf.d/cuda-.conf` + +#. Navigate into the directory where the NVIDIA installer was downloaded: + + .. code-block:: bash + + cd ~/Downloads/ + + +#. Run the installer with the advanced options below: + + .. code-block:: bash + + sudo ./cuda__linux.run \ + --toolkit \ + --installpath=/opt/cuda \ + --no-man-page \ + --override \ + --silent + +#. Validate the CUDA Toolkit was installed by checking the NVIDIA CUDA + compiler version: + + .. code-block:: bash + + /opt/cuda/bin/nvcc --version + + +The CUDA Toolkit is now installed and can be used to compile and run CUDA +applications. + + +Using the NVIDIA CUDA Toolkit +***************************** + +#. Verify that the NVIDIA device characters files /dev/nvidia* exist and have + the correct (0666) file permissions. The character devices should be + automatically created on system with the NVIDIA driver loaded through X + server, but will not be on systems that do not automatically load the + NVIDIA driver. + + .. code:: + + ls -l /dev/nvidia* + + +#. If your system does not have the NVIDIA character devices created + automatically, run the `script from NVIDIA documentation + `_ + with root privileges. + + + Alternatively a setuid utility, :command:`nvidia-modprobe`, can be compiled + and installed to automatically create the device character files on-demand. + + + .. code:: + + wget https://download.nvidia.com/XFree86/nvidia-modprobe/nvidia-modprobe-.tar.bz2 + tar -xvf nvidia-modprobe-.tar.bz2 + cd nvidia-modprobe-/ + make + sudo make install PREFIX=/usr/local/cuda/ + + +#. When the CUDA toolkit is needed, export PATH variables pointing to the CUDA + directories. This will temporarily add CUDA files to the PATH and use the + specified linked version of GCC for the terminal session. + + .. code:: bash + + export PATH=/usr/local/cuda/bin:$PATH + export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH + + +Source code for CUDA sample located at +:file:`/usr/local/cuda/NVIDIA_CUDA-_Samples`. See the `CUDA +documentation on compiling samples +`_ +to learn more. + + +Uninstalling +************ + +The NVIDIA drivers and associated software can be uninstalled and nouveau +driver restored by: + +#. Run the :command:`sudo /usr/local/cuda/bin/cuda-uninstaller`. + +#. Follow the prompts on the screen and reboot the system. + + +Debugging +********* + +* The NVIDIA CUDA installer places logs under + :file:`/tmp/cuda-installer.log`. + + +Additional resources +******************** +* `NVIDIA CUDA Toolkit Documentation `_ + +* `Why aren't the NVIDIA Linux drivers open source? `_ + +* `Where can I get support for NVIDIA Linux drivers? `_ + +* `NVIDIA Accelerated Linux Graphics Driver Installation Guides `_ + +.. _`nouveau project`: https://nouveau.freedesktop.org/wiki/ + +.. _`NVIDIA CUDA downloads website`: https://developer.nvidia.com/cuda-downloads + + diff --git a/_sources/tutorials/nvidia.rst.txt b/_sources/tutorials/nvidia.rst.txt new file mode 100644 index 00000000..f838ec3c --- /dev/null +++ b/_sources/tutorials/nvidia.rst.txt @@ -0,0 +1,531 @@ +.. _nvidia: + +NVIDIA\* Drivers +################ + +NVIDIA manufactures graphics processing units (GPU), also known as +graphics cards. + +NVIDIA devices on Linux\* have two popular device driver options: the +opensource drivers from the `nouveau project`_ or the proprietary drivers +published by NVIDIA. The nouveau drivers are built into the |CL-ATTR| +kernel and are loaded automatically at system boot if a compatible card +is detected. + +These instructions show how to use the proprietary NVIDIA drivers, which +require a manual installation. + +.. warning:: + + Software installed outside of :ref:`swupd ` is not updated + with |CL| updates and must be updated and maintained manually. + + For example, the file :file:`/usr/lib/libGL.so` conflicts with the file + provided by the mesa package in |CL| and the file NVIDIA provides. If a + |CL| update or repair overwrites these files, a reinstallation of the + NVIDIA driver might be required. + +.. contents:: :local: + :depth: 1 + +Prerequisites +************* + +* A |CL| system with a desktop installed +* An NVIDIA device installed + + +Known issues +============ + +Systems with multiple graphics devices, including integrated graphics (iGPU), +are known to be problematic. + +.. note:: NVIDIA Optimus\* + + Some systems come with a hybrid graphics configuration for a balanced power + and performance profile. This configuration is commonly found on + laptops. `NVIDIA Optimus technology + `_, is designed to + allow switching seamlessly between a NVIDIA device and another graphics + devices sharing the same display. + + Getting NVIDIA Optimus on Linux working well with both graphics devices + adds an additional level of complexity with platform specific steps and may + require additional software. Installation for systems with NVIDIA Optimus + with both graphics devices operating is not covered by the scope of this + documentation. As a simple workaround, some systems can disable one of the + graphics devices or NVIDIA Optimus in the system firmware. + +.. note:: + The :ref:`Long Term Support (LTS) kernel ` variant is + more likely to be compatible with proprietary NVIDIA drivers. + +**See the** `Troubleshooting`_ **section for more known issues and solutions.** + + +Installation +************ + +Configure workarounds +===================== + +Some workarounds are required for the NVIDIA proprietary drivers to be usable +and sustainable on |CL|. + + +#. Remove the kernel command-line parameter *intel_iommu=igfx_off* or disable + input–output memory management unit (IOMMU), also known as Intel® + Virtualization Technology (Intel® VT) for Directed I/O (Intel® VT-d), in your system EFI/BIOS. + See `this GitHub report + `_ and the NVIDIA + documentation on `DMA issues + `_ + for more information. + + The *intel_iommu-igfx_off* kernel parameter can be removed with the + commands below: + + .. code-block:: bash + + sudo mkdir -p /etc/kernel/cmdline-removal.d/ + echo "intel_iommu=igfx_off" | sudo tee /etc/kernel/cmdline-removal.d/intel-iommu.conf + +#. Create a custom systemd unit that overwrites the :file:`libGL` library + after every |CL| update with a pointer to the NVIDIA provided copy instead + of the version provided by |CL|. These libraries conflict causing the + NVIDIA driver to break when |CL| updates mesa. See the NVIDIA documentation + on `installed components + `_ + for more information. + + a. Create a systemd service unit to overwrite the |CL| provided + :file:`libGL.so.1` files with a symlink to the NVIDIA copies. + + .. code-block:: bash + + sudo tee /etc/systemd/system/fix-nvidia-libGL-trigger.service > /dev/null <<'EOF' + [Unit] + Description=Fixes libGL symlinks for the NVIDIA proprietary driver + BindsTo=update-triggers.target + + [Service] + Type=oneshot + ExecStart=/usr/bin/ln -sfv /opt/nvidia/lib/libGL.so.1 /usr/lib/libGL.so.1 + ExecStart=/usr/bin/ln -sfv /opt/nvidia/lib32/libGL.so.1 /usr/lib32/libGL.so.1 + EOF + + b. Reload the systemd manager configuration to pickup the new serivce. + + .. code-block:: bash + + sudo systemctl daemon-reload + + c. Add the service as a dependency to the |CL| updates trigger causing the + service to run after every update. + + .. code-block:: bash + + sudo systemctl add-wants update-triggers.target fix-nvidia-libGL-trigger.service + +Install DKMS +============ + +The :ref:`Dynamic Kernel Module System (DKMS) ` allows +the NVIDIA kernel modules to be automatically integrated when kernel updates +occur in |CL|. Install the appropriate DKMS bundle using the instructions +below: + +.. note:: + The Long Term Support (LTS) kernel variant is more likely to remain + compatible between updates with NVIDIA drivers. + +.. include:: /guides/kernel/kernel-modules-dkms.rst + :start-after: kernel-modules-dkms-install-begin: + :end-before: kernel-modules-dkms-install-end: + +Download the NVIDIA drivers +=========================== + +#. Identify the NVIDIA GPU model that is installed. + + .. code-block:: bash + + sudo lshw -C display + +#. Go to the `NVIDIA Driver Downloads website`_ . Search for and download the + appropriate driver based on the NVIDIA GPU model you have with *Linux + 64-bit* selected as the Operating System. + + .. code-block:: bash + + wget https://download.nvidia.com/XFree86/Linux-x86_64//NVIDIA-Linux-x86_64-.run + + If you already know the appropriate driver version for your device, you can + also obtain a download link directly from one of the links below: + + - https://www.nvidia.com/en-us/drivers/unix/ + - https://download.nvidia.com/XFree86/Linux-x86_64/ + + +Disable the nouveau driver +========================== + +The proprietary NVIDIA driver is incompatible with the nouveau driver and +must be disabled before installation can continue. + +#. Disable the nouveau driver by creating a blacklist file under + :file:`/etc/modprobe.d` and reboot. + + .. code-block:: bash + + sudo mkdir /etc/modprobe.d + + printf "blacklist nouveau \noptions nouveau modeset=0 \n" | sudo tee --append /etc/modprobe.d/disable-nouveau.conf + +#. Reboot the system and log back in. It is normal for the graphical + environment not to start without the NVIDIA driver loaded. + +Configure alternative software paths +==================================== + +The NVIDIA installer is directed to install files under +:file:`/opt/nvidia` as much as possible to keep its contents isolated from the +rest of the |CL| system files under :file:`/usr`. The dynamic linker and X +server must be configured to use the content under +:file:`/opt/nvidia`. + +#. Configure the dynamic linker to look for and to cache shared libraries under + :file:`/opt/nvidia/lib` and :file:`/opt/nvidia/lib32` in addition to the + default paths. + + .. code-block:: bash + + echo "include /etc/ld.so.conf.d/*.conf" | sudo tee --append /etc/ld.so.conf + + sudo mkdir /etc/ld.so.conf.d + printf "/opt/nvidia/lib \n/opt/nvidia/lib32 \n" | sudo tee --append /etc/ld.so.conf.d/nvidia.conf + +#. Reload the dynamic linker run-time bindings and library cache. + + .. code-block:: bash + + sudo ldconfig + +#. Create a Xorg configuration file to search for modules under + :file:`/opt/nvidia` in addition to the default path. + + .. code-block:: bash + + sudo mkdir -p /etc/X11/xorg.conf.d/ + + sudo tee /etc/X11/xorg.conf.d/nvidia-files-opt.conf > /dev/null <<'EOF' + Section "Files" + ModulePath "/usr/lib64/xorg/modules" + ModulePath "/opt/nvidia/lib64/xorg/modules" + EndSection + EOF + +Install the NVIDIA drivers +========================== + +#. A terminal not running on */dev/tty1* is useful to view uninterrupted + installation progress. Switch to a secondary virtual terminal by pushing + :command:`CTRL + ALT + F2` or remotely login over SSH. + +#. Navigate to the directory where the NVIDIA installer was downloaded. In + this example, it was saved in the :file:`Downloads` folder. + + .. code-block:: bash + + cd ~/Downloads/ + + +#. Run the installer with the advanced options below. + + .. code-block:: bash + + sudo sh NVIDIA-Linux-x86_64-.run \ + --utility-prefix=/opt/nvidia \ + --opengl-prefix=/opt/nvidia \ + --compat32-prefix=/opt/nvidia \ + --compat32-libdir=lib32 \ + --x-prefix=/opt/nvidia \ + --x-module-path=/opt/nvidia/lib64/xorg/modules \ + --x-library-path=/opt/nvidia/lib64 \ + --x-sysconfig-path=/etc/X11/xorg.conf.d \ + --documentation-prefix=/opt/nvidia \ + --application-profile-path=/etc/nvidia/nvidia-application-profiles-rc.d \ + --no-precompiled-interface \ + --no-nvidia-modprobe \ + --no-distro-scripts \ + --force-libglx-indirect \ + --glvnd-egl-config-path=/etc/glvnd/egl_vendor.d \ + --egl-external-platform-config-path=/etc/egl/egl_external_platform.d \ + --dkms \ + --silent + +#. The graphical interface may automatically start after the NVIDIA driver + is loaded. Return to the working terminal and log back in if necessary. + +#. Confirm that the NVIDIA kernel modules are loaded. + + .. code-block:: bash + + lsmod | grep ^nvidia + +#. Optional: Create a link for the nvidia-settings desktop entry to + :file:`~/.local/share/applications` so that it appears in the launcher for easy access. + + .. code-block:: bash + + ln -sv /opt/nvidia/share/applications/nvidia-settings.desktop $HOME/.local/share/applications + + +Updating +******** + +The proprietary NVIDIA drivers are installed manually outside of +:ref:`swupd ` and must be updated manually when needed. + +Updating the NVIDIA drivers follows the same steps as initial installation, +however the desktop environment must first be stopped so that the drivers are +not in use. + +#. Follow the steps in the `Download the NVIDIA drivers`_ section + to get the latest NVIDIA drivers. + +#. Temporarily set the default boot target to the *multi-user*, which is + a non-graphical runtime. + + .. code-block:: bash + + sudo systemctl set-default multi-user.target + +#. Reboot the system and log back in. It is normal for the graphical + environment not to start. + +#. Follow the steps in the `Install the NVIDIA Drivers`_ section to update + the NVIDIA drivers. This installation will overwrite the previous NVIDIA + drivers and files. + +#. Set the default boot target back to the *graphical* target. + + .. code-block:: bash + + sudo systemctl set-default graphical.target + +#. Reboot the system and log back in. + +#. Trigger a :command:`flatpak update` to download the runtime corresponding + with the new NVIDIA drivers for the flatpak apps that require it. + + .. important:: + + Some flatpak applications won't start after updating the NVIDIA drivers + until the flatpak runtime is updated with the corresponding driver + version. + + .. code-block:: bash + + flatpak update + + + + + +Uninstallation +************** + +The NVIDIA drivers and associated software can be uninstalled and nouveau +driver restored with the instructions in this section. + +#. Remove the files created for workarounds. + + .. code-block:: bash + + sudo rm /etc/kernel/cmdline-removal.d/intel-iommu.conf + sudo rm /etc/systemd/system/fix-nvidia-libGL-trigger.service + sudo rm /etc/systemd/system/update-triggers.target.wants/fix-nvidia-libGL-trigger.service + sudo systemctl daemon-reload + + +#. Remove the :file:`modprobe.d` file that prevents nouveau from loading. + + .. code-block:: bash + + sudo rm /etc/modprobe.d/disable-nouveau.conf + +#. Remove the :file:`nvidia.conf` file so that dynamic linker does not + look for cached libraries under :file:`/opt/nvidia/lib` and :file:`/opt/nvidia/lib32`. + + .. code-block:: bash + + sudo rm /etc/ld.so.conf.d/nvidia.conf + sudo ldconfig + + Optionally, restore :file:`ld.so.conf` to default if no other configuration files under :file:`/etc/ld.so.conf.d` + needs to be included. + + .. code-block:: bash + + sudo sed -i '/^include \/etc\/ld\.so\.conf\.d\/\*\.conf$/d' /etc/ld.so.conf + + +#. Remove the :file:`xorg.conf.d` file that adds a search path for X modules. + + .. code:: bash + + sudo rm /etc/X11/xorg.conf.d/nvidia-files-opt.conf + +#. Remove the nvidia-settings desktop entry file if it was linked to + :file:`~/.local/share/applications`. + + .. code:: bash + + unlink -v $HOME/.local/share/applications/nvidia-settings.desktop + + +#. Run the :command:`nvidia-uninstall` command. + + .. code:: bash + + sudo /opt/nvidia/bin/nvidia-uninstall + +#. Follow the prompts on the screen and reboot the system. + + +Troubleshooting +*************** + +* The NVIDIA driver places installer and uninstaller logs under + :file:`/var/log/nvidia-install` and :file:`/var/log/nvidia-uninstall`. + +* :file:`NVIDIA-Linux-x86_64-.run --advanced-options` shows many + parameters to control installation behavior. + +* :file:`NVIDIA-Linux-x86_64-.run --extract-only` extracts + installation files into a directory named + :file:`NVIDIA-Linux-x86_64-`. + +* The X server logs under :file:`/var/log/X*` contain useful + information about display and driver loading. Check all the files and + timestamps when troubleshooting. + +* The DKMS build logs under :file:`/var/lib/dkms/nvidia*` contain information + about kernel module builds which can be useful if the NVIDIA driver breaks + between kernel upgrades. + + +No display or blank screen +========================== + +Check to see if the display has come up on another graphics device, including +the integrated graphics device. + +You might get a black screen or the login screen might not come up after +installing the NVIDIA drivers until an Xorg configuration has been defined for +your monitors. + + +"Oh no! Something has gone wrong" GNOME\* crash +=============================================== + + +.. figure:: /_figures/nvidia/nvidia-gnome-crash.png + :alt: NVIDIA driver GNOME crash on Clear Linux OS + :align: center + + NVIDIA driver GNOME crash dialogue on Clear Linux OS. + +There have been reports of GNOME crashing with an "Oh no! Something has gone +wrong" error message with NVIDIA drivers installed while other graphics +devices are enabled. + +Try disabling other graphics devices, including integrated graphics, in your +system's EFI/BIOS. + + +Slow boot times +=============== + + +There have been reports of slow boot times with NVIDIA drivers installed. +Normally, when GDM detects NVIDIA proprietary drivers, it will disable Wayland +and enable X11. Should GDM fail to disbale Wayland, it may results in slow boot +times, according to `this GitHub reprot +`_. + +To manually disable Wayland: + +.. code-block:: bash + + sudo tee /etc/gdm/custom.conf > /dev/null <<'EOF' + [daemon] + WaylandEnable=false + EOF + + +Brightness control +================== + +If you can't control the screen brightness with the NVIDIA driver installed, +try one of the solutions below: + +- Add a kernel parameter *acpi_osi=* which disables the ACPI Operating System + Identification function. Some system firmware may manipulate brightness + control keys based on the reported operating system. Disabling the + identification mechanism can cause the system firmware to expose brightness + controls that are recognizable in Linux. + + .. code:: bash + + sudo mkdir -p /etc/kernel/cmdline.d + echo "acpi_osi=" | sudo tee /etc/kernel/cmdline.d/acpi-backlight.conf + sudo clr-boot-manager update + + + +- Add a kernel parameter for the nvidia driver: + *NVreg_EnableBacklightHandler=1*. This handler overrides the ACPI-based one + provided by the video.ko kernel module. This option is available with NVIDIA + driver version 387.22 and above. + + .. code:: bash + + sudo mkdir -p /etc/kernel/cmdline.d + echo "nvidia.NVreg_EnableBacklightHandler=1" | sudo tee /etc/kernel/cmdline.d/nvidia-backlight.conf + sudo clr-boot-manager update + + +- Add the *EnableBrightnessControl=1* options to the *Device* + section of your xorg config. Below is an example: + + .. code:: bash + + sudo mkdir -p /etc/X11/xorg.conf.d/ + + sudo tee /etc/X11/xorg.conf.d/nvidia-brightness.conf > /dev/null <<'EOF' + Section "Device" + Identifier "Device0" + Driver "nvidia" + Option "RegistryDwords" "EnableBrightnessControl=1" + EndSection + EOF + +Additional resources +==================== + +* `Why aren't the NVIDIA Linux drivers open source? `_ + +* `Where can I get support for NVIDIA Linux drivers? `_ + +* `NVIDIA Accelerated Linux Graphics Driver Installation Guides `_ + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _`nouveau project`: https://nouveau.freedesktop.org/wiki/ + +.. _`NVIDIA Driver Downloads website`: https://www.nvidia.com/download/index.aspx + + diff --git a/_sources/tutorials/openfaas.rst.txt b/_sources/tutorials/openfaas.rst.txt new file mode 100644 index 00000000..831ef643 --- /dev/null +++ b/_sources/tutorials/openfaas.rst.txt @@ -0,0 +1,388 @@ +.. _openfaas: + +OpenFaaS\* +########## + +This tutorial shows how to set up OpenFaaS running on top of a Kubernetes\* +cluster on |CL|, obtain |CL| based OpenFaaS templates, and develop an example +function. + +.. contents:: :local: + :depth: 1 + +Background +********** + +Functions as a Service (FaaS) is a framework for building serverless functions +that are ephemeral, automatically scalable, and focused pieces of code running +within containers to allow developers to focus on application code rather than +infrastructure nuances. + +Many cloud service providers have ready-to-use FaaS offerings which offer a +high degree of convenience for developers and granular billing based on +per-second usage. + +If you want an on-premise or self-hosted serverless capability to avoid vendor +lock-in or simply want more development, OpenFaaS is currently the most +popular solution in the space based on the number of Github stars on the +project. + +Prerequisites +************* + +For simplicity, this tutorial assumes you have a Kubernetes single node +cluster with only master node running |CL|. + +- For detailed instructions on how to install |CL|, see the :ref:`getting + started ` section. + +- For a detailed guide on how to set up Kubernetes, see the documentation on + :ref:`Kubernetes `. + +.. note:: + + Please note that in this example the `master node was tainted + `_ + to be able to be scheduled, which means containers are able to be deployed + to the master node. + + .. code:: bash + + kubectl taint nodes --all node-role.kubernetes.io/master- + + +Deploy OpenFaaS +*************** + +#. Install the official command line tool for using OpenFaas, *faas-cli*, by + installing the *faas-cli* bundle. + + .. code:: bash + + sudo swupd bundle-add faas-cli + +#. Download the `faas-netes `_, the + OpenFaaS provider templates that enable Kubernetes for OpenFaaS. + + .. code:: bash + + git clone https://github.com/openfaas/faas-netes.git + +#. Set variables for the OpenFaaS admin user and password. + + .. warning:: + + For simplicity, this tutorial uses a basic authentication with an + unecrypted username and password. For production environments, see the + OpenFaaS documentation on `Deploying OpenFaas in Production + `_. + + .. code:: bash + + export FAAS_USER=admin + export FAAS_PASSWD=clearlinux + +#. Deploy the OpenFaaS stack on Kubernetes using :command:`kubectl`. + + .. code:: bash + + kubectl apply -f faas-netes/namespaces.yml + + .. code:: bash + + kubectl -n openfaas create secret generic basic-auth \ + --from-literal=basic-auth-user=$FAAS_USER \ + --from-literal=basic-auth-password="$FAAS_PASSWD" + + .. code:: bash + + kubectl apply -f faas-netes/yaml/ + + Wait for the OpenFaaS pods and services to get ready. This involves + downloading container images from the Internet and may take some time + depending on your Internet connection. You can enter the commands below to + have the terminal wait until services are ready to use. + + .. code:: bash + + kubectl wait --for=condition=available --timeout=600s deployment/gateway -n openfaas + kubectl wait --for=condition=available --timeout=600s deployment/faas-idler -n openfaas + +#. Login to the OpenFaaS instance. *31112* is the default port. + + - You can login over the command-line: + + .. code:: bash + + export OPENFAAS_URL=http://127.0.0.1:31112 + echo -n $FAAS_PASSWD | faas-cli login --password-stdin + + + - You can also login to the OpenFaaS web interface by navigating to + **http://${master_node_IP}:31112** + + + .. figure:: /_figures/openfaas/openfaas-login.png + :scale: 100% + :align: center + :alt: OpenFaaS web interface login page + + Figure 1: OpenFaaS web interface login page + +OpenFaaS templates +****************** + +OpenFaaS templates, though not necessary, abstract configurations for +running functions in common programming languages. Templates allows developers +to better focus on writing the code for their function. + +OpenFaaS has dozens of `templates in the official store +`_. There are also |CL|-based templates +available for download. + +#. You can list all the official templates in the store using + :command:`faas-cli`. + + .. code:: bash + + faas-cli template store list + + NAME SOURCE DESCRIPTION + csharp openfaas Classic C# template + dockerfile openfaas Classic Dockerfile template + go openfaas Classic Golang template + java8 openfaas Classic Java 8 template + node openfaas Classic NodeJS 8 template + php7 openfaas Classic PHP 7 template + python openfaas Classic Python 2.7 template + python3 openfaas Classic Python 3.6 template + ... + +#. Create and enter a workspace. + + .. code-block:: bash + + mkdir ~/faas-example + cd ~/faas-example + +#. Download the |CL|-based OpenFaaS templates which are stored in the + https://github.com/clearlinux/dockerfiles repository and copy them into your + working directory. + + .. code:: bash + + git clone https://github.com/clearlinux/dockerfiles.git + cp -r dockerfiles/FaaS/OpenFaaS/template/ . + +#. After the |CL| based templates have been retrieved, they will show up in + the same repository and available to use locally. + + .. code:: bash + + faas-cli new --list + + Languages available as templates: + - dockerfile-clearlinux + - python3-clearlinux + +OpenFaaS is ready to use at this point. See the `OpenFaaS documentation +`_ to learn more about deploying functions. + + +Example: Develop a function +*************************** + +In this example, we'll imagine a FaaS solution where: a user provides a URL to +a pictures, which invokes a function to do image classification and outputs +the result. + +We will use the `OpenVINO™ toolkit - Deep Learning Deployment Toolkit (DLDT) +`_ to do the image +inference. As inference development is not the focus of this example, we will +just use the built-in sample "`classification_sample_async +`_" +for this function. + + +We'll use the *python3-clearlinux* template as a base and customize it by: + +* Adding additional |CL| bundles (*bundles.txt*) +* Adding additional required python packages (*requirements.txt*) +* Adding a script to download and convert DLDT models (*helper_script.sh*) +* Finally, we'll develop the python function to be run (*handler.py*) + +More ways to customize the |CL| based OpenFaaS templates can be found in the +`README on GitHub +`_. + +#. Enter the previously created working directory. + + .. code-block:: bash + + cd ~/faas-example + +#. Create a new function skeleton + + .. code-block:: bash + + faas-cli new --lang python3-clearlinux classification-sample --prefix="" + + This will create the directory structure below: + + .. code-block:: console + + tree . + ├── classification-sample + │ ├── bundles.txt + │ ├── handler.py + │ ├── helper_script.sh + │ ├── __init__.py + │ └── requirements.txt + ├── classification-sample.yml + +#. Add the required |CL| bundles to the :file:`bundles.txt` file. + + .. code-block:: bash + + echo "computer-vision-openvino" >> classification-sample/bundles.txt + +#. Add the required python packages to the :file:`requirements.txt` file. + + .. code-block:: bash + + echo "glob3" >> classification-sample/requirements.txt + echo "urllib3" >> classification-sample/requirements.txt + echo "networkx==2.3" >> classification-sample/requirements.txt + + +#. OpenCV has a `model downloader and other automation tools + `_ + to help downloading models and converting them into different formats. + Customize the OpenFaaS template to use the model-downloader in the + :file:`helper_script.sh` file. The :file:`helper_script.sh` file script + gets executed during the build process. + + .. code-block:: bash + :caption: helper_script.sh + :name: helper_script.sh + + cat classification-sample/helper_script.sh + #!/bin/bash + + # Download and convert models + export MODEL_DIR="/models" + export MO_PATH="/usr/share/openvino/model-optimizer/mo.py" + export MODEL_NAME="resnet-50-int8-tf-0001" + + # Download and convert models + model-downloader --name $MODEL_NAME -o $MODEL_DIR + model-converter --name $MODEL_NAME -d $MODEL_DIR -o $MODEL_DIR --mo $MO_PATH + + +#. With the requirements added to the template. Write a python in the + :file:`handler.py` file. This function will parse the input picture URL, + find the model path, and call “classification_sample_async” to do image + classification. + + .. code-block:: python + :caption: handler.py + :name: handler.py + + #!/usr/bin/python3 + + import os + import glob + import urllib.request + from urllib.parse import urlparse + from os.path import splitext + + ALLOWED_IMAGE_TYPE = [".bmp", ".BMP"] + + def get_ext(url): + parsed = urlparse(url) + _, ext = splitext(parsed.path) + return ext + + def get_image_from_url(url): + """get image and save to local path""" + + ext = get_ext(url) + + local_file_path = "/tmp/image" + ext + urllib.request.urlretrieve(url, local_file_path) + return local_file_path + + def find_model_path(): + """ return model xml path """ + + model_dir = os.getenv('MODEL_DIR', '/models') + model_name = os.getenv('MODEL_NAME', 'resnet-50-int8-tf-0001') + ".xml" + precision = os.getenv('MODEL_PRECISION', 'FP32') + + pattern = model_dir + '/**/' + precision + '/' + model_name + paths = glob.glob(pattern, recursive=True) + + if not len(paths): + print("No " + model_name + " found") + return None + + return paths[0] + + def do_classification(image, model_path): + """ Use dldt sample classification_sample_async """ + + cmd = "classification_sample_async -i " + image + " -m " + model_path + return os.system(cmd) + + def handle(req): + """handle a request to the function + Args: + req (str): request body + """ + if not len(req): + print("Request body is missing.") + return + + model_path = find_model_path() + if model_path is None: + return + + if get_ext(req) not in ALLOWED_IMAGE_TYPE: + print("Only " + ALLOWED_IMAGE_TYPE + " images are allowed.") + return + + file_path = get_image_from_url(req) + do_classification(file_path, model_path) + + + + + +#. Build and deploy the function to the OpenFaaS instance. + + .. code-block:: bash + + faas-cli build --build-arg http_proxy=$http_proxy --build-arg https_proxy=$https_proxy -f classification-sample.yml + faas-cli deploy --env=http_proxy=$http_proxy --env=https_proxy=$https_proxy -f classification-sample.yml + +#. Finally, test the function by going to the OpenFaaS web interface at + **http://${master_node_IP}:31112** and Invoking the *classification-sample* + function with a URL to any BMP image. The result should show the what the + image has been identified as and probability. + + + .. figure:: /_figures/openfaas/openfaas-invoke-function.png + :scale: 100% + :align: center + :alt: OpenFaaS web interface invoke function + + Figure 2: OpenFaaS web interface invoke function + + .. figure:: /_figures/openfaas/openfaas-function-output.png + :scale: 100% + :align: center + :alt: OpenFaaS web interface function output + + Figure 3: OpenFaaS web interface invoke function + +*Intel, OpenVINO, and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* \ No newline at end of file diff --git a/_sources/tutorials/php.rst.txt b/_sources/tutorials/php.rst.txt new file mode 100644 index 00000000..becbe84f --- /dev/null +++ b/_sources/tutorials/php.rst.txt @@ -0,0 +1,253 @@ +.. _php: + +PHP and PHP-FPM +############### + +This tutorial describes how to configure and use :abbr:`PHP (PHP: Hypertext Preprocessor)` +and :abbr:`PHP-FPM (PHP FastCGI Process Manager)` on |CL-ATTR|. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +The `PHP language`_ is an Open Source general-purpose scripting language that is popular +with web-developers who leverage its ability to create dynamically generated +web pages. `PHP-FPM`_ is a PHP FastCGI implementation that controls process +management, workers, and logging for PHP. The two applications work together, +but each one has its own configuration. + +This tutorial specifically addresses PHP and PHP-FPM, however, it provides a +general guide for working with applications in the |CL| :ref:`stateless` +environment. + + +Background +********** + +By default, PHP looks for configuration settings in the :file:`php.ini` file, +which resides in the :file:`usr/share/defaults/php/` path. Because |CL| is a +:ref:`stateless` operating system, you must create an optional configuration +file to override the default values. Every time :command:`swupd` updates the +system, it overwrites changes to the :file:`/usr/share/defaults` file +structure. To save your configuration options through updates, you must create +a PHP configuration file in a location that will not be overwritten. The +recommended location is within the :file:`/etc` file structure, which is why this +tutorial creates a :file:`/etc/php.d` directory. + +The PHP-FPM configuration file is separate from the :file:`php.ini` file used +by PHP, however it has a similar default path restriction. |CL| installs the +default :file:`php-fpm.conf` file in :file:`/usr/share/defaults/php`. This +file with its default values is overwritten during each software update. +However, PHP-FPM requires that the configuration file exist in that location, +and, by design, does not read configuration options from a different path. +This tutorial describes a solution to changing PHP-FPM configuration options +in |CL|, by manually overriding the php-fpm.service unit in systemd to pass +an explicit location to a custom :file:`php-fpm.conf` file. + + +Prerequisites +************* + +* :ref:`Install ` |CL| on your host system. +* Use :command:`swupd` to install the :file:`php-basic` bundle: + + .. code-block:: bash + + sudo swupd bundle-add php-basic + +.. note:: + + PHP does not require a web server for operation. If you need a web + server, refer to :ref:`lamp-server-install` for instructions on setting + up a :abbr:`LAMP (Linux, Apache\*, MySQL\*, PHP)` server, or use + :command:`swupd` to install :file:`nginx` or similar. + + +Configure PHP +************* + +.. important:: + + This section does **not** describe configuration for the PHP-FPM service, which is + described later in this guide. + +This section creates a :file:`/etc/php.d` directory for all PHP configuration +files to work around the default path restriction. + +#. Create a :file:`php.ini` file: + + .. code-block:: bash + + sudo mkdir -p /etc/php.d + sudo touch /etc/php.d/my-php.ini + + This file can be edited with any of your specific configuration + requirements, and will not be overwritten when :command:`swupd` performs + an update. The `PHP configuration file`_ documentation contains details + about what you can set in this file. + +#. Verify the location of the PHP configuration files: + + .. code-block:: bash + + php --ini + + You should see output like this: + + .. code-block:: console + + Configuration File (php.ini) Path: /usr/share/defaults/php/ + Loaded Configuration File: /usr/share/defaults/php/php.ini + Scan for additional .ini files in: /etc/php.d + Additional .ini files parsed: + + + This output indicates that PHP will read the :file:`php.ini` file from + :file:`/usr/share/defaults/php` and will then load any further + configuration from :file:`.ini` files in :file:`/etc/php.d/`. The + :file:`my-php.ini` file in :file:`/etc/php.d` contains your configuration + details, and allows the defaults to be read from + :file:`/usr/share/defaults/php/`. Note that the :file:`my-php.ini` file + has not been parsed, because the file has no content at this + point and is disregarded. + +Install PHP extensions +********************** + +PHP extensions are compiled libraries designed to enable specific functions in +your PHP code. |CL| provides PHP extensions in the :file:`php-extras` bundle. + +#. Install the bundle with swupd: + + .. code-block:: bash + + sudo swupd bundle-add php-extras + +#. Find the list of extensions included in the `php-extras`_ bundle on the + |CL| `Store`_. + + +Enable PHP extensions +********************* + +To enable an installed extension, you must add it to the :file:`php.ini` file +for the composer. + +#. Create the :file:`my-php.ini` file using the directive to load the + php-imagick extension: + + .. code-block:: bash + + sudo echo "extension=imagick.so" >> /etc/php.d/my-php.ini + +#. Restart the php-fpm service for PHP to pick up the modification to the + :file:`/etc/php.d/my-php.ini` file: + + .. code-block:: bash + + sudo systemctl restart php-fpm + +#. Verify that the imagick extension has been loaded by searching through the + runtime list of loaded PHP Modules: + + .. code-block:: bash + + php -m | grep imagick + + +.. note:: + + To enable an extension, you must install it, add it to the + :file:`my-php.ini` file, and restart the :file:`php-fpm` service. However, + some extensions may have configuration options, which will be documented by + the extension maintainer. Add the options you need to the + :file:`/etc/php.d/my-php.ini` file as described in the extension's + documentation. Be sure to restart :file:`php-fpm` after changing the file. + +Configure PHP-FPM +***************** + +The PHP-FPM configuration file is separate from the :file:`php.ini` file used +by PHP, however, it has a similar default path restriction. Follow the steps +below to configure PHP-FPM. + + +#. Copy the :file:`/usr/share/defaults/php/php-fpm.conf` file to the + :file:`/etc/php.d` file: + + .. code-block:: bash + + sudo cp /usr/share/defaults/php/php-fpm.conf /etc/php.d/php-fpm.conf + +#. Make changes to the :file:`php-fpm.conf` file as needed. The + `FPM documentation`_ has details on the configuration options available + to PHP-FPM. + +#. Edit the systemd service unit file: + + .. code-block:: bash + + sudo systemctl edit --full php-fpm.service + + This opens the :file:`php-fpm.service` file for systemd in your editor. + +#. Change the :command:`ExecStart` configuration to add the + :command:`--fpm-config` option to point to the custom location: + + .. code-block:: console + + [Unit] + Description=The PHP FastCGI Process Manager + After=syslog.target network.target + + [Service] + Type=notify + PIDFile=/run/php-fpm.pid + ExecStart=/usr/sbin/php-fpm --nodaemonize --fpm-config /etc/php.d/php-fpm.conf + ExecReload=/bin/kill -USR2 $MAINPID + PrivateTmp=true + + [Install] + WantedBy=multi-user.target + +#. Restart the service: + + .. code-block:: bash + + sudo systemctl restart php-fpm.service + +#. Verify that the new path has been picked up: + + .. code-block:: bash + + sudo systemctl status php-fpm.service + + You should see the new path in the output: + + .. code-block:: console + + ● php-fpm.service - The PHP FastCGI Process Manager + Loaded: loaded (/etc/systemd/system/php-fpm.service; enabled; vendor preset: disabled) + Active: active (running) since Thu 2019-10-17 13:19:34 PDT; 8min ago + Main PID: 14452 (php-fpm) + Status: "Processes active: 0, idle: 0, Requests: 0, slow: 0, Traffic: 0req/sec" + Tasks: 1 + Memory: 11.1M + CGroup: /system.slice/php-fpm.service + └─14452 php-fpm: master process (/etc/php.d/php-fpm.conf) + + +.. _PHP language: https://www.php.net/ + +.. _PHP-FPM: https://php-fpm.org/ + +.. _php-extras: https://clearlinux.org/software/bundle/php-extras + +.. _Store: https://clearlinux.org/software/ + +.. _PHP configuration file: https://www.php.net/manual/en/configuration.file.php + +.. _FPM documentation: https://www.php.net/manual/en/install.fpm.configuration.php diff --git a/_sources/tutorials/redis.rst.txt b/_sources/tutorials/redis.rst.txt new file mode 100644 index 00000000..3c74640b --- /dev/null +++ b/_sources/tutorials/redis.rst.txt @@ -0,0 +1,223 @@ +.. _redis: + +Redis\* +####### + +In this tutorial, you'll install :abbr:`Redis (REmote DIctionary Server)` +and launch a `Redis-server` on |CL-ATTR|. We invite you to pull our +`Clear Linux Redis instance`_ on Docker Hub\* for application or +infrastructure development. + +.. contents:: + :local: + :depth: 1 + +Description +*********** + +Redis is an in-memory key:value store designed for quick lookups, accessible +over a network. While the `redis data structure store`_ can serve +as a NoSQL database for a web application, it's also easy to integrate into an +existing stack. For example, you could use the Redis caching layer for +real-time responses on a leaderboard in a gaming app. Redis offers many client +libraries with language-specific bindings for Python\*, Perl\*, Ruby, and more. + +Install the Redis bundle +************************ + +#. Log in as a user with administrative privilege. + +#. Open a terminal. + +#. Update your |CL| to the latest version. + + .. code-block:: bash + + sudo swupd update + +#. Install the `redis-native` bundle. + + .. code-block:: bash + + sudo swupd bundle-add redis-native + +Start the Redis-server +********************** + +A :command:`systemd` service unit is available to control the Redis-server. +By default, Redis runs on port 6379. + +#. Start the service and set it to start automatically on boot. + + .. code-block:: bash + + sudo systemctl enable --now redis + +#. Confirm the service is running. + + .. code-block:: bash + + sudo systemctl status redis + +#. Verify that the Redis-server sends a reply. + + .. code-block:: bash + + redis-cli ping + + Expected output: + + .. code-block:: console + + PONG + +.. note:: + + If you wish to customize settings for Redis, copy the + default :file:`/usr/share/defaults/etc/redis.conf` file into the + /etc/ directory, make changes as needed, and restart the service. + + .. code-block:: bash + + sudo cp -v /usr/share/defaults/etc/redis.conf /etc/ + +The Redis-server is now ready to use on |CL|. Try some of the examples shown +below. + +Example 1: Use the redis-cli and commands +***************************************** + +One advantage of Redis over other NoSQL databases is that developers can +easily access data structures like lists, sets, sorted sets, strings, and +hashes using collection operations commands similar to those found in many +programming languages. These exercises are inspired by `try redis io`_. + +After your Redis-server is running, try some basic commands. + +#. Start `redis-cli`. It provides syntax suggestions as you type. + + .. code-block:: bash + + redis-cli + +#. :command:`SET` a key to hold a string value. In the set, create connections + and increment. + + .. code-block:: none + + SET server:name "clearlinux" + + .. code-block:: none + + MGET server:name + + .. note:: + + If the key does not exist or hold a key value, `nil` is returned. + + .. code-block:: none + + SET connections 100 + + .. code-block:: none + + INCR connections + + .. code-block:: none + + INCR connections + + .. code-block:: none + + DEL connections + +#. Create a `friends` list and insert new values at the end of the list. + + .. code-block:: none + + RPUSH friends "Deb" + + .. code-block:: none + + RPUSH friends "David" + + .. code-block:: none + + RPUSH friends "Mary" + +#. Modify the `friends` list, using a common slice method with a 0-index. + + .. code-block:: none + + LRANGE friends 0 1 + + .. code-block:: none + + LLEN friends + + .. code-block:: none + + LPOP friends + + .. code-block:: none + + RPOP friends + + .. code-block:: none + + LLEN friends + +#. Consider using a hash, which maps string fields and string values, and + offers multiple lookup methods. + + Enter many user key:values with `HMSET`. Then try `HGET` and `HGETALL`. + + .. code-block:: none + + HMSET user:1000 name "Robert Noyce" password "SuperEngi9eer" email "robert.noyce@intel.com" + + .. code-block:: none + + HGET user:1000 name + + .. code-block:: none + + HGET user:1000 email + + .. code-block:: none + + HGETALL user:1000 + + +Example 2: Run the |CL| Redis Docker\* image +******************************************** + +We also provide a `Clear Linux Redis instance`_, which is +updated continuously and maintained by |CL| development. + +.. code-block:: bash + + sudo swupd bundle-add containers-basic + +.. code-block:: bash + + sudo systemctl start docker + +.. code-block:: bash + + sudo -E docker pull clearlinux/redis + +Next Steps +********** + +* Follow the `redis quickstart tutorial`_ to expand potential uses. + +* Learn how to use :ref:`docker`. + +.. _try redis io: http://try.redis.io/ + +.. _Clear Linux Redis instance: https://hub.docker.com/r/clearlinux/redis + +.. _redis data structure store: https://redis.io/ + +.. _redis quickstart tutorial: https://redis.io/topics/quickstart diff --git a/_sources/tutorials/smb-desktop.rst.txt b/_sources/tutorials/smb-desktop.rst.txt new file mode 100644 index 00000000..1858c96b --- /dev/null +++ b/_sources/tutorials/smb-desktop.rst.txt @@ -0,0 +1,84 @@ +.. _smb-desktop: + +Samba\* as a client +################### + +This tutorial explains how to access an existing shared drive on +Windows\* via Server Message Block (SMB) / Common Internet File System (CIFS) +from the |CL| GNOME\* desktop. A CIFS filesystem is generally used to access +shared storage locations, or share files. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* You have already `set up a shared location on Windows`_ + +Connect to Windows shared location with Nautilus +************************************************ + +#. From the desktop, select :guilabel:`Files` from the application menu. + + .. note:: + + GNOME Files is also known as `Nautilus`. + +#. In :guilabel:`Files`, select :guilabel:`Other Locations`. + + .. figure:: /_figures/samba/smb-desktop-1.png + :scale: 100% + :alt: Files, Other Locations + + Figure 1: Files, Other Locations + +#. In the lower taskbar, beside :guilabel:`Connect to Server`, + enter the file-sharing address using the Windows sharing schema: + + .. code-block:: bash + + smb://servername/Share + + .. figure:: /_figures/samba/smb-desktop-2.png + :scale: 100% + :alt: Connect to Server + + Figure 2: Connect to Server + +#. Optional: If there are issues with DNS, you can use an IP address in + place of the `servername` above. You must still specify the share. + +#. Optional: On the Windows machine, in a CLI, retrieve the IP address by + entering the command: + + .. code-block:: bash + + ifconfig + + .. note:: + + If using the IP address, assure that it is accessible and secure. + +#. Select :guilabel:`Connect`. + + The server requests authentication, as shown in Figure 3. + + .. figure:: /_figures/samba/smb-desktop-3.png + :scale: 100% + :alt: Authentication + + Figure 3: Authentication + +#. Log in with the same Windows system credentials for which you granted + access to the share. + +#. Select the appropriate checkbox to save your credentials. Consider + carefully the security risks as a result of your selection. + + +.. _set up a shared location on Windows: https://www.howtogeek.com/176471/how-to-share-files-between-windows-and-linux/ + + + diff --git a/_sources/tutorials/smb-server.rst.txt b/_sources/tutorials/smb-server.rst.txt new file mode 100644 index 00000000..81306b63 --- /dev/null +++ b/_sources/tutorials/smb-server.rst.txt @@ -0,0 +1,150 @@ +.. _smb-server: + +Samba Server +############ + +This tutorial describes how to enable simple file sharing on a system +running |CL-ATTR| and how to access the share from clients on other operating +systems. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +* |CL| is installed on your host system. + + For detailed instructions, follow one of these guides: + + * *Desktop* version: :ref:`bare-metal-install-desktop` + * *Server* version: :ref:`bare-metal-install-server` + +* Update your |CL| system to the latest version. + + .. code-block:: bash + + sudo swupd update + +Set up file sharing +******************* + +#. Add the :command:`samba` bundle, which includes the Samba binaries. + + .. code-block:: bash + + sudo swupd bundle-add samba + +#. Create the :file:`/etc/samba` directory. + + .. code-block:: bash + + sudo mkdir /etc/samba + +#. Create a configuration file named :file:`/etc/samba/smb.conf`. In this + simple example, :envvar:`[SharedDrive]` enables a folder share located in + :file:`/home/clear/mysharedrive` granting users `samba-user-1` and + `samba-user-2` access. + + If :envvar:`valid users` is not specified, then anyone with a user account + on the machine and with their Samba password already set can access the + folder. However, the account is only able to access files and folders for + which they have appropriate permissions. + + For more advanced sharing, refer to the `Samba guide`_. + + .. code-block:: console + + [Global] + map to guest = Bad User + + [SharedDrive] + path=/home/clear/myshareddrive + read only = no + guest ok = no + browsable = yes + valid users = samba-user-1 samba-user-2 + +#. Start the Samba service and set it to start automatically on boot. + + .. code-block:: bash + + sudo systemctl enable --now smb + +#. Verify the service started properly. + + .. code-block:: bash + + sudo systemctl status smb + +#. Use :command:`smbpasswd` to add the initial password for the user + account to access the share. Be aware that Samba maintains its own list of + passwords for user accounts. The Samba password list can be different from + the password used to log in. + + For example: + + .. code-block:: bash + + sudo smbpasswd -a samba-user-1 + +Access the shared drive +*********************** + +Depending on your operating system, connect to the shared drive using one +of the methods belows: + +* On |CL|: + + a. Add the `samba` bundle. + + .. code-block:: bash + + sudo swupd bundle-add samba + + #. List available shares. + + .. code-block:: bash + + smbclient -L // + + #. Connect to a shared drive. + + .. code-block:: bash + + smbclient /// -U + +* On Windows: + + a. Open `File Explorer`. + + #. Enter :command:`\\\` in the + URL field. See Figure 1. + + .. rst-class:: dropshadow + + .. figure:: ../_figures/samba/smb-server-01.png + :scale: 100% + :alt: Windows > File Explorer > Connect to Samba share + + `Figure 1: Windows > File Explorer > Connect to Samba share` + +* On macOS: + + a. Open the `Finder`. + + #. Press :kbd:`Command` + :kbd:`K` to open the dialog box for + connecting to a server. + + #. Enter :command:`smb:///`. + in the URL field. See Figure 2. + + .. figure:: ../_figures/samba/smb-server-02.png + :scale: 100% + :alt: macOS > Finder > Connect to Samba share + + `Figure 2: macOS > Finder > Connect to Samba share` + +.. _Samba guide: + https://www.samba.org/samba/docs/using_samba/ch00.html diff --git a/_sources/tutorials/tensorflow-machine-learning.rst.txt b/_sources/tutorials/tensorflow-machine-learning.rst.txt new file mode 100644 index 00000000..745a56d1 --- /dev/null +++ b/_sources/tutorials/tensorflow-machine-learning.rst.txt @@ -0,0 +1,248 @@ +.. _machine-learning: + +TensorFlow\* machine learning +############################# + +This tutorial demonstrates the installation and execution of a TensorFlow\* +machine learning example on |CL-ATTR|. It uses a Jupyter\* Notebook and MNIST +data for handwriting recognition.  + +The initial steps show how to set up a Jupyter kernel and run a Notebook +on a bare-metal |CL| system. + +.. contents:: + :local: + :depth: 1 + +Prerequisites +************* + +This tutorial assumes you have installed |CL| on your host system. For +detailed instructions on installing |CL| on a bare metal system, follow the +:ref:`bare metal installation tutorial`. + +Before you install any new packages, update |CL| with the following command: + +.. code-block:: bash + + sudo swupd update + +After your system is updated, add the following bundles to your system: + +* :command:`machine-learning-web-ui`: This bundle contains the Jupyter application. + +* :command:`machine-learning-basic`: This bundle contains TensorFlow + and other useful tools. + +To install the bundles, run the following commands in your :file:`$HOME` +directory: + +.. code-block:: bash + + sudo swupd bundle-add machine-learning-web-ui + + sudo swupd bundle-add machine-learning-basic + +Set up a Jupyter Notebook +************************* + +With all required packages and libraries installed, set up the file structure +for the Jupyter Notebook. + +#. In the :file:`$HOME` directory, create a directory for the Jupyter + Notebooks named :file:`Notebooks`. + + .. code-block:: bash + + mkdir Notebooks + +#. Within :file:`Notebooks`, create a directory named :file:`Handwriting`. + + .. code-block:: bash + + mkdir Notebooks/Handwriting + +#. Change to the new directory. + + .. code-block:: bash + + cd Notebooks/Handwriting + +#. Copy the :file:`MNIST_example.ipynb` file into the :file:`Handwriting` + directory. + + .. note:: + + After installing the :command:`machine-learning basic` bundle, you can find the example code under + :file:`/usr/share/doc/tensorflow/MNIST_example.ipynb`. + + +The example code downloads and decompresses the MNIST data directly into the +:file:`./mnist` directory. Alternatively, download the four files directly +from the Yann LeCun’s `MNIST Database website`_ and save them into a +:file:`mnist` directory within the :file:`Handwriting` directory. + +The files needed are: + +* `train-images-idx3-ubyte.gz`_: Training set images (9912422 bytes) + +* `train-labels-idx1-ubyte.gz`_: Training set labels (28881 bytes) + +* `t10k-images-idx3-ubyte.gz`_: Test set images (1648877 bytes) + +* `t10k-labels-idx1-ubyte.gz`_: Test set labels (4542 bytes) + +Run the Jupyter machine learning example code +********************************************* + +With |CL|, Jupyter, and TensorFlow installed and configured, you can +run the example code. + +#. Go to the :file:`($HOME)/Notebooks` directory and start Jupyter with the + following commands: + + .. code-block:: bash + + cd ~/Notebooks + + jupyter notebook + + The Jupyter server starts and opens a web browser showing the Jupyter file + manager with a list of files in the current directory, as shown in + Figure 1. + + .. figure:: /_figures/tensorflow/machine-learning-1.png + :alt: Jupyter file manager + + Figure 1: The Jupyter file manager shows the list of available files. + +#. Click on the :file:`Handwriting` directory. The :file:`MNIST_example.ipynb` + file created earlier should be listed there, as shown in Figure 2. + + .. figure:: /_figures/tensorflow/machine-learning-2.png + :alt: Example file within the Jupyter file manager + + Figure 2: The example file within the Jupyter file manager. + +#. To run the handwriting example, click on the :file:`MNIST_example.ipynb` + file to load the notebook, as shown in Figure 3. + + .. figure:: /_figures/tensorflow/machine-learning-3.png + :alt: The loaded MNIST_example notebook + + Figure 3: The loaded MNIST_example notebook within the Jupyter file + manager. + +#. Click the |run-cell| button to execute the code in the current cell and + move to the next. + +#. Select the :guilabel:`In [2]` cell and click the |run-cell| button to load + the MNIST data. The successful output is shown on Figure 4. + + .. figure:: /_figures/tensorflow/machine-learning-4.png + :alt: Successful import of MNIST data + + Figure 4: Output after successfully importing the MNIST data. + + + After the MNIST data is successfully downloaded and extracted into the + :file:`mnist` directory within the :file:`($HOME)/Notebooks/Handwriting` + directory, four .gz files are present and the four data sets are created: + `trainX`, `trainY`, `testX` and `testY`. + +#. To inspect the imported data, the function in :guilabel:`In [3]` first + instructs Jupyter to reshape the data into an array of 28 x 28 images and to + plot the area in a 28 x 28 grid. Click the |run-cell| button twice to show + the first two digits in the `trainX` dataset. An example is shown in + Figure 5. + + .. figure:: /_figures/tensorflow/machine-learning-5.png + :alt: Function to reshape data. + + Figure 5: A function reshapes the data and displays the first two + digits in the `trainX` dataset. + +#. The :guilabel:`In [4]` cell defines the neural network. It provides the + inputs, defines the hidden layers, runs the training model, and sets up + the output layer, as shown in Figure 6. Click the |run-cell| button four + times to perform these operations. + + .. figure:: /_figures/tensorflow/machine-learning-6.png + :alt: Defining, building and training the neural network model + + Figure 6: Defining, building, and training the neural network model. + +#. To test the accuracy of the prediction that the system makes, select the + :guilabel:`In [8]` cell and click the |run-cell| button. In this example, + the number 6 was predicted with a 99% accuracy, as shown in Figure 7. + + .. figure:: /_figures/tensorflow/machine-learning-7.png + :alt: Prediction example + + Figure 7: The system predicts a number providing the accuracy of the + prediction. + + .. note:: + + To retest the accuracy of a random data point's prediction, run the + cell :guilabel:`In [8]` again. It will take another random data point + and predict its value. + +#. To check the accuracy for the whole dataset, select the :guilabel:`In [10]` + cell and click the |run-cell| button. Our example's accuracy is + calculated as 97.17%, as shown in Figure 8. + + .. figure:: /_figures/tensorflow/machine-learning-8.png + :alt: System's accuracy + + Figure 8: The system's accuracy for the entire data set. + +For more in-depth information on the model used and the mathematics it entails, +visit the TensorFlow tutorials +`TensorFlow MNIST beginners demo`_ and `TensorFlow MNIST pros demo`_. + +**Congratulations!** + +You have successfully installed a Jupyter kernel on |CL|. In addition, you +trained a neural network to successfully predict the values contained in a +data set of hand-written number images. + +Related topics +************** + +* `MNIST Database website`_ +* `TensorFlow MNIST beginners demo`_ +* `TensorFlow MNIST pros demo`_ +* `Jupyter main website`_ +* `Jupyter documentation`_ +* `MNIST at Wikipedia`_ + +.. _MNIST Database website: + http://yann.lecun.com/exdb/mnist/ + +.. _train-images-idx3-ubyte.gz: + http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz + +.. _train-labels-idx1-ubyte.gz: + http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz + +.. _t10k-images-idx3-ubyte.gz: + http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz + +.. _t10k-labels-idx1-ubyte.gz: + http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz + +.. _Jupyter main website: http://jupyter.org/ + +.. _Jupyter documentation:  https://jupyter.readthedocs.io/en/latest/index.html + +.. _TensorFlow MNIST beginners demo: + https://www.tensorflow.org/get_started/mnist/beginners + +.. _TensorFlow MNIST pros demo: + https://www.tensorflow.org/get_started/mnist/pros + +.. _MNIST at Wikipedia: + https://en.wikipedia.org/wiki/MNIST_database + +.. |run-cell| image:: /_figures/tensorflow/run-cell-button.png diff --git a/_sources/tutorials/vmware-workstation.rst.txt b/_sources/tutorials/vmware-workstation.rst.txt new file mode 100644 index 00000000..d777d7cd --- /dev/null +++ b/_sources/tutorials/vmware-workstation.rst.txt @@ -0,0 +1,141 @@ + +.. _vmware-workstation: + +VMware\* Workstation Pro +######################## + +VMware Workstation Pro allows you to run multiple operating systems as +:abbr:`VMs (virtual machines)` on a single host. It is a more advanced version +of VMware Workstation Player. + +This tutorial shows how to do a manual installation of VMware Workstation +Pro on a |CL-ATTR| host using the console plus the VMware Workstation Pro GUI. + +VMware Workstation Pro on Linux installs two major components: + +#. VMware hypervisor software +#. VMware kernel modules + +.. note:: + + |CL| is not an officially supported host OS for VMware Workstation Pro. This + tutorial follows the generic Linux installation instructions with details + specific to a manual installation on |CL|. + +Prerequisites +************* + +Enable virtualization in the BIOS before installing VMware Workstation Pro. + +Install VMware Workstation Pro +****************************** + +Use the console installer to install the VMware Workstation Pro hypervisor +software: + +#. Download + `VMware Workstation Pro for Linux `_. + + .. note:: + + The downloaded file is named with a :file:`.bundle` file extension but + this is not a |CL| :ref:`bundle `! + +#. In a terminal, :command:`cd` to the directory where the `.bundle` installation + file was saved. + + .. code-block:: bash + + cd ~/Downloads/ + +#. Make the `.bundle` installation file executable. Replace + [VMware-Workstation-xxxx-xxxx.architecture] with the actual file name. + + .. code-block:: bash + + chmod +x ./[VMware-Workstation-xxxx-xxxx.architecture].bundle + +#. Start the installation. + + .. code-block:: bash + + sudo ./VMware-Workstation-[xxxx-xxxx].architecture.bundle + + .. code-block:: console + + Extracting VMware Installer...done. + +#. When prompted to specify the system service scripts directory, use the common + suggestion: + + .. code-block:: console + + System service scripts directory (commonly /etc/init.d).: /etc/init.d + + You will see output similar to the following: + + .. code-block:: console + + Installing VMware VMX 15.5.0 + Configuring...No rc*.d style init script directories were given to the installer. + You must manually add the necessary links to ensure that the vmware ] 49% + service at /etc/init.d/vmware is automatically started and stopped on + Installing VMware Workstation 15.5.0 + Configuring... + [######################################################################] 100% + Installation was successful. + + .. note:: + + During installation you will get an error about "No rc*.d style init + script directories" being given. This can safely be ignored as |CL| uses + :command:`systemd`. + +Install VMware kernel modules +***************************** + +After installing VMware Workstation Pro, additional VMware kernel modules must +be compiled and installed. Before installing the VMware kernel modules, install +DKMS, which will provide the necessary tools to add the VMware kernel modules. + +Install DKMS +============ + +.. include:: ../guides/kernel/kernel-modules-dkms.rst + :start-after: kernel-modules-dkms-install-begin: + :end-before: kernel-modules-dkms-install-end: + +Install kernel modules +====================== + +Launch the VMware Workstation Pro GUI to finish the installation and build the +needed kernel modules. + +#. On the |CL| desktop, find the VMware Workstation Pro icon and click to launch. + +#. Click through the installation customization screens as directed in the + VMware Kernel Module Updater. + +#. On the last screen click :guilabel:`Finish`. VMware Workstation Pro will launch. + +#. Create and configure a new VM! + +Troubleshooting +*************** + +If problems occur during installation, the recommended first step is to identify +which major component the issues are occurring in (hypervisor or kernel modules). +This will help direct further troubleshooting. + +Troubleshooting tips: + +* If the issue is with compiling the kernel modules and you are running the native + kernel, try installing the LTS kernel instead. + +* If modules failed to install or load, check logs in :file:`/tmp/vmware-/*.log` + +* Try re-installing all VMware modules with the following command: + + .. code-block:: bash + + sudo vmware-modconfig --console --install-all diff --git a/_sources/tutorials/wordpress.rst.txt b/_sources/tutorials/wordpress.rst.txt new file mode 100644 index 00000000..e4c86340 --- /dev/null +++ b/_sources/tutorials/wordpress.rst.txt @@ -0,0 +1,214 @@ +.. _wp-install: + +WordPress\* Server +################## + +This tutorial shows you how to install the WordPress\* components on your |CL| +:abbr:`LAMP (Linux, Apache\*, MySQL\*, PHP)` server. At the end of +:ref:`lamp-server-install`, you created a WordPress-ready database using +phpMyAdmin\* and MariaDB\*. Now that the LAMP server is up and running, you +can add the WordPress components needed to host a WordPress website on your system. + +Before you begin +**************** + +This tutorial assumes that you have successfully completed +:ref:`bare-metal-install-desktop` and you have :ref:`lamp-server-install`. + +Create a WordPress server +************************* + +WordPress can be installed in a variety of ways. These instructions are +written for users who have followed our instructions for installing phpMyAdmin +when they :ref:`set up a LAMP web server `. Note that +all steps in this tutorial have been tested using a NUC6i5SYH Intel® NUC. +Visit the `NUC6i5SYH product page`_ for detailed information. + +Numerous online articles are available to help you name your website and +acquire the necessary certificates. Those tasks are beyond the scope of this tutorial. + +You can take several actions to harden your website from attacks. The security +of your website and the data it contains are complex and ever-evolving tasks. +Prioritize security if you plan to expose your website to the outside world. +This tutorial does not address security measures that you can take to harden +your site but we strongly encourage you to take action. + +.. note:: + + Throughout this tutorial, we reference your website name as . + + +Download WordPress and manage directories +========================================= + +For this tutorial, you will create a WordPress blog that can be accessed at: +\http:///blog. + +To accomplish this setup, you must add WordPress components to the :file:`/var/www/html/blog` +directory. + +Follow these steps: + + +#. Navigate to the top level of the website’s root directory: + + .. code-block:: bash + + cd /var/www/html + +#. Download the latest version of WordPress: + + .. code-block:: bash + + sudo curl -O https://wordpress.org/latest.tar.gz + +#. Extract all files and directories from the downloaded file: + + .. code-block:: bash + + sudo tar –xzvf latest.tar.gz + +#. Rename the top-level WordPress directory to “blog”: + + .. code-block:: bash + + sudo mv wordpress blog + +#. Remove the downloaded tar file: + + .. code-block:: bash + + sudo rm latest.tar.gz + +Set up WordPress with web-based GUI +=================================== + +Recall that you created a database and user when you installed phpMyAdmin when you +set up a |CL| based :ref:`web server `. Next, you must +connect WordPress to the database and install WordPress. + +To continue with the setup, go to: \http:///blog/wp-admin/install.php. +The WordPress language option screen appears, as shown in figure 1. + +#. Select :guilabel:`English` and click :guilabel:`Continue`. + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/wp-install-1.png + :alt: WordPress language selection + :width: 600 + + `Figure 1: WordPress language selection screen.` + + + The WordPress installation continues until the Welcome screen appears, as shown in + figure 2: + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/wp-install-2.png + :alt: WordPress welcome screen + :width: 600 + + `Figure 2: WordPress Welcome screen.` + +#. Click :guilabel:`Let’s go!`. + +#. Enter database connection specifics in the screen that appears, as shown in figure 3 + below. + + * Database name: WordPress + * Database username: wordpressuser + * Database password: wp-example (asterisks will not appear in the text box) + * Database host: localhost + * Table prefix: wp\_ + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/wp-install-3.png + :alt: Database connection details + :width: 600 + + `Figure 3: Information necessary for WordPress to connect to the database.` + +#. Click :guilabel:`Submit` to complete the setup. + + Figure 4 shows the confirmation screen that verifies a successful setup. WordPress + is connected to the MariaDB database. + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/wp-install-4.png + :alt: Successful database connection. + :width: 600 + + `Figure 4: Successful WordPress connection.` + +#. Click :guilabel:`Run the install`. + The installer runs until WordPress is fully installed on your system. + +Complete successful login +========================= + +Once the installation is complete, you can name your blog and create a WordPress username +and password. See figure 5. + +.. rst-class:: dropshadow + +.. figure:: ../_figures/wordpress/wp-install-5.png + :alt: WordPress user creation + :width: 600 + + `Figure 5: WordPress site information screen.` + + +#. Enter all required information. +#. Click :guilabel:`Install WordPress`. +#. Verify that the initial login screen appears once the installation is complete. See figure 6: + + .. rst-class:: dropshadow + + .. figure:: ../_figures/wordpress/wp-install-6.png + :alt: WordPress login + :width: 600 + + `Figure 6: The WordPress login screen.` + +#. Enter your WordPress username and password. +#. Check :guilabel:`Remember me` to save your credentials. +#. Click :guilabel:`Log in`. + +Figure 7 shows the WordPress dashboard after a successful login: + +.. rst-class:: dropshadow + +.. figure:: ../_figures/wordpress/wp-install-7.png + :alt: WordPress Dashboard + :width: 600 + + `Figure 7: The WordPress dashboard.` + +You are ready to go! + +To check out your blog as it is seen by the outside world, enter: +\http:///blog on your browser. Figure 8 shows the result: + +.. figure:: ../_figures/wordpress/wp-install-8.png + :alt: WordPress blog + :width: 600 + + `Figure 8: Your WordPress blog.` + +**Congratulations, your WordPress blog is up and running!** + +You have successfully installed WordPress on a host system. + +Add new entries to your blog and share them with the world using |CL|! + +*Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries.* + +.. _`step-by-step guide`: + https://codex.wordpress.org/Installing_WordPress#Famous_5-Minute_Install + +.. _`NUC6i5SYH product page`: + http://www.intel.com/content/www/us/en/nuc/nuc-kit-nuc6i5syh.html diff --git a/_sources/tutorials/yubikey-u2f.rst.txt b/_sources/tutorials/yubikey-u2f.rst.txt new file mode 100644 index 00000000..2a8c1e89 --- /dev/null +++ b/_sources/tutorials/yubikey-u2f.rst.txt @@ -0,0 +1,123 @@ +.. _yubikey-u2f: + +YubiKey\* Support +################# + +This tutorial explains how to configure a YubiKey\* for U2F authentication +through a web browser on a |CL-ATTR| system. + +.. contents:: :local: + :depth: 1 + +Description +*********** + +YubiKey is a USB security token manufactured by `Yubico`_. Depending on the +model, a YubiKey can support different authentication protocols including +One-Time Password (OTP), Smart card, FIDO2, and Universal 2nd Factor (U2F). + +A list of `websites that accept U2F authentication with the YubiKey`_ +is available on the Yubico website. See the Yubico website to learn more about +the Yubikey: https://www.yubico.com/getstarted/ + +Prerequisites +************* + +This tutorial assumes you have: + +#. |CL| installed and running. + +#. Mozilla Firefox installed on |CL|. + +#. A YubiKey. + +Enable Linux udev rules for YubiKey +*********************************** + +:command:`udev` is the Linux device manager that handles events when USB +devices are added and removed. Custom rules needs to be created to properly +identify the YubiKey and provide applications access. + +These instructions are derived from: `Yubico support article Using Your U2F +YubiKey with Linux +`_ + + +#. Create the udev rules folder under :file:`/etc`. + + .. code:: bash + + sudo mkdir -p /etc/udev/rules.d/ + + +#. Download the u2f rules from the Yubico GitHub: + + .. code:: bash + + curl -O https://raw.githubusercontent.com/Yubico/libu2f-host/master/70-u2f.rules + + +#. Move the downloaded :file:`70-u2f.rules` file into the :file:`/etc/udev` + folder. + + .. code:: bash + + sudo mv 70-u2f.rules /etc/udev/rules.d/ + + +#. The udev rules should automatically be reloaded. However, they can be + manually reloaded or you can reboot the system. + + .. code:: bash + + sudo udevadm control --reload-rules && sudo udevadm trigger + + +#. Plugin and validate the YubiKey appears as a USB device. + + .. code:: bash + + lsusb + + + +Enable U2F in Mozilla Firefox +***************************** + +Firefox comes with U2F web authentication support disabled by default. U2F +needs to be enabled in the advanced settings. + +These instructions are derived from: `Yubico support article Enabling U2F +support in Mozilla Firefox +`_ + + +#. Launch Mozilla Firefox + +#. In the URL bar, type :command:`about:config` to access the advanced + settings. + + .. code:: bash + + about:config + +#. Click the *I accept the risk!* button to continue to the advanced settings + +#. Search for the :command:`security.webauth.u2f` and double-click it + so *Value* becomes **true**. + + +Your YubiKey is now usable on |CL| with Mozilla Firefox with websites that +support U2F authentication. + + +Related topics +************** + +- |CL| :ref:`security` + + +.. _`Yubico`: https://www.yubico.com/ + +.. _`websites that accept U2F authentication with the YubiKey`: +.. https://www.yubico.com/works-with-yubikey/catalog/#protocol=universal-2nd-factor-(u2f)&usecase=all&key=all diff --git a/_sources/tutorials/zfs.rst.txt b/_sources/tutorials/zfs.rst.txt new file mode 100644 index 00000000..b5f49120 --- /dev/null +++ b/_sources/tutorials/zfs.rst.txt @@ -0,0 +1,225 @@ +.. _zfs: + +OpenZFS* +######## + +This tutorial shows how to set up `OpenZFS* file system and volume manager`_ on |CL-ATTR|, using a non-root device for zpools. + +.. contents:: + :local: + :depth: 1 + +Background +********** + +The OpenZFS storage platform provides volume management, snapshot capabilities, and redundancy detection. |CL| **does not** ship with a binary ZFS kernel module (zfs.ko). |CL| users who wish to incorporate the zfs.ko kernel module must build and maintain this work themselves. + +.. CAUTION:: + + Use of the OpenZFS kernel module in connection with |CL| is neither recommended nor officially endorsed by the Clear Linux* Project. Users who follow this tutorial and build zfs.ko kernel module are encouraged to seek independent legal counsel regarding any plan to redistribute a software package containing zfs.ko and |CL|. + +Known Issues +************ + +Using a long-term-support (LTS) kernel when running OpenZFS reduces the risk of incompatibilities with kernel updates. When new kernels or new versions of OpenZFS are released, users bear the responsibility to test those releases and ensure compatibility before deploying any updates. + +Prerequisites +************* + +* Learn to :ref:`kernel-modules-dkms` +* Learn to use :ref:`swupd-guide` + +Install the DKMS kernel +======================= + +.. include:: ../guides/kernel/kernel-modules-dkms.rst + :start-after: kernel-modules-dkms-install-begin-alt: + :end-before: kernel-modules-dkms-install-end: + +Bundles +======= + +Before installing OpenZFS, install the bundles that contain the build dependencies. + +.. code-block:: console + + sudo swupd bundle-add wget devpkg-openssl devpkg-util-linux + +Install +******* + +Download OpenZFS release +======================== + +In this section, we download release 2.0.0 directly from the `OpenZFS repository` (the latest available as of the latest revision of this page). + +Download release 2.0.0 + + .. code-block:: console + + cd /usr/src + sudo wget https://github.com/openzfs/zfs/releases/download/zfs-2.0.0/zfs-2.0.0.tar.gz + sudo tar -xvf zfs-2.0.0.tar.gz + +Compile the module +================== + +We will build the module using DKMS. This will enable us to keep the module up to date as new kernels are released in the future. + +The ZFS distribution provides a script to build a suitable dkms.conf file. + + Build dkms.conf and install it into the DKMS tree. + + .. code-block:: console + + cd /usr/src/zfs-2.0.0 + sudo scripts/dkms.mkconf -n zfs -v 2.0.0 -f dkms.conf + sudo dkms add -m zfs -v 2.0.0 + sudo dkms build -m zfs -v 2.0.0 + sudo dkms install -m zfs -v 2.0.0 + +Observe that this install the zfs kernel modules to: + + :file:`/usr/lib/modules//extra/zfs` + +Compile userspace tools +======================= + +Here we compile and install the zfs userspace tools (e.g., zpool, zfs, etc.). + + .. code-block:: console + + cd /usr/src/zfs-2.0.0 + sudo ./configure + sudo make + sudo make install + +The binaries are installed at the following directory. While not required, it's recommended to add :file:`/usr/local/sbin` to your path variable. + + .. code-block:: console + + + /usr/local/ + |--+ bin/ + |--zvol_wait + |--zgenhostid + |--raidz_test + |--+ etc/ + |--+ zfs/ + |--* zed.d/ + |--+ zpool.d/ + |--zfs-functions + |--+ include/libzfs/ [contents omitted] + |--+ lib/ + |--+ libexec/ + |--+ zfs/ + |-- zpool.d/ + |-- zed.d + |--+ share/zfs/ [contents omitted] + |--+ sbin/ + |--fsck.zfs + |--zpool + |--zdb + |--zed + |--zfs + |--zhack + |--zinject + |--zpool + |--ztest + |--zstreamdump + |--+ src/ + |--+ zfs-2.0.0/ + |--+ spl-2.0.0/ + + +Set up systemd +============== + +We now have these unit files available. + +.. code-block:: console + + zfs-env-bootfs.service + zfs-zed.service + zfs-import-cache.service + zfs-import-scan.service + zfs-mount.service + zfs-share.service + zfs-volume-wait.service + + +OpenZFS requires that we explicitly install and enable the services desired. + +To use ZFS automatic zpool import and filesystem mount services, enable them. + +.. code-block:: console + + sudo systemctl enable zfs-import-cache + sudo systemctl enable zfs-import.target + sudo systemctl enable zfs-import-scan + sudo systemctl enable zfs-mount + sudo systemctl enable zfs.target + + +Load the kernel module at boot +============================== + +OpenZFS kernel modules must be loaded before any OpenZFS filesystems are mounted. For convenience, load the kernel modules at boot. + +Systemd uses the `/etc/modules-load.d/` directory to load out-of-tree kernel modules. Make sure that the directory exists: + + .. code-block:: bash + + sudo mkdir -p /etc/modules-load.d + +Create the configuration file: + + .. code-block:: bash + + echo "zfs" | sudo tee /etc/modules-load.d/01-zfs.conf + +Reboot your system. zfs.ko should be loaded automatically (the module should appear in the outout of command :file:`lsmod`). + +.. CAUTION:: + + When the |CL| kernel is upgraded, DKMS will attempt to rebuild the OpenZFS module for the new kernel. + + - DKMS may not have rebuilt the module + - DKMS may not have auto-installed the module + - The new kernel might introduce breaking changes that prevent zfs + from compiling + +To fix this situation, recompile zfs.ko with the new kernel code. OpenZFS *might* not compile at all with the new kernel. + +.. CAUTION:: + + **Be sure not to put anything on an OpenZFS pool that will be needed to rebuild kernel modules.** Ensure compatibility of OpenZFS with new Linux kernels when released. + +Troubleshooting +=============== + +If you suspect an issue with DKMS rebuilding your module, you can check two places for information. The dkms-new-kernel service will show status that may help in troubleshooting: + +.. code-block:: console + + systemctl status dkms-new-kernel.service + +The systemd journal may also have important information: + +.. code-block:: console + + journalctl -xe + +.. CAUTION:: + As of OpenZFS 2.0.0, the included file :file:`script/dkms.mkconf` contains a minor incompability -- it calls the command :file:`lsb_release`, which is not available on Clear Linux by default. It is trivial to edit :file:`dkms.mkconf` and remove the singular reference to :file:`lsb_release` without any ill effects, and then execute the :file:`dkms` commands above. However, keeping the file as provided is perfectly fine, but **will result in :file:`dkms` warnings**. + +Next steps +********** +You're now ready to create zpools and datasets! For more information on using ZFS, see: + +* `FreeBSD Handbook chapter on ZFS`_ +* `ZFS-on-Linux issue tracker`_ + +.. _FreeBSD Handbook chapter on ZFS: https://www.freebsd.org/doc/handbook/zfs.html +.. _ZFS-on-Linux issue tracker: https://github.com/openzfs/zfs/issues/10068 +.. _ZFS on Linux repository: https://github.com/openzfs/zfs +.. _OpenZFS* file system and volume manager: https://github.com/openzfs/zfs diff --git a/_static/background_b01.png b/_static/background_b01.png new file mode 100644 index 00000000..353f26dd Binary files /dev/null and b/_static/background_b01.png differ diff --git a/_static/basic.css b/_static/basic.css new file mode 100644 index 00000000..5e1cb131 --- /dev/null +++ b/_static/basic.css @@ -0,0 +1,914 @@ +/* + * Sphinx stylesheet -- basic theme. + */ + +/* -- main layout ----------------------------------------------------------- */ + +div.clearer { + clear: both; +} + +div.section::after { + display: block; + content: ''; + clear: left; +} + +/* -- relbar ---------------------------------------------------------------- */ + +div.related { + width: 100%; + font-size: 90%; +} + +div.related h3 { + display: none; +} + +div.related ul { + margin: 0; + padding: 0 0 0 10px; + list-style: none; +} + +div.related li { + display: inline; +} + +div.related li.right { + float: right; + margin-right: 5px; +} + +/* -- sidebar --------------------------------------------------------------- */ + +div.sphinxsidebarwrapper { + padding: 10px 5px 0 10px; +} + +div.sphinxsidebar { + float: left; + width: 210px; + margin-left: -100%; + font-size: 90%; + word-wrap: break-word; + overflow-wrap : break-word; +} + +div.sphinxsidebar ul { + list-style: none; +} + +div.sphinxsidebar ul ul, +div.sphinxsidebar ul.want-points { + margin-left: 20px; + list-style: square; +} + +div.sphinxsidebar ul ul { + margin-top: 0; + margin-bottom: 0; +} + +div.sphinxsidebar form { + margin-top: 10px; +} + +div.sphinxsidebar input { + border: 1px solid #98dbcc; + font-family: sans-serif; + font-size: 1em; +} + +div.sphinxsidebar #searchbox form.search { + overflow: hidden; +} + +div.sphinxsidebar #searchbox input[type="text"] { + float: left; + width: 80%; + padding: 0.25em; + box-sizing: border-box; +} + +div.sphinxsidebar #searchbox input[type="submit"] { + float: left; + width: 20%; + border-left: none; + padding: 0.25em; + box-sizing: border-box; +} + + +img { + border: 0; + max-width: 100%; +} + +/* -- search page ----------------------------------------------------------- */ + +ul.search { + margin-top: 10px; +} + +ul.search li { + padding: 5px 0; +} + +ul.search li a { + font-weight: bold; +} + +ul.search li p.context { + color: #888; + margin: 2px 0 0 30px; + text-align: left; +} + +ul.keywordmatches li.goodmatch a { + font-weight: bold; +} + +/* -- index page ------------------------------------------------------------ */ + +table.contentstable { + width: 90%; + margin-left: auto; + margin-right: auto; +} + +table.contentstable p.biglink { + line-height: 150%; +} + +a.biglink { + font-size: 1.3em; +} + +span.linkdescr { + font-style: italic; + padding-top: 5px; + font-size: 90%; +} + +/* -- general index --------------------------------------------------------- */ + +table.indextable { + width: 100%; +} + +table.indextable td { + text-align: left; + vertical-align: top; +} + +table.indextable ul { + margin-top: 0; + margin-bottom: 0; + list-style-type: none; +} + +table.indextable > tbody > tr > td > ul { + padding-left: 0em; +} + +table.indextable tr.pcap { + height: 10px; +} + +table.indextable tr.cap { + margin-top: 10px; + background-color: #f2f2f2; +} + +img.toggler { + margin-right: 3px; + margin-top: 3px; + cursor: pointer; +} + +div.modindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +div.genindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +/* -- domain module index --------------------------------------------------- */ + +table.modindextable td { + padding: 2px; + border-collapse: collapse; +} + +/* -- general body styles --------------------------------------------------- */ + +div.body { + min-width: 360px; + max-width: 800px; +} + +div.body p, div.body dd, div.body li, div.body blockquote { + -moz-hyphens: auto; + -ms-hyphens: auto; + -webkit-hyphens: auto; + hyphens: auto; +} + +a.headerlink { + visibility: hidden; +} + +a:visited { + color: #551A8B; +} + +h1:hover > a.headerlink, +h2:hover > a.headerlink, +h3:hover > a.headerlink, +h4:hover > a.headerlink, +h5:hover > a.headerlink, +h6:hover > a.headerlink, +dt:hover > a.headerlink, +caption:hover > a.headerlink, +p.caption:hover > a.headerlink, +div.code-block-caption:hover > a.headerlink { + visibility: visible; +} + +div.body p.caption { + text-align: inherit; +} + +div.body td { + text-align: left; +} + +.first { + margin-top: 0 !important; +} + +p.rubric { + margin-top: 30px; + font-weight: bold; +} + +img.align-left, figure.align-left, .figure.align-left, object.align-left { + clear: left; + float: left; + margin-right: 1em; +} + +img.align-right, figure.align-right, .figure.align-right, object.align-right { + clear: right; + float: right; + margin-left: 1em; +} + +img.align-center, figure.align-center, .figure.align-center, object.align-center { + display: block; + margin-left: auto; + margin-right: auto; +} + +img.align-default, figure.align-default, .figure.align-default { + display: block; + margin-left: auto; + margin-right: auto; +} + +.align-left { + text-align: left; +} + +.align-center { + text-align: center; +} + +.align-default { + text-align: center; +} + +.align-right { + text-align: right; +} + +/* -- sidebars -------------------------------------------------------------- */ + +div.sidebar, +aside.sidebar { + margin: 0 0 0.5em 1em; + border: 1px solid #ddb; + padding: 7px; + background-color: #ffe; + width: 40%; + float: right; + clear: right; + overflow-x: auto; +} + +p.sidebar-title { + font-weight: bold; +} + +nav.contents, +aside.topic, +div.admonition, div.topic, blockquote { + clear: left; +} + +/* -- topics ---------------------------------------------------------------- */ + +nav.contents, +aside.topic, +div.topic { + border: 1px solid #ccc; + padding: 7px; + margin: 10px 0 10px 0; +} + +p.topic-title { + font-size: 1.1em; + font-weight: bold; + margin-top: 10px; +} + +/* -- admonitions ----------------------------------------------------------- */ + +div.admonition { + margin-top: 10px; + margin-bottom: 10px; + padding: 7px; +} + +div.admonition dt { + font-weight: bold; +} + +p.admonition-title { + margin: 0px 10px 5px 0px; + font-weight: bold; +} + +div.body p.centered { + text-align: center; + margin-top: 25px; +} + +/* -- content of sidebars/topics/admonitions -------------------------------- */ + +div.sidebar > :last-child, +aside.sidebar > :last-child, +nav.contents > :last-child, +aside.topic > :last-child, +div.topic > :last-child, +div.admonition > :last-child { + margin-bottom: 0; +} + +div.sidebar::after, +aside.sidebar::after, +nav.contents::after, +aside.topic::after, +div.topic::after, +div.admonition::after, +blockquote::after { + display: block; + content: ''; + clear: both; +} + +/* -- tables ---------------------------------------------------------------- */ + +table.docutils { + margin-top: 10px; + margin-bottom: 10px; + border: 0; + border-collapse: collapse; +} + +table.align-center { + margin-left: auto; + margin-right: auto; +} + +table.align-default { + margin-left: auto; + margin-right: auto; +} + +table caption span.caption-number { + font-style: italic; +} + +table caption span.caption-text { +} + +table.docutils td, table.docutils th { + padding: 1px 8px 1px 5px; + border-top: 0; + border-left: 0; + border-right: 0; + border-bottom: 1px solid #aaa; +} + +th { + text-align: left; + padding-right: 5px; +} + +table.citation { + border-left: solid 1px gray; + margin-left: 1px; +} + +table.citation td { + border-bottom: none; +} + +th > :first-child, +td > :first-child { + margin-top: 0px; +} + +th > :last-child, +td > :last-child { + margin-bottom: 0px; +} + +/* -- figures --------------------------------------------------------------- */ + +div.figure, figure { + margin: 0.5em; + padding: 0.5em; +} + +div.figure p.caption, figcaption { + padding: 0.3em; +} + +div.figure p.caption span.caption-number, +figcaption span.caption-number { + font-style: italic; +} + +div.figure p.caption span.caption-text, +figcaption span.caption-text { +} + +/* -- field list styles ----------------------------------------------------- */ + +table.field-list td, table.field-list th { + border: 0 !important; +} + +.field-list ul { + margin: 0; + padding-left: 1em; +} + +.field-list p { + margin: 0; +} + +.field-name { + -moz-hyphens: manual; + -ms-hyphens: manual; + -webkit-hyphens: manual; + hyphens: manual; +} + +/* -- hlist styles ---------------------------------------------------------- */ + +table.hlist { + margin: 1em 0; +} + +table.hlist td { + vertical-align: top; +} + +/* -- object description styles --------------------------------------------- */ + +.sig { + font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace; +} + +.sig-name, code.descname { + background-color: transparent; + font-weight: bold; +} + +.sig-name { + font-size: 1.1em; +} + +code.descname { + font-size: 1.2em; +} + +.sig-prename, code.descclassname { + background-color: transparent; +} + +.optional { + font-size: 1.3em; +} + +.sig-paren { + font-size: larger; +} + +.sig-param.n { + font-style: italic; +} + +/* C++ specific styling */ + +.sig-inline.c-texpr, +.sig-inline.cpp-texpr { + font-family: unset; +} + +.sig.c .k, .sig.c .kt, +.sig.cpp .k, .sig.cpp .kt { + color: #0033B3; +} + +.sig.c .m, +.sig.cpp .m { + color: #1750EB; +} + +.sig.c .s, .sig.c .sc, +.sig.cpp .s, .sig.cpp .sc { + color: #067D17; +} + + +/* -- other body styles ----------------------------------------------------- */ + +ol.arabic { + list-style: decimal; +} + +ol.loweralpha { + list-style: lower-alpha; +} + +ol.upperalpha { + list-style: upper-alpha; +} + +ol.lowerroman { + list-style: lower-roman; +} + +ol.upperroman { + list-style: upper-roman; +} + +:not(li) > ol > li:first-child > :first-child, +:not(li) > ul > li:first-child > :first-child { + margin-top: 0px; +} + +:not(li) > ol > li:last-child > :last-child, +:not(li) > ul > li:last-child > :last-child { + margin-bottom: 0px; +} + +ol.simple ol p, +ol.simple ul p, +ul.simple ol p, +ul.simple ul p { + margin-top: 0; +} + +ol.simple > li:not(:first-child) > p, +ul.simple > li:not(:first-child) > p { + margin-top: 0; +} + +ol.simple p, +ul.simple p { + margin-bottom: 0; +} + +aside.footnote > span, +div.citation > span { + float: left; +} +aside.footnote > span:last-of-type, +div.citation > span:last-of-type { + padding-right: 0.5em; +} +aside.footnote > p { + margin-left: 2em; +} +div.citation > p { + margin-left: 4em; +} +aside.footnote > p:last-of-type, +div.citation > p:last-of-type { + margin-bottom: 0em; +} +aside.footnote > p:last-of-type:after, +div.citation > p:last-of-type:after { + content: ""; + clear: both; +} + +dl.field-list { + display: grid; + grid-template-columns: fit-content(30%) auto; +} + +dl.field-list > dt { + font-weight: bold; + word-break: break-word; + padding-left: 0.5em; + padding-right: 5px; +} + +dl.field-list > dd { + padding-left: 0.5em; + margin-top: 0em; + margin-left: 0em; + margin-bottom: 0em; +} + +dl { + margin-bottom: 15px; +} + +dd > :first-child { + margin-top: 0px; +} + +dd ul, dd table { + margin-bottom: 10px; +} + +dd { + margin-top: 3px; + margin-bottom: 10px; + margin-left: 30px; +} + +.sig dd { + margin-top: 0px; + margin-bottom: 0px; +} + +.sig dl { + margin-top: 0px; + margin-bottom: 0px; +} + +dl > dd:last-child, +dl > dd:last-child > :last-child { + margin-bottom: 0; +} + +dt:target, span.highlighted { + background-color: #fbe54e; +} + +rect.highlighted { + fill: #fbe54e; +} + +dl.glossary dt { + font-weight: bold; + font-size: 1.1em; +} + +.versionmodified { + font-style: italic; +} + +.system-message { + background-color: #fda; + padding: 5px; + border: 3px solid red; +} + +.footnote:target { + background-color: #ffa; +} + +.line-block { + display: block; + margin-top: 1em; + margin-bottom: 1em; +} + +.line-block .line-block { + margin-top: 0; + margin-bottom: 0; + margin-left: 1.5em; +} + +.guilabel, .menuselection { + font-family: sans-serif; +} + +.accelerator { + text-decoration: underline; +} + +.classifier { + font-style: oblique; +} + +.classifier:before { + font-style: normal; + margin: 0 0.5em; + content: ":"; + display: inline-block; +} + +abbr, acronym { + border-bottom: dotted 1px; + cursor: help; +} + +.translated { + background-color: rgba(207, 255, 207, 0.2) +} + +.untranslated { + background-color: rgba(255, 207, 207, 0.2) +} + +/* -- code displays --------------------------------------------------------- */ + +pre { + overflow: auto; + overflow-y: hidden; /* fixes display issues on Chrome browsers */ +} + +pre, div[class*="highlight-"] { + clear: both; +} + +span.pre { + -moz-hyphens: none; + -ms-hyphens: none; + -webkit-hyphens: none; + hyphens: none; + white-space: nowrap; +} + +div[class*="highlight-"] { + margin: 1em 0; +} + +td.linenos pre { + border: 0; + background-color: transparent; + color: #aaa; +} + +table.highlighttable { + display: block; +} + +table.highlighttable tbody { + display: block; +} + +table.highlighttable tr { + display: flex; +} + +table.highlighttable td { + margin: 0; + padding: 0; +} + +table.highlighttable td.linenos { + padding-right: 0.5em; +} + +table.highlighttable td.code { + flex: 1; + overflow: hidden; +} + +.highlight .hll { + display: block; +} + +div.highlight pre, +table.highlighttable pre { + margin: 0; +} + +div.code-block-caption + div { + margin-top: 0; +} + +div.code-block-caption { + margin-top: 1em; + padding: 2px 5px; + font-size: small; +} + +div.code-block-caption code { + background-color: transparent; +} + +table.highlighttable td.linenos, +span.linenos, +div.highlight span.gp { /* gp: Generic.Prompt */ + user-select: none; + -webkit-user-select: text; /* Safari fallback only */ + -webkit-user-select: none; /* Chrome/Safari */ + -moz-user-select: none; /* Firefox */ + -ms-user-select: none; /* IE10+ */ +} + +div.code-block-caption span.caption-number { + padding: 0.1em 0.3em; + font-style: italic; +} + +div.code-block-caption span.caption-text { +} + +div.literal-block-wrapper { + margin: 1em 0; +} + +code.xref, a code { + background-color: transparent; + font-weight: bold; +} + +h1 code, h2 code, h3 code, h4 code, h5 code, h6 code { + background-color: transparent; +} + +.viewcode-link { + float: right; +} + +.viewcode-back { + float: right; + font-family: sans-serif; +} + +div.viewcode-block:target { + margin: -1px -10px; + padding: 0 10px; +} + +/* -- math display ---------------------------------------------------------- */ + +img.math { + vertical-align: middle; +} + +div.body div.math p { + text-align: center; +} + +span.eqno { + float: right; +} + +span.eqno a.headerlink { + position: absolute; + z-index: 1; +} + +div.math:hover a.headerlink { + visibility: visible; +} + +/* -- printout stylesheet --------------------------------------------------- */ + +@media print { + div.document, + div.documentwrapper, + div.bodywrapper { + margin: 0 !important; + width: 100%; + } + + div.sphinxsidebar, + div.related, + div.footer, + #top-link { + display: none; + } +} \ No newline at end of file diff --git a/_static/bizstyle.css b/_static/bizstyle.css new file mode 100644 index 00000000..2b46ec39 --- /dev/null +++ b/_static/bizstyle.css @@ -0,0 +1,505 @@ +/* + * Sphinx stylesheet -- business style theme. + */ + +@import url("basic.css"); + +/* -- page layout ----------------------------------------------------------- */ + +body { + font-family: 'Lucida Grande', 'Lucida Sans Unicode', 'Geneva', + 'Verdana', sans-serif; + font-size: 14px; + letter-spacing: -0.01em; + line-height: 150%; + text-align: center; + background-color: white; + background-image: url(background_b01.png); + color: black; + padding: 0; + border-right: 1px solid #336699; + border-left: 1px solid #336699; + + margin: 0px 40px 0px 40px; +} + +div.document { + background-color: white; + text-align: left; + background-repeat: repeat-x; + + -moz-box-shadow: 2px 2px 5px #000; + -webkit-box-shadow: 2px 2px 5px #000; +} + +div.documentwrapper { + float: left; + width: 100%; +} + +div.bodywrapper { + margin: 0 0 0 240px; + border-left: 1px solid #ccc; +} + +div.body { + margin: 0; + padding: 0.5em 20px 20px 20px; +} +div.bodywrapper { + margin: 0 0 0 calc(210px + 30px); +} + +div.related { + font-size: 1em; + + -moz-box-shadow: 2px 2px 5px #000; + -webkit-box-shadow: 2px 2px 5px #000; +} + +div.related ul { + background-color: #336699; + height: 100%; + overflow: hidden; + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; +} + +div.related ul li { + color: white; + margin: 0; + padding: 0; + height: 2em; + float: left; +} + +div.related ul li.right { + float: right; + margin-right: 5px; +} + +div.related ul li a { + margin: 0; + padding: 0 5px 0 5px; + line-height: 1.75em; + color: #fff; +} + +div.related ul li a:hover { + color: #fff; + text-decoration: underline; +} + +div.sphinxsidebarwrapper { + padding: 0; +} + +div.sphinxsidebar { + padding: 0.5em 12px 12px 12px; + width: 210px; + font-size: 1em; + text-align: left; +} + +div.sphinxsidebar h3, div.sphinxsidebar h4 { + margin: 1em 0 0.5em 0; + font-size: 1em; + padding: 0.1em 0 0.1em 0.5em; + color: white; + border: 1px solid #336699; + background-color: #336699; +} + +div.sphinxsidebar h3 a { + color: white; +} + +div.sphinxsidebar ul { + padding-left: 1.5em; + margin-top: 7px; + padding: 0; + line-height: 130%; +} + +div.sphinxsidebar ul ul { + margin-left: 20px; +} + +div.sphinxsidebar input { + border: 1px solid #336699; +} + +div.footer { + background-color: white; + color: #336699; + padding: 3px 8px 3px 0; + clear: both; + font-size: 0.8em; + text-align: right; + border-bottom: 1px solid #336699; + + -moz-box-shadow: 2px 2px 5px #000; + -webkit-box-shadow: 2px 2px 5px #000; +} + +div.footer a { + color: #336699; + text-decoration: underline; +} + +/* -- body styles ----------------------------------------------------------- */ + +p { + margin: 0.8em 0 0.5em 0; +} + +a { + color: #336699; + text-decoration: none; +} + +a:hover { + color: #336699; + text-decoration: underline; +} + +a:visited { + color: #551a8b; +} + +div.body a { + text-decoration: underline; +} + +h1, h2, h3 { + color: #336699; +} + +h1 { + margin: 0; + padding: 0.7em 0 0.3em 0; + font-size: 1.5em; +} + +h2 { + margin: 1.3em 0 0.2em 0; + font-size: 1.35em; + padding-bottom: .5em; + border-bottom: 1px solid #336699; +} + +h3 { + margin: 1em 0 -0.3em 0; + font-size: 1.2em; + padding-bottom: .3em; + border-bottom: 1px solid #CCCCCC; +} + +div.body h1 a, div.body h2 a, div.body h3 a, +div.body h4 a, div.body h5 a, div.body h6 a { + color: black!important; +} + +h1 a.anchor, h2 a.anchor, h3 a.anchor, +h4 a.anchor, h5 a.anchor, h6 a.anchor { + display: none; + margin: 0 0 0 0.3em; + padding: 0 0.2em 0 0.2em; + color: #aaa!important; +} + +h1:hover a.anchor, h2:hover a.anchor, h3:hover a.anchor, h4:hover a.anchor, +h5:hover a.anchor, h6:hover a.anchor { + display: inline; +} + +h1 a.anchor:hover, h2 a.anchor:hover, h3 a.anchor:hover, h4 a.anchor:hover, +h5 a.anchor:hover, h6 a.anchor:hover { + color: #777; + background-color: #eee; +} + +a.headerlink { + color: #c60f0f!important; + font-size: 1em; + margin-left: 6px; + padding: 0 4px 0 4px; + text-decoration: none!important; +} + +a.headerlink:hover { + background-color: #ccc; + color: white!important; +} + +cite, code, tt { + font-family: 'Consolas', 'Deja Vu Sans Mono', + 'Bitstream Vera Sans Mono', monospace; + font-size: 0.95em; + letter-spacing: 0.01em; +} + +code { + background-color: #F2F2F2; + border-bottom: 1px solid #ddd; + color: #333; +} + +code.descname, code.descclassname, code.xref { + border: 0; +} + +hr { + border: 1px solid #abc; + margin: 2em; +} + +a code { + border: 0; + color: #CA7900; +} + +a code:hover { + color: #2491CF; +} + +pre { + background-color: transparent !important; + font-family: 'Consolas', 'Deja Vu Sans Mono', + 'Bitstream Vera Sans Mono', monospace; + font-size: 0.95em; + letter-spacing: 0.015em; + line-height: 120%; + padding: 0.5em; + border-right: 5px solid #ccc; + border-left: 5px solid #ccc; +} + +pre a { + color: inherit; + text-decoration: underline; +} + +td.linenos pre { + padding: 0.5em 0; +} + +div.quotebar { + background-color: #f8f8f8; + max-width: 250px; + float: right; + padding: 2px 7px; + border: 1px solid #ccc; +} + +nav.contents, +aside.topic, +div.topic { + background-color: #f8f8f8; +} + +table { + border-collapse: collapse; + margin: 0 -0.5em 0 -0.5em; +} + +table td, table th { + padding: 0.2em 0.5em 0.2em 0.5em; +} + +div.admonition { + font-size: 0.9em; + margin: 1em 0 1em 0; + border: 3px solid #cccccc; + background-color: #f7f7f7; + padding: 0; +} + +div.admonition p { + margin: 0.5em 1em 0.5em 1em; + padding: 0; +} + +div.admonition li p { + margin-left: 0; +} + +div.admonition pre, div.warning pre { + margin: 0; +} + +div.highlight { + margin: 0.4em 1em; +} + +div.admonition p.admonition-title { + margin: 0; + padding: 0.1em 0 0.1em 0.5em; + color: white; + border-bottom: 3px solid #cccccc; + font-weight: bold; + background-color: #165e83; +} + +div.danger { border: 3px solid #f0908d; background-color: #f0cfa0; } +div.error { border: 3px solid #f0908d; background-color: #ede4cd; } +div.warning { border: 3px solid #f8b862; background-color: #f0cfa0; } +div.caution { border: 3px solid #f8b862; background-color: #ede4cd; } +div.attention { border: 3px solid #f8b862; background-color: #f3f3f3; } +div.important { border: 3px solid #f0cfa0; background-color: #ede4cd; } +div.note { border: 3px solid #f0cfa0; background-color: #f3f3f3; } +div.hint { border: 3px solid #bed2c3; background-color: #f3f3f3; } +div.tip { border: 3px solid #bed2c3; background-color: #f3f3f3; } + +div.danger p.admonition-title, div.error p.admonition-title { + background-color: #b7282e; + border-bottom: 3px solid #f0908d; +} + +div.caution p.admonition-title, +div.warning p.admonition-title, +div.attention p.admonition-title { + background-color: #f19072; + border-bottom: 3px solid #f8b862; +} + +div.note p.admonition-title, div.important p.admonition-title { + background-color: #f8b862; + border-bottom: 3px solid #f0cfa0; +} + +div.hint p.admonition-title, div.tip p.admonition-title { + background-color: #7ebea5; + border-bottom: 3px solid #bed2c3; +} + +div.admonition ul, div.admonition ol, +div.warning ul, div.warning ol { + margin: 0.1em 0.5em 0.5em 3em; + padding: 0; +} + +div.versioninfo { + margin: 1em 0 0 0; + border: 1px solid #ccc; + background-color: #DDEAF0; + padding: 8px; + line-height: 1.3em; + font-size: 0.9em; +} + +.viewcode-back { + font-family: 'Lucida Grande', 'Lucida Sans Unicode', 'Geneva', + 'Verdana', sans-serif; +} + +div.viewcode-block:target { + background-color: #f4debf; + border-top: 1px solid #ac9; + border-bottom: 1px solid #ac9; +} + +p.versionchanged span.versionmodified { + font-size: 0.9em; + margin-right: 0.2em; + padding: 0.1em; + background-color: #DCE6A0; +} + +dl.field-list > dt { + color: white; + background-color: #82A0BE; +} + +dl.field-list > dd { + background-color: #f7f7f7; +} + +/* -- table styles ---------------------------------------------------------- */ + +table.docutils { + margin: 1em 0; + padding: 0; + border: 1px solid white; + background-color: #f7f7f7; +} + +table.docutils td, table.docutils th { + padding: 1px 8px 1px 5px; + border-top: 0; + border-left: 0; + border-right: 1px solid white; + border-bottom: 1px solid white; +} + +table.docutils td p { + margin-top: 0; + margin-bottom: 0.3em; +} + +table.field-list td, table.field-list th { + border: 0 !important; + word-break: break-word; +} + +table.footnote td, table.footnote th { + border: 0 !important; +} + +th { + color: white; + text-align: left; + padding-right: 5px; + background-color: #82A0BE; +} + +div.literal-block-wrapper div.code-block-caption { + background-color: #EEE; + border-style: solid; + border-color: #CCC; + border-width: 1px 5px; +} + +/* WIDE DESKTOP STYLE */ +@media only screen and (min-width: 1176px) { +body { + margin: 0 40px 0 40px; +} +} + +/* TABLET STYLE */ +@media only screen and (min-width: 768px) and (max-width: 991px) { +body { + margin: 0 40px 0 40px; +} +} + +/* MOBILE LAYOUT (PORTRAIT/320px) */ +@media only screen and (max-width: 767px) { +body { + margin: 0; +} +div.bodywrapper { + margin: 0; + width: 100%; + border: none; +} +div.sphinxsidebar { + display: none; +} +} + +/* MOBILE LAYOUT (LANDSCAPE/480px) */ +@media only screen and (min-width: 480px) and (max-width: 767px) { +body { + margin: 0 20px 0 20px; +} +} + +/* RETINA OVERRIDES */ +@media +only screen and (-webkit-min-device-pixel-ratio: 2), +only screen and (min-device-pixel-ratio: 2) { +} + +/* -- end ------------------------------------------------------------------- */ \ No newline at end of file diff --git a/_static/bizstyle.js b/_static/bizstyle.js new file mode 100644 index 00000000..14d1c453 --- /dev/null +++ b/_static/bizstyle.js @@ -0,0 +1,24 @@ +/* + * Sphinx javascript -- for bizstyle theme. + * + * This theme was created by referring to 'sphinxdoc' + */ +const initialiseBizStyle = () => { + if (navigator.userAgent.indexOf("iPhone") > 0 || navigator.userAgent.indexOf("Android") > 0) { + document.querySelector("li.nav-item-0 a").innerText = "Top" + } + const truncator = item => {if (item.textContent.length > 20) { + item.title = item.innerText + item.innerText = item.innerText.substr(0, 17) + "..." + } + } + document.querySelectorAll("div.related:first ul li:not(.right) a").slice(1).forEach(truncator); + document.querySelectorAll("div.related:last ul li:not(.right) a").slice(1).forEach(truncator); +} + +window.addEventListener("resize", + () => (document.querySelector("li.nav-item-0 a").innerText = (window.innerWidth <= 776) ? "Top" : "Documentation for Clear Linux* project") +) + +if (document.readyState !== "loading") initialiseBizStyle() +else document.addEventListener("DOMContentLoaded", initialiseBizStyle) \ No newline at end of file diff --git a/_static/check-solid.svg b/_static/check-solid.svg new file mode 100644 index 00000000..92fad4b5 --- /dev/null +++ b/_static/check-solid.svg @@ -0,0 +1,4 @@ + + + + diff --git a/_static/clearlinux.png b/_static/clearlinux.png new file mode 100644 index 00000000..af518c0e Binary files /dev/null and b/_static/clearlinux.png differ diff --git a/_static/clipboard.min.js b/_static/clipboard.min.js new file mode 100644 index 00000000..54b3c463 --- /dev/null +++ b/_static/clipboard.min.js @@ -0,0 +1,7 @@ +/*! + * clipboard.js v2.0.8 + * https://clipboardjs.com/ + * + * Licensed MIT © Zeno Rocha + */ +!function(t,e){"object"==typeof exports&&"object"==typeof module?module.exports=e():"function"==typeof define&&define.amd?define([],e):"object"==typeof exports?exports.ClipboardJS=e():t.ClipboardJS=e()}(this,function(){return n={686:function(t,e,n){"use strict";n.d(e,{default:function(){return o}});var e=n(279),i=n.n(e),e=n(370),u=n.n(e),e=n(817),c=n.n(e);function a(t){try{return document.execCommand(t)}catch(t){return}}var f=function(t){t=c()(t);return a("cut"),t};var l=function(t){var e,n,o,r=1 + + + + diff --git a/_static/copybutton.css b/_static/copybutton.css new file mode 100644 index 00000000..f1916ec7 --- /dev/null +++ b/_static/copybutton.css @@ -0,0 +1,94 @@ +/* Copy buttons */ +button.copybtn { + position: absolute; + display: flex; + top: .3em; + right: .3em; + width: 1.7em; + height: 1.7em; + opacity: 0; + transition: opacity 0.3s, border .3s, background-color .3s; + user-select: none; + padding: 0; + border: none; + outline: none; + border-radius: 0.4em; + /* The colors that GitHub uses */ + border: #1b1f2426 1px solid; + background-color: #f6f8fa; + color: #57606a; +} + +button.copybtn.success { + border-color: #22863a; + color: #22863a; +} + +button.copybtn svg { + stroke: currentColor; + width: 1.5em; + height: 1.5em; + padding: 0.1em; +} + +div.highlight { + position: relative; +} + +/* Show the copybutton */ +.highlight:hover button.copybtn, button.copybtn.success { + opacity: 1; +} + +.highlight button.copybtn:hover { + background-color: rgb(235, 235, 235); +} + +.highlight button.copybtn:active { + background-color: rgb(187, 187, 187); +} + +/** + * A minimal CSS-only tooltip copied from: + * https://codepen.io/mildrenben/pen/rVBrpK + * + * To use, write HTML like the following: + * + *

Short

+ */ + .o-tooltip--left { + position: relative; + } + + .o-tooltip--left:after { + opacity: 0; + visibility: hidden; + position: absolute; + content: attr(data-tooltip); + padding: .2em; + font-size: .8em; + left: -.2em; + background: grey; + color: white; + white-space: nowrap; + z-index: 2; + border-radius: 2px; + transform: translateX(-102%) translateY(0); + transition: opacity 0.2s cubic-bezier(0.64, 0.09, 0.08, 1), transform 0.2s cubic-bezier(0.64, 0.09, 0.08, 1); +} + +.o-tooltip--left:hover:after { + display: block; + opacity: 1; + visibility: visible; + transform: translateX(-100%) translateY(0); + transition: opacity 0.2s cubic-bezier(0.64, 0.09, 0.08, 1), transform 0.2s cubic-bezier(0.64, 0.09, 0.08, 1); + transition-delay: .5s; +} + +/* By default the copy button shouldn't show up when printing a page */ +@media print { + button.copybtn { + display: none; + } +} diff --git a/_static/copybutton.js b/_static/copybutton.js new file mode 100644 index 00000000..d06852f6 --- /dev/null +++ b/_static/copybutton.js @@ -0,0 +1,248 @@ +// Localization support +const messages = { + 'en': { + 'copy': 'Copy', + 'copy_to_clipboard': 'Copy to clipboard', + 'copy_success': 'Copied!', + 'copy_failure': 'Failed to copy', + }, + 'es' : { + 'copy': 'Copiar', + 'copy_to_clipboard': 'Copiar al portapapeles', + 'copy_success': '¡Copiado!', + 'copy_failure': 'Error al copiar', + }, + 'de' : { + 'copy': 'Kopieren', + 'copy_to_clipboard': 'In die Zwischenablage kopieren', + 'copy_success': 'Kopiert!', + 'copy_failure': 'Fehler beim Kopieren', + }, + 'fr' : { + 'copy': 'Copier', + 'copy_to_clipboard': 'Copier dans le presse-papier', + 'copy_success': 'Copié !', + 'copy_failure': 'Échec de la copie', + }, + 'ru': { + 'copy': 'Скопировать', + 'copy_to_clipboard': 'Скопировать в буфер', + 'copy_success': 'Скопировано!', + 'copy_failure': 'Не удалось скопировать', + }, + 'zh-CN': { + 'copy': '复制', + 'copy_to_clipboard': '复制到剪贴板', + 'copy_success': '复制成功!', + 'copy_failure': '复制失败', + }, + 'it' : { + 'copy': 'Copiare', + 'copy_to_clipboard': 'Copiato negli appunti', + 'copy_success': 'Copiato!', + 'copy_failure': 'Errore durante la copia', + } +} + +let locale = 'en' +if( document.documentElement.lang !== undefined + && messages[document.documentElement.lang] !== undefined ) { + locale = document.documentElement.lang +} + +let doc_url_root = DOCUMENTATION_OPTIONS.URL_ROOT; +if (doc_url_root == '#') { + doc_url_root = ''; +} + +/** + * SVG files for our copy buttons + */ +let iconCheck = ` + ${messages[locale]['copy_success']} + + +` + +// If the user specified their own SVG use that, otherwise use the default +let iconCopy = ``; +if (!iconCopy) { + iconCopy = ` + ${messages[locale]['copy_to_clipboard']} + + + +` +} + +/** + * Set up copy/paste for code blocks + */ + +const runWhenDOMLoaded = cb => { + if (document.readyState != 'loading') { + cb() + } else if (document.addEventListener) { + document.addEventListener('DOMContentLoaded', cb) + } else { + document.attachEvent('onreadystatechange', function() { + if (document.readyState == 'complete') cb() + }) + } +} + +const codeCellId = index => `codecell${index}` + +// Clears selected text since ClipboardJS will select the text when copying +const clearSelection = () => { + if (window.getSelection) { + window.getSelection().removeAllRanges() + } else if (document.selection) { + document.selection.empty() + } +} + +// Changes tooltip text for a moment, then changes it back +// We want the timeout of our `success` class to be a bit shorter than the +// tooltip and icon change, so that we can hide the icon before changing back. +var timeoutIcon = 2000; +var timeoutSuccessClass = 1500; + +const temporarilyChangeTooltip = (el, oldText, newText) => { + el.setAttribute('data-tooltip', newText) + el.classList.add('success') + // Remove success a little bit sooner than we change the tooltip + // So that we can use CSS to hide the copybutton first + setTimeout(() => el.classList.remove('success'), timeoutSuccessClass) + setTimeout(() => el.setAttribute('data-tooltip', oldText), timeoutIcon) +} + +// Changes the copy button icon for two seconds, then changes it back +const temporarilyChangeIcon = (el) => { + el.innerHTML = iconCheck; + setTimeout(() => {el.innerHTML = iconCopy}, timeoutIcon) +} + +const addCopyButtonToCodeCells = () => { + // If ClipboardJS hasn't loaded, wait a bit and try again. This + // happens because we load ClipboardJS asynchronously. + if (window.ClipboardJS === undefined) { + setTimeout(addCopyButtonToCodeCells, 250) + return + } + + // Add copybuttons to all of our code cells + const COPYBUTTON_SELECTOR = 'div.highlight pre'; + const codeCells = document.querySelectorAll(COPYBUTTON_SELECTOR) + codeCells.forEach((codeCell, index) => { + const id = codeCellId(index) + codeCell.setAttribute('id', id) + + const clipboardButton = id => + `` + codeCell.insertAdjacentHTML('afterend', clipboardButton(id)) + }) + +function escapeRegExp(string) { + return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string +} + +/** + * Removes excluded text from a Node. + * + * @param {Node} target Node to filter. + * @param {string} exclude CSS selector of nodes to exclude. + * @returns {DOMString} Text from `target` with text removed. + */ +function filterText(target, exclude) { + const clone = target.cloneNode(true); // clone as to not modify the live DOM + if (exclude) { + // remove excluded nodes + clone.querySelectorAll(exclude).forEach(node => node.remove()); + } + return clone.innerText; +} + +// Callback when a copy button is clicked. Will be passed the node that was clicked +// should then grab the text and replace pieces of text that shouldn't be used in output +function formatCopyText(textContent, copybuttonPromptText, isRegexp = false, onlyCopyPromptLines = true, removePrompts = true, copyEmptyLines = true, lineContinuationChar = "", hereDocDelim = "") { + var regexp; + var match; + + // Do we check for line continuation characters and "HERE-documents"? + var useLineCont = !!lineContinuationChar + var useHereDoc = !!hereDocDelim + + // create regexp to capture prompt and remaining line + if (isRegexp) { + regexp = new RegExp('^(' + copybuttonPromptText + ')(.*)') + } else { + regexp = new RegExp('^(' + escapeRegExp(copybuttonPromptText) + ')(.*)') + } + + const outputLines = []; + var promptFound = false; + var gotLineCont = false; + var gotHereDoc = false; + const lineGotPrompt = []; + for (const line of textContent.split('\n')) { + match = line.match(regexp) + if (match || gotLineCont || gotHereDoc) { + promptFound = regexp.test(line) + lineGotPrompt.push(promptFound) + if (removePrompts && promptFound) { + outputLines.push(match[2]) + } else { + outputLines.push(line) + } + gotLineCont = line.endsWith(lineContinuationChar) & useLineCont + if (line.includes(hereDocDelim) & useHereDoc) + gotHereDoc = !gotHereDoc + } else if (!onlyCopyPromptLines) { + outputLines.push(line) + } else if (copyEmptyLines && line.trim() === '') { + outputLines.push(line) + } + } + + // If no lines with the prompt were found then just use original lines + if (lineGotPrompt.some(v => v === true)) { + textContent = outputLines.join('\n'); + } + + // Remove a trailing newline to avoid auto-running when pasting + if (textContent.endsWith("\n")) { + textContent = textContent.slice(0, -1) + } + return textContent +} + + +var copyTargetText = (trigger) => { + var target = document.querySelector(trigger.attributes['data-clipboard-target'].value); + + // get filtered text + let exclude = '.linenos'; + + let text = filterText(target, exclude); + return formatCopyText(text, '$ ', false, true, true, true, '', '') +} + + // Initialize with a callback so we can modify the text before copy + const clipboard = new ClipboardJS('.copybtn', {text: copyTargetText}) + + // Update UI with error/success messages + clipboard.on('success', event => { + clearSelection() + temporarilyChangeTooltip(event.trigger, messages[locale]['copy'], messages[locale]['copy_success']) + temporarilyChangeIcon(event.trigger) + }) + + clipboard.on('error', event => { + temporarilyChangeTooltip(event.trigger, messages[locale]['copy'], messages[locale]['copy_failure']) + }) +} + +runWhenDOMLoaded(addCopyButtonToCodeCells) \ No newline at end of file diff --git a/_static/copybutton_funcs.js b/_static/copybutton_funcs.js new file mode 100644 index 00000000..dbe1aaad --- /dev/null +++ b/_static/copybutton_funcs.js @@ -0,0 +1,73 @@ +function escapeRegExp(string) { + return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); // $& means the whole matched string +} + +/** + * Removes excluded text from a Node. + * + * @param {Node} target Node to filter. + * @param {string} exclude CSS selector of nodes to exclude. + * @returns {DOMString} Text from `target` with text removed. + */ +export function filterText(target, exclude) { + const clone = target.cloneNode(true); // clone as to not modify the live DOM + if (exclude) { + // remove excluded nodes + clone.querySelectorAll(exclude).forEach(node => node.remove()); + } + return clone.innerText; +} + +// Callback when a copy button is clicked. Will be passed the node that was clicked +// should then grab the text and replace pieces of text that shouldn't be used in output +export function formatCopyText(textContent, copybuttonPromptText, isRegexp = false, onlyCopyPromptLines = true, removePrompts = true, copyEmptyLines = true, lineContinuationChar = "", hereDocDelim = "") { + var regexp; + var match; + + // Do we check for line continuation characters and "HERE-documents"? + var useLineCont = !!lineContinuationChar + var useHereDoc = !!hereDocDelim + + // create regexp to capture prompt and remaining line + if (isRegexp) { + regexp = new RegExp('^(' + copybuttonPromptText + ')(.*)') + } else { + regexp = new RegExp('^(' + escapeRegExp(copybuttonPromptText) + ')(.*)') + } + + const outputLines = []; + var promptFound = false; + var gotLineCont = false; + var gotHereDoc = false; + const lineGotPrompt = []; + for (const line of textContent.split('\n')) { + match = line.match(regexp) + if (match || gotLineCont || gotHereDoc) { + promptFound = regexp.test(line) + lineGotPrompt.push(promptFound) + if (removePrompts && promptFound) { + outputLines.push(match[2]) + } else { + outputLines.push(line) + } + gotLineCont = line.endsWith(lineContinuationChar) & useLineCont + if (line.includes(hereDocDelim) & useHereDoc) + gotHereDoc = !gotHereDoc + } else if (!onlyCopyPromptLines) { + outputLines.push(line) + } else if (copyEmptyLines && line.trim() === '') { + outputLines.push(line) + } + } + + // If no lines with the prompt were found then just use original lines + if (lineGotPrompt.some(v => v === true)) { + textContent = outputLines.join('\n'); + } + + // Remove a trailing newline to avoid auto-running when pasting + if (textContent.endsWith("\n")) { + textContent = textContent.slice(0, -1) + } + return textContent +} diff --git a/_static/css3-mediaqueries.js b/_static/css3-mediaqueries.js new file mode 100644 index 00000000..59735f59 --- /dev/null +++ b/_static/css3-mediaqueries.js @@ -0,0 +1 @@ +if(typeof Object.create!=="function"){Object.create=function(e){function t(){}t.prototype=e;return new t}}var ua={toString:function(){return navigator.userAgent},test:function(e){return this.toString().toLowerCase().indexOf(e.toLowerCase())>-1}};ua.version=(ua.toString().toLowerCase().match(/[\s\S]+(?:rv|it|ra|ie)[\/: ]([\d.]+)/)||[])[1];ua.webkit=ua.test("webkit");ua.gecko=ua.test("gecko")&&!ua.webkit;ua.opera=ua.test("opera");ua.ie=ua.test("msie")&&!ua.opera;ua.ie6=ua.ie&&document.compatMode&&typeof document.documentElement.style.maxHeight==="undefined";ua.ie7=ua.ie&&document.documentElement&&typeof document.documentElement.style.maxHeight!=="undefined"&&typeof XDomainRequest==="undefined";ua.ie8=ua.ie&&typeof XDomainRequest!=="undefined";var domReady=function(){var e=[];var t=function(){if(!arguments.callee.done){arguments.callee.done=true;for(var t=0;t=200&&r.status<300||r.status===304||navigator.userAgent.indexOf("Safari")>-1&&typeof r.status==="undefined"){t(r.responseText)}else{n()}document.documentElement.style.cursor="";r=null}};r.send("")};var l=function(t){t=t.replace(e.REDUNDANT_COMPONENTS,"");t=t.replace(e.REDUNDANT_WHITESPACE,"$1");t=t.replace(e.WHITESPACE_IN_PARENTHESES,"($1)");t=t.replace(e.MORE_WHITESPACE," ");t=t.replace(e.FINAL_SEMICOLONS,"}");return t};var c={stylesheet:function(t){var n={};var r=[],i=[],s=[],o=[];var u=t.cssHelperText;var a=t.getAttribute("media");if(a){var f=a.toLowerCase().split(",")}else{var f=["all"]}for(var l=0;l-1&&a.href&&a.href.length!==0&&!a.disabled){r[r.length]=a}}if(r.length>0){var c=0;var d=function(){c++;if(c===r.length){i()}};var v=function(t){var n=t.href;f(n,function(r){r=l(r).replace(e.RELATIVE_URLS,"url("+n.substring(0,n.lastIndexOf("/"))+"/$1)");t.cssHelperText=r;d()},d)};for(u=0;u0){r.setAttribute("media",t.join(","))}document.getElementsByTagName("head")[0].appendChild(r);if(r.styleSheet){r.styleSheet.cssText=e}else{r.appendChild(document.createTextNode(e))}r.addedWithCssHelper=true;if(typeof n==="undefined"||n===true){cssHelper.parsed(function(t){var n=p(r,e);for(var i in n){if(n.hasOwnProperty(i)){g(i,n[i])}}a("newStyleParsed",r)})}else{r.parsingDisallowed=true}return r},removeStyle:function(e){return e.parentNode.removeChild(e)},parsed:function(e){if(n){s(e)}else{if(typeof t!=="undefined"){if(typeof e==="function"){e(t)}}else{s(e);d()}}},stylesheets:function(e){cssHelper.parsed(function(t){e(m.stylesheets||y("stylesheets"))})},mediaQueryLists:function(e){cssHelper.parsed(function(t){e(m.mediaQueryLists||y("mediaQueryLists"))})},rules:function(e){cssHelper.parsed(function(t){e(m.rules||y("rules"))})},selectors:function(e){cssHelper.parsed(function(t){e(m.selectors||y("selectors"))})},declarations:function(e){cssHelper.parsed(function(t){e(m.declarations||y("declarations"))})},properties:function(e){cssHelper.parsed(function(t){e(m.properties||y("properties"))})},broadcast:a,addListener:function(e,t){if(typeof t==="function"){if(!u[e]){u[e]={listeners:[]}}u[e].listeners[u[e].listeners.length]=t}},removeListener:function(e,t){if(typeof t==="function"&&u[e]){var n=u[e].listeners;for(var r=0;r=a||s&&l0}}else if("device-height"===e.substring(r-13,r)){c=screen.height;if(t!==null){if(u==="length"){return i&&c>=a||s&&c0}}else if("width"===e.substring(r-5,r)){l=document.documentElement.clientWidth||document.body.clientWidth;if(t!==null){if(u==="length"){return i&&l>=a||s&&l0}}else if("height"===e.substring(r-6,r)){c=document.documentElement.clientHeight||document.body.clientHeight;if(t!==null){if(u==="length"){return i&&c>=a||s&&c0}}else if("device-aspect-ratio"===e.substring(r-19,r)){return u==="aspect-ratio"&&screen.width*a[1]===screen.height*a[0]}else if("color-index"===e.substring(r-11,r)){var h=Math.pow(2,screen.colorDepth);if(t!==null){if(u==="absolute"){return i&&h>=a||s&&h0}}else if("color"===e.substring(r-5,r)){var p=screen.colorDepth;if(t!==null){if(u==="absolute"){return i&&p>=a||s&&p0}}else if("resolution"===e.substring(r-10,r)){var d;if(f==="dpcm"){d=o("1cm")}else{d=o("1in")}if(t!==null){if(u==="resolution"){return i&&d>=a||s&&d0}}else{return false}};var a=function(e){var t=e.getValid();var n=e.getExpressions();var r=n.length;if(r>0){for(var i=0;i0){u=false;for(var f=0;f0){l[c++]=","}l[c++]=h}}if(l.length>0){r[r.length]=cssHelper.addStyle("@media "+l.join("")+"{"+e.getCssText()+"}",t,false)}};var l=function(e,t){for(var n=0;n0}}var o=[],u=[];for(var f in i){if(i.hasOwnProperty(f)){o[o.length]=f;if(i[f]){u[u.length]=f}if(f==="all"){n=true}}}if(u.length>0){r[r.length]=cssHelper.addStyle(e.getCssText(),u,false)}var c=e.getMediaQueryLists();if(n){l(c)}else{l(c,o)}};var h=function(e){for(var t=0;td||Math.abs(s-t)>d){e=n;t=s;clearTimeout(r);r=setTimeout(function(){if(!i()){p()}else{cssHelper.broadcast("cssMediaQueriesTested")}},500)}};window.onresize=function(){var e=window.onresize||function(){};return function(){e();s()}}()};var m=document.documentElement;m.style.marginLeft="-32767px";setTimeout(function(){m.style.marginLeft=""},5e3);return function(){if(!i()){cssHelper.addListener("newStyleParsed",function(e){c(e.cssHelperParsed.stylesheet)});cssHelper.addListener("cssMediaQueriesTested",function(){if(ua.ie){m.style.width="1px"}setTimeout(function(){m.style.width="";m.style.marginLeft=""},0);cssHelper.removeListener("cssMediaQueriesTested",arguments.callee)});s();p()}else{m.style.marginLeft=""}v()}}());try{document.execCommand("BackgroundImageCache",false,true)}catch(e){} diff --git a/_static/css3-mediaqueries_src.js b/_static/css3-mediaqueries_src.js new file mode 100644 index 00000000..8aedaab9 --- /dev/null +++ b/_static/css3-mediaqueries_src.js @@ -0,0 +1,1104 @@ +/* +css3-mediaqueries.js - CSS Helper and CSS3 Media Queries Enabler + +author: Wouter van der Graaf +version: 1.0 (20110330) +license: MIT +website: https://code.google.com/p/css3-mediaqueries-js/ + +W3C spec: https://www.w3.org/TR/css3-mediaqueries/ + +Note: use of embedded "].join(""),l.id=h,(m?l:n).innerHTML+=f,n.appendChild(l),m||(n.style.background="",n.style.overflow="hidden",k=g.style.overflow,g.style.overflow="hidden",g.appendChild(n)),i=c(l,a),m?l.parentNode.removeChild(l):(n.parentNode.removeChild(n),g.style.overflow=k),!!i},z=function(b){var c=a.matchMedia||a.msMatchMedia;if(c)return c(b).matches;var d;return y("@media "+b+" { #"+h+" { position: absolute; } }",function(b){d=(a.getComputedStyle?getComputedStyle(b,null):b.currentStyle)["position"]=="absolute"}),d},A=function(){function d(d,e){e=e||b.createElement(a[d]||"div"),d="on"+d;var f=d in e;return f||(e.setAttribute||(e=b.createElement("div")),e.setAttribute&&e.removeAttribute&&(e.setAttribute(d,""),f=F(e[d],"function"),F(e[d],"undefined")||(e[d]=c),e.removeAttribute(d))),e=null,f}var a={select:"input",change:"input",submit:"form",reset:"form",error:"img",load:"img",abort:"img"};return d}(),B={}.hasOwnProperty,C;!F(B,"undefined")&&!F(B.call,"undefined")?C=function(a,b){return B.call(a,b)}:C=function(a,b){return b in a&&F(a.constructor.prototype[b],"undefined")},Function.prototype.bind||(Function.prototype.bind=function(b){var c=this;if(typeof c!="function")throw new TypeError;var d=w.call(arguments,1),e=function(){if(this instanceof e){var a=function(){};a.prototype=c.prototype;var f=new a,g=c.apply(f,d.concat(w.call(arguments)));return Object(g)===g?g:f}return c.apply(b,d.concat(w.call(arguments)))};return e}),s.flexbox=function(){return J("flexWrap")},s.canvas=function(){var a=b.createElement("canvas");return!!a.getContext&&!!a.getContext("2d")},s.canvastext=function(){return!!e.canvas&&!!F(b.createElement("canvas").getContext("2d").fillText,"function")},s.webgl=function(){return!!a.WebGLRenderingContext},s.touch=function(){var c;return"ontouchstart"in a||a.DocumentTouch&&b instanceof DocumentTouch?c=!0:y(["@media (",n.join("touch-enabled),("),h,")","{#modernizr{top:9px;position:absolute}}"].join(""),function(a){c=a.offsetTop===9}),c},s.geolocation=function(){return"geolocation"in navigator},s.postmessage=function(){return!!a.postMessage},s.websqldatabase=function(){return!!a.openDatabase},s.indexedDB=function(){return!!J("indexedDB",a)},s.hashchange=function(){return A("hashchange",a)&&(b.documentMode===c||b.documentMode>7)},s.history=function(){return!!a.history&&!!history.pushState},s.draganddrop=function(){var a=b.createElement("div");return"draggable"in a||"ondragstart"in a&&"ondrop"in a},s.websockets=function(){return"WebSocket"in a||"MozWebSocket"in a},s.rgba=function(){return D("background-color:rgba(150,255,150,.5)"),G(j.backgroundColor,"rgba")},s.hsla=function(){return D("background-color:hsla(120,40%,100%,.5)"),G(j.backgroundColor,"rgba")||G(j.backgroundColor,"hsla")},s.multiplebgs=function(){return D("background:url(https://),url(https://),red url(https://)"),/(url\s*\(.*?){3}/.test(j.background)},s.backgroundsize=function(){return J("backgroundSize")},s.borderimage=function(){return J("borderImage")},s.borderradius=function(){return J("borderRadius")},s.boxshadow=function(){return J("boxShadow")},s.textshadow=function(){return b.createElement("div").style.textShadow===""},s.opacity=function(){return E("opacity:.55"),/^0.55$/.test(j.opacity)},s.cssanimations=function(){return J("animationName")},s.csscolumns=function(){return J("columnCount")},s.cssgradients=function(){var a="background-image:",b="gradient(linear,left top,right bottom,from(#9f9),to(white));",c="linear-gradient(left top,#9f9, white);";return D((a+"-webkit- ".split(" ").join(b+a)+n.join(c+a)).slice(0,-a.length)),G(j.backgroundImage,"gradient")},s.cssreflections=function(){return J("boxReflect")},s.csstransforms=function(){return!!J("transform")},s.csstransforms3d=function(){var a=!!J("perspective");return a&&"webkitPerspective"in g.style&&y("@media (transform-3d),(-webkit-transform-3d){#modernizr{left:9px;position:absolute;height:3px;}}",function(b,c){a=b.offsetLeft===9&&b.offsetHeight===3}),a},s.csstransitions=function(){return J("transition")},s.fontface=function(){var a;return y('@font-face {font-family:"font";src:url("https://")}',function(c,d){var e=b.getElementById("smodernizr"),f=e.sheet||e.styleSheet,g=f?f.cssRules&&f.cssRules[0]?f.cssRules[0].cssText:f.cssText||"":"";a=/src/i.test(g)&&g.indexOf(d.split(" ")[0])===0}),a},s.generatedcontent=function(){var a;return y(["#",h,"{font:0/0 a}#",h,':after{content:"',l,'";visibility:hidden;font:3px/1 a}'].join(""),function(b){a=b.offsetHeight>=3}),a},s.video=function(){var a=b.createElement("video"),c=!1;try{if(c=!!a.canPlayType)c=new Boolean(c),c.ogg=a.canPlayType('video/ogg; codecs="theora"').replace(/^no$/,""),c.h264=a.canPlayType('video/mp4; codecs="avc1.42E01E"').replace(/^no$/,""),c.webm=a.canPlayType('video/webm; codecs="vp8, vorbis"').replace(/^no$/,"")}catch(d){}return c},s.audio=function(){var a=b.createElement("audio"),c=!1;try{if(c=!!a.canPlayType)c=new Boolean(c),c.ogg=a.canPlayType('audio/ogg; codecs="vorbis"').replace(/^no$/,""),c.mp3=a.canPlayType("audio/mpeg;").replace(/^no$/,""),c.wav=a.canPlayType('audio/wav; codecs="1"').replace(/^no$/,""),c.m4a=(a.canPlayType("audio/x-m4a;")||a.canPlayType("audio/aac;")).replace(/^no$/,"")}catch(d){}return c},s.localstorage=function(){try{return localStorage.setItem(h,h),localStorage.removeItem(h),!0}catch(a){return!1}},s.sessionstorage=function(){try{return sessionStorage.setItem(h,h),sessionStorage.removeItem(h),!0}catch(a){return!1}},s.webworkers=function(){return!!a.Worker},s.applicationcache=function(){return!!a.applicationCache},s.svg=function(){return!!b.createElementNS&&!!b.createElementNS(r.svg,"svg").createSVGRect},s.inlinesvg=function(){var a=b.createElement("div");return a.innerHTML="",(a.firstChild&&a.firstChild.namespaceURI)==r.svg},s.smil=function(){return!!b.createElementNS&&/SVGAnimate/.test(m.call(b.createElementNS(r.svg,"animate")))},s.svgclippaths=function(){return!!b.createElementNS&&/SVGClipPath/.test(m.call(b.createElementNS(r.svg,"clipPath")))};for(var L in s)C(s,L)&&(x=L.toLowerCase(),e[x]=s[L](),v.push((e[x]?"":"no-")+x));return e.input||K(),e.addTest=function(a,b){if(typeof a=="object")for(var d in a)C(a,d)&&e.addTest(d,a[d]);else{a=a.toLowerCase();if(e[a]!==c)return e;b=typeof b=="function"?b():b,typeof f!="undefined"&&f&&(g.className+=" "+(b?"":"no-")+a),e[a]=b}return e},D(""),i=k=null,function(a,b){function k(a,b){var c=a.createElement("p"),d=a.getElementsByTagName("head")[0]||a.documentElement;return c.innerHTML="x",d.insertBefore(c.lastChild,d.firstChild)}function l(){var a=r.elements;return typeof a=="string"?a.split(" "):a}function m(a){var b=i[a[g]];return b||(b={},h++,a[g]=h,i[h]=b),b}function n(a,c,f){c||(c=b);if(j)return c.createElement(a);f||(f=m(c));var g;return f.cache[a]?g=f.cache[a].cloneNode():e.test(a)?g=(f.cache[a]=f.createElem(a)).cloneNode():g=f.createElem(a),g.canHaveChildren&&!d.test(a)?f.frag.appendChild(g):g}function o(a,c){a||(a=b);if(j)return a.createDocumentFragment();c=c||m(a);var d=c.frag.cloneNode(),e=0,f=l(),g=f.length;for(;e",f="hidden"in a,j=a.childNodes.length==1||function(){b.createElement("a");var a=b.createDocumentFragment();return typeof a.cloneNode=="undefined"||typeof a.createDocumentFragment=="undefined"||typeof a.createElement=="undefined"}()}catch(c){f=!0,j=!0}})();var r={elements:c.elements||"abbr article aside audio bdi canvas data datalist details figcaption figure footer header hgroup mark meter nav output progress section summary time video",shivCSS:c.shivCSS!==!1,supportsUnknownElements:j,shivMethods:c.shivMethods!==!1,type:"default",shivDocument:q,createElement:n,createDocumentFragment:o};a.html5=r,q(b)}(this,b),e._version=d,e._prefixes=n,e._domPrefixes=q,e._cssomPrefixes=p,e.mq=z,e.hasEvent=A,e.testProp=function(a){return H([a])},e.testAllProps=J,e.testStyles=y,e.prefixed=function(a,b,c){return b?J(a,b,c):J(a,"pfx")},g.className=g.className.replace(/(^|\s)no-js(\s|$)/,"$1$2")+(f?" js "+v.join(" "):""),e}(this,this.document),function(a,b,c){function d(a){return"[object Function]"==o.call(a)}function e(a){return"string"==typeof a}function f(){}function g(a){return!a||"loaded"==a||"complete"==a||"uninitialized"==a}function h(){var a=p.shift();q=1,a?a.t?m(function(){("c"==a.t?B.injectCss:B.injectJs)(a.s,0,a.a,a.x,a.e,1)},0):(a(),h()):q=0}function i(a,c,d,e,f,i,j){function k(b){if(!o&&g(l.readyState)&&(u.r=o=1,!q&&h(),l.onload=l.onreadystatechange=null,b)){"img"!=a&&m(function(){t.removeChild(l)},50);for(var d in y[c])y[c].hasOwnProperty(d)&&y[c][d].onload()}}var j=j||B.errorTimeout,l=b.createElement(a),o=0,r=0,u={t:d,s:c,e:f,a:i,x:j};1===y[c]&&(r=1,y[c]=[]),"object"==a?l.data=c:(l.src=c,l.type=a),l.width=l.height="0",l.onerror=l.onload=l.onreadystatechange=function(){k.call(this,r)},p.splice(e,0,u),"img"!=a&&(r||2===y[c]?(t.insertBefore(l,s?null:n),m(k,j)):y[c].push(l))}function j(a,b,c,d,f){return q=0,b=b||"j",e(a)?i("c"==b?v:u,a,b,this.i++,c,d,f):(p.splice(this.i++,0,a),1==p.length&&h()),this}function k(){var a=B;return a.loader={load:j,i:0},a}var l=b.documentElement,m=a.setTimeout,n=b.getElementsByTagName("script")[0],o={}.toString,p=[],q=0,r="MozAppearance"in l.style,s=r&&!!b.createRange().compareNode,t=s?l:n.parentNode,l=a.opera&&"[object Opera]"==o.call(a.opera),l=!!b.attachEvent&&!l,u=r?"object":l?"script":"img",v=l?"script":u,w=Array.isArray||function(a){return"[object Array]"==o.call(a)},x=[],y={},z={timeout:function(a,b){return b.length&&(a.timeout=b[0]),a}},A,B;B=function(a){function b(a){var a=a.split("!"),b=x.length,c=a.pop(),d=a.length,c={url:c,origUrl:c,prefixes:a},e,f,g;for(f=0;f"),i("table.docutils.footnote").wrap("
"),i("table.docutils.citation").wrap("
"),i(".wy-menu-vertical ul").not(".simple").siblings("a").each(function(){var e=i(this);expand=i(''),expand.on("click",function(n){return t.toggleCurrent(e),n.stopPropagation(),!1}),e.prepend(expand)})},reset:function(){var n=encodeURI(window.location.hash)||"#";try{var e=$(".wy-menu-vertical"),i=e.find('[href="'+n+'"]');if(0===i.length){var t=$('.document [id="'+n.substring(1)+'"]').closest("div.section");0===(i=e.find('[href="#'+t.attr("id")+'"]')).length&&(i=e.find('[href="#"]'))}0this.docHeight||(this.navBar.scrollTop(i),this.winPosition=n)},onResize:function(){this.winResize=!1,this.winHeight=this.win.height(),this.docHeight=$(document).height()},hashChange:function(){this.linkScroll=!0,this.win.one("hashchange",function(){this.linkScroll=!1})},toggleCurrent:function(n){var e=n.closest("li");e.siblings("li.current").removeClass("current"),e.siblings().find("li.current").removeClass("current"),e.find("> ul li.current").removeClass("current"),e.toggleClass("current")}},"undefined"!=typeof window&&(window.SphinxRtdTheme={Navigation:e.exports.ThemeNav,StickyNav:e.exports.ThemeNav}),function(){for(var r=0,n=["ms","moz","webkit","o"],e=0;e= 0) + return; + function pulse() { + var i; + Search._pulse_status = (Search._pulse_status + 1) % 4; + var dotString = ''; + for (i = 0; i < Search._pulse_status; i++) + dotString += '.'; + Search.dots.text(dotString); + if (Search._pulse_status > -1) + window.setTimeout(pulse, 500); + } + pulse(); + }, + + /** + * perform a search for something (or wait until index is loaded) + */ + performSearch : function(query) { + // create the required interface elements + this.out = $('#search-results'); + this.title = $('

' + _('Searching') + '

').appendTo(this.out); + this.dots = $('').appendTo(this.title); + this.status = $('

').appendTo(this.out); + this.output = $('