Files
2024-11-04 18:56:31 +00:00

409 lines
23 KiB
HTML
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html>
<html lang="en" data-content_root="../../">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Kubernetes* migration &#8212; Documentation for Clear Linux* project</title>
<link rel="stylesheet" type="text/css" href="../../_static/pygments.css?v=fa44fd50" />
<link rel="stylesheet" type="text/css" href="../../_static/bizstyle.css?v=5283bb3d" />
<link rel="stylesheet" type="text/css" href="../../_static/copybutton.css?v=76b2166b" />
<script src="../../_static/documentation_options.js?v=5929fcd5"></script>
<script src="../../_static/doctools.js?v=9bcbadda"></script>
<script src="../../_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="../../_static/clipboard.min.js?v=a7894cd8"></script>
<script src="../../_static/copybutton.js?v=a56c686a"></script>
<script src="../../_static/bizstyle.js"></script>
<link rel="canonical" href="https://clearlinux.github.io/clear-linux-documentation/guides/clear/k8s-migration.html" />
<link rel="icon" href="../../_static/favicon.ico"/>
<link rel="author" title="About these documents" href="../../about.html" />
<link rel="index" title="Index" href="../../genindex.html" />
<link rel="search" title="Search" href="../../search.html" />
<link rel="next" title="mixer" href="mixer.html" />
<link rel="prev" title="Debug system" href="debug.html" />
<meta name="viewport" content="width=device-width,initial-scale=1.0" />
<!--[if lt IE 9]>
<script src="_static/css3-mediaqueries.js"></script>
<![endif]-->
</head><body>
<div class="related" role="navigation" aria-label="Related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="mixer.html" title="mixer"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="debug.html" title="Debug system"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">Documentation for Clear Linux* project</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" accesskey="U">Guides</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Kubernetes* migration</a></li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<section id="kubernetes-migration">
<span id="id1"></span><h1>Kubernetes* migration<a class="headerlink" href="#kubernetes-migration" title="Link to this heading"></a></h1>
<p>This guide describes how to migrate <a class="reference external" href="https://kubernetes.io/">Kubernetes container orchestration system</a> on Clear Linux* OS from 1.17.x to 1.19.x.</p>
<nav class="contents local" id="contents">
<ul class="simple">
<li><p><a class="reference internal" href="#background" id="id3">Background</a></p></li>
<li><p><a class="reference internal" href="#prerequisites" id="id4">Prerequisites</a></p></li>
<li><p><a class="reference internal" href="#upgrade-1-17-x-1-18-15" id="id5">Upgrade 1.17.x —&gt; 1.18.15</a></p></li>
<li><p><a class="reference internal" href="#upgrade-1-18-15-1-19-x" id="id6">Upgrade 1.18.15 —&gt; 1.19.x</a></p></li>
<li><p><a class="reference internal" href="#related-topics" id="id7">Related topics</a></p></li>
</ul>
</nav>
<section id="background">
<h2><a class="toc-backref" href="#id3" role="doc-backlink">Background</a><a class="headerlink" href="#background" title="Link to this heading"></a></h2>
<p>The version of Kubernetes* was bumped from 1.17.7 to 1.19.4 in Clear Linux* OS
release 34090. This guide and the Clear Linux OS bundle <cite>k8s-migration</cite> were created
to help facilitate migration of a cluster from 1.17.x to the latest 1.19.x .</p>
<p>The new Clear Linux OS bundle <cite>k8s-migration</cite> was added in Clear Linux* OS release 34270.</p>
</section>
<section id="prerequisites">
<h2><a class="toc-backref" href="#id4" role="doc-backlink">Prerequisites</a><a class="headerlink" href="#prerequisites" title="Link to this heading"></a></h2>
<ul class="simple">
<li><p>Make sure you check any updates to kubernetes upgrade doc for caveats related to the version that is running in the cluster.</p></li>
<li><p>Make sure ALL the nodes are in Ready state. Without that, the cluster cannot be upgraded.
Either fix the broken nodes or remove them from the cluster.</p></li>
</ul>
</section>
<section id="upgrade-1-17-x-1-18-15">
<h2><a class="toc-backref" href="#id5" role="doc-backlink">Upgrade 1.17.x —&gt; 1.18.15</a><a class="headerlink" href="#upgrade-1-17-x-1-18-15" title="Link to this heading"></a></h2>
<ol class="arabic">
<li><p>Upgrade Control Node to 1.18.15 first</p>
<p>First step would be to upgrade one of the main control node and
update kubernetes components on them. You will need to have a newer
version of <strong class="command">kubeadm</strong> for the upgrade to work. Please consult
<a class="reference external" href="https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/">kubeadm upgrade guide</a>
for any caveats from your current version to the new one.</p>
<p>Update Clear Linux OS to the latest release to update the kubernetes version.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>swupd<span class="w"> </span>update
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Note: PLEASE DO NOT REBOOT YOUR SYSTEM AT THIS TIME. Clear Linux OS is awesome and
your stuff will work just fine.</p>
</div>
</li>
<li><p>Add the new Kubernetes migration bundle which contains the 1.18.15 binaries.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>swupd<span class="w"> </span>bundle-add<span class="w"> </span>k8s-migration
</pre></div>
</div>
</li>
<li><p>Find the upgrade version of kubeadm that can used. This should be 1.18.15.</p>
<p>This command will show the command and possible jumps that can be made from the current kubernetes version.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>/usr/k8s-migration/bin/kubeadm<span class="w"> </span>upgrade<span class="w"> </span>plan
</pre></div>
</div>
<p>Sample output:</p>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="go">[upgrade/config] Making sure the configuration is correct:</span>
<span class="go">[upgrade/config] Reading configuration from the cluster...</span>
<span class="go">[upgrade/config] FYI: You can look at this config file with &#39;kubectl -n kube-system get cm kubeadm-config -oyaml&#39;</span>
<span class="go">[preflight] Running pre-flight checks.</span>
<span class="go">[upgrade] Running cluster health checks</span>
<span class="go">[upgrade] Fetching available versions to upgrade to</span>
<span class="go">[upgrade/versions] Cluster version: v1.17.17</span>
<span class="go">[upgrade/versions] kubeadm version: v1.18.15</span>
<span class="go">I0209 21:12:49.868786 832739 version.go:252] remote version is much newer: v1.20.2; falling back to: stable-1.18</span>
<span class="go">[upgrade/versions] Latest stable version: v1.18.15</span>
<span class="go">[upgrade/versions] Latest stable version: v1.18.15</span>
<span class="go">[upgrade/versions] Latest version in the v1.17 series: v1.17.17</span>
<span class="go">[upgrade/versions] Latest version in the v1.17 series: v1.17.17</span>
<span class="go">Components that must be upgraded manually after you have upgraded the control plane with &#39;kubeadm upgrade apply&#39;:</span>
<span class="go">COMPONENT CURRENT AVAILABLE</span>
<span class="go">Kubelet 3 x v1.17.7 v1.18.15</span>
<span class="go">Upgrade to the latest stable version:</span>
<span class="go">COMPONENT CURRENT AVAILABLE</span>
<span class="go">API Server v1.17.17 v1.18.15</span>
<span class="go">Controller Manager v1.17.17 v1.18.15</span>
<span class="go">Scheduler v1.17.17 v1.18.15</span>
<span class="go">Kube Proxy v1.17.17 v1.18.15</span>
<span class="go">CoreDNS 1.6.5 1.6.7</span>
<span class="go">Etcd 3.4.3 3.4.3-0</span>
<span class="go">You can now apply the upgrade by executing the following command:</span>
<span class="go"> kubeadm upgrade apply v1.18.15</span>
</pre></div>
</div>
</li>
<li><p>Upgrade the node to the intermediate 1.18.15 version of Kubernetes.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>/usr/k8s-migration/bin/kubeadm<span class="w"> </span>upgrade<span class="w"> </span>apply<span class="w"> </span>v1.18.15
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Note: Do <strong>not</strong> reboot the system yet.</p>
</div>
</li>
<li><p>Upgrade Additional Control Nodes to 1.18.15</p>
<p>In multi-node control plane, verify all the control plane nodes are updated prior to upgrading the worker nodes/SUTs.</p>
</li>
<li><p>Upgrade Other Nodes to 1.18.15</p>
<p>For each of the other nodes:</p>
<ol class="loweralpha">
<li><p>Update Clear Linux OS to the latest release to update the kubernetes version.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>swupd<span class="w"> </span>update
</pre></div>
</div>
</li>
<li><p>Add the new Kubernetes migration bundle which contains the 1.18.15 binaries.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>swupd<span class="w"> </span>bundle-add<span class="w"> </span>k8s-migration
</pre></div>
</div>
</li>
<li><p>On the <strong>Admin node</strong>, drain the Client node <em>FIRST</em></p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>/usr/k8s-migration/bin/kubectl<span class="w"> </span>drain<span class="w"> </span>&lt;CLIENT_NODE_NAME&gt;<span class="w"> </span>--ignore-daemonsets<span class="w"> </span>--delete-local-data
</pre></div>
</div>
</li>
<li><p>Back on the <strong>Client node</strong>, upgrade Kubernetes on the Client</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>/usr/k8s-migration/bin/kubeadm<span class="w"> </span>upgrade<span class="w"> </span>node
</pre></div>
</div>
</li>
<li><p>On the <strong>Admin node</strong>, re-enable the Client</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>/usr/k8s-migration/bin/kubectl<span class="w"> </span>uncordon<span class="w"> </span>&lt;CLIENT_NODE_NAME&gt;
</pre></div>
</div>
</li>
<li><p>Back on the <strong>Client node</strong>, restart Kubernetes on the Client</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>systemctl<span class="w"> </span>restart<span class="w"> </span>kubelet
</pre></div>
</div>
</li>
</ol>
</li>
<li><p>Restart Kubernetes on the Admin node(s) to finish the 1.18.x upgrade</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>systemctl<span class="w"> </span>restart<span class="w"> </span>kubelet
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Note: Wait for all nodes to be Ready and showing the 1.19.x version.
This version will now show as it is the released version the
service files will see and use, but the Nodes are <em>not</em> upgraded yet.</p>
</div>
</li>
</ol>
</section>
<section id="upgrade-1-18-15-1-19-x">
<h2><a class="toc-backref" href="#id6" role="doc-backlink">Upgrade 1.18.15 —&gt; 1.19.x</a><a class="headerlink" href="#upgrade-1-18-15-1-19-x" title="Link to this heading"></a></h2>
<ol class="arabic">
<li><p>Upgrade Control Node to 1.19.x</p>
<p>Now that systems are upgraded to the intermediate release of 1.18.15
each of the nodes can be upgraded to the latest 1.19.x release.</p>
</li>
<li><p>Find the upgrade version of kubeadm that can used. This should be 1.19.x.</p>
<p>This command will show the command and possible jumps that can be made from the current kubernetes version.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>kubeadm<span class="w"> </span>upgrade<span class="w"> </span>plan
</pre></div>
</div>
<p>Sample output:</p>
<div class="highlight-console notranslate"><div class="highlight"><pre><span></span><span class="go">[upgrade/config] Making sure the configuration is correct:</span>
<span class="go">[upgrade/config] Reading configuration from the cluster...</span>
<span class="go">[upgrade/config] FYI: You can look at this config file with &#39;kubectl -n kube-system get cm kubeadm-config -oyaml&#39;</span>
<span class="go">[preflight] Running pre-flight checks.</span>
<span class="go">[upgrade] Running cluster health checks</span>
<span class="go">[upgrade] Fetching available versions to upgrade to</span>
<span class="go">[upgrade/versions] Cluster version: v1.18.15</span>
<span class="go">[upgrade/versions] kubeadm version: v1.19.7</span>
<span class="go">I0209 23:08:23.810900 925910 version.go:252] remote version is much newer: v1.20.2; falling back to: stable-1.19</span>
<span class="go">[upgrade/versions] Latest stable version: v1.19.7</span>
<span class="go">[upgrade/versions] Latest stable version: v1.19.7</span>
<span class="go">[upgrade/versions] Latest version in the v1.18 series: v1.18.15</span>
<span class="go">[upgrade/versions] Latest version in the v1.18 series: v1.18.15</span>
<span class="go">Components that must be upgraded manually after you have upgraded the control plane with &#39;kubeadm upgrade apply&#39;:</span>
<span class="go">COMPONENT CURRENT AVAILABLE</span>
<span class="go">kubelet 3 x v1.17.7 v1.19.7</span>
<span class="go">Upgrade to the latest stable version:</span>
<span class="go">COMPONENT CURRENT AVAILABLE</span>
<span class="go">kube-apiserver v1.18.15 v1.19.7</span>
<span class="go">kube-controller-manager v1.18.15 v1.19.7</span>
<span class="go">kube-scheduler v1.18.15 v1.19.7</span>
<span class="go">kube-proxy v1.18.15 v1.19.7</span>
<span class="go">CoreDNS 1.6.7 1.7.0</span>
<span class="go">etcd 3.4.3-0 3.4.13-0</span>
<span class="go">You can now apply the upgrade by executing the following command:</span>
<span class="go"> kubeadm upgrade apply v1.19.7</span>
<span class="go">The table below shows the current state of component configs as understood by this version of kubeadm.</span>
<span class="go">Configs that have a &quot;yes&quot; mark in the &quot;MANUAL UPGRADE REQUIRED&quot; column require manual config upgrade or</span>
<span class="go">resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually</span>
<span class="go">upgrade to is denoted in the &quot;PREFERRED VERSION&quot; column.</span>
<span class="go">API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED</span>
<span class="go">kubeproxy.config.k8s.io v1alpha1 v1alpha1 no</span>
<span class="go">kubelet.config.k8s.io v1beta1 v1beta1 no</span>
</pre></div>
</div>
</li>
<li><p>Upgrade the node to the latest 1.19.x version of Kubernetes.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>/usr/bin/kubeadm<span class="w"> </span>upgrade<span class="w"> </span>apply<span class="w"> </span>v1.19.7
</pre></div>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Note: Do <strong>not</strong> reboot the system yet.</p>
</div>
</li>
<li><p>Upgrade Additional Control Nodes to 1.19.x</p>
<p>In multi-node control plane, verify all the control plane nodes are updated prior to upgrading the worker nodes/SUTs.</p>
</li>
<li><p>Upgrade Other Nodes to 1.19.x</p>
<p>For each of the other nodes:</p>
<ol class="loweralpha">
<li><p>On the <strong>Admin node</strong>, drain the Client <em>FIRST</em></p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kubectl<span class="w"> </span>drain<span class="w"> </span>&lt;CLIENT_NODE_NAME&gt;<span class="w"> </span>--ignore-daemonsets
</pre></div>
</div>
</li>
<li><p>Back on the <strong>Client node</strong>, upgrade Kubernetes on the Client</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>kubeadm<span class="w"> </span>upgrade<span class="w"> </span>node
</pre></div>
</div>
</li>
<li><p>On the <strong>Admin node</strong>, re-enable the Client</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kubectl<span class="w"> </span>uncordon<span class="w"> </span>&lt;CLIENT_NODE_NAME&gt;
</pre></div>
</div>
</li>
<li><p>Back on the <strong>Client node</strong>, if you wish reboot the Client, it is now safe to do so.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>reboot
</pre></div>
</div>
</li>
</ol>
</li>
<li><p>Reboot the Control Node (optional)</p></li>
</ol>
<blockquote>
<div><p><em>If you wish reboot the nodes, it is now safe to do so.</em></p>
<blockquote>
<div><div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>reboot
</pre></div>
</div>
</div></blockquote>
</div></blockquote>
<p><strong>Congratulations!</strong></p>
<p>Youve successfully installed and set up Kubernetes in Clear Linux OS using CRI-O and kata-runtime. You are now ready to follow on-screen instructions to deploy a pod network to the cluster and join worker nodes with the displayed token and IP information.</p>
<p>Clean up: Remove the migration bundle for each node</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>sudo<span class="w"> </span>-E<span class="w"> </span>swupd<span class="w"> </span>bundle-remove<span class="w"> </span>k8s-migration
</pre></div>
</div>
</section>
<section id="related-topics">
<h2><a class="toc-backref" href="#id7" role="doc-backlink">Related topics</a><a class="headerlink" href="#related-topics" title="Link to this heading"></a></h2>
<p>Read the Kubernetes documentation to learn more about:</p>
<ul class="simple">
<li><p><a class="reference external" href="tutorials/kubernetes">Kubernetes tutorial</a></p></li>
<li><p><a class="reference external" href="tutorials/kubernetes-bp">Kubernetes best practices</a></p></li>
<li><p>Deploying Kubernetes with a <a class="reference external" href="https://github.com/clearlinux/cloud-native-setup/tree/master/clr-k8s-examples">cloud-native-setup</a></p></li>
<li><p><a class="reference external" href="https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-3">Understanding basic Kubernetes architecture</a></p></li>
<li><p><a class="reference external" href="https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-2">Deploying an application to your cluster</a></p></li>
<li><p>Installing a <a class="reference external" href="https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network">pod network add-on</a></p></li>
<li><p><a class="reference external" href="https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#join-nodes">Joining your nodes</a></p></li>
</ul>
</section>
</section>
<div class="clearer"></div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="Main">
<div class="sphinxsidebarwrapper">
<p class="logo"><a href="../../index.html">
<img class="logo" src="../../_static/clearlinux.png" alt="Logo of Clear Linux* Project Docs"/>
</a></p>
<div>
<h3><a href="../../index.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Kubernetes* migration</a><ul>
<li><a class="reference internal" href="#background">Background</a></li>
<li><a class="reference internal" href="#prerequisites">Prerequisites</a></li>
<li><a class="reference internal" href="#upgrade-1-17-x-1-18-15">Upgrade 1.17.x —&gt; 1.18.15</a></li>
<li><a class="reference internal" href="#upgrade-1-18-15-1-19-x">Upgrade 1.18.15 —&gt; 1.19.x</a></li>
<li><a class="reference internal" href="#related-topics">Related topics</a></li>
</ul>
</li>
</ul>
</div>
<div>
<h4>Previous topic</h4>
<p class="topless"><a href="debug.html"
title="previous chapter">Debug system</a></p>
</div>
<div>
<h4>Next topic</h4>
<p class="topless"><a href="mixer.html"
title="next chapter">mixer</a></p>
</div>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../../_sources/guides/clear/k8s-migration.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<search id="searchbox" style="display: none" role="search">
<h3 id="searchlabel">Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
<input type="submit" value="Go" />
</form>
</div>
</search>
<script>document.getElementById('searchbox').style.display = "block"</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="Related">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="mixer.html" title="mixer"
>next</a> |</li>
<li class="right" >
<a href="debug.html" title="Debug system"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../../index.html">Documentation for Clear Linux* project</a> &#187;</li>
<li class="nav-item nav-item-1"><a href="../index.html" >Guides</a> &#187;</li>
<li class="nav-item nav-item-this"><a href="">Kubernetes* migration</a></li>
</ul>
</div>
<div class="footer" role="contentinfo">
&#169; Copyright 2022 Intel Corporation. All Rights Reserved..
Last updated on Nov 04, 2024.
Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 8.1.3.
</div>
</body>
</html>