Commit Graph

100 Commits

Author SHA1 Message Date
David Lyle d8a8d77d1b Adding charting support for multinode 2019-08-02 17:39:44 +01:00
David Lyle 510dd83b09 adding multinode utilization tracking 2019-08-02 17:39:44 +01:00
Graham Whaley 42359e1f8e metrics: scaling: store json int results as non-strings
We were storing some JSON data (such as NUM_PODS) as strings.
This would default their type into the Elastic DB as strings,
and not ints. May as well store ints as ints, so make that so.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-07-30 13:13:35 -06:00
Graham Whaley 5c9a5df264 metrics: report: Fix scaling points for boot chart
The boot 'points' code was accidentally updating the cpu plot
(copy and paste error). Correct it to update the correct chart.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-07-30 13:13:35 -06:00
Graham Whaley e73beb18e1 metrics: parallel: Test deployment parallel pod scaling
Test how long it takes to launch, and delete, a deployment
running up 'n' pods. Used to ascertain if parallel launch times
are linear with number of pods or not.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-07-30 13:13:35 -06:00
Graham Whaley 59a1f36e37 metrics: common: remove un-used docker-esque code
We were carrying a bunch of legacy docker based functions
that we do not use. Drop them. We can always add them back
in later if we find we do need some.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-07-30 13:13:35 -06:00
Graham Whaley fa94c2ce90 metrics: api: move some api code out to a common file
We are very likely to re-use the k8s api code in many/all of the
tests, so move common code out to a common libray file.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-07-30 13:13:35 -06:00
Graham Whaley 293a2b32d2 metrics: json: add k8s info sections
Capture some of the k8s setup information in the JSON results
files. Add sections for:
 - kubectl version
 - kubectl get nodes

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-07-30 13:13:35 -06:00
Graham Whaley bead9c7dfc metrics: use RUNTIME to determine RuntimeClass
If we have $RUNTIME set in the env, then we set up the pod
RuntimeClass, otherwise, just use the default node runtimes.

Clean up some of the un-used library code around docker/RUNTIME
as well whilst we are there.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-07-30 13:13:35 -06:00
Graham Whaley 6e238d7cd9 metrics: scaling: change check for 'first deploy' to non-1 indexed
We need to check when we are launching our first instance of a
deployment when using the k8s API directly, as we have to modify
the URL for the first instance. Rather than check for '1', let's
check for 'STEP', as technically you can run the loop starting at
a non-1 index (particularly useful to run with NUM_PODS==STEP, as
you then get a 'single shot' test of 'n' containers).

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-07-30 11:51:06 -07:00
David Lyle c4b6824f07 Fixing proxy process management 2019-07-26 16:14:55 +01:00
David Lyle d7d40598b4 Add option to use API to launch 2019-07-26 16:14:55 +01:00
Graham Whaley 351540049d metrics: Add the start of the local report generator
Add the first basic report generator code, utilising R, Rmarkdown
and pandoc to do the data processing and pdf report generation.

Fixes: #121

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-07-24 10:52:13 -06:00
David Lyle 0f39518f8d Moving to milliseconds rather than seconds 2019-07-24 17:46:21 +01:00
David Lyle c67566e41e Initial commit of metrics for scaling 2019-07-24 17:46:21 +01:00
Ganesh Maharaj Mahalingam 39a3f46ec8 Add swupd update back
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
v1.6
2019-07-22 14:00:48 -07:00
NitinAtIntel 0e27cf06a3 As scripts will be distributed with bundle in clear, user can control their clear version if using scripts from git. 2019-07-22 10:35:37 -07:00
NitinAtIntel 41e9b05cc0 Enabling exit on error at top of script. Switched method to obtain k8s source with check for existing repo. Switched version check to look at client version as we want to test using same client k8s version as came bundled 2019-07-22 10:35:37 -07:00
NitinAtIntel 7106afcd9f Adding a script to execute upstream k8s e2e tests on the cloud native stack 2019-07-22 10:35:37 -07:00
Ganesh Maharaj Mahalingam 859da50c94 Move to containerd as the default manager
This setup scripts have been using cri-o all this while as the pod
controller/manager system. Recent past cri-o has had some issues with
kata-deploy (A restart of the service will not be able to re-connect
with the existing pods and restart all of them including kata-deploy,
which will hit an endless loop). Moving to containerd as default for
now.

Firecracker cannot be used with a released version of containerd as
there is no default block based snapshotter available today. If you wish
to use cri-o make sure you set `RUNNER=crio` in your environment prior
to using the script.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2019-07-19 17:32:01 -07:00
Manohar Castelino 0ffacbd904 Versioning: Update clearlinux version to 30270
Clearlinux release 30270 is now a known good version compatible
with kubernetes.

RuntimeName:  cri-o
RuntimeVersion:  1.14.4
RuntimeApiVersion:  v1alpha1

Kubernetes v1.15.0

runc version 1.0.0-rc5
spec: 1.0.0

systemd 242 (242)
+PAM +AUDIT -SELINUX +IMA -APPARMOR -SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 -IDN -PCRE2 default-hierarchy=legacy

Kernel: 4.19.57-60.lts2018

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2019-07-10 17:09:45 -07:00
Saikrishna Edupuganti bbeb447a8a Pinning to working box version and clear version
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-06-25 10:04:49 -07:00
Saikrishna Edupuganti 8ab98c53db Configure canal to ignore rpf instead of host
Modifying rp_filter setting on host seems to be causing a routing issue
when connecting to a pod from remote node. Until we figure out the best
way to solve the issue, we are configuring canal to ignore default
setting of 2

Fixes: #102

Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-06-20 17:03:48 -07:00
Ganesh Maharaj Mahalingam 0e280f40fa cri-o update fixes
ClearLinux now has cri-o version 1.14.1 which allows multiple plugin
locations and also creates /opt/cni/bin by default. We no longer need
the hacks for them

Fixes: #82
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
v1.5
2019-06-04 11:53:36 -07:00
Ganesh Maharaj Mahalingam d31d78c193 calico-node fails to come up cause of all.rp_filter
Currently net.ipv4.conf.all.rp_filter is set to 2 in Clear and
calico-node fails to come up unless that value is either 0 or 1.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2019-05-30 14:17:44 -07:00
Jose Carlos Venegas Munoz 04487a2cfe reset_stack: stop containerd and crio
- If stop CRI service if is running
- Restart the CRI service only if enabled
- Do not enable crio on reset_stack.sh this should be only part of the
setup.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-05-29 18:20:33 -07:00
Jose Carlos Venegas Munoz e67630b0bd create_stack: quote variables to avoid code expands
General fixes make my vim mark less warnings from shellcheck

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-05-24 11:40:49 -07:00
Jose Carlos Venegas Munoz e08c92f6d5 stack: do not provide CRI socket
kubeadm autodetects the socket path based on defaults from well known
CRI servers.

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-05-24 11:37:08 -07:00
Jose Carlos Venegas Munoz d00a5ea9d8 README: update flavor setup information.
Add docuementation to use ./clr-k8s-examples/reset_stack.sh help

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-05-23 20:38:45 -07:00
Jose Carlos Venegas Munoz a9b3b5c506 create_stack: Add init and cni subcommands
Add more subcommands to to increase granularity

init: start cluster
cni: setup network

This functionality already existed, we only handle
subcommands in a more dynamic way.

Fixes: #92

Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
2019-05-23 18:56:59 -07:00
Ganesh Maharaj Mahalingam 24f248b02f Set reverse path forwarding to strict.
Calico requires the default reverse path forwarding to be either 0 (no
validation) or 1 (strict validation). The default value of 2 (loose
validation) prevents calico from completing the setup and the pod is
always stuck getting ready.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
v1.4
2019-05-18 14:13:13 -07:00
Ganesh Maharaj Mahalingam d688cbb693 Download OVMF if it doesn't exist
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2019-05-01 09:57:04 -07:00
Ganesh Maharaj Mahalingam 4e82cf411d Fix readme to add k8s version dependency
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
v1.3
2019-04-04 10:56:23 -07:00
Saikrishna Edupuganti af56d51499 Remove applying runtimeclass CRD
CRD was removed as part of 1.14 update. Missed update to create_stack

Fixes: #84

Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-04-04 10:01:36 -07:00
Eric Ernst f863bb7f1f runtimeClass updates for kubernetes 1.14
RuntimeClass moved from alpha to beta. Remove the feature gates, remove
the runtimeClass CRD since it is a built in type, and take the new data
structure into account.

Fixes: #77

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2019-04-03 11:23:40 -07:00
Eric Ernst d0c7bb3513 kata-deploy: update to match what's on Kata master
We updated to use containerd v2 shim - pull in latest from master

Signed-off-by: Eric Ernst <eric.ernst@intel.com>
2019-03-28 13:09:39 -07:00
Saikrishna Edupuganti 4f38e14245 Turn on static policy for CPU manager
This is finally supported in Kata 1.6
Update memory limit for kata to run in guaranteed test case

Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-03-27 10:27:12 -07:00
Saikrishna Edupuganti 3b7e7f75d1 Updated top-level README to point to admit-kata
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-03-27 10:26:54 -07:00
Saikrishna Edupuganti 39b454045d Copied the README too
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-03-27 10:26:54 -07:00
Saikrishna Edupuganti fe57aa93ff Upstream artifact for kata admission controller
Get upstream artifacts of kata admission controller from
https://github.com/kata-containers/tests/tree/master/kata-webhook
Commit: 5ad2cec

Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-03-27 10:26:54 -07:00
Saikrishna Edupuganti 0417be1cdd Update README.md
Add note about environment variables that can be used to customize the setup
2019-03-26 16:35:25 -07:00
Ganesh Maharaj Mahalingam 02750e991c Make firecracker setup configurable
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2019-03-22 17:11:57 -07:00
Saikrishna Edupuganti dc345d4bdf Do not restart containerd, remove stop kubelet
In case of containerd there is no need to restart. Restart for crio is needed
when it manages the networks ns lifecycle. Removed the kubelet stop step which
was a leftover from earlier device plugin install.

Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-03-20 13:23:16 -07:00
Manohar Castelino 935bb69cfd Eliminate legacy annotation
We only support kubernetes versions with support for runtimeClass.
Remove the legacy annotations to reduce confusion.

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2019-03-14 10:03:37 -07:00
Manohar Castelino e5c55c3c0d Update webhook setup and documentation
The admission controller is now able to access full context.
Hence explicit tagging of pod manifests to exclude them from
kata is no longer required.

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2019-03-14 10:03:37 -07:00
Manohar Castelino a6d2f487d8 Disable proxy
Disable proxy setup if the host does not have proxy setup in
the enviornment variables

Signed-off-by: Manohar Castelino <manohar.r.castelino@intel.com>
2019-03-12 09:57:44 -07:00
Saikrishna Edupuganti ed9f71ee92 Update README.md 2019-03-10 09:23:50 -07:00
Saikrishna Edupuganti c2e932ac7b Hotfix for rsync update in new clear (box?)
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-02-26 17:31:55 -08:00
Saikrishna Edupuganti 13226c0c72 Update README.md 2019-02-22 19:35:41 -08:00
Saikrishna Edupuganti be17bc3ab7 Use kata-deploy to install and configure kata
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2019-02-22 19:34:49 -08:00