Commit Graph

177 Commits

Author SHA1 Message Date
Julio Rivera 276ff274db Kustomize kubeadm.yaml config file (#193)
Signed-off-by: Rivera Gonzalez, Julio C <julio.c.rivera.gonzalez@intel.com>
2019-10-11 11:02:12 -07:00
Justin Scott a7f92a918b Update rook to v1.1.1 (#199)
Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-10-09 10:52:02 -07:00
Syed Ahsan ebff530fa4 Vagrant Libvirt Instructions update (#189)
* Vagrant Libvirt Instructions update

This patch adds the users to the default group so we dont
have to use sudo to start the vagrant, as using sudo is not
the most optimal route for gems to be properly configured.

Signed-off: Syed Ahsan<ahsanmohsin04@yahoo.com>
2019-10-08 11:10:50 -07:00
Syed Ahsan 859237b189 Setting the RUNNER to use crio (#208)
This change helps with setting up the RUNNER to use crio
by default instead of using containerd. If the user wants
to use containerd, they need to specify when running the
setup_system script.

Signed-off: Syed Ahsan <syed.ahsan.shamim.zaidi@intel.com>
2019-10-08 11:05:21 -07:00
Syed Ahsan Shamim Zaidi d95d82e102 Typo in vagrant instruction
This patch fixes the typo in vagrant instructions, it should be
CPUS.

Signed-off: Syed Ahsan <syed.ahsan.shamim.zaidi@intel.com>
2019-10-08 09:24:45 -06:00
Antti Kervinen 0325a44d6f scaling: fix syntax error in results/k8s-scaling-nc.json
Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
2019-10-07 11:21:04 -06:00
David Lyle 11c6d011cf global variables in function aren't global
A recent change put global variable declarations in a function,
the variables no longer are globally visible. This change removes
the encapsulation in a function.
2019-10-03 17:01:22 -05:00
Justin Scott 7c352427a2 Add section about code conventions to DEVELOP.md (#185)
This adds details about shfmt and shellcheck settings.

Closes #180

Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-10-02 11:19:22 -07:00
Justin Scott 5c8f4e89a1 Update Prometheus to v0.2.0 (#196)
Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-10-02 11:13:03 -07:00
Antti Kervinen 6213a499c1 metrics: scaling: add pod response latency test
- Send n requests for each of the running pods.
- Measure response time to response.
- Report response time percentiles: 5-25-50-75-95 %.
- Refactor scaling/*.sh variables to scaling/common.bash.

Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
2019-10-01 14:30:33 -06:00
Mikko Ylinen 2332542376 kubelet: use systemd cgroup driver (#192)
Fixes: clearlinux/distribution/issues/1267 (partial)

Signed-off-by: Mikko Ylinen <mikko.ylinen@intel.com>
2019-09-30 16:11:32 -07:00
Marcela Morales Q 38d8d058d9 Add main README file for cloud-native-setup repository. (#195)
Signed-off-by: Morales Quispe, Marcela <marcela.morales.quispe@intel.com>
2019-09-30 10:12:06 -07:00
Morales Quispe, Marcela 908a9f72f5 Add step-by-step guide for scaling execution into README file.
Signed-off-by: Morales Quispe, Marcela <marcela.morales.quispe@intel.com>
2019-09-27 20:51:30 -05:00
Justin Scott 6170fe3d3b Update metrics to v0.3.5 (#191)
Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-09-26 11:56:53 -07:00
Justin Scott d9f5318d4d Update canal to v3.9 (#187)
This replaces previous canal v3.3 which is no available.

Closes #186

Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-09-25 16:31:23 -07:00
Syed Ahsan Shamim Zaidi 1f9a8cf3ef Adding instructions to modify CPU/MEMORY for VM's
This change add's a note to specify env vars to use desired
CPU and Memory for individual VM's as required

Signed-off: Syed Ahsan<syed.ahsan.shamim.zaidi@intel.com>
2019-09-25 17:15:44 -06:00
Justin Scott 4dd5d51248 Update kata to v1.8.2 (#184)
Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-09-19 19:16:13 -07:00
Justin Scott 651510a51d Update metallb to v0.8.1 (#183)
Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-09-19 11:18:27 -07:00
Justin Scott 48ba56142d Update ingress-nginx to v0.25.1 (#182)
Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-09-19 10:59:04 -07:00
Justin Scott bd6ac270e6 Update to rook v1.1.0 (#169)
Update to rook v1.1.0 which has CSI support by default.

Closes #30

Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-09-19 09:40:56 -07:00
David Lyle dc48b000d5 fixing timing of kubelet configuration 2019-09-13 17:36:38 -05:00
David Lyle 43a402e45b splitting config to appropriate script 2019-09-13 17:36:38 -05:00
David Lyle b476b5d6e5 Moving from metrics config to setup_system.sh 2019-09-13 17:36:38 -05:00
David Lyle 332ec89af9 Adding node-config.sh
This script configures a node in the cluster for pod scaling
testing. This configuration is necessary for pushing beyond the
110 pod max default in Kubernetes.
2019-09-13 17:36:38 -05:00
David Lyle ccddc591dc Adding metric for inode usage
Adds support for tracking inode usage on the nodes. Results also
added to report.

The stats pod is updated as well to use net=host.
2019-09-12 09:45:05 -05:00
Khanak Nangia 044146703f adding detailed instructions for vagrant setup on clear linux (#173)
* adding detailed instructions for vagrant setup on clear linux

* adding all the dependencies together
2019-09-04 12:22:06 -07:00
CraigSterrett 23712beac2 Fixes no_proxy issues with Vagrant file (#172)
Vagrant file blows up if http_proxy is set but no_proxy is not set. This
was discovered during the testing of the initial patch which was to add
the default CIDR for Vagrant to the no_proxy settings.

Closes issue #136

Signed-off-by: craigsterrett <craig.sterrett@intel.com>
2019-09-04 11:44:34 -07:00
Justin Scott 98f8e3eb5e WIP Add node problem detector (#160)
Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-09-03 10:39:38 -07:00
David Lyle 20c7eccf1f Fixing number of pods in report
In the rewrite of the report generation code, we regressed to
using the passed in requested number of pods rather than the
actual. Since we typically attempt to launch more pods than the
system is capable of, the math for the stats are done incorrectly
and misreported.

This changes the behavior back to using the correct value of the
total number of pods launched.
2019-09-02 10:57:55 -05:00
Graham Whaley be164144e4 metrics: json: use tr for newline/whitespace conversion
Use `tr` rather than `sed` for doing the line/whitespace conversions
on the JSON stream, if asked. Slightly more efficient, and we also drop
the wrapping ""'s, to allow the data to land in logstash without error.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-29 12:33:41 -06:00
Antti Kervinen 18fde35232 metrics: scaling: require jq is installed
Also remove trailing whitespace

Signed-off-by: Antti Kervinen <antti.kervinen@intel.com>
2019-08-29 09:01:00 -06:00
David Lyle a61deb736c Fix array declaration
Used as associative array, not declared that way. The result
was indexing errors.
2019-08-29 14:31:50 +01:00
Syed Ahsan Zaidi 270bbc6f90 Support for Virtualbox
This change adds the env variable for the support for
virtualbox to support kubernetes communication between
master and worker nodes. Since KVM gives default ip
address to the spinned up VM's, whereas virtualbox uses
same ip address for all, the kubelet was not functioning
, this change fixes it and assigns the ip address to env
variable.
2019-08-27 12:10:56 -07:00
Graham Whaley 8787955188 metrics: report: generate PNGs alongside the pdf
To aid in incorporating the results graphs and tables into other
documents, generate PNG files of the tables and plots alongside
the full PDF report, and place them in the output directory.

Achieve this by processing an html as well as a pdf targetted
output document, and grabbing the png's generated from that
operation.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-27 12:46:11 -06:00
Graham Whaley 2d1f8551b0 metrics: report: update README with new debug mode info
Debug mode of the report generator now volume mounts the scripts
into the container (rather than just COPYing them in), to allow
interactive edit/run debug cycles. Document it.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-27 09:41:44 -06:00
Graham Whaley 659600b9a6 metrics: report: drop deprecated scaling.R
Now the new tidy_scaling.R seems to be functional, drop the
deprecated scaling.R.
Yes, we could/should probably have just replaced scaling.R, but
that is just the way it panned out.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-27 09:41:44 -06:00
Graham Whaley 0e6596bed6 metrics: report: do not account noschedule. Show boot in S
Do not account for 'noschedule' nodes in the stats calculations,
as they don't host the pods we are measuring.
Show the boot times in seconds, rather than milliseconds.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-27 09:41:44 -06:00
Graham Whaley 6d5c782fce metrics: scaling: check we have bc installed
We now use `bc` for some of the stats calculations. Ensure it
is installed, or we silently generate bad field data.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-27 09:41:44 -06:00
Graham Whaley 6a7eb8bbc9 metrics: report: scaling: use a more 'tidyverse' flow
Due to the nature of the shape of the JSON data import, the scaling
R had some hard wired constructs bound to the data shape. This limited
our extending the JSON data.
recode to be more 'R tidyverse' style.
Also add in a launch time stats table, and split the graphing
across 3 pages to de-cramp them (particularly when rendering >1
test run).

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-27 09:41:44 -06:00
Graham Whaley 6e189ff2be metrics: report: mount scripts in debug mode
When in debug mode, mount the scripts dir into the container to
allow 'live' editing and reloading into R. This makes the dev cycle
much faster.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-27 09:41:44 -06:00
Graham Whaley 5e53ce9afb metrics: scaling: calculate mem consumed and pod/Gb
Keep track of memory consumed on the nodes, log that, and
generate an overall cluster value for 'pods-per-Gb'.

Generate this data here, whilst we have the data to hand, as
it can get much more difficult to calculate this information
the further you get down the monitoring stream (such as the
farther you get into an ELK stack).

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-27 09:41:44 -06:00
Graham Whaley d3cab6b06e metrics: report: switch scaling to tidy R
Switch the rendering of the scaling page to use the tidy R version.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-27 09:41:44 -06:00
craigsterrett ba52442d0c Removed vagrant box pinning
Removed hardcoded vagrant box version. This change will now
allow the override of the version, or if no override is passed in
it will take the version on the system, or download the latest version.

Signed-off-by: craigsterrett <craig.sterrett@intel.com>
2019-08-22 13:48:15 -07:00
Justin Scott 535680bce2 Update setup_system.sh to include alt OS file (#141)
This udpate allows us to specify an OS specific
include file with function overrides. Useful when using
setup_system.sh on other OSes.

Signed-off-by: Justin Scott <justin.a.scott@intel.com>
2019-08-19 15:04:53 -07:00
Graham Whaley 68be049f98 metrics: README: update to be accurate and more descriptive
The README has not kept upto date with the code. Update to make it
more useful, but try to keep fairly neutral so it does not rot so
fast.

Signed-off-by: Graham Whaley <graham.whaley@intel.com>
2019-08-19 10:50:36 -06:00
Ganesh Maharaj Mahalingam 78e07e2c4f Fix script error cause of missing variables
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2019-08-17 21:38:05 -07:00
Khanak Nangia 202c8ec007 adding bullets points for distro names 2019-08-16 15:48:52 -07:00
Khanak Nangia 82795e7850 adding distros as headers 2019-08-16 15:48:52 -07:00
Khanak Nangia 5267dac92e updating with the raw URL 2019-08-16 15:48:52 -07:00
Khanak Nangia 53ae4e0227 updating vagrant steps for Clear 2019-08-16 15:48:52 -07:00