DontReview Garnitin/add gke load testing/v1 #2225

gargnitingoogle · 2024-07-26T21:27:18Z

Description

Link to the issue in case of a bug fix.

NA

Testing details

Manual - NA
Unit tests - NA
Integration tests - NA

codecov · 2024-07-26T21:32:04Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.11%. Comparing base (8662b3e) to head (74b5bd8).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2225      +/-   ##
==========================================
+ Coverage   77.07%   77.11%   +0.04%     
==========================================
  Files         113      113              
  Lines       15910    15910              
==========================================
+ Hits        12262    12269       +7     
+ Misses       3123     3117       -6     
+ Partials      525      524       -1

Flag	Coverage Δ
unittests	`77.11% <ø> (+0.04%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests

gargnitingoogle · 2024-07-26T21:29:20Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+fi
+
+function validateMachineConfig() {
+  echo "Validiting input parameters ..."


gargnitingoogle · 2024-07-26T21:29:52Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  local cluster_name=${1}
+  local zone=${2}
+  local node_pool=${3}
+  if [ $(gcloud container node-pools list --cluster=${cluster_name} --zone=${zone} | grep -ow ${node_pool} | wc -l) -gt 0 ] ; then


simplify this with grep -q

gargnitingoogle · 2024-07-26T21:30:59Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  if ClusterExists ${cluster_name} ; then
+    gcloud container clusters update ${cluster_name} --location=${zone} --workload-pool=${project_id}.svc.id.goog
+  else
+    # gcloud container --project "${project_id}" clusters create ${cluster_name} --zone "${zone}" --cluster-version "${cluster_version}" --workload-pool=${project_id}.svc.id.goog


remove commented code

gargnitingoogle · 2024-07-26T21:32:10Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  echo "Enabling/disabling csi add-on ..."
+  # By default, disable the managed csi driver.
+  if ${useCustomCsiDriver}; then
+    # gcloud -q container clusters update ${cluster_name} \


re-enable this

gargnitingoogle · 2024-07-26T21:32:29Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  gcloud container clusters get-credentials ${cluster_name} --location=${zone}
+  kubectl create namespace ${appnamespace}
+  kubectl create serviceaccount ${ksa} --namespace ${appnamespace}
+  for workload_bucket in ${buckets} ; do


add code to get buckets from somewhere.

gargnitingoogle · 2024-07-26T21:35:00Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+}
+
+# validateMachineConfig ${machine_type} ${num_nodes} ${num_ssd}
+# installDependencies


re-enable all these disabled steps.

gargnitingoogle · 2024-07-26T21:36:51Z

perfmetrics/scripts/testing_on_gke/utils/utils.py

+def get_cpu(pod_name: str, start: str, end: str) -> Tuple[float, float]:
+    # for some reason, the mash filter does not always work, so we fetch all the metrics for all the pods and filter later.
+    result = subprocess.run(["mash", "--namespace=cloud_prod", "--output=csv", 
+                             f"Query(Fetch(Raw('cloud.kubernetes.K8sContainer', 'kubernetes.io/container/cpu/core_usage_time'), {{'project': '927584127901'}})| Window(Rate('10m'))| GroupBy(['pod_name', 'container_name'], Max()), TimeInterval('{start}', '{end}'), '5s')"], 


put the project-number back to the original and change it by code during runtime and then revert it back when done during runtime.

gargnitingoogle · 2024-07-26T21:37:31Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+function updateMachineTypeInPodConfigs() {
+  for file in ${gke_testing_dir}/examples/fio/loading-test/values.yaml ${gke_testing_dir}/examples/dlio/unet3d-loading-test/values.yaml ; do
+    test -f ${file}
+    sed -i -E "s/nodeType: [0-9a-z_-]+$/nodeType: ${machine_type}/g" ${file}


add code to revert this back

gargnitingoogle · 2024-07-26T21:37:42Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

+  for file in ${gke_testing_dir}/examples/fio/loading-test/values.yaml ${gke_testing_dir}/examples/dlio/unet3d-loading-test/values.yaml ; do
+    test -f ${file}
+    # sed -i -E "s/mountOptions: [0-9a-zA-Z,\:\"-_]+$/mountOptions: \"${gcsfuse_mount_options}\"/g" ${file}
+    sed -i -E "s/mountOptions:[ \t]*\"?[a-zA-Z0-9,:_-]+\"? *$/mountOptions: \"${gcsfuse_mount_options}\"/g" "${file}"


add code to revert this back when done

gargnitingoogle · 2024-07-27T04:27:38Z

perfmetrics/scripts/testing_on_gke/utils/utils.py

+    return utc_timestamp_string
+
+def standard_timestamp(timestamp: int) -> str:
+    return timestamp.split('.')[0].replace('T', ' ') + " UTC"


insert newline after this line

gargnitingoogle · 2024-07-27T04:28:16Z

perfmetrics/scripts/testing_on_gke/examples/run-gke-tests.sh

@@ -0,0 +1,425 @@
+#!/bin/bash
+


Add a header and a help option at the top.

Utilities: 1. Append given tabular data to the given gsheet id and worksheet name. 2. Return url for a gsheet given its ID. 3. Adds unit tests for the above utilities.

add generic utility to append to a gsheet Add utility for gsheet improve gsheet utility export fio output to gsheet encapsulate cpu/memory calculation in fio disable repeat operations for quick testing add dlio output export to gsheet fix a bug in dlio output parsing fix a column-name in fio csv output Revert "disable repeat operations for quick testing" This reverts commit 04bf834. add log of successful addition to gsheet clean-up code changes added some error-checking wip wip fix calls to download_gcs_objects support key-file on gcs in gsheet put back cpu/memory metrics fix couple of logs fix couple of help messages put back accidentally deleted command

Adds row with "ERROR" for all values rather than crashing during printing in CSV file.

Purposes. * Consistent behavior across all machines * Monitoring API has faster runtime than mash. * Monitoring API is supported on GCE VM too.

gargnitingoogle commented Jul 26, 2024

View reviewed changes

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch from 2958a3c to ad97f82 Compare July 29, 2024 10:47

gargnitingoogle commented Jul 29, 2024

View reviewed changes

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 7 times, most recently from ff7ee55 to 478921c Compare August 6, 2024 09:53

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 9 times, most recently from 1e4353a to b6a0e76 Compare August 16, 2024 14:45

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 9 times, most recently from b62b517 to d008a84 Compare August 22, 2024 11:53

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch from 0daf723 to 6dbb552 Compare August 26, 2024 09:05

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 2 times, most recently from ba27f43 to 6c7f199 Compare October 30, 2024 10:09

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 5 times, most recently from c914b7d to 6721f53 Compare November 9, 2024 10:26

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 4 times, most recently from 743af55 to e284c50 Compare November 18, 2024 10:27

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch 3 times, most recently from f8955cd to 6d56362 Compare November 22, 2024 06:04

gargnitingoogle added 15 commits November 23, 2024 06:07

Add gsheet utilities

0228c49

Utilities: 1. Append given tabular data to the given gsheet id and worksheet name. 2. Return url for a gsheet given its ID. 3. Adds unit tests for the above utilities.

fix gsheet unit tests

6a18b04

address self-review comments

895e2ab

address some more self-review comments

3c55ac5

address a self-review comment

f273dd4

fix bug in gsheet utility code

b05a353

rename some utilities

94f9b10

address some self-review comments

06c280e

Add try-catch for exception during parsing

1b3ec7f

Adds row with "ERROR" for all values rather than crashing during printing in CSV file.

add some useful comments

c475333

Replace mash with monitoring-api everywhere

34d2b98

Purposes. * Consistent behavior across all machines * Monitoring API has faster runtime than mash. * Monitoring API is supported on GCE VM too.

fix some var names

d7194ca

expose force_update_gcsfuse_code

e3cdbc2

empty commit

74b5bd8

gargnitingoogle force-pushed the garnitin/add-gke-load-testing/v1 branch from 6d56362 to 74b5bd8 Compare November 23, 2024 06:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DontReview Garnitin/add gke load testing/v1 #2225

DontReview Garnitin/add gke load testing/v1 #2225

gargnitingoogle commented Jul 26, 2024

codecov bot commented Jul 26, 2024 •

edited

Loading

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 26, 2024

gargnitingoogle Jul 27, 2024

gargnitingoogle Jul 27, 2024

DontReview Garnitin/add gke load testing/v1 #2225

Are you sure you want to change the base?

DontReview Garnitin/add gke load testing/v1 #2225

Conversation

gargnitingoogle commented Jul 26, 2024

Description

Link to the issue in case of a bug fix.

Testing details

codecov bot commented Jul 26, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jul 26, 2024 •

edited

Loading