-
Notifications
You must be signed in to change notification settings - Fork 430
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enabling debug logs for rename perf test #2509
base: master
Are you sure you want to change the base?
Conversation
perfmetrics/scripts/hns_rename_folders_metrics/run_rename_benchmark.sh
Outdated
Show resolved
Hide resolved
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark.py
Outdated
Show resolved
Hide resolved
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark.py
Outdated
Show resolved
Hide resolved
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark_test.py
Outdated
Show resolved
Hide resolved
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark_test.py
Outdated
Show resolved
Hide resolved
3b8989b
to
a3c5ced
Compare
else : | ||
# Creating config file for mounting with hns disabled. | ||
with open("/tmp/config.yml",'w') as mount_config: | ||
mount_config.write("enable-hns: false") | ||
mount_flags = "--config-file=/tmp/config.yml --implicit-dirs --rename-dir-limit=1000000 --stackdriver-export-interval=30s" | ||
mount_flags = "--config-file=/tmp/config.yml --log-severity=trace --log-format \"text\" --log-file {} --implicit-dirs --rename-dir-limit=1000000 --stackdriver-export-interval=30s".format(LOG_FILE) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding trace-logging affect the performance of the test run. Is that accounted for in thresholds ?
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark.py
Outdated
Show resolved
Hide resolved
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark.py
Outdated
Show resolved
Hide resolved
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark.py
Outdated
Show resolved
Hide resolved
@@ -133,7 +133,7 @@ def test_perform_testing_flat(self, mock_log, mock_record_time_of_operation, | |||
test_type = "flat" | |||
num_samples = 4 | |||
results = {} | |||
mount_flags = "--config-file=/tmp/config.yml --implicit-dirs --rename-dir-limit=1000000 --stackdriver-export-interval=30s" | |||
mount_flags = "--config-file=/tmp/config.yml --log-severity=trace --log-format \"text\" --log-file /tmp/gcsfuse-logs-rename.txt --implicit-dirs --rename-dir-limit=1000000 --stackdriver-export-interval=30s" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same question as above about the performance issue with log-severity trace.
export PATH=/usr/local/google-cloud-sdk/bin:$PATH | ||
echo 'export PATH=/usr/local/google-cloud-sdk/bin:$PATH' >> ~/.bashrc | ||
gcloud version && rm gcloud.tar.gz | ||
sudo /usr/local/google-cloud-sdk/bin/gcloud components update |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need the full path to gcloud here as we have already update the PATH variable locally.
sudo /usr/local/google-cloud-sdk/bin/gcloud components update | |
sudo gcloud components update |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While trying to do this, I was facing this issue
ERROR: (gcloud.components.update) You cannot perform this action because this Google Cloud CLI installation is managed by an external package manager.
Please consider using a separate installation of the Google Cloud CLI created through the default mechanism described at: https://cloud.google.com/sdk/
but this did not come up when using sudo /usr/local/google-cloud-sdk/bin/gcloud components update
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark.py
Outdated
Show resolved
Hide resolved
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark.py
Outdated
Show resolved
Hide resolved
perfmetrics/scripts/hns_rename_folders_metrics/renaming_benchmark_test.py
Show resolved
Hide resolved
perfmetrics/scripts/hns_rename_folders_metrics/run_rename_benchmark.sh
Outdated
Show resolved
Hide resolved
a3c5ced
to
b79a9c9
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2509 +/- ##
==========================================
+ Coverage 78.85% 78.90% +0.05%
==========================================
Files 106 106
Lines 11686 11686
==========================================
+ Hits 9215 9221 +6
+ Misses 1992 1987 -5
+ Partials 479 478 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
ddf6c78
to
117db8d
Compare
Description
Enabled debug logs for rename perf test
Link to the issue in case of a bug fix.
NA
Testing details