Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[occm] Support Octavia/Amphora Prometheus endpoint creation using annotations #2633

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[occm] Support Octavia/Amphora Prometheus endpoint creation using annotations #2633

wants to merge 1 commit into from

Conversation

Lucasgranet
Copy link
Contributor

@Lucasgranet Lucasgranet commented Aug 22, 2024 •
edited
Loading

What this PR does / why we need it:
Adds the ability to add a Prometheus listener on the Octavia LoadBalancer in order to fetch it with any Prometheus scrapper to get metrics from the LoadBalancer.

This PR brings 4 new annotations:

loadbalancer.openstack.org/metrics-enable: "true" # Enable the listener endpoint on the Octavia LoadBalancer (default false)
loadbalancer.openstack.org/metrics-port: "9101" # Listener's port (default 9100)
loadbalancer.openstack.org/metrics-allow-cidrs: "10.0.0.0/8, fe80::/10" # Listener's allowed cidrs (default none) - see below
loadbalancer.openstack.org/load-balancer-vip-address: "10.4.2.3" #  Auto-computed field based on Octavia VIP

Why you should have to use loadbalancer.openstack.org/metrics-allow-cidrs

It's strongly recommended to apply an allowed cidrs on the listener. If a Floating IP is attached to the Octavia, the metric endpoint will be exposed publicly. Apply a restriction to the K8S's subnet is recommended.

More detail on the Prometheus listener:
https://docs.openstack.org/octavia/latest/user/guides/monitoring.html#monitoring-with-prometheus

Which issue this PR fixes(if applicable):
fixes #2465

Special notes for reviewers:
A new E2E test has been added.

Release note:

[openstack-cloud-controller-manager] Support Octavia/Amphora Prometheus endpoint creation using annotations

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Aug 22, 2024
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 22, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @Lucasgranet. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 22, 2024
@Lucasgranet Lucasgranet force-pushed the feat/occm/metric-annotations branch 6 times, most recently from ea89e68 to 1219946 Compare August 26, 2024 10:05
@nicolaslacroux nicolaslacroux mentioned this pull request Aug 27, 2024
Open
@Lucasgranet Lucasgranet marked this pull request as ready for review August 27, 2024 14:14
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 27, 2024
@Lucasgranet Lucasgranet force-pushed the feat/occm/metric-annotations branch 4 times, most recently from fd375fb to 569454a Compare September 2, 2024 12:22
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 2, 2024
@jichenjc
Copy link
Contributor

jichenjc commented Sep 3, 2024

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 3, 2024
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 3, 2024
@Lucasgranet
Copy link
Contributor Author

Hello,

Have you any feedback or questions about this PR?

@Lucasgranet
Copy link
Contributor Author

Hello, @jichenjc @dulek,

If you have some time, would you mind reviewing this PR please?

Thank you,

kayrus
kayrus reviewed Oct 15, 2024
View reviewed changes
Copy link
Contributor

@kayrus kayrus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. See my comments. I think we need to rewrite the ensureOctaviaListener func to avoid excessive code duplication.

pkg/openstack/loadbalancer.go Outdated Show resolved Hide resolved
pkg/openstack/loadbalancer.go Outdated Show resolved Hide resolved
pkg/openstack/loadbalancer.go Outdated Show resolved Hide resolved
pkg/openstack/loadbalancer.go Outdated Show resolved Hide resolved
pkg/openstack/loadbalancer.go Outdated Show resolved Hide resolved
pkg/openstack/loadbalancer.go
if svcConf.metricEnabled && openstackutil.IsOctaviaFeatureSupported(lbaas.lb, openstackutil.OctaviaFeaturePrometheusListener, lbaas.opts.LBProvider) {
// Only a LB owner can add the prometheus listener (to avoid conflict with a shared loadbalancer)
if isLBOwner {
svcConf.metricPort = getIntFromServiceAnnotation(service, ServiceAnnotationLoadBalancerMetricsPort, 9100)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we check in advance whether the metricPort intersects with the service.Spec.Ports?

pkg/openstack/loadbalancer.go
// Only a LB owner can add the prometheus listener (to avoid conflict with a shared loadbalancer)
if isLBOwner {
svcConf.metricPort = getIntFromServiceAnnotation(service, ServiceAnnotationLoadBalancerMetricsPort, 9100)
lbaas.updateServiceAnnotation(service, ServiceAnnotationLoadBalancerMetricsPort, strconv.Itoa(svcConf.metricPort))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be a read-only annotation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This force the metric-port annotation to be pushed on the Service.
It's useful when using a metric scrapper like Prometheus to get the metric port.

pkg/openstack/loadbalancer.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 31, 2024
@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Oct 31, 2024
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 8, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kayrus for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 20, 2024
@k8s-ci-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kayrus kayrus kayrus left review comments

@FengyunPan2 FengyunPan2 Awaiting requested review from FengyunPan2

@mdbooth mdbooth Awaiting requested review from mdbooth

@anguslees anguslees Awaiting requested review from anguslees

@dulek dulek Awaiting requested review from dulek

Assignees
No one assigned
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[occm] Support Octavia/Amphora Prometheus endpoint creation using annotations
4 participants
@Lucasgranet @k8s-ci-robot @jichenjc @kayrus