Mapr deprecation #441
Open
Mapr deprecation #441
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Build failures: |
|
I believe the primary failure is actually during the #440 contains the proposal to address this issue. Re-expressing the next steps that need to happen: 1- ome/ansible-role-iptables-raw#13 needs to be reviewed, merged and released |
|
Seb: see https://github.com/ome/ansible-role-nginx-proxy/blob/1a2eb446951fe831126128c747beb6221d142047/templates/nginx-confd-proxy.j2#L50 for nginx setup. "Could put 410.html under `about" or "static" locations. |
|
As discussed this morning in answer to my question above about where to put: It seems that the equivalent 50x handling is part of this ansible role: However, it's not clear how/where to add the custom This is somewhat similar to the 'maintenance page' we have specified at However, to show a similar page for the So, the options are:
So, probably best to go for option 1. This may not be precisely the correct response, but it will be most user-friendly and easiest for us, bearing in mind it would be nice to release this next week or soon after. |
|
Build is green now 👍 . |
|
@will-moore with #440 in, can you merge |
|
@sbesson Done |
|
Deployed to TASK [ome.nginx_proxy : nginx | proxy cache config] ****************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-cache.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmp3t9v16lg/nginx-confd-proxy-cache.j2
@@ -5,7 +5,6 @@
proxy_cache_path /var/cache/nginx/omerorender levels=1:2 keys_zone=omerorender:35m max_size=40g inactive=180d use_temp_path=off;
proxy_cache_path /var/cache/nginx/omerometadata levels=1:2 keys_zone=omerometadata:65m max_size=25g inactive=180d use_temp_path=off;
proxy_cache_path /var/cache/nginx/omeroapi levels=1:2 keys_zone=omeroapi:320m max_size=10g inactive=180d use_temp_path=off;
-proxy_cache_path /var/cache/nginx/omeromapr levels=1:2 keys_zone=omeromapr:100m max_size=5g inactive=180d use_temp_path=off;
proxy_cache_path /var/cache/nginx/omeroviewers levels=1:2 keys_zone=omeroviewers:1m max_size=1g inactive=180d use_temp_path=off;
proxy_cache_path /var/cache/nginx/grafana levels=1:2 keys_zone=grafana:1m max_size=100m inactive=1m use_temp_path=off;
@@ -18,7 +17,6 @@
"~web(client|gateway)/get_thumbnail*" 0;
"~(webclient/)?api/*" 0;
"~static/*" 0;
- "~mapr/*" 0;
"~gallery-api/*" 0;
"~webclient/img_detail/*" 0;
"~iviewer/*" 0;
@@ -46,7 +44,6 @@
"~web(client|gateway)/render_*/*" omerorender;
"~webclient/metadata_*" omerometadata;
"~(webclient/)?api/*" omeroapi;
- # "~mapr/*" omeromapr;
"~webclient/img_detail/*" omeroviewers;
"~iviewer/*" omeroviewers;
"~grafana/*" grafana;
changed: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy cache config] ****************************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | check websockets configuration] ****************************************************************************************************************
skipping: [test125-proxy] => (item={'name': 'omerocached', 'location': '~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*', 'server': 'http://omeroreadonly', 'cache_validity': '1d', 'maintenance_flag': '/srv/www/omero-maintenance.flag', 'maintenance_uri': '/about/omero-maintenance.html'})
skipping: [test125-proxy] => (item={'name': 'omeromapr', 'location': '~ /mapr/*', 'server': 'http://omeroreadonly', 'cache_validity': '180d', 'maintenance_flag': '/srv/www/omero-maintenance.flag', 'maintenance_uri': '/about/omero-maintenance.html'})
skipping: [test125-proxy] => (item={'name': 'omerostatic', 'location': '~ /static/*', 'server': 'http://omeroreadonly', 'cache_validity': '1d'})
skipping: [test125-proxy] => (item={'name': 'omero', 'location': '/', 'server': 'http://omeroreadonly', 'maintenance_flag': '/srv/www/omero-maintenance.flag', 'maintenance_uri': '/about/omero-maintenance.html'})
skipping: [test125-proxy] => (item={'name': 'omerowebsockets', 'location': '= /omero-ws', 'server': 'http://omeroreadonlywebsockets', 'websockets': True, 'websocketsonly': True, 'read_timeout': 86400})
skipping: [test125-proxy] => (item={'name': 'grafanarender', 'location': '^~ /grafana/', 'server': 'http://192.168.2.92:3000/render/dashboard/db/', 'cache_validity': '1m'})
skipping: [test125-proxy] => (item={'name': 'prometheusfederate', 'location': '^~ /prometheus/federate', 'server': 'http://192.168.2.92:9090/federate', 'cache_validity': '15s'})
skipping: [test125-proxy] => (item={'name': 'searchengine', 'location': '^~ /searchengine', 'server': 'http://192.168.2.197:5577/searchengine', 'host_header': '$host/searchengine'})
TASK [ome.nginx_proxy : nginx | proxy websockets config] ***********************************************************************************************************************
ok: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy websockets config] ***********************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | check redirect configuration] ******************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy redirects config] ************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-redirect.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmpjl5hj5mc/nginx-confd-proxy-redirect.j2
@@ -14,8 +14,4 @@
~/pgpc /webclient/?show=screen-1151;
/study/idr0083/figure/4i /webclient/img_detail/9822151/?dataset=10201&x=34105&y=84808&zm=25&c=1|318:9927$808080&m=g;
/study/idr0083/figure/4r /webclient/img_detail/9822152/?dataset=10201&x=80560&y=77440&zm=66&c=1|495:9204$808080&m=g;
- /mapr/api/antibody /searchengine/apidocs/;
- /mapr/api/compound /searchengine/apidocs/;
- /mapr/api/gene /searchengine/apidocs/;
- /mapr/api/config /mapr/api/config;
}
changed: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy redirects config] ************************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | stream-conf.d directory] ***********************************************************************************************************************
ok: [test125-proxy]
TASK [ome.nginx_proxy : nginx | stream proxy config] ***************************************************************************************************************************
skipping: [test125-proxy]
TASK [ome.nginx_proxy : nginx | stream proxy config] ***************************************************************************************************************************
ok: [test125-proxy]
TASK [ome.nginx_proxy : nginx | proxy config] **********************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-default.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmpnatinmsu/nginx-confd-proxy.j2
@@ -12,7 +12,9 @@
ssl_certificate /etc/ssl/localcerts/bundled.crt;
ssl_certificate_key /etc/ssl/localcerts/server.key;
- ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # don't use SSLv3 ref: POODLE
+ ssl_protocols TLSv1.2 TLSv1.3; # don't use SSLv3 ref: POODLE
+ ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305;
+
#charset koi8-r;
#access_log /var/log/nginx/log/host.access.log main;
@@ -44,9 +46,8 @@
location ~ ^/(mito|tara|pgpc|study)($|/) {
return 302 $redirect_uri;
}
-
location ~ ^/mapr/api/(?!config).*($|/) {
- return 302 /searchengine/apidocs/;
+ return 302 $redirect_uri;
}
location = /webclient {
changed: [test125-proxy] => (item={'nginx_proxy_is_default': True, 'nginx_proxy_additional_directives': ['add_header Access-Control-Allow-Origin $allow_origin']})
--- before: /etc/nginx/conf.d/proxy-cachebuster.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmp2pvaxxro/nginx-confd-proxy.j2
@@ -33,6 +33,9 @@
port_in_redirect off;
location ~ ^/(mito|tara|pgpc|study)($|/) {
+ return 302 $redirect_uri;
+ }
+ location ~ ^/mapr/api/(?!config).*($|/) {
return 302 $redirect_uri;
}
changed: [test125-proxy] => (item={'nginx_proxy_server_name': 'cachebuster', 'nginx_proxy_listen_http': 0, 'nginx_proxy_ssl': False, 'nginx_proxy_cachebuster_enabled': True, 'nginx_proxy_backends': [{'name': 'omerocached', 'location': '~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*|/mapr/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d', 'read_timeout': 900}, {'name': 'omerostatic', 'location': '~ /static/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d'}, {'name': 'omero', 'location': '/', 'server': 'http://omeroreadwrite'}]})
--- before: /etc/nginx/conf.d/proxy-idr-demo.openmicroscopy.org.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-552739yr2e5xa/tmpifpytgda/nginx-confd-proxy.j2
@@ -10,7 +10,9 @@
ssl_certificate /etc/ssl/localcerts/bundled.crt;
ssl_certificate_key /etc/ssl/localcerts/server.key;
- ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # don't use SSLv3 ref: POODLE
+ ssl_protocols TLSv1.2 TLSv1.3; # don't use SSLv3 ref: POODLE
+ ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305;
+
#charset koi8-r;
#access_log /var/log/nginx/log/host.access.log main;
changed: [test125-proxy] => (item={'nginx_proxy_server_name': 'idr-demo.openmicroscopy.org', 'nginx_proxy_ssl': True, 'nginx_proxy_redirect_map_locations': [], 'nginx_proxy_direct_locations': [{'location': '/', 'redirect301': '$scheme://idr.openmicroscopy.org$request_uri'}], 'nginx_proxy_backends': []}) |
|
@sbesson - mostly that diff looks as expected (but not at I had manually added this myself by editing of /etc/nginx/conf.d/proxy-default.conf: so it looks like the diff is comparing my manual changes here to the changes after your update? But I also don't understand in This looks like removal of redirects to |
That's correct, this is capturing the changes was applied to
Clearly someone edited this file as these changes are neither on If you don't want to use the redirect map, then you probably need to put your 302 redirect under |
|
Ah - understood! Done, thanks. |
|
After running the TASK [ome.nginx_proxy : nginx | proxy config] **************************************************************************************************************************************************************************************************************
--- before: /etc/nginx/conf.d/proxy-default.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-67117nlkwjgz4/tmp4odhrluu/nginx-confd-proxy.j2
@@ -46,9 +46,6 @@
location ~ ^/(mito|tara|pgpc|study)($|/) {
return 302 $redirect_uri;
}
- location ~ ^/mapr/api/(?!config).*($|/) {
- return 302 $redirect_uri;
- }
location = /webclient {
return 302 /webclient/?experimenter=-1;
@@ -76,6 +73,9 @@
}
location = /submitter-survey {
return 302 https://forms.gle/beR4vqLrgLgRzgSX9;
+ }
+ location ~ ^/mapr/api/(?!config).*($|/) {
+ return 302 /searchengine/apidocs/;
}
location ~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/* {
changed: [test125-proxy] => (item={'nginx_proxy_is_default': True, 'nginx_proxy_additional_directives': ['add_header Access-Control-Allow-Origin $allow_origin']})
--- before: /etc/nginx/conf.d/proxy-cachebuster.conf
+++ after: /Users/sbesson/.ansible/tmp/ansible-local-67117nlkwjgz4/tmp4y0krw4k/nginx-confd-proxy.j2
@@ -35,9 +35,6 @@
location ~ ^/(mito|tara|pgpc|study)($|/) {
return 302 $redirect_uri;
}
- location ~ ^/mapr/api/(?!config).*($|/) {
- return 302 $redirect_uri;
- }
location = /webclient {
return 302 /webclient/?experimenter=-1;
@@ -65,6 +62,9 @@
}
location = /submitter-survey {
return 302 https://forms.gle/beR4vqLrgLgRzgSX9;
+ }
+ location ~ ^/mapr/api/(?!config).*($|/) {
+ return 302 /searchengine/apidocs/;
}
location ~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*|/mapr/* {
changed: [test125-proxy] => (item={'nginx_proxy_server_name': 'cachebuster', 'nginx_proxy_listen_http': 0, 'nginx_proxy_ssl': False, 'nginx_proxy_cachebuster_enabled': True, 'nginx_proxy_backends': [{'name': 'omerocached', 'location': '~ /webclient/metadata_*|/webclient/render_*|/webclient/get_thumbnail*|/webgateway/metadata_*|/webgateway/render_*|/webgateway/get_thumbnail*|/webclient/api/*|/webclient/search/*|/api/*|/webclient/img_detail/*|/iviewer/*|/figure/*|/gallery-api/*|/mapr/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d', 'read_timeout': 900}, {'name': 'omerostatic', 'location': '~ /static/*', 'server': 'http://omeroreadwrite', 'cache_validity': '1d'}, {'name': 'omero', 'location': '/', 'server': 'http://omeroreadwrite'}]})
ok: [test125-proxy] => (item={'nginx_proxy_server_name': 'idr-demo.openmicroscopy.org', 'nginx_proxy_ssl': True, 'nginx_proxy_redirect_map_locations': [], 'nginx_proxy_direct_locations': [{'location': '/', 'redirect301': '$scheme://idr.openmicroscopy.org$request_uri'}], 'nginx_proxy_backends': []}) |
|
@sbesson Thanks - that looks like it's working fine now 👍 . |
👍 Do I understand correctly that this PR is now only waiting on a release of the deployment/ansible/group_vars/omero-hosts.yml Line 245 in cb12a87 |
|
Yes, I think so |
|
I'll wait for idr-gallery release (and mapr too) then bump them both in this PR. |
sbesson
reviewed
Nov 25, 2024
Merged
sbesson
approved these changes
Nov 27, 2024
There was a problem hiding this comment.
The components and NGINX changes are consistent with the discussions around the deprecation of the mapr endpoints from the last few weeks and have been successfully deployed at test125.
TASK [ome.omero_web : omero web | install requirements] *************************************************************************************************************************************************
changed: [test125-omeroreadonly-2]
changed: [test125-omeroreadonly-1]
changed: [test125-omeroreadonly-4]
changed: [test125-omeroreadwrite]
changed: [test125-omeroreadonly-3]
...
TASK [ome.omero_web : omero web | flush systemd handlers] ***********************************************************************************************************************************************
RUNNING HANDLER [ome.omero_web : omero-web rewrite omero-web configuration] *****************************************************************************************************************************
changed: [test125-omeroreadonly-1]
changed: [test125-omeroreadonly-3]
changed: [test125-omeroreadonly-4]
changed: [test125-omeroreadwrite]
changed: [test125-omeroreadonly-2]
RUNNING HANDLER [ome.omero_web : omero-web restart omero-web] *******************************************************************************************************************************************
changed: [test125-omeroreadonly-1]
changed: [test125-omeroreadonly-4]
changed: [test125-omeroreadwrite]
changed: [test125-omeroreadonly-2]
changed: [test125-omeroreadonly-3]
Approving from a deployment perspective and assigning a few reviewers for the functional testing before we apply this on prod125
|
Testing on idr-testing looks good for me. Search, redirects etc. |
|
Tested on idr-testing. Looks good. Search, redirects and pop-up working as expected. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This represents the changes I have been making manually on idr-testing with the replacement of mapr searches with searchengine, described at IDR/idr-gallery#45
Summary of changes:
/mapr/gene/get handled by idr-gallery instead of mapr/mapr/api/[key]/requests we now response with a 410 response. [EDIT] Now redirects to /searchengine/apidocs/ 302 response - see below.NB: I couldn't see where to add the file I created with:
sudo vi /usr/share/nginx/html/410.html(based on the 50x.html)Also the config for this file looks like:
In my testing, I had added this in
proxy-default.confright before thelocationentry that I have added above:The
error_page 410about follows on fromerror_page 50xetc section in the same file, but I don't see that this is specified anywhere in this repo, so maybe that comes from existing nginx config somewhere?