Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Learning maintenance: evaluate script moving users to a disabled group #363

Open
sbesson opened this issue Aug 22, 2022 · 6 comments
Open
Assignees

Comments

@sbesson
Copy link
Member

sbesson commented Aug 22, 2022

The UoD SLS learning system is getting heavily used during the academic term with new students logging into the resource using their University credentials and being mapped into the default group containing the course supporting data to allow access - see

omero.ldap.base: "{{ omero_server_ldap_base }}"
omero.ldap.username: "{{ omero_server_ldap_username }}"
omero.ldap.password: "{{ omero_server_ldap_password | default('') }}"
omero.ldap.user_filter: "{{ omero_server_ldap_user_filter }}"
omero.ldap.new_user_group: "{{ omero_server_ldap_new_user_group }}"
.

Eventually, many teaching semesters lead to the creation of several 100s of users in the single group. This causes performance degradation on the OMERO.web deployment as several of the queries start to run more slowly.

At the moment, the OME team runs periodically ad-hoc maintenance scripts on the system to reduce the number of users in the group. On the image.sc forum, a similar problem was discussed with a script being shared which allows to move stale users into a "graveyard" group - see https://forum.image.sc/t/omero-user-scripted-inactivation/70767/3.

For the next maintenance, we might want to evaluate whether this script could be adapted to the requirements of the SLS learning deployment and used to move students from the former year to another group to restore performance.

@pwalczysko
Copy link
Member

pwalczysko commented Aug 22, 2022

Tested on a local omero server with 61 users in 79 groups.

The script is working without problems, the situation before

omero user list
Using session for root@localhost:4064. Idle timeout: 10 min. Current group: system
 id | login                                | first name  | last name  | email | active | ldap  | admin | member of | owner of 
----+--------------------------------------+-------------+------------+-------+--------+-------+-------+-----------+----------
 0  | root                                 | root        | root       |       | Yes    | False | Yes   | 3         |          
 1  | guest                                | Guest       | Account    |       |        | False |       | 2         |          
 2  | user-1                               | user-1      | user-1     |       | Yes    | False |       |           | 3        
 3  | user-2                               | user-2      | user-2     |       | Yes    | False |       | 3,4       |          
 4  | user-3                               | user-3      | user-3     |       | Yes    | False |       | 5         | 4        
 5  | user-4                               | user-4      | user-4     |       | Yes    | False |       | 4         | 6        
 6  | user-5                               | user-5      | user-5     |       | Yes    | False |       | 6         | 5        
 7  | user-6                               | user-6      | user-6     |       | Yes    | False | Yes   | 3,4,5,6   |          
 8  | user-7                               | user-7      | user-7     |       | Yes    | False |       |           | 3        
 9  | user-8                               | user-8      | user-8     |       | Yes    | False |       | 3,4       |          
 10 | user-9                               | user-9      | user-9     |       | Yes    | False |       | 5         | 4        
 11 | user-10                              | user-10     | user-10    |       | Yes    | False |       | 4         | 6        
 12 | user-11                              | user-11     | user-11    |       | Yes    | False |       | 6         | 5        
 13 | user-12                              | user-12     | user-12    |       | Yes    | False | Yes   | 3,4,5,6   |          
 14 | adm-user-1                           | adm-user-1  | adm-user-1 |       | Yes    | False |       |           | 7        
 15 | adm-user-2                           | adm-user-2  | adm-user-2 |       | Yes    | False |       | 7,8       |          
 16 | adm-user-3                           | adm-user-3  | adm-user-3 |       | Yes    | False |       | 9         | 8        
 17 | adm-user-4                           | adm-user-4  | adm-user-4 |       | Yes    | False |       | 8         | 10       
 18 | adm-user-5                           | adm-user-5  | adm-user-5 |       | Yes    | False |       | 10        | 9        
 19 | adm-user-6                           | adm-user-6  | adm-user-6 |       | Yes    | False | Yes   | 7,8,9,10  |          
 52 | 364e383a-6dc4-456f-a714-a0e7a9a4fc28 | integration | tester     |       | Yes    | False |       | 53        |          
 53 | 93c4347c-67b3-487d-87be-dee90c9d9053 | integration | tester     |       | Yes    | False |       |           | 54       
 54 | 0884834e-5c96-486f-b7b6-3896da90f342 | integration | tester     |       | Yes    | False |       | 55,54     |          
 55 | cf599c11-b462-4a03-b23d-d07bd14636f9 | integration | tester     |       | Yes    | False | Yes   |           | 56       
 56 | 7a54f780-0c29-4098-850e-87c6444705a4 | integration | tester     |       | Yes    | False |       | 57,56     |          
 57 | 8830379a-9e0e-4abc-be81-7de098121c1e | integration | tester     |       | Yes    | False | Yes   |           | 58       
 58 | 39f8cc5e-5f08-4090-8829-ccec34f1185f | integration | tester     |       | Yes    | False |       | 59,58     |          
 59 | c826120c-9d35-46ec-8365-0606c6c6f2f1 | integration | tester     |       | Yes    | False |       | 60        |          
 60 | 6037f119-c9c7-437a-b5fe-6284485aca8b | integration | tester     |       | Yes    | False |       |           | 61       
 61 | f6df8848-faee-43e9-a2b4-aba4e80e6500 | a           | user       |       | Yes    | False |       | 61        |          
 62 | e8f43db7-775c-4bd7-99c9-b893ad20ac2f | integration | tester     |       | Yes    | False |       |           | 62       
 63 | 0ea1b3b8-2f3f-4c18-9064-cff1b167659c | a           | user       |       | Yes    | False |       | 62        |          
 64 | 5a2caab3-f285-47b2-98a2-6d19548c2d66 | integration | tester     |       | Yes    | False |       |           | 63       
 65 | e13978ce-da76-49cc-bb88-b306283a5d13 | a           | user       |       | Yes    | False |       | 63        |          
 66 | 37dd3bb3-db17-48f3-b80c-932ce2eb7d01 | integration | tester     |       | Yes    | False |       |           | 64       
 67 | e85840ea-7e0c-4c14-9441-11b252330962 | a           | user       |       | Yes    | False |       | 64        |          
 68 | 938e3121-574e-499b-b088-1b8e65936f7f | integration | tester     |       | Yes    | False |       |           | 65       
 69 | 3135fc9c-824e-4191-983e-dc93a71c8e2e | integration | tester     |       | Yes    | False |       | 65        |          
 70 | 73e745d4-015a-49fb-8fe5-f80a08304f4d | integration | tester     |       | Yes    | False |       |           | 66       
 71 | 16ee4b03-b1a0-453d-a6db-2a141054185e | a           | user       |       | Yes    | False |       | 66        |          
 72 | b9eb9948-21ce-4e0a-baf9-b050891bb084 | integration | tester     |       | Yes    | False |       |           | 67       
 73 | 5fc7c388-1ba4-45b3-a3a4-e10a1c72c06c | a           | user       |       | Yes    | False |       | 67        |          
 74 | 2e3d764c-d680-47c8-b596-10896185758e | integration | tester     |       | Yes    | False |       | 68        |          
 75 | 0e28f224-5928-4a02-b046-92ecc927fb16 | integration | tester     |       | Yes    | False |       | 69        |          
 76 | 322fd2d4-3c16-40cd-a052-97135d7c6a12 | integration | tester     |       | Yes    | False |       | 70        |          
 77 | 1e8fa56c-062d-4d86-9b9e-d2790367123a | integration | tester     |       | Yes    | False |       |           | 71       
 78 | 3048698d-5655-4193-bed7-9ce00d67eb2b | a           | user       |       | Yes    | False |       | 71        |          
 79 | 2022d769-14a3-49f0-b618-afdc407ce81c | integration | tester     |       | Yes    | False |       |           | 72       
 80 | 9a3f51d6-8f22-4ddb-8bd6-63b52fc78043 | a           | user       |       | Yes    | False |       | 72        |          
 81 | 1976e5bc-d1c1-49ad-a389-cc4db421cdf9 | integration | tester     |       | Yes    | False |       |           | 73       
 82 | ac3a7053-0f80-4f7e-acb9-3c0287bee36c | a           | user       |       | Yes    | False |       | 73        |          
 83 | 1999865b-e21f-4f0f-9d99-0ad52d1a466c | integration | tester     |       | Yes    | False |       |           | 74       
 84 | 23fa9d7e-7228-4de3-889e-0cecebb18979 | a           | user       |       | Yes    | False |       | 74        |          
 85 | 9adb5f03-cc08-4bed-9b23-46e3d7d242ec | integration | tester     |       | Yes    | False |       |           | 75       
 86 | bbb177c5-22f8-42ab-be69-979077f1cfba | integration | tester     |       | Yes    | False |       | 75        |          
 87 | eafc9ef7-9aa9-4b6c-a2d6-a334bf5d75ee | integration | tester     |       | Yes    | False |       |           | 76       
 88 | 83e7a307-713c-4dcf-957d-3bd7189d9b79 | a           | user       |       | Yes    | False |       | 76        |          
 89 | a3f9c91e-58ab-4394-b480-7e37ae4d9df2 | integration | tester     |       | Yes    | False |       |           | 77       
 90 | 93d63cc8-365e-47f4-92c8-750ba9a8d589 | a           | user       |       | Yes    | False |       | 77        |          
 91 | 52350d61-d76a-4a27-9b4e-0c43eee8c8d9 | integration | tester     |       | Yes    | False |       | 78        |          
 92 | 3c05ca72-323b-48a2-8fd3-a8789ee566da | integration | tester     |       | Yes    | False |       | 79        |         


omero group list
Using session for root@localhost:4064. Idle timeout: 10 min. Current group: system
 id | name                                 | perms  | ldap  | # of owners | # of members 
----+--------------------------------------+--------+-------+-------------+--------------
 0  | system                               | rw---- | False | 1           | 5            
 1  | user                                 | rwr-r- | False | 0           | 60           
 2  | guest                                | rw---- | False | 0           | 1            
 3  | private-2                            | rw---- | False | 2           | 5            
 4  | read-only-1                          | rwr--- | False | 2           | 6            
 5  | read-annotate-1                      | rwra-- | False | 2           | 4            
 6  | read-write-1                         | rwrw-- | False | 2           | 4            
 7  | adm-private-2                        | rw---- | False | 1           | 2            
 8  | adm-read-only-1                      | rwr--- | False | 1           | 3            
 9  | adm-read-annotate-1                  | rwra-- | False | 1           | 2            
 10 | adm-read-write-1                     | rwrw-- | False | 1           | 2            
 53 | aac57b72-ba1d-4900-869a-57aec5db668b | rw---- | False | 0           | 1            
 54 | 0913db98-9096-4cd2-b528-50662bdf426d | rwr--- | False | 1           | 1            
 55 | 8d123466-c1b5-4429-9135-db16da7e746a | rwr--- | False | 0           | 1            
 56 | 31f8a6fa-0e41-4447-8f75-ee668cc9b472 | rwr--- | False | 1           | 1            
 57 | 5a2a53d5-2624-402f-8485-cc9938453f51 | rwr--- | False | 0           | 1            
 58 | 85e54459-ddad-41d4-bb61-836f80e6e2e9 | rwr--- | False | 1           | 1            
 59 | 92fd7d0b-3698-496f-9412-1d32b211b054 | rwr--- | False | 0           | 1            
 60 | acc4112d-c7d6-4122-9db0-da7922c8d723 | rw---- | False | 0           | 1            
 61 | c1ab635d-f238-4cb7-9ec8-98d3509999f7 | rwr--- | False | 1           | 1            
 62 | 7ec9e07d-20c8-4aa7-9c5c-bc5beddef9a6 | rwr--- | False | 1           | 1            
 63 | c2bcfcbf-e4b6-4725-ac85-af9dadfa81cf | rwr--- | False | 1           | 1            
 64 | 94b0c3cc-7c8c-4184-858c-673ed306b5e2 | rwr--- | False | 1           | 1            
 65 | 45dbefda-d63f-4930-bcf1-b35e255824ad | rwr--- | False | 1           | 1            
 66 | c6e5ddcf-97d8-4405-be83-e89ae302a424 | rwr--- | False | 1           | 1            
 67 | 58dd561c-8419-4e0f-a7c0-75b220901426 | rwr--- | False | 1           | 1            
 68 | 9d63e4e4-2aba-472c-a636-bcb275d18fb9 | rw---- | False | 0           | 1            
 69 | 7d2a95c2-a929-41ec-9844-456cb0d32583 | rwr--- | False | 0           | 1            
 70 | 1977e9f9-6b62-403e-ba1a-c2030c216114 | rw---- | False | 0           | 1            
 71 | c408630e-6301-436c-90ca-7b7c166dbb24 | rwr--- | False | 1           | 1            
 72 | 8c3b27d5-7e0b-463e-b865-dad34970ed61 | rwr--- | False | 1           | 1            
 73 | 6c5c72fc-9ad6-4552-b51a-35a2b56d2bb6 | rwr--- | False | 1           | 1            
 74 | 5596e598-33f2-48fc-b017-f48f9f9e789b | rwr--- | False | 1           | 1            
 75 | ee4223ed-3a4f-4818-ad67-47514427b756 | rwr--- | False | 1           | 1            
 76 | 2d2b858b-a25d-4d6a-8c39-3312e79fa79a | rwr--- | False | 1           | 1            
 77 | 4c612944-3f1f-4a84-a963-5f6c61a53d67 | rwr--- | False | 1           | 1            
 78 | c65b6089-9072-443a-8f64-0248f5c8c4f2 | rw---- | False | 0           | 1            
 79 | 9fab258b-3bad-4a78-b3b6-b7f9bee6552e | rwr--- | False | 0           | 1            
(38 rows)

The group with ID 79 was used as graveyard group in the test. The setting of necessary inactive days was 300 (this is a not-often used server and DB). This singled out user-3 (very plausible, the most used user) as active . The script also omitted root, Guest and the logged-in user (user-6).
See the output of the script below

python inactivate_users.py
Enter username:user-6
Password:
Enter host IP:localhost
Enter the group ID for the inactive users:79
Enter the minimum amount of days a user must have been inactive:300
Ignoring "user-6" (#7) who is logged in.
Ignoring "user-3" (#4) who logged in recently.
found these 57 users {2: 'user-1', 3: 'user-2', 5: 'user-4', 6: 'user-5', 8: 'user-7', 9: 'user-8', 10: 'user-9', 11: 'user-10', 12: 'user-11', 13: 'user-12', 14: 'adm-user-1', 15: 'adm-user-2', 16: 'adm-user-3', 17: 'adm-user-4', 18: 'adm-user-5', 19: 'adm-user-6', 52: '364e383a-6dc4-456f-a714-a0e7a9a4fc28', 53: '93c4347c-67b3-487d-87be-dee90c9d9053', 54: '0884834e-5c96-486f-b7b6-3896da90f342', 56: '7a54f780-0c29-4098-850e-87c6444705a4', 55: 'cf599c11-b462-4a03-b23d-d07bd14636f9', 67: 'e85840ea-7e0c-4c14-9441-11b252330962', 58: '39f8cc5e-5f08-4090-8829-ccec34f1185f', 57: '8830379a-9e0e-4abc-be81-7de098121c1e', 59: 'c826120c-9d35-46ec-8365-0606c6c6f2f1', 60: '6037f119-c9c7-437a-b5fe-6284485aca8b', 61: 'f6df8848-faee-43e9-a2b4-aba4e80e6500', 62: 'e8f43db7-775c-4bd7-99c9-b893ad20ac2f', 68: '938e3121-574e-499b-b088-1b8e65936f7f', 63: '0ea1b3b8-2f3f-4c18-9064-cff1b167659c', 64: '5a2caab3-f285-47b2-98a2-6d19548c2d66', 65: 'e13978ce-da76-49cc-bb88-b306283a5d13', 66: '37dd3bb3-db17-48f3-b80c-932ce2eb7d01', 69: '3135fc9c-824e-4191-983e-dc93a71c8e2e', 70: '73e745d4-015a-49fb-8fe5-f80a08304f4d', 71: '16ee4b03-b1a0-453d-a6db-2a141054185e', 72: 'b9eb9948-21ce-4e0a-baf9-b050891bb084', 73: '5fc7c388-1ba4-45b3-a3a4-e10a1c72c06c', 74: '2e3d764c-d680-47c8-b596-10896185758e', 75: '0e28f224-5928-4a02-b046-92ecc927fb16', 76: '322fd2d4-3c16-40cd-a052-97135d7c6a12', 77: '1e8fa56c-062d-4d86-9b9e-d2790367123a', 78: '3048698d-5655-4193-bed7-9ce00d67eb2b', 79: '2022d769-14a3-49f0-b618-afdc407ce81c', 80: '9a3f51d6-8f22-4ddb-8bd6-63b52fc78043', 81: '1976e5bc-d1c1-49ad-a389-cc4db421cdf9', 82: 'ac3a7053-0f80-4f7e-acb9-3c0287bee36c', 83: '1999865b-e21f-4f0f-9d99-0ad52d1a466c', 84: '23fa9d7e-7228-4de3-889e-0cecebb18979', 85: '9adb5f03-cc08-4bed-9b23-46e3d7d242ec', 86: 'bbb177c5-22f8-42ab-be69-979077f1cfba', 87: 'eafc9ef7-9aa9-4b6c-a2d6-a334bf5d75ee', 88: '83e7a307-713c-4dcf-957d-3bd7189d9b79', 89: 'a3f9c91e-58ab-4394-b480-7e37ae4d9df2', 90: '93d63cc8-365e-47f4-92c8-750ba9a8d589', 91: '52350d61-d76a-4a27-9b4e-0c43eee8c8d9', 92: '3c05ca72-323b-48a2-8fd3-a8789ee566da'}
______________________________________________
______________________________________________
user user-1 (2) removed from 2 groups [{1: 'user'}, {3: 'private-2'}]
user user-2 (3) removed from 3 groups [{1: 'user'}, {3: 'private-2'}, {4: 'read-only-1'}]
user user-4 (5) removed from 3 groups [{1: 'user'}, {4: 'read-only-1'}, {6: 'read-write-1'}]
user user-5 (6) removed from 3 groups [{1: 'user'}, {5: 'read-annotate-1'}, {6: 'read-write-1'}]
user user-7 (8) removed from 2 groups [{1: 'user'}, {3: 'private-2'}]
user user-8 (9) removed from 3 groups [{1: 'user'}, {3: 'private-2'}, {4: 'read-only-1'}]
user user-9 (10) removed from 3 groups [{1: 'user'}, {4: 'read-only-1'}, {5: 'read-annotate-1'}]
user user-10 (11) removed from 3 groups [{1: 'user'}, {4: 'read-only-1'}, {6: 'read-write-1'}]
user user-11 (12) removed from 3 groups [{1: 'user'}, {5: 'read-annotate-1'}, {6: 'read-write-1'}]
user user-12 (13) removed from 6 groups [{0: 'system'}, {1: 'user'}, {3: 'private-2'}, {4: 'read-only-1'}, {5: 'read-annotate-1'}, {6: 'read-write-1'}]
user adm-user-1 (14) removed from 2 groups [{1: 'user'}, {7: 'adm-private-2'}]
user adm-user-2 (15) removed from 3 groups [{1: 'user'}, {7: 'adm-private-2'}, {8: 'adm-read-only-1'}]
user adm-user-3 (16) removed from 3 groups [{1: 'user'}, {8: 'adm-read-only-1'}, {9: 'adm-read-annotate-1'}]
user adm-user-4 (17) removed from 3 groups [{1: 'user'}, {8: 'adm-read-only-1'}, {10: 'adm-read-write-1'}]
user adm-user-5 (18) removed from 3 groups [{1: 'user'}, {9: 'adm-read-annotate-1'}, {10: 'adm-read-write-1'}]
user adm-user-6 (19) removed from 6 groups [{0: 'system'}, {1: 'user'}, {7: 'adm-private-2'}, {8: 'adm-read-only-1'}, {9: 'adm-read-annotate-1'}, {10: 'adm-read-write-1'}]
user 364e383a-6dc4-456f-a714-a0e7a9a4fc28 (52) removed from 2 groups [{1: 'user'}, {53: 'aac57b72-ba1d-4900-869a-57aec5db668b'}]
user 93c4347c-67b3-487d-87be-dee90c9d9053 (53) removed from 2 groups [{1: 'user'}, {54: '0913db98-9096-4cd2-b528-50662bdf426d'}]
user 0884834e-5c96-486f-b7b6-3896da90f342 (54) removed from 3 groups [{1: 'user'}, {54: '0913db98-9096-4cd2-b528-50662bdf426d'}, {55: '8d123466-c1b5-4429-9135-db16da7e746a'}]
user 7a54f780-0c29-4098-850e-87c6444705a4 (56) removed from 3 groups [{1: 'user'}, {56: '31f8a6fa-0e41-4447-8f75-ee668cc9b472'}, {57: '5a2a53d5-2624-402f-8485-cc9938453f51'}]
user cf599c11-b462-4a03-b23d-d07bd14636f9 (55) removed from 3 groups [{0: 'system'}, {1: 'user'}, {56: '31f8a6fa-0e41-4447-8f75-ee668cc9b472'}]
user e85840ea-7e0c-4c14-9441-11b252330962 (67) removed from 2 groups [{1: 'user'}, {64: '94b0c3cc-7c8c-4184-858c-673ed306b5e2'}]
user 39f8cc5e-5f08-4090-8829-ccec34f1185f (58) removed from 3 groups [{1: 'user'}, {58: '85e54459-ddad-41d4-bb61-836f80e6e2e9'}, {59: '92fd7d0b-3698-496f-9412-1d32b211b054'}]
user 8830379a-9e0e-4abc-be81-7de098121c1e (57) removed from 3 groups [{0: 'system'}, {1: 'user'}, {58: '85e54459-ddad-41d4-bb61-836f80e6e2e9'}]
user c826120c-9d35-46ec-8365-0606c6c6f2f1 (59) removed from 2 groups [{1: 'user'}, {60: 'acc4112d-c7d6-4122-9db0-da7922c8d723'}]
user 6037f119-c9c7-437a-b5fe-6284485aca8b (60) removed from 2 groups [{1: 'user'}, {61: 'c1ab635d-f238-4cb7-9ec8-98d3509999f7'}]
user f6df8848-faee-43e9-a2b4-aba4e80e6500 (61) removed from 2 groups [{1: 'user'}, {61: 'c1ab635d-f238-4cb7-9ec8-98d3509999f7'}]
user e8f43db7-775c-4bd7-99c9-b893ad20ac2f (62) removed from 2 groups [{1: 'user'}, {62: '7ec9e07d-20c8-4aa7-9c5c-bc5beddef9a6'}]
user 938e3121-574e-499b-b088-1b8e65936f7f (68) removed from 2 groups [{1: 'user'}, {65: '45dbefda-d63f-4930-bcf1-b35e255824ad'}]
user 0ea1b3b8-2f3f-4c18-9064-cff1b167659c (63) removed from 2 groups [{1: 'user'}, {62: '7ec9e07d-20c8-4aa7-9c5c-bc5beddef9a6'}]
user 5a2caab3-f285-47b2-98a2-6d19548c2d66 (64) removed from 2 groups [{1: 'user'}, {63: 'c2bcfcbf-e4b6-4725-ac85-af9dadfa81cf'}]
user e13978ce-da76-49cc-bb88-b306283a5d13 (65) removed from 2 groups [{1: 'user'}, {63: 'c2bcfcbf-e4b6-4725-ac85-af9dadfa81cf'}]
user 37dd3bb3-db17-48f3-b80c-932ce2eb7d01 (66) removed from 2 groups [{1: 'user'}, {64: '94b0c3cc-7c8c-4184-858c-673ed306b5e2'}]
user 3135fc9c-824e-4191-983e-dc93a71c8e2e (69) removed from 2 groups [{1: 'user'}, {65: '45dbefda-d63f-4930-bcf1-b35e255824ad'}]
user 73e745d4-015a-49fb-8fe5-f80a08304f4d (70) removed from 2 groups [{1: 'user'}, {66: 'c6e5ddcf-97d8-4405-be83-e89ae302a424'}]
user 16ee4b03-b1a0-453d-a6db-2a141054185e (71) removed from 2 groups [{1: 'user'}, {66: 'c6e5ddcf-97d8-4405-be83-e89ae302a424'}]
user b9eb9948-21ce-4e0a-baf9-b050891bb084 (72) removed from 2 groups [{1: 'user'}, {67: '58dd561c-8419-4e0f-a7c0-75b220901426'}]
user 5fc7c388-1ba4-45b3-a3a4-e10a1c72c06c (73) removed from 2 groups [{1: 'user'}, {67: '58dd561c-8419-4e0f-a7c0-75b220901426'}]
user 2e3d764c-d680-47c8-b596-10896185758e (74) removed from 2 groups [{1: 'user'}, {68: '9d63e4e4-2aba-472c-a636-bcb275d18fb9'}]
user 0e28f224-5928-4a02-b046-92ecc927fb16 (75) removed from 2 groups [{1: 'user'}, {69: '7d2a95c2-a929-41ec-9844-456cb0d32583'}]
user 322fd2d4-3c16-40cd-a052-97135d7c6a12 (76) removed from 2 groups [{1: 'user'}, {70: '1977e9f9-6b62-403e-ba1a-c2030c216114'}]
user 1e8fa56c-062d-4d86-9b9e-d2790367123a (77) removed from 2 groups [{1: 'user'}, {71: 'c408630e-6301-436c-90ca-7b7c166dbb24'}]
user 3048698d-5655-4193-bed7-9ce00d67eb2b (78) removed from 2 groups [{1: 'user'}, {71: 'c408630e-6301-436c-90ca-7b7c166dbb24'}]
user 2022d769-14a3-49f0-b618-afdc407ce81c (79) removed from 2 groups [{1: 'user'}, {72: '8c3b27d5-7e0b-463e-b865-dad34970ed61'}]
user 9a3f51d6-8f22-4ddb-8bd6-63b52fc78043 (80) removed from 2 groups [{1: 'user'}, {72: '8c3b27d5-7e0b-463e-b865-dad34970ed61'}]
user 1976e5bc-d1c1-49ad-a389-cc4db421cdf9 (81) removed from 2 groups [{1: 'user'}, {73: '6c5c72fc-9ad6-4552-b51a-35a2b56d2bb6'}]
user ac3a7053-0f80-4f7e-acb9-3c0287bee36c (82) removed from 2 groups [{1: 'user'}, {73: '6c5c72fc-9ad6-4552-b51a-35a2b56d2bb6'}]
user 1999865b-e21f-4f0f-9d99-0ad52d1a466c (83) removed from 2 groups [{1: 'user'}, {74: '5596e598-33f2-48fc-b017-f48f9f9e789b'}]
user 23fa9d7e-7228-4de3-889e-0cecebb18979 (84) removed from 2 groups [{1: 'user'}, {74: '5596e598-33f2-48fc-b017-f48f9f9e789b'}]
user 9adb5f03-cc08-4bed-9b23-46e3d7d242ec (85) removed from 2 groups [{1: 'user'}, {75: 'ee4223ed-3a4f-4818-ad67-47514427b756'}]
user bbb177c5-22f8-42ab-be69-979077f1cfba (86) removed from 2 groups [{1: 'user'}, {75: 'ee4223ed-3a4f-4818-ad67-47514427b756'}]
user eafc9ef7-9aa9-4b6c-a2d6-a334bf5d75ee (87) removed from 2 groups [{1: 'user'}, {76: '2d2b858b-a25d-4d6a-8c39-3312e79fa79a'}]
user 83e7a307-713c-4dcf-957d-3bd7189d9b79 (88) removed from 2 groups [{1: 'user'}, {76: '2d2b858b-a25d-4d6a-8c39-3312e79fa79a'}]
user a3f9c91e-58ab-4394-b480-7e37ae4d9df2 (89) removed from 2 groups [{1: 'user'}, {77: '4c612944-3f1f-4a84-a963-5f6c61a53d67'}]
user 93d63cc8-365e-47f4-92c8-750ba9a8d589 (90) removed from 2 groups [{1: 'user'}, {77: '4c612944-3f1f-4a84-a963-5f6c61a53d67'}]
user 52350d61-d76a-4a27-9b4e-0c43eee8c8d9 (91) removed from 2 groups [{1: 'user'}, {78: 'c65b6089-9072-443a-8f64-0248f5c8c4f2'}]
user 3c05ca72-323b-48a2-8fd3-a8789ee566da (92) removed from 1 groups [{1: 'user'}]
#################
#######DONE######
#################

After the run of the script, the output of group list and user list was

omero group list
Using session for root@localhost:4064. Idle timeout: 10 min. Current group: system
 id | name                                 | perms  | ldap  | # of owners | # of members 
----+--------------------------------------+--------+-------+-------------+--------------
 0  | system                               | rw---- | False | 1           | 1            
 1  | user                                 | rwr-r- | False | 0           | 3            
 2  | guest                                | rw---- | False | 0           | 1            
 3  | private-2                            | rw---- | False | 0           | 2            
 4  | read-only-1                          | rwr--- | False | 1           | 1            
 5  | read-annotate-1                      | rwra-- | False | 0           | 2            
 6  | read-write-1                         | rwrw-- | False | 0           | 1            
 7  | adm-private-2                        | rw---- | False | 0           | 0            
 8  | adm-read-only-1                      | rwr--- | False | 0           | 0            
 9  | adm-read-annotate-1                  | rwra-- | False | 0           | 0            
 10 | adm-read-write-1                     | rwrw-- | False | 0           | 0            
 53 | aac57b72-ba1d-4900-869a-57aec5db668b | rw---- | False | 0           | 0            
 54 | 0913db98-9096-4cd2-b528-50662bdf426d | rwr--- | False | 0           | 0            
 55 | 8d123466-c1b5-4429-9135-db16da7e746a | rwr--- | False | 0           | 0            
 56 | 31f8a6fa-0e41-4447-8f75-ee668cc9b472 | rwr--- | False | 0           | 0            
 57 | 5a2a53d5-2624-402f-8485-cc9938453f51 | rwr--- | False | 0           | 0            
 58 | 85e54459-ddad-41d4-bb61-836f80e6e2e9 | rwr--- | False | 0           | 0            
 59 | 92fd7d0b-3698-496f-9412-1d32b211b054 | rwr--- | False | 0           | 0            
 60 | acc4112d-c7d6-4122-9db0-da7922c8d723 | rw---- | False | 0           | 0            
 61 | c1ab635d-f238-4cb7-9ec8-98d3509999f7 | rwr--- | False | 0           | 0            
 62 | 7ec9e07d-20c8-4aa7-9c5c-bc5beddef9a6 | rwr--- | False | 0           | 0            
 63 | c2bcfcbf-e4b6-4725-ac85-af9dadfa81cf | rwr--- | False | 0           | 0            
 64 | 94b0c3cc-7c8c-4184-858c-673ed306b5e2 | rwr--- | False | 0           | 0            
 65 | 45dbefda-d63f-4930-bcf1-b35e255824ad | rwr--- | False | 0           | 0            
 66 | c6e5ddcf-97d8-4405-be83-e89ae302a424 | rwr--- | False | 0           | 0            
 67 | 58dd561c-8419-4e0f-a7c0-75b220901426 | rwr--- | False | 0           | 0            
 68 | 9d63e4e4-2aba-472c-a636-bcb275d18fb9 | rw---- | False | 0           | 0            
 69 | 7d2a95c2-a929-41ec-9844-456cb0d32583 | rwr--- | False | 0           | 0            
 70 | 1977e9f9-6b62-403e-ba1a-c2030c216114 | rw---- | False | 0           | 0            
 71 | c408630e-6301-436c-90ca-7b7c166dbb24 | rwr--- | False | 0           | 0            
 72 | 8c3b27d5-7e0b-463e-b865-dad34970ed61 | rwr--- | False | 0           | 0            
 73 | 6c5c72fc-9ad6-4552-b51a-35a2b56d2bb6 | rwr--- | False | 0           | 0            
 74 | 5596e598-33f2-48fc-b017-f48f9f9e789b | rwr--- | False | 0           | 0            
 75 | ee4223ed-3a4f-4818-ad67-47514427b756 | rwr--- | False | 0           | 0            
 76 | 2d2b858b-a25d-4d6a-8c39-3312e79fa79a | rwr--- | False | 0           | 0            
 77 | 4c612944-3f1f-4a84-a963-5f6c61a53d67 | rwr--- | False | 0           | 0            
 78 | c65b6089-9072-443a-8f64-0248f5c8c4f2 | rw---- | False | 0           | 0            
 79 | 9fab258b-3bad-4a78-b3b6-b7f9bee6552e | rwr--- | False | 0           | 57           
(38 rows)
(cli) pwalczysko@ls31618~/inactivate-users$ omero user list
Using session for root@localhost:4064. Idle timeout: 10 min. Current group: system
 id | login                                | first name  | last name  | email | active | ldap  | admin | member of | owner of 
----+--------------------------------------+-------------+------------+-------+--------+-------+-------+-----------+----------
 0  | root                                 | root        | root       |       | Yes    | False | Yes   | 3         |          
 1  | guest                                | Guest       | Account    |       |        | False |       | 2         |          
 2  | user-1                               | user-1      | user-1     |       |        | False |       | 79        |          
 3  | user-2                               | user-2      | user-2     |       |        | False |       | 79        |          
 4  | user-3                               | user-3      | user-3     |       | Yes    | False |       | 5         | 4        
 5  | user-4                               | user-4      | user-4     |       |        | False |       | 79        |          
 6  | user-5                               | user-5      | user-5     |       |        | False |       | 79        |          
 7  | user-6                               | user-6      | user-6     |       | Yes    | False | Yes   | 3,4,5,6   |          
 8  | user-7                               | user-7      | user-7     |       |        | False |       | 79        |          
 9  | user-8                               | user-8      | user-8     |       |        | False |       | 79        |          
 10 | user-9                               | user-9      | user-9     |       |        | False |       | 79        |          
 11 | user-10                              | user-10     | user-10    |       |        | False |       | 79        |          
 12 | user-11                              | user-11     | user-11    |       |        | False |       | 79        |          
 13 | user-12                              | user-12     | user-12    |       |        | False |       | 79        |          
 14 | adm-user-1                           | adm-user-1  | adm-user-1 |       |        | False |       | 79        |          
 15 | adm-user-2                           | adm-user-2  | adm-user-2 |       |        | False |       | 79        |          
 16 | adm-user-3                           | adm-user-3  | adm-user-3 |       |        | False |       | 79        |          
 17 | adm-user-4                           | adm-user-4  | adm-user-4 |       |        | False |       | 79        |          
 18 | adm-user-5                           | adm-user-5  | adm-user-5 |       |        | False |       | 79        |          
 19 | adm-user-6                           | adm-user-6  | adm-user-6 |       |        | False |       | 79        |          
 52 | 364e383a-6dc4-456f-a714-a0e7a9a4fc28 | integration | tester     |       |        | False |       | 79        |          
 53 | 93c4347c-67b3-487d-87be-dee90c9d9053 | integration | tester     |       |        | False |       | 79        |          
 54 | 0884834e-5c96-486f-b7b6-3896da90f342 | integration | tester     |       |        | False |       | 79        |          
 55 | cf599c11-b462-4a03-b23d-d07bd14636f9 | integration | tester     |       |        | False |       | 79        |          
 56 | 7a54f780-0c29-4098-850e-87c6444705a4 | integration | tester     |       |        | False |       | 79        |          
 57 | 8830379a-9e0e-4abc-be81-7de098121c1e | integration | tester     |       |        | False |       | 79        |          
 58 | 39f8cc5e-5f08-4090-8829-ccec34f1185f | integration | tester     |       |        | False |       | 79        |          
 59 | c826120c-9d35-46ec-8365-0606c6c6f2f1 | integration | tester     |       |        | False |       | 79        |          
 60 | 6037f119-c9c7-437a-b5fe-6284485aca8b | integration | tester     |       |        | False |       | 79        |          
 61 | f6df8848-faee-43e9-a2b4-aba4e80e6500 | a           | user       |       |        | False |       | 79        |          
 62 | e8f43db7-775c-4bd7-99c9-b893ad20ac2f | integration | tester     |       |        | False |       | 79        |          
 63 | 0ea1b3b8-2f3f-4c18-9064-cff1b167659c | a           | user       |       |        | False |       | 79        |          
 64 | 5a2caab3-f285-47b2-98a2-6d19548c2d66 | integration | tester     |       |        | False |       | 79        |          
 65 | e13978ce-da76-49cc-bb88-b306283a5d13 | a           | user       |       |        | False |       | 79        |          
 66 | 37dd3bb3-db17-48f3-b80c-932ce2eb7d01 | integration | tester     |       |        | False |       | 79        |          
 67 | e85840ea-7e0c-4c14-9441-11b252330962 | a           | user       |       |        | False |       | 79        |          
 68 | 938e3121-574e-499b-b088-1b8e65936f7f | integration | tester     |       |        | False |       | 79        |          
 69 | 3135fc9c-824e-4191-983e-dc93a71c8e2e | integration | tester     |       |        | False |       | 79        |          
 70 | 73e745d4-015a-49fb-8fe5-f80a08304f4d | integration | tester     |       |        | False |       | 79        |          
 71 | 16ee4b03-b1a0-453d-a6db-2a141054185e | a           | user       |       |        | False |       | 79        |          
 72 | b9eb9948-21ce-4e0a-baf9-b050891bb084 | integration | tester     |       |        | False |       | 79        |          
 73 | 5fc7c388-1ba4-45b3-a3a4-e10a1c72c06c | a           | user       |       |        | False |       | 79        |          
 74 | 2e3d764c-d680-47c8-b596-10896185758e | integration | tester     |       |        | False |       | 79        |          
 75 | 0e28f224-5928-4a02-b046-92ecc927fb16 | integration | tester     |       |        | False |       | 79        |          
 76 | 322fd2d4-3c16-40cd-a052-97135d7c6a12 | integration | tester     |       |        | False |       | 79        |          
 77 | 1e8fa56c-062d-4d86-9b9e-d2790367123a | integration | tester     |       |        | False |       | 79        |          
 78 | 3048698d-5655-4193-bed7-9ce00d67eb2b | a           | user       |       |        | False |       | 79        |          
 79 | 2022d769-14a3-49f0-b618-afdc407ce81c | integration | tester     |       |        | False |       | 79        |          
 80 | 9a3f51d6-8f22-4ddb-8bd6-63b52fc78043 | a           | user       |       |        | False |       | 79        |          
 81 | 1976e5bc-d1c1-49ad-a389-cc4db421cdf9 | integration | tester     |       |        | False |       | 79        |          
 82 | ac3a7053-0f80-4f7e-acb9-3c0287bee36c | a           | user       |       |        | False |       | 79        |          
 83 | 1999865b-e21f-4f0f-9d99-0ad52d1a466c | integration | tester     |       |        | False |       | 79        |          
 84 | 23fa9d7e-7228-4de3-889e-0cecebb18979 | a           | user       |       |        | False |       | 79        |          
 85 | 9adb5f03-cc08-4bed-9b23-46e3d7d242ec | integration | tester     |       |        | False |       | 79        |          
 86 | bbb177c5-22f8-42ab-be69-979077f1cfba | integration | tester     |       |        | False |       | 79        |          
 87 | eafc9ef7-9aa9-4b6c-a2d6-a334bf5d75ee | integration | tester     |       |        | False |       | 79        |          
 88 | 83e7a307-713c-4dcf-957d-3bd7189d9b79 | a           | user       |       |        | False |       | 79        |          
 89 | a3f9c91e-58ab-4394-b480-7e37ae4d9df2 | integration | tester     |       |        | False |       | 79        |          
 90 | 93d63cc8-365e-47f4-92c8-750ba9a8d589 | a           | user       |       |        | False |       | 79        |          
 91 | 52350d61-d76a-4a27-9b4e-0c43eee8c8d9 | integration | tester     |       |        | False |       | 79        |          
 92 | 3c05ca72-323b-48a2-8fd3-a8789ee566da | integration | tester     |       |        | False |       | 79        |          
(61 rows)

which is plausible imho.
The login of user-3 and user-6 and root went fine, the login of user-4 was blocked as expected.

In summary, I think this is a very useful script.
Nevertheless, in larger production systems, it must be executed with caution, indeed, it would make a good sense to have a reverse all back to original state provision in the script - the error (for example putting in too low inactive days number, or having the named users list which should not be inactivated incorrectly) can cause a maintenance problem.
At least there should be a
Proceed:y/N prompt after the facts were gathered and displayed before the script executes the move.
Definitely good to have the pre-run of the script captured first.

@pwalczysko
Copy link
Member

@sbesson : When consulting the workflow with @will-moore , an important consideration came up. If we delete users, and the deleted user is logging in again using LDAP, this will re-create their account and they will have no detrimental user experience. This is not a given for the workflow of the graveyard group suggested here. The disabled user

  • will not be able to log in at all (present state of the script) or
  • they will be unable to view the data (because they are in the graveyard group (in case we would tailor the script not to disable the users)

@pwalczysko
Copy link
Member

The test on a local OMERO and OMERO.web clearly shows that

  • in a situation where there are over 600 users in one group, the users in that group are experiencing slow performance
    • in RHP of webclient
    • in viewers (old viewer tested on my setup, but learning has problems with iviewer too).
  • Both of these parameters improve markedly by removal of 630 users into graveyard group using the script here.

The RHP load speed is:

  • 9184 ms with 700 users in the group
  • 341 ms with ca 10 users in the group

This is an improvement of more than order of magnitude, but mainly, it also completely restores the feeling of responsiveness in webclient for a user using the UI to acceptable levels.

I think this shows that the script would be very useful, except for #363 (comment)

@pwalczysko
Copy link
Member

Note that the slowness is not perceived by an admin which is not a member of the group where the data are located.
Unfortunately, unless we adjust the permissions of all the relevant users to be restricted admins on the learning system, we cannot exploit that quirk I guess...

@pwalczysko
Copy link
Member

pwalczysko commented Aug 24, 2022

A new idea: the #363 (comment) could be solved by forcing the graveyard user to unhook from LDAP, and changing its loginname. In this way, when the same LDAP user logs in again, the account will be re-created ? What do you think @sbesson ?

@sbesson
Copy link
Member Author

sbesson commented Aug 24, 2022

Interesting and the approach makes sense to me. I assume we will have to handle the scenario where an archived LDAP user is recreated and archived a second time as it might lead to name conflict.

Note the current maintenance process i.e. the deletion of old users with no associated data via a custom SQL script, meets the minimal requirements in terms of restoring performance of a system with a growing number of temporary users in a single group. The workflow discussed above would have the benefit of keeping the users in the DB which offers advantages mostly in terms of reporting as it allows to introspect historical data while collecting usage metrics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants