Documentation for nextResults and bulk delete

site/docs/src/json/entities/responses.json

@@ -22,5 +22,7 @@
         "name": "Customers"
       }
     }
-  ]
+  ],
+  "nextResults": "eyJscyI6WyIxLjAxNzM5MTciLCJlbnRpdHktMSIsImE3OTA0NTZmLTFmZTUtNGE2ZS04MWNkLWZmOTA3MmJmMzRmYiJdLCJxcyI6IioiLCJzZiI6W119",
+  "total": 1
 }

site/docs/src/json/entities/search-nextResults-request.json

@@ -0,0 +1,6 @@
+{
+  "search": {
+    "numberOfResults": 25,
+    "nextResults": "eyJscyI6WyIxLjAxNzM5MTciLCJlbnRpdHktMSIsImE3OTA0NTZmLTFmZTUtNGE2ZS04MWNkLWZmOTA3MmJmMzRmYiJdLCJxcyI6IioiLCJzZiI6W119"
+  }
+}

site/docs/src/json/users/bulk-delete-queryString-group-request.json

@@ -1,5 +1,6 @@
 {
   "dryRun": true,
   "hardDelete": true,
+  "limit": 10000,
   "queryString": "memberships.groupId:7f318a9d-3de4-4ded-9b53-1d97fd730868"
 }

site/docs/src/json/users/search-nextResults-request.json

@@ -0,0 +1,6 @@
+{
+  "search": {
+    "numberOfResults": 25,
+    "nextResults": "eyJscyI6WyIxLjAwMTQ2MTkiLG51bGwsInRlc3R1c2VyOTkwOUBsb2NhbC5jb20iLCJjNmI4ZjQyNC0wOTRjLTQ1MWYtYWMxNS05Y2ZkODI3NTZlNGEiXSwicXMiOiIqIiwic2YiOltdfQ"
+  }
+}

site/docs/src/json/users/search-queryString-all-users-elasticsearch-request.json

@@ -0,0 +1,15 @@
+{
+  "search": {
+    "accurateTotal": true,
+    "numberOfResults": 25,
+    "queryString": "*",
+    "sortFields": [
+      {
+        "missing": "_first",
+        "name": "email",
+        "order": "asc"
+      }
+    ],
+    "startRow": 0
+  }
+}

site/docs/src/json/users/users-search-response.json

@@ -1,4 +1,5 @@
 {
+  "nextResults": "eyJscyI6WyIxLjAwMTQ2MTkiLG51bGwsInRlc3R1c2VyOTkwOUBsb2NhbC5jb20iLCJjNmI4ZjQyNC0wOTRjLTQ1MWYtYWMxNS05Y2ZkODI3NTZlNGEiXSwicXMiOiIqIiwic2YiOltdfQ",
   "total": 1,
   "users": [
     {

site/docs/v1/tech/apis/_user-bulk-delete-request-body.adoc

@@ -7,6 +7,11 @@ To preview the user Ids to be deleted by the request without applying the reques
 [field]#hardDelete# [type]#[Boolean]# [optional]#Optional# [default]#defaults to `false`#::
 To Permanently delete a user from FusionAuth set this value to `true`. Once a user has been permanently deleted, the action cannot be undone. When this value is set to `false` the user is marked as inactive and the user will be unable log into FusionAuth. This action may be undone by reactivating the user.
 
+[field]#limit# [type]#[Integer]# [optional]#Optional# [since]#Available since 1.48.0# [default]#defaults to `10,000`# ::
+The maximum number of users to delete in one call.
++
+You may use this parameter to process deletes in batches in order to limit individual request processing time and the number of user Ids on the response.
+
 [field]#query# [type]#[String]# [optional]#Optional# [since]#Available since 1.13.0#::
 The raw JSON Elasticsearch query that is used to search for Users. The [field]#userIds#, [field]#query#, and [field]#queryString#  parameters are mutually exclusive, they are listed here in order of precedence.
 +
@@ -79,7 +84,7 @@ include::../../../src/json/users/bulk-delete-queryString-email-request.json[]
 ----
 
 [source,json]
-.Example Request JSON searching by `queryString` for users belonging to a specific group
+.Example Request JSON searching by `queryString` for users belonging to a specific group with a limit
 ----
 include::../../../src/json/users/bulk-delete-queryString-group-request.json[]
 ----

site/docs/v1/tech/apis/_user-search-request-body-elasticsearch-examples.adoc

@@ -127,3 +127,9 @@ include::../../../src/json/users/search-registration-data-range-query.json[]
 include::../../../src/json/users/search-data-exists-query.json[]
 ----
 
+[source,json]
+.Example Request JSON searching by [field]#nextResults# for users using a token returned in a previous search response
+----
+include::../../../src/json/users/search-nextResults-request.json[]
+----
+

site/docs/v1/tech/apis/_user-search-request-parameters.adoc

@@ -37,6 +37,17 @@ Using this parameter will not hit the Elasticsearch index, instead it will cause
 endif::[]
 endif::[]
 
+ifdef::elasticsearch_search_engine_type[]
+[field]#{parameter_prefix}nextResults# [type]#[String]# [optional]#Optional# [since]#Available since 1.48.0#::
+The encoded token returned in the [field]#nextResults# field of a previous search response. Providing this token will return a result set for the page following the last result from the previous search response.
++
+This parameter can be used to successfully page past the `max_result_window` limitation.
++
+This parameter cannot be used with the [field]#ids#, [field]#query#, [field]#queryString#, or [field]#sortFields# parameters. The [field]#startRow# parameter must be 0 or omitted.
++
+See link:/docs/v1/tech/core-concepts/search#extended-pagination[Extended Pagination] for more information.
+endif::[]
+
 [field]#{parameter_prefix}numberOfResults# [type]#[Integer]# [optional]#Optional# [default]#defaults to `25`#::
 The number of search results to return. Used for pagination.
 

site/docs/v1/tech/apis/_users-response-body.adoc

@@ -6,6 +6,11 @@ ifdef::search_result[]
 The number of users that matched the search criteria. The number of results in the response will be equal to the `numberOfResults` provided on the request, if not provided the default number of results will be returned.
 +
 This value may cap out at 10,000 even if more users are found based upon the provided search query. Use the `accurateTotal` request parameter if you require this value to represent the actual number of matching users in the search index when your query may match more than 10,000 users.
+
+[field]#nextResults# [type]#[String]# [since]#Available since 1.48.0#::
+An encoded token that can be used in subsequent search requests to page forward after the last result of the current response. Will only be returned if using the Elasticsearch engine.
++
+See link:/docs/v1/tech/core-concepts/search#extended-pagination[Extended Pagination] for more information.
 endif::[]
 
 [field]#users# [type]#[Array]#::

site/docs/v1/tech/apis/entity-management/_entity-response-body-base.adoc

@@ -1,4 +1,16 @@
 [api]
+ifdef::search_result[]
+[field]#total# [type]#[Long]#::
+The number of entities that matched the search criteria. The number of results in the response will be equal to the `numberOfResults` provided on the request, if not provided the default number of results will be returned.
++
+This value may cap out at 10,000 even if more entities are found based upon the provided search query. Use the `accurateTotal` request parameter if you require this value to represent the actual number of matching entities in the search index when your query may match more than 10,000 entities.
+
+[field]#nextResults# [type]#[String]# [since]#Available since 1.48.0#::
+An encoded token that can be used in subsequent search requests to page forward after the last result of the current response. Will only be returned if using the Elasticsearch engine.
++
+See link:/docs/v1/tech/core-concepts/search#extended-pagination[Extended Pagination] for more information.
+endif::[]
+
 ifeval::["{base_field_name}" != "entity"]
 [field]#{base_field_name}# [type]#[Array]#::
 The list of Entity objects.

site/docs/v1/tech/apis/entity-management/_entity-search-request-body-elasticsearch-examples.adoc

@@ -27,3 +27,9 @@ include::../../../../src/json/entities/search-queryString-name-request.json[]
 ----
 include::../../../../src/json/entities/search-queryString-tenant-request.json[]
 ----
+
+[source,json]
+.Example Request JSON searching by [field]#nextResults# for entities using a token returned in a previous search response
+----
+include::../../../../src/json/entities/search-queryString-tenant-request.json[]
+----

site/docs/v1/tech/apis/entity-management/_entity-search-request-parameters.adoc

@@ -37,6 +37,17 @@ Using this parameter will not hit the Elasticsearch index, instead it will cause
 endif::[]
 endif::[]
 
+ifdef::elasticsearch_search_engine_type[]
+[field]#{parameter_prefix}nextResults# [type]#[String]# [optional]#Optional# [since]#Available since 1.48.0#::
+The encoded token returned in the [field]#nextResults# of a previous search response. Providing this token will return a result set for the page following the last result from the previous search response.
++
+This parameter can be used to successfully page past the `max_result_window` limitation.
++
+This parameter cannot be used with the [field]#ids#, [field]#query#, [field]#queryString#, or [field]#sortFields# parameters. The [field]#startRow# parameter must be 0 or omitted.
++
+See link:/docs/v1/tech/core-concepts/search#extended-pagination[Extended Pagination] for more information.
+endif::[]
+
 [field]#{parameter_prefix}numberOfResults# [type]#[Integer]# [optional]#Optional# [default]#defaults to `25`#::
 The number of search results to return. Used for pagination.
 

site/docs/v1/tech/apis/entity-management/entities.adoc

@@ -271,6 +271,14 @@ link:/docs/v1/tech/apis/authentication#api-key-authentication[icon:lock[type=fas
 [method]#GET# [uri]#/api/entity/search``?query=\{query\}``#
 --
 
+[.api-authentication]
+link:/docs/v1/tech/apis/authentication#api-key-authentication[icon:lock[type=fas]] Search for Entities by a `nextResults` token
+[.endpoint]
+.URI
+--
+[method]#GET# [uri]#/api/entity/search``?nextResults=\{nextResults\}``#
+--
+
 ==== Request Parameters
 
 :query_string_request: 

site/docs/v1/tech/apis/users.adoc

@@ -316,15 +316,15 @@ link:/docs/v1/tech/apis/authentication#api-key-authentication[icon:lock[type=fas
 [.endpoint]
 .URI
 --
-[method]#DELETE# [uri]#/api/user/bulk``?queryString=\{queryString}&dryRun=false``#
+[method]#DELETE# [uri]#/api/user/bulk``?queryString=\{queryString}&dryRun=false&limit=10000``#
 --
 
 [.api-authentication]
 link:/docs/v1/tech/apis/authentication#api-key-authentication[icon:lock[type=fas]] Deactivate Users by Raw JSON Search Query
 [.endpoint]
 .URI
 --
-[method]#DELETE# [uri]#/api/user/bulk``?query=\{query\}&dryRun=false``#
+[method]#DELETE# [uri]#/api/user/bulk``?query=\{query\}&dryRun=false&limit=10000``#
 --
 
 [.api-authentication]
@@ -340,15 +340,15 @@ link:/docs/v1/tech/apis/authentication#api-key-authentication[icon:lock[type=fas
 [.endpoint]
 .URI
 --
-[method]#DELETE# [uri]#/api/user/bulk``?queryString=\{queryString\}&hardDelete=true&dryRun=false``#
+[method]#DELETE# [uri]#/api/user/bulk``?queryString=\{queryString\}&hardDelete=true&dryRun=false&limit=10000``#
 --
 
 [.api-authentication]
 link:/docs/v1/tech/apis/authentication#api-key-authentication[icon:lock[type=fas]] Permanently Delete Users by Raw JSON Search Query
 [.endpoint]
 .URI
 --
-[method]#DELETE# [uri]#/api/user/bulk``?query=\{query\}&hardDelete=true&dryRun=false``#
+[method]#DELETE# [uri]#/api/user/bulk``?query=\{query\}&hardDelete=true&dryRun=false&limit=10000``#
 --
 
 ==== Request Parameters
@@ -360,6 +360,11 @@ To preview the user Ids to be deleted by the request without applying the reques
 [field]#hardDelete# [type]#[Boolean]# [optional]#Optional# [default]#defaults to `false`#::
 To Permanently delete a user from FusionAuth set this value to `true`. Once a user has been permanently deleted, the action cannot be undone. When this value is set to `false` the user is marked as inactive and the user will be unable log into FusionAuth. This action may be undone by reactivating the user.
 
+[field]#limit# [type]#[Integer]# [optional]#Optional# [default]#defaults to `10,000`# [since]#Available since 1.48.0#::
+The maximum number of users to delete in one call.
++
+You may use this parameter to process deletes in batches in order to limit individual request processing time and the number of user Ids on the response.
+
 [field]#query# [type]#[String]# [optional]#Optional# [since]#Available since 1.13.0#::
 The raw JSON Elasticsearch query that is used to search for Users. The [field]#userId#, [field]#query#, and [field]#queryString#  parameters are mutually exclusive, they are listed here in order of precedence.
 +
@@ -682,6 +687,14 @@ link:/docs/v1/tech/apis/authentication#api-key-authentication[icon:lock[type=fas
 [method]#GET# [uri]#/api/user/search``?query=\{query\}``#
 --
 
+[.api-authentication]
+link:/docs/v1/tech/apis/authentication#api-key-authentication[icon:lock[type=fas]] Search for Users with a `nextResults` token
+[.endpoint]
+.URI
+--
+[method]#GET# [uri]#/api/user/search``?nextResults=\{nextResults\}``#
+--
+
 ==== Request Parameters
 
 :query_string_request:

site/docs/v1/tech/core-concepts/search.adoc

@@ -9,12 +9,13 @@ navcategory: admin
 
 == Overview
 
-FusionAuth ships with two options for search: a simple search via the database search engine and the Elasticsearch database engine.
+FusionAuth ships with two options for search: a simple search via the database search engine and the Elasticsearch search engine.
 
 * <<Configuration>>
 * <<Using the Database Search Engine>>
 * <<Using the Elasticsearch Search Engine>>
 * <<Pagination>>
+** <<Extended Pagination>>
 
 == Configuration
 
@@ -81,4 +82,29 @@ include::docs/v1/tech/shared/_re-indexing-elasticsearch.adoc[]
 
 include::docs/v1/tech/shared/_paginating-search-results.adoc[]
 
-Note that you'll only be able to get back 10000 results no matter how you paginate. See the link:/docs/v1/tech/reference/limitations#user-searches[Known Limitations] section for more and some workarounds.
+Note that prior to version 1.48.0 you'll only be able to get back 10,000 results no matter how you paginate. See the link:/docs/v1/tech/reference/limitations#user-searches[Known Limitations] section for more and some workarounds.
+
+As of version 1.48.0, FusionAuth you can use a [field]#nextResults# token to page past the 10,000 results limitation when using the Elasticsearch engine.
+
+See the <<Extended Pagination>> section for more details.
+
+=== Extended Pagination
+
+[NOTE.since]
+====
+Available since 1.48.0
+====
+
+Starting in version 1.48.0 of FusionAuth every search using the Elasticsearch engine will return a [field]#nextResults# token in the response. You may use this token on subsequent calls to continue paging forward in the result set and go past the 10,000 results limit with Elasticsearch. Every search request made using a [field]#nextResults# token will return a page of results and another [field]#nextResults# token that can be used to retrieve the next page. These calls can be made indefinitely until the number of returned results is 0.
+
+Internally FusionAuth uses this token to make a `search_after` parameter on the request to Elasticsearch. See link:https://www.elastic.co/guide/en/elasticsearch/reference/7.17/paginate-search-results.html[the Elasticsearch documentation] for more information.
+
+[NOTE]
+====
+Because this call relies on the parameters of a previous search request, you cannot supply new parameters in a search request with a [field]#nextResults# token which would change the query to Elasticsearch.
+
+As such, when using a [field]#nextResults# token in a search request you may only supply the [field]#numberOfResults# as an additional request parameter.
+====
+
+For more detail on extended pagination see the link:/docs/v1/tech/guides/user-search-with-elasticsearch[Searching Users With Elasticsearch] guide.
+

site/docs/v1/tech/core-concepts/users.adoc

@@ -731,6 +731,27 @@ image::core-concepts/user-search-json-query.png[User Search with JSON Query.,wid
 
 To learn more about the Elasticsearch search engine in general, view the link:/docs/v1/tech/guides/user-search-with-elasticsearch[Elasticsearch search guide].
 
+[NOTE.since]
+====
+Prior to version 1.48.0 if you attempted to page past 10,000 results in the Admin UI you would encounter an error.
+
+Starting in version 1.48.0 you are able to page past the 10,000 results limit one page at a time.
+====
+
+When there are more than 10,000 results when using the Elasticsearch engine you have the option to navigate past the limit, however you may only do so one page at a time.
+
+image::core-concepts/user-search-extended-a.png[User Search with over 10000 results first page,width=1200,role=bottom-cropped]
+
+Clicking on the [uielement]#Last# pagination button will take you to the page with the 10,000th result. You will then have the option to click on the [uielement]#Next# button to continue paging forward.
+
+image::core-concepts/user-search-extended-b.png[User Search with over 10000 results paged to 10000,width=1200,role=bottom-cropped]
+
+You may continue to page forward using [uielement]#Next# until you reach the last of the search results.
+
+image::core-concepts/user-search-extended-c.png[User Search with over 10000 results paged to end,width=1200,role=bottom-cropped]
+
+For more information see link:/docs/v1/tech/core-concepts/search#extended-pagination[Extended Pagination]
+
 ==== Limitations On Changing Data Field Types
 
 include::docs/v1/tech/shared/_data-field-data-type-changes.adoc[]