Search API max count is 10k... (docs, UI)

[ChristopherChudzicki]

The https://mitopen-rc.odl.mit.edu/api/v1/content_file_search/ API endpoint includes a count: number property in its response. However, this value maxes out at 10k.

See https://mitopen-rc.odl.mit.edu/api/v1/content_file_search/ for example.

The 10k value is inaccurate.

To-do:

This may be a technical limitation of opensearch. From https://opensearch.org/docs/latest/search-plugins/searching-data/paginate/

The from and size parameters allow you to paginate your search results but with a limit of 10,000 results at a time.

Though there appear to be other pagination options.

If supporting pagination beyond 10k is a pain, we could at least document the fact that 10k is the max count returned from our APIs. Our UIs could show 10,000+ in these cases.