Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert using objectId as tie breaker in infinite scrolling #1799

Closed
4 tasks done
mtrezza opened this issue Sep 14, 2021 · 6 comments
Closed
4 tasks done

Revert using objectId as tie breaker in infinite scrolling #1799

mtrezza opened this issue Sep 14, 2021 · 6 comments
Labels
type:bug Impaired feature or lacking behavior that is likely assumed

Comments

@mtrezza
Copy link
Member

mtrezza commented Sep 14, 2021

New Issue Checklist

Issue Description

Since #1706, creating an index to optimize dashboard queries requires compound indices including objectId. For example, when browsing a class sorted by createdAt a compound index createdAt, objectId is required.

if (field !== 'objectId') {
if (sortDir === '-') {
query.addDescending('objectId');
} else {
query.addAscending('objectId');
}
}

This approach requires special indices just for dashboard browsing. If an app is using a complex or resource intensive index, dashboard requires the same index plus the additional objectId in the index compound. I don't think this is a feasible solution, given the high cost of indices on a DB. Maybe we need to look for an alternative solution.

Additionally, #1706 broke existing "query optimization" and should have been treated as a breaking change. The reason this was released as non-breaking was that we didn't have the policy that a changelog entry has to be written as part of the PR back then. It was not noticeable without reading the PR discussion in detail that it was a breaking change. This should not happen anymore as we are now requiring a changelog entry with the PR.

Steps to reproduce

  1. Create new call
  2. Browse class, sorted by createdAt
  3. Query createdAt, objectId sent to server

Actual Outcome

Dashboard currently paginates by issuing an entirely new query which requires to construct a complicated pagination mechanism (#1706), which creates the indexing issue described here.

Expected Outcome

In MongoDB at least, the proper and resource efficient way to paginate is to use the query cursor, see getMore. That may require some changes on the server side, for example sending the cursor ID to dashboard as query results meta data. And not sure whether Postgres supports cursor based pagination.

In any case, I think we should revert #1706 ASAP and release as a fix, or at least the part that adds the object ID to the query, because that can render the dashboard unusable or cause stability issues for some deployments.

@sadakchap @davimacedo @Klaitos What do you think?

Environment

n/a

Dashboard

  • Parse Dashboard version: 2.2.0, master

Logs

n/a

@parse-github-assistant
Copy link

parse-github-assistant bot commented Sep 14, 2021

Thanks for opening this issue!

  • 🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing.

@mtrezza mtrezza added the docs label Sep 14, 2021
@mtrezza mtrezza changed the title Default indices missing for sort by createdAt docs: add indexing considerations to optimize dashboard queries Sep 14, 2021
@mtrezza mtrezza changed the title docs: add indexing considerations to optimize dashboard queries Add indexing considerations to optimize dashboard queries Sep 14, 2021
@mtrezza mtrezza added enhancement and removed docs labels Sep 14, 2021
@mtrezza mtrezza changed the title Add indexing considerations to optimize dashboard queries Revert using objectId as tie breaker in infinite scrolling Sep 14, 2021
@mtrezza mtrezza added type:bug Impaired feature or lacking behavior that is likely assumed and removed enhancement labels Sep 14, 2021
@sadakchap
Copy link
Member

@mtrezza, yeah.. we should revert #1706 . Could you please do that? And then we can tackle #1551 later separately (probably using getMore).

@Klaitos
Copy link
Contributor

Klaitos commented Sep 14, 2021

I'm glad you changed your mind about it 🙂.

@mtrezza
Copy link
Member Author

mtrezza commented Sep 14, 2021

It's unfortunate that we need to revert it, because that opens the original scrolling issue again, and @sadakchap put some good work into fixing this. However, I think the implications we discovered now do not justify the type of fix.

@Klaitos Would you want to give it a try to open a PR to revert this?

@Klaitos
Copy link
Contributor

Klaitos commented Sep 15, 2021

Done here #1800

@mtrezza
Copy link
Member Author

mtrezza commented Sep 16, 2021

Closing via #1800

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug Impaired feature or lacking behavior that is likely assumed
Projects
None yet
Development

No branches or pull requests

3 participants