Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove the need of SELECT COUNT(*) during snapshot #1386

Closed
altmannmarcelo opened this issue Oct 9, 2024 · 0 comments
Closed

Remove the need of SELECT COUNT(*) during snapshot #1386

altmannmarcelo opened this issue Oct 9, 2024 · 0 comments
Assignees
@altmannmarcelo
Labels
High priority Created by Linear-GitHub Sync
Milestone
v.53

Comments

@altmannmarcelo
Copy link
Contributor

Description

Currently we do a full index scan to get the exact number of rows during MySQL snapshot. This is a costly operation for big tables. We should find a way to not require it.

Change in user-visible behavior

Requires documentation change
@altmannmarcelo altmannmarcelo added this to the v.46 milestone Oct 10, 2024
@altmannmarcelo altmannmarcelo added the High priority Created by Linear-GitHub Sync label Oct 10, 2024
@altmannmarcelo altmannmarcelo modified the milestones: v.46, v.47, v.48 Oct 14, 2024
@altmannmarcelo altmannmarcelo modified the milestones: v.48, v.49 Oct 28, 2024
@altmannmarcelo altmannmarcelo modified the milestones: v.49, v.50 Nov 4, 2024
@altmannmarcelo altmannmarcelo modified the milestones: v.50, v.51, v.52 Nov 11, 2024
@altmannmarcelo altmannmarcelo modified the milestones: v.52, v.53 Nov 25, 2024
@altmannmarcelo altmannmarcelo self-assigned this Nov 25, 2024
readysetbot pushed a commit that referenced this issue Nov 25, 2024
@altmannmarcelo
Remove the need for exact row count
cc4d0ef 
This commit removes the requirement for MySQL snapshot for requiring
the exact number of rows it will snapshot in advance. We use the
number of rows to show snapshot progress and also to know for sure
that we have snapshoted the required number of rows. Querying the
table to get the number of rows in MySQL requires a full index scan.
This can be time consuming for large tables.
We don't necessarlly need to know the exact number of tables. In a
server with up to date table statistics the estimation of table rows
will be pretty much accurate.

Fixes: #REA-4886
Closes: #1386

Release-Note-Core: Improve MySQL snapshot performance by removing the
  need for exact row count for snapshot.
Change-Id: Idf887a0b8abb8e20113b3585af49c75d3da8b05f
readysetbot pushed a commit that referenced this issue Nov 25, 2024
@altmannmarcelo
Remove the need for exact row count
1d280b2 
This commit removes the requirement for MySQL snapshot for requiring
the exact number of rows it will snapshot in advance. We use the
number of rows to show snapshot progress and also to know for sure
that we have snapshoted the required number of rows. Querying the
table to get the number of rows in MySQL requires a full index scan.
This can be time consuming for large tables.
We don't necessarlly need to know the exact number of tables. In a
server with up to date table statistics the estimation of table rows
will be pretty much accurate.

Fixes: #REA-4886
Closes: #1386

Release-Note-Core: Improve MySQL snapshot performance by removing the
  need for exact row count for snapshot.
Change-Id: Idf887a0b8abb8e20113b3585af49c75d3da8b05f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@altmannmarcelo altmannmarcelo

Labels
High priority Created by Linear-GitHub Sync
Projects
None yet
Milestone
v.53
Development

No branches or pull requests
1 participant
@altmannmarcelo