Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gcTS and alertGCTS aren't meaningful #1262

Open
AricSu opened this issue Jul 11, 2023 · 0 comments
Open

gcTS and alertGCTS aren't meaningful #1262

AricSu opened this issue Jul 11, 2023 · 0 comments

Comments

@AricSu
Copy link

AricSu commented Jul 11, 2023

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?
    Because of refine gc strategy of pump #646
    We can see that a formula alertGCTS := gcTS + int64(storage.EncodeTSO(alertGCMS)) adds extra 20h to avoid the scenarios of disk space of the pump is full and much impact on core system formation.
    But, the metric and error log aren't accurate and easy understand to user.
drainer's checkpoint is older than pump gc ts, some binlogs are purged
  1. What did you expect to see?

something likes this one

after 20h, drainer's checkpoint will be older than pump gc ts, some binlogs will be purged, please handle it manually based on gc ts as soon as possible
  1. What did you see instead?
drainer's checkpoint is older than pump gc ts, some binlogs are purged

and gcTs == alertGCTS --> https://github.com/pingcap/tidb-binlog/pull/646/files#diff-d73c6b7c1637ab38367029c88ed0bf596762627453dab635429f991ae9dfc9acR589-R596

		if drainer.MaxCommitTS < gcTS {
			log.Error("drainer's checkpoint is older than pump gc ts, some binlogs are purged",
				zap.String("drainer", drainer.NodeID),
				zap.Int64("gc ts", gcTS),
				zap.Int64("drainer checkpoint", drainer.MaxCommitTS),
			)
			// will add test when binlog have failpoint
			detectedDrainerBinlogPurged.WithLabelValues(drainer.NodeID).Inc()
		}
	}
}
  1. Please provide the relate downstream type and version of drainer.
    (run drainer -V in terminal to get drainer's version)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant