Update ScanServerRefFile format to sort on UUID #4691
Conversation
|
Currently in draft since I need to add tests for upgrade code and make sure existing tests still pass. |
| @@ -209,4 +216,16 @@ static void upgradeDataFileCF(final Key key, final Value value, final Mutation m | |||
| } | |||
| } | |||
|
|
|||
| public void removeScanServerRange(ServerContext context, String tableName) { | |||
There was a problem hiding this comment.
The upgrade code will also need to create the new table. In #4690 it creates the new table in the init code. Should try to reuse code for that.
There was a problem hiding this comment.
Found this in the elasticity upgrade code to create a fate table. But not sure if that is doing the same thing init does when it creates the fate table.
There was a problem hiding this comment.
We can leave #4652 open if the table creation is not done. Table creation in upgrade does not need to block this PR, we just need to ensure its not forgotten.
There was a problem hiding this comment.
Added the table creation bits to this for zk table node creation, populating of table properties, and initial tablet file creation.
There was a problem hiding this comment.
In order to reuse the init code I had to make the class public.
I'm not a huge fan of that as a solution so I'm open to suggestions on where this table creation code should live.
server/manager/src/main/java/org/apache/accumulo/manager/upgrade/Upgrader11to12.java
Outdated
Show resolved
Hide resolved
ddanielr
force-pushed
the
sserver-ref-change-main
branch
4 times, most recently
from
4db22dc
to
63b898b
Compare
|
After we merge this forward into Elasticity, I think we could follow the same pattern here and use it to fix up the upgrade for the Fate table as part of #4692 |
test/src/main/java/org/apache/accumulo/test/upgrade/ScanServerUpgrade11to12TestIT.java
Show resolved
Hide resolved
server/base/src/main/java/org/apache/accumulo/server/init/FileSystemInitializer.java
Outdated
Show resolved
Hide resolved
server/base/src/main/java/org/apache/accumulo/server/init/FileSystemInitializer.java
Outdated
Show resolved
Hide resolved
server/base/src/main/java/org/apache/accumulo/server/init/FileSystemInitializer.java
Outdated
Show resolved
Hide resolved
server/base/src/main/java/org/apache/accumulo/server/init/FileSystemInitializer.java
Outdated
Show resolved
Hide resolved
server/base/src/main/java/org/apache/accumulo/server/init/FileSystemInitializer.java
Outdated
Show resolved
Hide resolved
server/manager/src/main/java/org/apache/accumulo/manager/upgrade/Upgrader11to12.java
Outdated
Show resolved
Hide resolved
server/base/src/main/java/org/apache/accumulo/server/init/FileSystemInitializer.java
Outdated
Show resolved
Hide resolved
Updates the ScanServerRef format and moves the UUID entry forward so sorting is done on UUID instead of filename. Also adds the removal of the scan serverref range from the metadata table
* Moves the scanRefTablet creation to a separate method so it can be called from the upgrader code * Handle scanRef table creation in Upgrader code * Cleans up the FileSystemInitializer code * Upgrader can now call the scanRef tablet creation codepath * Separates the scanRef table node creation into a separate method that can be called from the upgrader code
ScanRefs are not written to the root table, only the metadata table so handling scanref upgrades for the root table is not needed
* Removes the directory creation from the upgrade code * Modifies the test to shutdown the cluster after removing the scan ref table bits
…SystemInitializer.java Co-authored-by: Keith Turner <[email protected]>
ddanielr
force-pushed
the
sserver-ref-change-main
branch
from
c5db1f0
to
4b767dc
Compare
|
@ddanielr this change has caused some test failures in org.apache.accumulo.test.ScanServerMetadataEntriesIT |
Updates the ScanServerRef format and moves the UUID entry forward so sorting is done on UUID instead of filename.
Also adds the removal of the scan serverref range from the metadata table
This PR is ensuring that the changes from #4682 are merged into main without introducing a large number of merge conflicts.
closes #4652 and #4493