Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CNDB-11508 main-5.0 branch #1408

Open
wants to merge 16 commits into
base: main-5.0
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .build/parent-pom-template.xml
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@
</license>
</licenses>
<properties>
<bytebuddy.version>1.12.13</bytebuddy.version>
<byteman.version>4.0.20</byteman.version>
<bytebuddy.version>1.14.17</bytebuddy.version>
<byteman.version>4.0.23</byteman.version>
<ohc.version>0.5.1</ohc.version>

<!-- These are referenced in build.xml, so need to be propagated from there -->
Expand Down Expand Up @@ -930,7 +930,7 @@
<dependency>
<groupId>com.google.code.findbugs</groupId>
<artifactId>jsr305</artifactId>
<version>2.0.2</version>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>com.clearspring.analytics</groupId>
Expand Down
13 changes: 10 additions & 3 deletions bin/cassandra.in.sh
Original file line number Diff line number Diff line change
Expand Up @@ -122,11 +122,16 @@ jvmver=`echo "$java_ver_output" | grep '[openjdk|java] version' | awk -F'"' 'NR=
JVM_VERSION=${jvmver%_*}
short=$(echo "${jvmver}" | cut -c1-2)

JAVA_VERSION=17
JAVA_VERSION=22
if [ "$short" = "11" ] ; then
JAVA_VERSION=11
elif [ "$JVM_VERSION" \< "17" ] ; then
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this correct and consistent with what we have in main? I suggested to allow only LTS versions but eventually @JeremiahDJordan opposed and we allow for any >= 11; While here I can see 12..16 and 18..21 seem to be disallowed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a patch in trunk that adds a flag to be able to allow those. We should port it and not allow it by default. But let's not do it here?
This comment I wrote in the issue:
UPDATE: I will pull the patch from CASSANDRA-18688 in a separate ticket. We can add it when we fix all tests and enable main branch for testing with JDK22 in CI. Otherwise, there will be confusion that the branch is JDK22 ready when it is still not fully ready.

echo "DSE DB 5.0 requires Java 11 or Java 17."
echo "DSE DB 5.0 requires Java 11 or higher."
exit 1;
elif [ "$short" = "17" ] ; then
JAVA_VERSION=17
elif [ "$JVM_VERSION" \< "22" ] ; then
echo "DSE DB 5.0 requires Java 11 or higher."
exit 1;
fi

Expand All @@ -151,7 +156,9 @@ esac

# Read user-defined JVM options from jvm-server.options file
JVM_OPTS_FILE=$CASSANDRA_CONF/jvm${jvmoptions_variant:--clients}.options
if [ $JAVA_VERSION -ge 17 ] ; then
if [ $JAVA_VERSION -ge 22 ] ; then
JVM_DEP_OPTS_FILE=$CASSANDRA_CONF/jvm22${jvmoptions_variant:--clients}.options
elif [ $JAVA_VERSION -ge 17 ] ; then
JVM_DEP_OPTS_FILE=$CASSANDRA_CONF/jvm17${jvmoptions_variant:--clients}.options
elif [ $JAVA_VERSION -ge 11 ] ; then
JVM_DEP_OPTS_FILE=$CASSANDRA_CONF/jvm11${jvmoptions_variant:--clients}.options
Expand Down
131 changes: 124 additions & 7 deletions build.xml

Large diffs are not rendered by default.

5 changes: 4 additions & 1 deletion conf/jvm17-server.options
Original file line number Diff line number Diff line change
Expand Up @@ -92,12 +92,15 @@
--add-opens java.base/java.nio=ALL-UNNAMED
--add-opens jdk.compiler/com.sun.tools.javac=ALL-UNNAMED

# required for org.apache.cassandra.Util.getSupportedMTimeGranularity
--add-opens java.base/java.nio.file.attribute=ALL-UNNAMED

### GC logging options -- uncomment to enable

# Java 11 (and newer) GC logging options:
# See description of https://bugs.openjdk.java.net/browse/JDK-8046148 for details about the syntax
# The following is the equivalent to -XX:+PrintGCDetails -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M
-Xlog:gc=info,heap*=debug,age*=debug,safepoint=info,promotion*=debug:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760
# -Xlog:gc=info,heap*=debug,age*=debug,safepoint=info,promotion*=debug:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760
JeremiahDJordan marked this conversation as resolved.
Show resolved Hide resolved

# Notes for Java 8 migration:
#
Expand Down
32 changes: 32 additions & 0 deletions conf/jvm22-clients.options
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
###########################################################################
# jvm22-clients.options #
# #
# See jvm-clients.options. This file is specific for Java 22 and newer. #
###########################################################################

###################
# JPMS SETTINGS #
###################

-Djdk.attach.allowAttachSelf=true
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED
--add-exports java.base/jdk.internal.ref=ALL-UNNAMED
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
--add-exports java.management.rmi/com.sun.jmx.remote.internal.rmi=ALL-UNNAMED
--add-exports java.rmi/sun.rmi.registry=ALL-UNNAMED
--add-exports java.rmi/sun.rmi.server=ALL-UNNAMED
--add-exports java.sql/java.sql=ALL-UNNAMED
--add-exports jdk.attach/sun.tools.attach=ALL-UNNAMED

--add-opens java.base/java.io=ALL-UNNAMED
--add-opens java.base/java.lang.module=ALL-UNNAMED
--add-opens java.base/java.lang.reflect=ALL-UNNAMED
--add-opens java.base/jdk.internal.loader=ALL-UNNAMED
--add-opens java.base/jdk.internal.math=ALL-UNNAMED
--add-opens java.base/jdk.internal.module=ALL-UNNAMED
--add-opens java.base/jdk.internal.ref=ALL-UNNAMED
--add-opens java.base/jdk.internal.reflect=ALL-UNNAMED
--add-opens java.base/sun.nio.ch=ALL-UNNAMED
--add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED

# The newline in the end of file is intentional
109 changes: 109 additions & 0 deletions conf/jvm22-server.options
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
###########################################################################
# jvm22-server.options #
# #
# See jvm-server.options. This file is specific for Java 22 and newer. #
###########################################################################

#################
# GC SETTINGS #
#################

### G1 Settings
## Use the Hotspot garbage-first collector.
-XX:+UseG1GC
-XX:+ParallelRefProcEnabled

#
## Have the JVM do less remembered set work during STW, instead
## preferring concurrent GC. Reduces p99.9 latency.
-XX:G1RSetUpdatingPauseTimePercent=5
#
## Main G1GC tunable: lowering the pause target will lower throughput and vise versa.
## 200ms is the JVM default and lowest viable setting
## 1000ms increases throughput. Keep it smaller than the timeouts in cassandra.yaml.
-XX:MaxGCPauseMillis=500

## Optional G1 Settings
# Save CPU time on large (>= 16GB) heaps by delaying region scanning
# until the heap is 70% full. The default in Hotspot 8u40 is 40%.
#-XX:InitiatingHeapOccupancyPercent=70

# For systems with > 8 cores, the default ParallelGCThreads is 5/8 the number of logical cores.
# Otherwise equal to the number of cores when 8 or less.
# Machines with > 10 cores should try setting these to <= full cores.
#-XX:ParallelGCThreads=16
# By default, ConcGCThreads is 1/4 of ParallelGCThreads.
# Setting both to the same value can reduce STW durations.
#-XX:ConcGCThreads=16


### JPMS

-Djdk.attach.allowAttachSelf=true
-Djava.security.manager=allow
--add-exports java.base/jdk.internal.misc=ALL-UNNAMED
--add-exports java.base/jdk.internal.ref=ALL-UNNAMED
--add-exports java.base/jdk.internal.perf=ALL-UNNAMED
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
--add-exports java.management.rmi/com.sun.jmx.remote.internal.rmi=ALL-UNNAMED
--add-exports java.rmi/sun.rmi.registry=ALL-UNNAMED
--add-exports java.rmi/sun.rmi.server=ALL-UNNAMED
--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
--add-exports jdk.naming.dns/com.sun.jndi.dns=java.naming
--add-exports jdk.unsupported/sun.misc=ALL-UNNAMED

--add-opens java.base/java.io=ALL-UNNAMED
--add-opens java.base/java.lang.module=ALL-UNNAMED
--add-opens java.base/java.lang=ALL-UNNAMED
--add-opens java.base/java.lang.reflect=ALL-UNNAMED
--add-opens java.base/java.nio.file.spi=ALL-UNNAMED
--add-opens java.base/java.nio=ALL-UNNAMED
--add-opens java.base/java.net=ALL-UNNAMED
--add-opens java.base/java.util=ALL-UNNAMED
--add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED
--add-opens java.base/java.util.concurrent.locks=ALL-UNNAMED
--add-opens java.base/jdk.internal.loader=ALL-UNNAMED
--add-opens java.base/jdk.internal.math=ALL-UNNAMED
--add-opens java.base/jdk.internal.module=ALL-UNNAMED
--add-opens java.base/jdk.internal.ref=ALL-UNNAMED
--add-opens java.base/jdk.internal.reflect=ALL-UNNAMED
--add-opens java.base/jdk.internal.vm=ALL-UNNAMED
--add-opens java.base/sun.nio.ch=ALL-UNNAMED
--add-opens jdk.compiler/com.sun.tools.javac=ALL-UNNAMED
--add-opens jdk.management.jfr/jdk.management.jfr=ALL-UNNAMED
--add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED
--add-opens jdk.naming.dns/com.sun.jndi.dns=ALL-UNNAMED

# required for org.apache.cassandra.Util.getSupportedMTimeGranularity
--add-opens java.base/java.nio.file.attribute=ALL-UNNAMED

### GC logging options -- uncomment to enable

# Java 11 (and newer) GC logging options:
# See description of https://bugs.openjdk.java.net/browse/JDK-8046148 for details about the syntax
# The following is the equivalent to -XX:+PrintGCDetails -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=10M
# -Xlog:gc=info,heap*=debug,age*=debug,safepoint=info,promotion*=debug:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760

# Notes for Java 8 migration:
#
# -XX:+PrintGCDetails maps to -Xlog:gc*:... - i.e. add a '*' after "gc"
# -XX:+PrintGCDateStamps maps to decorator 'time'
#
# -XX:+PrintHeapAtGC maps to 'heap' with level 'trace'
# -XX:+PrintTenuringDistribution maps to 'age' with level 'debug'
# -XX:+PrintGCApplicationStoppedTime maps to 'safepoint' with level 'info'
# -XX:+PrintPromotionFailure maps to 'promotion' with level 'trace'
# -XX:PrintFLSStatistics=1 maps to 'freelist' with level 'trace'

### Netty Options

# On Java >= 9 Netty requires the io.netty.tryReflectionSetAccessible system property to be set to true to enable
# creation of direct buffers using Unsafe. Without it, this falls back to ByteBuffer.allocateDirect which has
# inferior performance and risks exceeding MaxDirectMemory
-Dio.netty.tryReflectionSetAccessible=true

### Enable vector incubator feature (simd support)

--add-modules jdk.incubator.vector

# The newline in the end of file is intentional
5 changes: 4 additions & 1 deletion ide/idea/workspace.xml
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,7 @@
-Djava.library.path=$PROJECT_DIR$/lib/sigar-bin
-Dlogback.configurationFile=file://$PROJECT_DIR$/conf/logback.xml
-XX:HeapDumpPath=build/test
-Dnet.bytebuddy.experimental=true
-ea" />
<option name="PROGRAM_PARAMETERS" value="" />
<option name="WORKING_DIRECTORY" value="" />
Expand Down Expand Up @@ -204,7 +205,8 @@
-XX:MaxMetaspaceSize=2G
-XX:SoftRefLRUPolicyMSPerMB=0
-Xmx4G
-ea" />
-ea
-Dnet.bytebuddy.experimental=true" />
<option name="PARAMETERS" value="" />
<fork_mode value="class" />
<option name="WORKING_DIRECTORY" value="" />
Expand Down Expand Up @@ -236,6 +238,7 @@
-Dlogback.configurationFile=file://$PROJECT_DIR$/conf/logback.xml
-XX:HeapDumpPath=build/test
-Xmx1G
-Dnet.bytebuddy.experimental=true
-ea" />
<option name="PROGRAM_PARAMETERS" value="" />
<option name="WORKING_DIRECTORY" value="file://$PROJECT_DIR$" />
Expand Down
3 changes: 0 additions & 3 deletions pylib/cassandra-cqlsh-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,6 @@ fi
if [[ -z "${java_version}" ]]; then
java_version=$(java -version 2>&1 | awk -F '"' '/version/ {print $2}' | sed 's/^1\.//g' | awk -F. '{print $1}')
fi
if [[ ${java_version} -ge 11 ]]; then
export CASSANDRA_USE_JDK11=true
fi

if [[ -z "${python_version}" ]]; then
python_version="3.8"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@
import com.google.common.io.ByteStreams;

import org.apache.commons.lang3.StringUtils;
import org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment;
import org.eclipse.jdt.internal.compiler.lookup.ModuleBinding;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -790,7 +790,8 @@ TrieEntry<K, V> nextEntryImpl(TrieEntry<K, V> start, TrieEntry<K, V> previous, T
* This is implemented by going always to the left until
* we encounter a valid uplink. That uplink is the first key.
*/
TrieEntry<K, V> firstEntry()
// @Override needed in JDK 21+.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if @Override is needed, then why don't you add it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it is needed only if we build on JDK21+. There is nothing to override on previous version and we still build on 11.

public TrieEntry<K, V> firstEntry()
{
// if Trie is empty, no first node.
return isEmpty() ? null : followLeft(root);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -414,7 +414,8 @@ private TrieEntry<K, V> subtree(K prefix)
* <p>This is implemented by going always to the right until
* we encounter a valid uplink. That uplink is the last key.
*/
private TrieEntry<K, V> lastEntry()
// @Override needed in JDK 21+.
public TrieEntry<K, V> lastEntry()
{
return followRight(root.left);
}
Expand Down
2 changes: 2 additions & 0 deletions src/java/org/apache/cassandra/service/StorageService.java
Original file line number Diff line number Diff line change
Expand Up @@ -338,6 +338,8 @@ private static int getSchemaDelay()

private final SamplingManager samplingManager = new SamplingManager();

// Newer versions of mockito contain mockito-inline which creates an issue in our test environment. Without this
// change, mocking of static methods is a problem with our DTest framework
@VisibleForTesting // this is used for dtests only, see CASSANDRA-18152
public volatile boolean skipNotificationListeners = false;

Expand Down
42 changes: 42 additions & 0 deletions src/java/org/apache/cassandra/utils/btree/BTreeSet.java
Original file line number Diff line number Diff line change
Expand Up @@ -312,6 +312,48 @@ public ListIterator<V> listIterator(int index)
throw new UnsupportedOperationException();
}

// @Override needed in JDK 21+.
public BTreeSet<V> reversed()
{
throw new UnsupportedOperationException();
}

// @Override needed in JDK 21+.
public V removeLast()
{
throw new UnsupportedOperationException();
}

// @Override needed in JDK 21+.
public V removeFirst()
{
throw new UnsupportedOperationException();
}

// @Override needed in JDK 21+.
public V getLast()
{
throw new UnsupportedOperationException();
}

// @Override needed in JDK 21+.
public V getFirst()
{
throw new UnsupportedOperationException();
}

// @Override needed in JDK 21+.
public void addLast(V v)
{
throw new UnsupportedOperationException();
}

// @Override needed in JDK 21+.
public void addFirst(V v)
{
throw new UnsupportedOperationException();
}

public static class BTreeRange<V> extends BTreeSet<V>
{
// both inclusive
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ public class JdkProperties extends Task
public void execute()
{
Project project = getProject();
project.setNewProperty("java.version." + project.getProperty("ant.java.version").replace("1.", ""), "true");
project.setNewProperty("use-jdk" + project.getProperty("ant.java.version").replace("1.", ""), "true");
project.setNewProperty("java.version." + project.getProperty("ant.java.version"), "true");
project.setNewProperty("use-jdk" + project.getProperty("ant.java.version"), "true");
}
}
3 changes: 2 additions & 1 deletion test/unit/org/apache/cassandra/index/sai/SAITester.java
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@
import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.service.snapshot.TableSnapshot;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.ReflectionUtils;
import org.apache.cassandra.utils.Throwables;
import org.apache.lucene.codecs.CodecUtil;
import org.awaitility.Awaitility;
Expand Down Expand Up @@ -898,7 +899,7 @@ protected static void setSegmentWriteBufferSpace(final int segmentSize) throws E
NamedMemoryLimiter limiter = (NamedMemoryLimiter) V1OnDiskFormat.class.getDeclaredField("SEGMENT_BUILD_MEMORY_LIMITER").get(null);
Field limitBytes = limiter.getClass().getDeclaredField("limitBytes");
limitBytes.setAccessible(true);
Field modifiersField = Field.class.getDeclaredField("modifiers");
Field modifiersField = ReflectionUtils.getField(Field.class, "modifiers");
modifiersField.setAccessible(true);
modifiersField.setInt(limitBytes, limitBytes.getModifiers() & ~Modifier.FINAL);
limitBytes.set(limiter, segmentSize);
Expand Down
3 changes: 2 additions & 1 deletion test/unit/org/apache/cassandra/index/sai/SAIUtil.java
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import java.lang.reflect.Field;

import org.apache.cassandra.index.sai.disk.format.Version;
import org.apache.cassandra.utils.ReflectionUtils;

public class SAIUtil
{
Expand All @@ -31,7 +32,7 @@ public static void setLatestVersion(Version version)
{
latest = Version.class.getDeclaredField("LATEST");
latest.setAccessible(true);
Field modifiersField = Field.class.getDeclaredField("modifiers");
Field modifiersField = ReflectionUtils.getField(Field.class, "modifiers");
modifiersField.setAccessible(true);
latest.set(null, version);
}
Expand Down
Loading