-
Notifications
You must be signed in to change notification settings - Fork 3.4k
HBASE-27594 [JDK17] SecurityManager is deprecated since JDK17 #6932
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
This is a simple attempt to get rid of System.setSecurityManager / System.getSecurityManager usages in hbase so that it makes our path to future JDK upgrades easier! Lets see what CI says. |
@@ -131,6 +131,13 @@ | |||
|
|||
<!-- Make the @SuppressWarnings annotations available to Checkstyle --> | |||
<module name="SuppressWarningsHolder"/> | |||
|
|||
<!-- Custom check to detect System.exit calls --> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just FYI: added this checkstyle rule to raise checkstyle an issue if anyone tries to directly use System.exit
hbase-common/src/test/java/org/apache/hadoop/hbase/SystemExitRule.java
Outdated
Show resolved
Hide resolved
@@ -59,6 +61,36 @@ public void testHBaseConfTool() { | |||
} | |||
} | |||
|
|||
@Test | |||
public void testHBaseConfToolError() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added a new test to check for failure scenario, should i drop this?
@Override | ||
public void checkExit(int status) throws SecurityException { | ||
exitCode = status; | ||
public void exit(int status) throws SecurityException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we still throw a SecurityException?
}; | ||
}; | ||
} | ||
|
||
// Exiting the JVM is not allowed in tests and this exception is thrown instead | ||
// when it is done | ||
public static class SystemExitInTestException extends SecurityException { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we still throw SecurityException?
public class ExitHandler { | ||
private static ExitHandler instance = new ExitHandler(); | ||
|
||
public static ExitHandler getInstance() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should I wrap into Double-Checked Locking. seems overkill for this use case. WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some kind of semaphore to ensure that only one thread in a JVM holds a test instance ?
That would also require releasing each test ExitHandler instance explicitily.
return instance; | ||
} | ||
|
||
public static void setInstance(ExitHandler handler) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should i let this be called from only tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR replaces calls to System.exit with a custom ExitHandler to prevent the JVM from terminating during tests and to eliminate deprecated SecurityManager usage.
- Replaces all System.exit calls with ExitHandler.getInstance().exit in production and test code
- Removes TestSecurityManager to further reduce deprecated security manager usage
- Updates SystemExitRule to leverage ExitHandler for controlled exit behavior
Reviewed Changes
Copilot reviewed 113 out of 114 changed files in this pull request and generated no comments.
Show a summary per file
File | Description |
---|---|
hbase-examples/src/main/java/org/apache/hadoop/hbase/mapreduce/SampleUploader.java | Replaces System.exit with ExitHandler.getInstance().exit |
hbase-examples/src/main/java/org/apache/hadoop/hbase/mapreduce/IndexBuilder.java | Replaces System.exit with ExitHandler.getInstance().exit in main and error handling |
hbase-endpoint/src/main/java/org/apache/hadoop/hbase/coprocessor/Export.java | Updates exit call to use ExitHandler |
hbase-diagnostics/src/main/java/org/apache/hadoop/hbase/wal/WALPerformanceEvaluation.java | Replaces System.exit with ExitHandler.getInstance().exit in usage and main |
hbase-diagnostics/src/main/java/org/apache/hadoop/hbase/master/balancer/LoadBalancerPerformanceEvaluation.java | Replaces System.exit with ExitHandler.getInstance().exit |
hbase-diagnostics/src/main/java/org/apache/hadoop/hbase/ScanPerformanceEvaluation.java | Replaces System.exit with ExitHandler.getInstance().exit in main |
hbase-diagnostics/src/main/java/org/apache/hadoop/hbase/PerformanceEvaluation.java | Replaces System.exit with ExitHandler.getInstance().exit in error usage and main |
hbase-compression/hbase-compression-zstd/src/test/java/org/apache/hadoop/hbase/io/compress/zstd/TestZstdDictionary.java | Replaces System.exit with ExitHandler.getInstance().exit in test usage |
hbase-common/src/test/java/org/apache/hadoop/hbase/TestSystemExitInTest.java | Uses ExitHandler.getInstance().exit instead of System.exit |
hbase-common/src/test/java/org/apache/hadoop/hbase/SystemExitRule.java | Updates rule to use ExitHandler for controlled test exits |
hbase-common/src/main/java/org/apache/hadoop/hbase/util/Random64.java | Replaces System.exit with ExitHandler.getInstance().exit on conflict detection |
hbase-common/src/main/java/org/apache/hadoop/hbase/util/JenkinsHash.java | Replaces System.exit with ExitHandler.getInstance().exit on usage errors |
hbase-common/src/main/java/org/apache/hadoop/hbase/util/ExitHandler.java | Introduces custom ExitHandler class to centralize exit functionality |
hbase-common/src/main/java/org/apache/hadoop/hbase/util/AbstractHBaseTool.java | Replaces System.exit with ExitHandler.getInstance().exit before termination |
hbase-client/src/test/java/org/apache/hadoop/hbase/ipc/TestCellBlockBuilder.java | Updates test exit usage to use ExitHandler.getInstance().exit |
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/mapreduce/MapReduceHFileSplitterJob.java | Replaces System.exit with ExitHandler.getInstance().exit in main |
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/RestoreDriver.java | Replaces System.exit with ExitHandler.getInstance().exit in main |
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/BackupDriver.java | Replaces System.exit with ExitHandler.getInstance().exit in main |
Files not reviewed (1)
- hbase-checkstyle/src/main/resources/hbase/checkstyle.xml: Language not supported
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
Two general comments: The description is too generic. SecurityManager is used in other places, mainly via HAdoop/UGI, which this patch does not fix. My other questions regarding JVM re-use in tests: |
What if there are System.exit calls in the libraries we used? Like hadoop? |
Hadoop uses similar ExitHandler mechanism(s). But you're right in that Hadoop should expose this functionality in a standard manner so that we can set this in our tests for Hadoop jobs. Unfortunately I'm not aware of a universal solution. |
# HBASE-27594: Replace SecurityManager with ExitHandler for JDK17 compatibility ## What? I've replaced the deprecated `SecurityManager` usage in HBase tests with a custom `ExitHandler` singleton pattern. This includes: - New `ExitHandler` class that intercepts `System.exit()` calls without using SecurityManager - Updated `SystemExitRule` to use the new ExitHandler approach - New `LauncherExitHandler` for main method testing scenarios - Removed deprecated `TestSecurityManager` and `LauncherSecurityManager` classes - Updated 15+ test files to use the new approach - Added checkstyle rule to prevent direct `System.exit()` calls The implementation maintains full backward compatibility - existing test code continues to work without any changes. ## Why? The `SecurityManager` class has been deprecated in JDK17 and will be removed in future JDK releases. HBase tests extensively use SecurityManager to intercept `System.exit()` calls for testing purposes, which prevents HBase from being fully compatible with JDK17. This change removes that dependency and makes our path to future JDK upgrades easier. This addresses the warnings we see in tests: ``` WARNING: System::setSecurityManager has been called by org.apache.hadoop.hbase.SystemExitRule WARNING: System::setSecurityManager will be removed in a future release ``` ## How? I created a singleton `ExitHandler` class that provides the same functionality as SecurityManager for exit interception: - **ExitHandler**: Thread-safe singleton that prevents `System.exit()` calls and throws `SecurityException` when exit is prevented - **LauncherExitHandler**: Replaces `LauncherSecurityManager` for main method testing, capturing exit codes and preventing actual JVM termination - **SystemExitRule**: Updated to use ExitHandler internally, maintaining the same API for existing tests The approach is simpler than SecurityManager (no reflection needed) and more performant (no security manager overhead). ## Testing? I've added comprehensive test coverage: - **TestExitHandler**: 6 tests covering singleton behavior, exit prevention, and exception handling - **TestLauncherExitHandler**: 3 tests verifying main method exit interception - **TestSystemExitInTest**: Verifies SystemExitRule works correctly with the new approach All tests pass successfully. The implementation is backward compatible - existing test code using `SystemExitRule` continues to work without changes. ## Screenshots N/A - Backend infrastructure change ## Anything Else? This implementation was inspired by [PR apache#6932](apache#6932) by @NihalJain, which first proposed the `ExitHandler` approach. Our implementation expands on that work with: - Enhanced `LauncherExitHandler` for main method testing - Comprehensive test coverage and unit tests - Checkstyle rules to prevent direct `System.exit()` calls - Updated documentation and migration guides The `ExitHandler` approach is more flexible than SecurityManager and can be extended for additional exit interception needs in the future. Consider migrating other projects that use SecurityManager to similar patterns. ## Related Issues - **HBASE-27594**: Replace SecurityManager with ExitHandler for JDK17 compatibility - **HBASE-26038**: Support JDK17 (parent issue)
ExitHandler
to wrapSystem.exit
calls, preventing JVM termination during tests and throwing an exception instead.System.exit
calls withExitHandler.getInstance().exit
.TestSecurityManager
to reduce deprecated security manager usage.SystemExitRule
to useExitHandler
for controlled exit behavior.System.setSecurityManager
/System.getSecurityManager
usages in HBase so that it makes our path to future JDK upgrades easierSystem.setSecurityManager
andSystem.getSecurityManager
calls.