Thanks to visit codestin.com
Credit goes to github.com

Skip to content

MAPREDUCE-7521. Fix TestUberAM failures after JUnit 5 migration #8017

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 10, 2025
Merged

MAPREDUCE-7521. Fix TestUberAM failures after JUnit 5 migration #8017

merged 1 commit into from
Oct 10, 2025

Conversation

@slfan1989
Copy link
Contributor

@slfan1989 slfan1989 commented Oct 8, 2025 •
edited
Loading

Description of PR

JIRA: MAPREDUCE-7521. Fix TestUberAM failures after JUnit 5 migration

Background

After migrating the project to JUnit 5, the TestUberAM unit test started failing intermittently. The error message observed during test execution is as follows:

Remote File /apps_staging_dir/jenkins/.staging/job_1759472296837_0013/job.jar could only be written to 0 of the 1 minReplication nodes. 
There are 2 datanode(s) running and 2 node(s) are excluded in this operation.

This error originates from

org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock

indicating that the MiniDFSCluster used in the test failed to find available data nodes for block writes during execution.

Change Summary

  • Removed redundant invocation of the parent class’s BeforeAll method in subclasses to prevent double initialization.
  • Increased MiniDFSCluster data node count from 2 to 5 to improve fault tolerance.
  • Verified that all affected tests pass successfully.

Impact Scope

  • Affects only AM tests based on MiniDFSCluster.
  • No impact on business logic or production code.

How was this patch tested?

Local testing was successful.

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.hadoop.mapreduce.v2.TestUberAM
[INFO] Tests run: 21, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 638.7 s -- in org.apache.hadoop.mapreduce.v2.TestUberAM
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 21, Failures: 0, Errors: 0, Skipped: 0

CI

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@slfan1989 slfan1989 force-pushed the fix-test-uber-am-error branch from 01b8111 to a3d6d0e Compare October 8, 2025 06:10
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 29m 42s trunk passed
+1 💚 compile 0m 22s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 compile 0m 19s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 checkstyle 0m 19s trunk passed
+1 💚 mvnsite 0m 26s trunk passed
+1 💚 javadoc 0m 20s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 19s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 0m 34s trunk passed
+1 💚 shadedclient 20m 27s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 17s the patch passed
+1 💚 compile 0m 16s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javac 0m 16s the patch passed
+1 💚 compile 0m 16s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 javac 0m 16s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 11s /results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: The patch generated 3 new + 37 unchanged - 3 fixed = 40 total (was 40)
+1 💚 mvnsite 0m 16s the patch passed
+1 💚 javadoc 0m 12s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 13s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 0m 32s the patch passed
+1 💚 shadedclient 20m 7s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 90m 9s /patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚 asflicense 0m 30s The patch does not generate ASF License warnings.
167m 27s
Reason Tests
Failed junit tests hadoop.mapreduce.v2.TestUberAM
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8017/2/artifact/out/Dockerfile
GITHUB PR #8017
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux b68c3f07092a 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / a3d6d0e
Default Java Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8017/2/testReport/
Max. process+thread count 1213 (vs. ulimit of 5500)
modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8017/2/console
versions git=2.25.1 maven=3.9.11 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 50s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 53m 43s trunk passed
+1 💚 compile 0m 38s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 compile 0m 35s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 checkstyle 0m 32s trunk passed
+1 💚 mvnsite 0m 39s trunk passed
+1 💚 javadoc 0m 34s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 28s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 0m 50s trunk passed
+1 💚 shadedclient 40m 10s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 27s the patch passed
+1 💚 compile 0m 29s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javac 0m 29s the patch passed
+1 💚 compile 0m 26s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 javac 0m 26s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 19s /results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: The patch generated 16 new + 37 unchanged - 3 fixed = 53 total (was 40)
+1 💚 mvnsite 0m 28s the patch passed
+1 💚 javadoc 0m 22s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 20s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 0m 51s the patch passed
+1 💚 shadedclient 39m 53s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 101m 58s /patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚 asflicense 0m 43s The patch does not generate ASF License warnings.
247m 25s
Reason Tests
Failed junit tests hadoop.mapreduce.v2.TestMRJobs
hadoop.mapreduce.v2.TestUberAM
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8017/1/artifact/out/Dockerfile
GITHUB PR #8017
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 1b4945748487 5.15.0-157-generic #167-Ubuntu SMP Wed Sep 17 21:35:53 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 01b8111
Default Java Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8017/1/testReport/
Max. process+thread count 1368 (vs. ulimit of 5500)
modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8017/1/console
versions git=2.25.1 maven=3.9.11 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989 slfan1989 force-pushed the fix-test-uber-am-error branch from a3d6d0e to 2f4cb93 Compare October 9, 2025 00:36
@slfan1989 slfan1989 marked this pull request as ready for review October 9, 2025 01:34
@slfan1989 slfan1989 changed the title [TEST] Fix TestUberAM Junit Error. MAPREDUCE-7521. Fix TestUberAM failures after JUnit 5 migration Oct 9, 2025
@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 52s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 58m 14s trunk passed
+1 💚 compile 0m 39s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 compile 0m 35s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 checkstyle 0m 34s trunk passed
+1 💚 mvnsite 0m 40s trunk passed
+1 💚 javadoc 0m 38s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 28s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 0m 54s trunk passed
+1 💚 shadedclient 40m 4s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 28s the patch passed
+1 💚 compile 0m 30s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javac 0m 30s the patch passed
+1 💚 compile 0m 27s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 javac 0m 27s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 21s the patch passed
+1 💚 mvnsite 0m 29s the patch passed
+1 💚 javadoc 0m 23s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 0m 19s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 spotbugs 0m 52s the patch passed
+1 💚 shadedclient 40m 5s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 120m 25s hadoop-mapreduce-client-jobclient in the patch passed.
+1 💚 asflicense 0m 44s The patch does not generate ASF License warnings.
270m 47s
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8017/3/artifact/out/Dockerfile
GITHUB PR #8017
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 3dfa522611a4 5.15.0-157-generic #167-Ubuntu SMP Wed Sep 17 21:35:53 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 2f4cb93
Default Java Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8017/3/testReport/
Max. process+thread count 1131 (vs. ulimit of 5500)
modules C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8017/3/console
versions git=2.25.1 maven=3.9.11 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989
Copy link
Contributor Author

@cnauroth @szetszwo Could you please help review this PR? Thanks a lot — it fixes the issue causing TestUberAM to fail during execution.

szetszwo
szetszwo approved these changes Oct 9, 2025
View reviewed changes
Copy link
Contributor

@szetszwo szetszwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 the change looks good.

@slfan1989
Copy link
Contributor Author

+1 the change looks good.

@szetszwo Thank you very much for reviewing the code!

@slfan1989 slfan1989 merged commit a170ff4 into apache:trunk Oct 10, 2025
4 checks passed
@slfan1989 slfan1989 mentioned this pull request Oct 10, 2025
Draft
75 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@szetszwo szetszwo szetszwo approved these changes

Assignees

No one assigned

Labels

MapReduce trunk

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@slfan1989 @hadoop-yetus @szetszwo