Thanks to visit codestin.com
Credit goes to github.com

Skip to content

HDFS-16417. RBF: StaticRouterRpcFairnessPolicyController init fails with division by 0 if concurrent ns handler count is configured #3871

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 10, 2022
Merged

HDFS-16417. RBF: StaticRouterRpcFairnessPolicyController init fails with division by 0 if concurrent ns handler count is configured #3871

merged 1 commit into from
Jan 10, 2022

Conversation

@kokonguyen191
Copy link
Contributor

@kokonguyen191 kokonguyen191 commented Jan 7, 2022

If dfs.federation.router.fairness.handler.count.concurrent is configured, unassignedNS is thus empty and handlerCount % unassignedNS.size() will throw a /0 exception.

Changed it to assigning extra handlers to concurrent ns in case it's configured.

@hadoop-yetus
Copy link

hadoop-yetus commented Jan 7, 2022

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 17m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 34m 23s trunk passed
+1 💚 compile 0m 44s trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 compile 0m 38s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 0m 28s trunk passed
+1 💚 mvnsite 0m 44s trunk passed
+1 💚 javadoc 0m 45s trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 0m 58s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 1m 21s trunk passed
+1 💚 shadedclient 21m 4s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 35s the patch passed
+1 💚 compile 0m 36s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javac 0m 36s the patch passed
+1 💚 compile 0m 34s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 0m 34s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 19s the patch passed
+1 💚 mvnsite 0m 35s the patch passed
+1 💚 javadoc 0m 33s the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 0m 52s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 1m 22s the patch passed
+1 💚 shadedclient 20m 54s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 38m 50s /patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 0m 36s The patch does not generate ASF License warnings.
146m 5s
Reason Tests
Failed junit tests hadoop.hdfs.server.federation.router.TestRouterFederationRename
hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3871/1/artifact/out/Dockerfile
GITHUB PR #3871
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux fe70f8b87fe4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 2c43833
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3871/1/testReport/
Max. process+thread count 2634 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3871/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@goiri
Copy link
Member

goiri commented Jan 7, 2022

There seems to fail in:

hadoop.hdfs.server.federation.router.TestRouterFederationRename
hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure

Can you take a look?

@kokonguyen191
Copy link
Contributor Author

kokonguyen191 commented Jan 7, 2022

Not sure why these 2 timed out, both pass normally when I run in local.

ferhui
ferhui approved these changes Jan 8, 2022
View reviewed changes
Copy link
Contributor

@ferhui ferhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@goiri
Copy link
Member

goiri commented Jan 8, 2022

@ferhui any idea on why those tests fail?

@ferhui ferhui closed this Jan 10, 2022
@ferhui ferhui reopened this Jan 10, 2022
@ferhui
Copy link
Contributor

ferhui commented Jan 10, 2022

@ferhui any idea on why those tests fail?

I think usually the CI environment has heavy load and some tests are time out

@ferhui
Copy link
Contributor

ferhui commented Jan 10, 2022

Changes are unrelated to failed tests and they passed locally.

@ferhui ferhui merged commit b88d5ae into apache:trunk Jan 10, 2022
@ferhui
Copy link
Contributor

ferhui commented Jan 10, 2022

@kokonguyen191 Thanks for your contribution. @goiri Thanks for your review! merged

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
@kokonguyen191
HDFS-16417. StaticRouterRpcFairnessPolicyController init (apache#3871)
4119948
LiuGuH pushed a commit to LiuGuH/hadoop that referenced this pull request Mar 26, 2024
@kokonguyen191
HDFS-16417. StaticRouterRpcFairnessPolicyController init (apache#3871)
1ac8e61
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ferhui ferhui ferhui approved these changes

@goiri goiri goiri approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kokonguyen191 @hadoop-yetus @goiri @ferhui