HADOOP-17377: ABFS: MsiTokenProvider doesn't retry HTTP 429/410 from the Instance Metadata Service #5273

anmolanmol1234 · 2023-01-04T11:54:14Z

ABFS: MsiTokenProvider doesn't retry HTTP 429 from the Instance Metadata Service

Resolution for the above mentioned issue where we should enable retries for HTTP error code 429 and HTTP error code 410 based on https://learn.microsoft.com/en-in/azure/virtual-machines/linux/instance-metadata-service?tabs=windows#errors-and-debugging and https://learn.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/how-to-use-vm-token#error-handling

...adoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java

steveloughran

commented

steveloughran · 2023-04-12T13:13:59Z

hadoop-tools/hadoop-azure/pom.xml

    <dependency>
      <groupId>org.mockito</groupId>
      <artifactId>mockito-core</artifactId>
+      <version>4.11.0</version>


sorry, hadoop-project defines the version, and through properties. revert this

again, cut this now; the version in hadoop project is the one you now expect

steveloughran · 2023-04-12T13:14:29Z

hadoop-tools/hadoop-azure/pom.xml

+    <dependency>
+      <groupId>org.mockito</groupId>
+      <artifactId>mockito-inline</artifactId>
+      <version>4.11.0</version>


if this is new to hadoop, declare it in hadoop-project/pom.xml, with versions and exclusions, then declare here without those

removed dependency

...ls/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/oauth2/AzureADAuthenticator.java

...adoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java

steveloughran · 2023-04-12T13:16:58Z

...adoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java

+   * https://learn.microsoft.com/en-us/azure/active-directory/
+   * managed-identities-azure-resources/how-to-use-vm-token#error-handling
+   */
+  private static final int HTTP_TOO_MANY_REQUESTS = 429;


make public and refer from tests, maybe put in a different file for this

steveloughran · 2023-04-12T13:17:13Z

...ools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsMsiTokenProvider.java

 public final class ITestAbfsMsiTokenProvider
    extends AbstractAbfsIntegrationTest {

+  private static final int HTTP_TOO_MANY_REQUESTS = 429;


refer to the value in the src/main code

...ools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAbfsMsiTokenProvider.java

HADOOP-17377: Add retry for HTTP 429 and HTTP 410 apache#5273

steveloughran · 2023-08-22T10:29:23Z

@anmolanmol1234 still need those (minor) changes -otherwise it is ready to merge

steveloughran

oh dear, this is a full mockito update now, which is always a PITA.

How about you split out the mockito update into its own JIRA "update mockito to 4.11.0" (and apply the comments i've done on the changes), then make the abfs change depend on it. That way a broader change is visible on its own.

...test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterRpcMultiDestination.java

steveloughran · 2023-08-22T18:05:58Z

.../hadoop/yarn/server/nodemanager/containermanager/linux/resources/TestCGroupsHandlerImpl.java

    //Since we enabled (deferred) cgroup controller mounting, no interactions
    //should have occurred, with this mock
-    verifyZeroInteractions(privilegedOperationExecutorMock);
+    Mockito.verifyNoInteractions(privilegedOperationExecutorMock);


use static import for consistency with the others.

...p/yarn/server/nodemanager/containermanager/linux/resources/gpu/TestGpuResourceAllocator.java

hadoop-project/pom.xml

steveloughran · 2023-11-13T20:20:20Z

I'll go with whatever @saxenapranav thinks here...we have seen this ourselves and need a fix.

However, that PR to update mockito bounced, so either

another attempt is made to update mockito, including the shaded client
this PR can be done without updating mockito (easier)

anmolanmol1234 · 2023-11-15T05:31:07Z

I'll go with whatever @saxenapranav thinks here...we have seen this ourselves and need a fix.

However, that PR to update mockito bounced, so either

another attempt is made to update mockito, including the shaded client

this PR can be done without updating mockito (easier)

The mockito upgrade was needed as part of this PR to mock static methods. So would it be fine if we remove that test method or if not I will attempt to upgrade mockito, including the shaded client.

anmolanmol1234 · 2023-11-15T05:37:09Z

I think this this PR is great, however there's still one related open problem: the default values (2) for fs.azure.oauth.token.fetch.retry.delta.backoff is incorrect. The value of 2 is consistent with MS recommendation (https://docs.microsoft.com/en-us/azure/active-directory/managed-service-identity/how-to-use-vm-token#retry-guidance), but it is assumed in seconds, but as this is used in Thread.sleep here, it will be measured in milliseconds. I think we should change the default to 2000. @steveloughran @anmolanmol1234 do you think we can implement this minimal change in this PR, or we should open a separate one?

Will update this change as an iteration of this PR, but will need some time for the mockito upgrade PR.

… into HADOOP-17377

hadoop-yetus · 2023-11-15T07:00:46Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
-1 ❌	mvninstall	0m 20s	/branch-mvninstall-root.txt	root in trunk failed.
-1 ❌	compile	0m 20s	/branch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt	hadoop-azure in trunk failed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.
-1 ❌	compile	0m 21s	/branch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt	hadoop-azure in trunk failed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05.
-0 ⚠️	checkstyle	0m 18s	/buildtool-branch-checkstyle-hadoop-tools_hadoop-azure.txt	The patch fails to run checkstyle in hadoop-azure
-1 ❌	mvnsite	0m 20s	/branch-mvnsite-hadoop-tools_hadoop-azure.txt	hadoop-azure in trunk failed.
-1 ❌	javadoc	0m 20s	/branch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt	hadoop-azure in trunk failed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.
+1 💚	javadoc	4m 44s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
-1 ❌	spotbugs	0m 47s	/branch-spotbugs-hadoop-tools_hadoop-azure.txt	hadoop-azure in trunk failed.
+1 💚	shadedclient	7m 10s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	7m 32s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
-1 ❌	mvninstall	0m 21s	/patch-mvninstall-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
-1 ❌	compile	0m 21s	/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt	hadoop-azure in the patch failed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.
-1 ❌	javac	0m 21s	/patch-compile-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt	hadoop-azure in the patch failed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.
-1 ❌	compile	0m 21s	/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt	hadoop-azure in the patch failed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05.
-1 ❌	javac	0m 21s	/patch-compile-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt	hadoop-azure in the patch failed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 19s	/buildtool-patch-checkstyle-hadoop-tools_hadoop-azure.txt	The patch fails to run checkstyle in hadoop-azure
-1 ❌	mvnsite	0m 24s	/patch-mvnsite-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
-1 ❌	javadoc	0m 9s	/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt	hadoop-azure in the patch failed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.
-1 ❌	javadoc	0m 9s	/patch-javadoc-hadoop-tools_hadoop-azure-jdkPrivateBuild-1.8.0_382-8u382-ga-1~20.04.1-b05.txt	hadoop-azure in the patch failed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05.
-1 ❌	spotbugs	0m 9s	/patch-spotbugs-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
-1 ❌	shadedclient	2m 52s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 9s	/patch-unit-hadoop-tools_hadoop-azure.txt	hadoop-azure in the patch failed.
+0 🆗	asflicense	0m 11s		ASF License check generated no output?
		15m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5273/19/artifact/out/Dockerfile
GITHUB PR	#5273
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 5348f26896a5 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `225541e`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5273/19/testReport/
Max. process+thread count	88 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5273/19/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

nandorKollar

LGTM, tested for our use case, where throttling caused constant 429 errors, this PR fixed the problem by doing proper retry.

steveloughran

This depends on mockito-inline to compile? Is this part of the mockito-upgrade? or is there a way to cope without it?

steveloughran · 2023-12-07T17:25:07Z

...op-azure/src/main/java/org/apache/hadoop/fs/azurebfs/constants/FileSystemConfigurations.java

  public static final int DEFAULT_AZURE_OAUTH_TOKEN_FETCH_RETRY_MIN_BACKOFF_INTERVAL = 0;
  public static final int DEFAULT_AZURE_OAUTH_TOKEN_FETCH_RETRY_MAX_BACKOFF_INTERVAL = SIXTY_SECONDS;
-  public static final int DEFAULT_AZURE_OAUTH_TOKEN_FETCH_RETRY_DELTA_BACKOFF = 2;
+  public static final int DEFAULT_AZURE_OAUTH_TOKEN_FETCH_RETRY_DELTA_BACKOFF = 2 * 1000;


steveloughran · 2023-12-07T17:28:29Z

hadoop-tools/hadoop-azure/pom.xml

      <scope>test</scope>
    </dependency>
+
+    <dependency>


is this needed? because its not in the base project pom.

I would rather this PR doesn't need that mockito upgrade as mockito upgrades are always a painful piece of work which never gets backported.

removed dependency

steveloughran · 2025-10-22T13:49:34Z

@anujmodi2021 can you revisit this so I can get it in?

we do appear to have been using it internally since december 2023, so I'm happy it works.

anmolanmol1234 · 2025-10-23T07:10:22Z

@steveloughran will backport the PR for merge and make the necessary test changes

steveloughran · 2025-10-23T15:43:18Z

sorry, commenting on wrong PR. will cut.

anujmodi2021

+1
LGTM.
Some comments need to be addressed, rest good to merge.

hadoop-yetus · 2025-10-23T18:38:55Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	38m 18s		trunk passed
+1 💚	compile	0m 45s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 48s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	checkstyle	0m 31s		trunk passed
+1 💚	mvnsite	0m 50s		trunk passed
+1 💚	javadoc	0m 40s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 37s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌	spotbugs	1m 26s	/branch-spotbugs-hadoop-tools_hadoop-azure-warnings.html	hadoop-tools/hadoop-azure in trunk has 178 extant spotbugs warnings.
+1 💚	shadedclient	27m 54s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	28m 14s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 37s		the patch passed
+1 💚	compile	0m 36s		the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 36s		the patch passed
+1 💚	compile	0m 36s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 36s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 20s		the patch passed
+1 💚	mvnsite	0m 40s		the patch passed
-1 ❌	javadoc	0m 30s	/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	hadoop-azure in the patch failed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04.
-1 ❌	javadoc	0m 29s	/patch-javadoc-hadoop-tools_hadoop-azure-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	hadoop-azure in the patch failed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04.
+1 💚	spotbugs	1m 23s		the patch passed
+1 💚	shadedclient	28m 0s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 55s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		110m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5273/22/artifact/out/Dockerfile
GITHUB PR	#5273
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux c145c3f5b775 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `fd26066`
Default Java	Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5273/22/testReport/
Max. process+thread count	629 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5273/22/console
versions	git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

anujmodi2021 · 2025-10-24T08:03:57Z

Thanks @anmolanmol1234 for refreshing up this.
@steveloughran this LGTM now and should be ready to merge.

The spotbugs adnd javadoc warnings are due to https://issues.apache.org/jira/browse/HADOOP-19731

anmolanmol1234 · 2025-10-24T08:10:31Z

:::: AGGREGATED TEST RESULT ::::

============================================================
HNS-OAuth-DFS

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 4
[WARNING] Tests run: 873, Failures: 0, Errors: 0, Skipped: 217
[WARNING] Tests run: 158, Failures: 0, Errors: 0, Skipped: 8
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 23

============================================================
HNS-SharedKey-DFS

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 5
[WARNING] Tests run: 876, Failures: 0, Errors: 0, Skipped: 169
[WARNING] Tests run: 158, Failures: 0, Errors: 0, Skipped: 8
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 10

============================================================
NonHNS-SharedKey-DFS

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 11
[WARNING] Tests run: 715, Failures: 0, Errors: 0, Skipped: 282
[WARNING] Tests run: 158, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 11

============================================================
AppendBlob-HNS-OAuth-DFS

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 4
[WARNING] Tests run: 873, Failures: 0, Errors: 0, Skipped: 228
[WARNING] Tests run: 135, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 23

============================================================
NonHNS-SharedKey-Blob

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 11
[WARNING] Tests run: 722, Failures: 0, Errors: 0, Skipped: 140
[WARNING] Tests run: 158, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 11

============================================================
NonHNS-OAuth-DFS

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 11
[WARNING] Tests run: 712, Failures: 0, Errors: 0, Skipped: 284
[WARNING] Tests run: 158, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 24

============================================================
NonHNS-OAuth-Blob

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 11
[WARNING] Tests run: 719, Failures: 0, Errors: 0, Skipped: 152
[WARNING] Tests run: 158, Failures: 0, Errors: 0, Skipped: 3
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 24

============================================================
AppendBlob-NonHNS-OAuth-Blob

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 11
[WARNING] Tests run: 714, Failures: 0, Errors: 0, Skipped: 198
[WARNING] Tests run: 135, Failures: 0, Errors: 0, Skipped: 4
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 24

============================================================
HNS-Oauth-DFS-IngressBlob

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 4
[WARNING] Tests run: 747, Failures: 0, Errors: 0, Skipped: 226
[WARNING] Tests run: 158, Failures: 0, Errors: 0, Skipped: 8
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 23

============================================================
NonHNS-OAuth-DFS-IngressBlob

[WARNING] Tests run: 194, Failures: 0, Errors: 0, Skipped: 11
[WARNING] Tests run: 712, Failures: 0, Errors: 0, Skipped: 281
[WARNING] Tests run: 158, Failures: 0, Errors: 0, Skipped: 9
[WARNING] Tests run: 271, Failures: 0, Errors: 0, Skipped: 24

anmolasrani123 added 2 commits January 4, 2023 03:51

Add retry for HTTP 429 and HTTP 410

3f6386a

Add retry for HTTP 429 and HTTP 410

56c6f7d

anmolanmol1234 changed the title ~~Add retry for HTTP 429 and HTTP 410~~ HADOOP-17377: Add retry for HTTP 429 and HTTP 410 Jan 4, 2023

anmolasrani123 added 4 commits January 4, 2023 04:04

Add retry for HTTP 429 and HTTP 410

2c25b39

Added tests to verify retry

88e51b6

Removing unused imports

3be0b00

Adding negative tests

e04011c

saxenapranav reviewed Jan 6, 2023

View reviewed changes

...adoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ExponentialRetryPolicy.java Outdated Show resolved Hide resolved

saxenapranav approved these changes Jan 11, 2023

View reviewed changes

Merge branch 'trunk' into HADOOP-17377

6283410

This comment was marked as outdated.

Sign in to view

steveloughran changed the title ~~HADOOP-17377: Add retry for HTTP 429 and HTTP 410~~ HADOOP-17377: AABFS: MsiTokenProvider doesn't retry HTTP 429/410 from the Instance Metadata Service Apr 12, 2023

steveloughran requested changes Apr 12, 2023

View reviewed changes

anmolanmol1234 added a commit to ABFSDriver/AbfsHadoop that referenced this pull request Apr 18, 2023

Merge pull request #16 from ABFSDriver/retry429fix

8904727

HADOOP-17377: Add retry for HTTP 429 and HTTP 410 apache#5273

Merge branch 'apache:trunk' into HADOOP-17377

cf73a70

github-actions bot added build trunk TOOLS ABFS labels Aug 22, 2023

Resolving PR comments

e6740a3

This comment was marked as outdated.

Sign in to view

Mockito fix

0688809

github-actions bot added YARN HDFS Common RBF labels Aug 22, 2023

steveloughran requested changes Aug 22, 2023

View reviewed changes

This comment was marked as outdated.

Sign in to view

Added in other PR

115e88a

apache deleted a comment from hadoop-yetus Nov 13, 2023

anmolanmol1234 added 3 commits November 14, 2023 22:14

Correct default value for backoff interval

240965d

Merge branch 'apache:trunk' into HADOOP-17377

b0563a1

Merge branch 'HADOOP-17377' of https://github.com/anmolanmol1234/hadoop…

225541e

… into HADOOP-17377

This comment was marked as outdated.

Sign in to view

nandorKollar approved these changes Nov 15, 2023

View reviewed changes

steveloughran reviewed Dec 7, 2023

View reviewed changes

steveloughran changed the title ~~HADOOP-17377: AABFS: MsiTokenProvider doesn't retry HTTP 429/410 from the Instance Metadata Service~~ HADOOP-17377: ABFS: MsiTokenProvider doesn't retry HTTP 429/410 from the Instance Metadata Service Dec 7, 2023

Modify tests and backmerge

c7d4671

anujmodi2021 reviewed Oct 23, 2025

View reviewed changes

This comment was marked as outdated.

Sign in to view

anmolanmol1234 added 2 commits October 23, 2025 09:44

Checkstyle and mockito fix

588ffb9

fix config value

fd26066

This comment was marked as outdated.

Sign in to view

anujmodi2021 approved these changes Oct 24, 2025

View reviewed changes

Uh oh!

HADOOP-17377: ABFS: MsiTokenProvider doesn't retry HTTP 429/410 from the Instance Metadata Service #5273

Are you sure you want to change the base?

HADOOP-17377: ABFS: MsiTokenProvider doesn't retry HTTP 429/410 from the Instance Metadata Service #5273

Conversation

anmolanmol1234 commented Jan 4, 2023 • edited by anujmodi2021 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

steveloughran left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

steveloughran commented Aug 22, 2023

Uh oh!

steveloughran left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

This comment was marked as outdated.

steveloughran commented Nov 13, 2023

Uh oh!

anmolanmol1234 commented Nov 15, 2023

Uh oh!

anmolanmol1234 commented Nov 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

hadoop-yetus commented Nov 15, 2023

Uh oh!

nandorKollar left a comment

Choose a reason for hiding this comment

Uh oh!

steveloughran left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

steveloughran commented Oct 22, 2025

Uh oh!

anmolanmol1234 commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steveloughran commented Oct 23, 2025

Uh oh!

anujmodi2021 left a comment

anmolanmol1234 commented Jan 4, 2023 •

edited by anujmodi2021

Loading

anmolanmol1234 commented Nov 15, 2023 •

edited

Loading

anmolanmol1234 commented Oct 23, 2025 •

edited

Loading

============================================================
HNS-OAuth-DFS

============================================================
HNS-SharedKey-DFS

============================================================
NonHNS-SharedKey-DFS

============================================================
AppendBlob-HNS-OAuth-DFS

============================================================
NonHNS-SharedKey-Blob

============================================================
NonHNS-OAuth-DFS

============================================================
NonHNS-OAuth-Blob

============================================================
AppendBlob-NonHNS-OAuth-Blob

============================================================
HNS-Oauth-DFS-IngressBlob

============================================================
NonHNS-OAuth-DFS-IngressBlob