22352 kerberos fallback #22531

mposolda · 2023-08-17T19:11:22Z

Summary of changes:

It adds new state FALLBACK to CredentialValidationOutput.Status. This means that federation provider was not able to finish authentication, and wants to fallback to other providers.
The status FAILED means that user storage provider wants to finish authentication with failure without fallback to other providers (current behaviour from Keycloak main). Status CONTINUE means that federation provider does not want to finish, but doesn't want fallback as another challenge response might need to be sent to the user (current behaviour from Keycloak main). Added also javadoc to what those status should mean.
@ahus1 This should raise your concern from the Make UserStorageManager check all applicable CredentialAuthenticators #22353 (comment) as it should be possible to acknowledge fallback as well as fail authentication right away.
There are 2 scenarios when there are multiple LDAP providers and fallback to next federation provider might be needed. Added automated tests to both of them in this PR:

Scenario when ldap-provider-1 (which is 1st in the chain) is not able to do SPNEGO authentication at all. For instance because token contains completely different kerberos realm. In this case SPNEGO authentication did not success, so fallback to another provider might be needed
Scenario when ldap-provider-1 (which is 1st in the chain) is able to do SPNEGO authentication, but that provider is not able to lookup the user. This is common scenario with various "forest trust" scenarios. For this case, I've introduced note KerberosConstants.AUTHENTICATED_SPNEGO_CONTEXT when the authenticated Spnego from 1st provider can be saved and then next provider in the chain (like ldap-provider-2) can lookup user.

It can be better if same provider always does both SPNEGO authentication and lookup the user, but repeating SPNEGO authentication with same token is not possible as this would replay error (Something like Caused by: KrbException: Request is a replay (34)) and authentication would not be possible to happen.

At the same time, I don't see much issues with provider1 doing SPNEGO authentication and provider2 doing eventual lookup the user. Assumption is that administrator has control over the user-storage providers configured in the realm and hence automatically doing fallback does not seem to me as an issue. We can eventually add an option to LDAP/Kerberos providers like Supports fallback for Kerberos authentication, which will allow both save spnego context for next providers as well as consume spnego context from previous providers. But so far, I did not messed with another option due I don't see issues with doing fallback automatically and the amount of options on LDAP-provider is already quite big.

I've updated only UserStorageManager in this PR, but did not updated MapUserProvider. I am not sure if it is needed to deal with the map storage and also wanted to first ask for the feedback in this PR. I can possibly update MapUserProvider as well if you think it is necessary and we're ok with those changes.

@ahus1 To your question from about detecting realm from spnego token - This is not easily possible. Spnego token is Base64 encoded token in ASN1 format with complicated structure. It might be possible to parse it with some 3rd party library or with BouncyCastle. Some hints are here https://stackoverflow.com/questions/4508555/decrypt-kerberos-ticket-using-spnego . I've tried to parse it with BouncyCastle (just for fun) in this branch https://github.com/mposolda/keycloak/blob/22352-spnego-parsing/crypto/default/src/test/java/org/keycloak/crypto/def/test/SPNEGOParseExample.java, but figured it is not trivial... At the same time, even if we figure parsing, announce which kerberos realms the LDAP provider supports won't be very useful as it is possible that realm MYREALM.ORG is supported by 2 LDAP providers ldap-provider-1 and ldap-provider-2 and hence fallback might be still needed as user authenticated by ldap-provider-1 might be available in the LDAP subtree of ldap-provider-2. Also requirement to announce realms might not be very useful in the scenarios with MSAD forest trust with many realms when realms are added to the forest ad-hoc and hence the change in the configuration of LDAP providers would be needed always when adding new realm. However it can help with the scenario 1 I've described above as the provider1 would not try to authenticate SPNEGO token, which is clearly for different realm.

cypress · 2023-08-17T19:43:51Z

2 flaky tests on run #8515 ↗︎

0	527	48	0	2

Details:

Merge 82c01841ebaedd5bab0056c724d2074af63ccde7 into `b99eb52`...
Project: Keycloak Admin UI	Commit: `cf58027886 ℹ️`
Status: Passed	Duration: 17:10 💡
Started: Aug 17, 2023 7:26 PM	Ended: Aug 17, 2023 7:43 PM

authentication_test.spec.ts • 1 flaky test • chrome

View Output Video

Test		Artifacts
Authentication test > should add a condition		`Output` `Screenshots`

realm_settings_general_tab_test.spec.ts • 1 flaky test • chrome

View Output Video

Test		Artifacts
Realm settings general tab tests > Test all general tab switches		`Output` `Screenshots`

_{This comment has been generated by cypress-bot as a result of this
project's GitHub integration settings.}

ullgren · 2023-08-18T18:34:42Z

@mposolda Thanks so much for looking in to this. Initial tests using our reproducer show that this PR does solve our issue, as reported in #22352

ahus1

@mposolda - thank you for the changes, they look good to me.

It might be worth adding this to the release notes, I assume some people have been waiting for this quite some time, and they'll be happy to find it there.

For the map storage, feel free to update the MapUserProvider or open a new ticket to add that support. It might be difficult (impossible) to write a test for the MapUserProvider yet, as the PR to add LDAP/Kerberos support for the map storage is still pending, see #10700.

Let me know how you decide, and then re-request a review and I'll approve it.

ullgren · 2023-08-20T06:40:53Z

@mposolda Done some more testing with our test environment and it confirmed that this PR solves our problem. Again thanks so much for the work.

pedroigor · 2023-08-21T11:58:32Z

server-spi/src/main/java/org/keycloak/models/CredentialValidationOutput.java

Not a blocker as it is related to naming. I think we can make the statues a bit more clear if:

VALID (AUTHENTICATED)

INVALID (FAILED)

VALID_USER (FALLBACK)

CHALLENGE (CONTINUE)

My point is:

AUTHENTICATED: Credentials are validated and authentication happens on the upper layer where the authentication logic is in

FAILED, the validator is actually returning that the credential is invalid

VALID_USER, the validator is actually returning if the user is valid but not the credential. The decision to try other providers to authenticate the user depends on the upper layer.

CONTINUE, this name indicates for me that validation should continue and not necessarily the authentication (somewhat related to the FALLBACK which indicates moving forward with the provider chain.

Thanks Pedro for the suggestions. Your naming is probably better, but there is also concern with backwards compatibility as CredentialValidationOutput class is part of the server-spi module, which is part of user-storage SPI and hence backwards compatibility might be a concern...

@ahus1 What do you think? Can we rename things or is it better to keep as it is?

I see options like:

Rename as @pedroigor suggested

Rename only FALLBACK to VALID_USER . This is fine for backwards compatibility though as FALLBACK is new thing added in this PR

Keep as it is

I vote for 2. I can do 1 if @ahus1 agrees that backwards compatibility is not an issue.

Update: In the end I've kept FALLBACK instead of VALID_USER. The VALID_USER is not accurate as it is used even for the cases when the user-storage provider is not able to recognize the credential nor the user and hence it just needs to fallback to next provider.

I see options like:

Rename statuses to VALID, INVALID, FALLBACK, CHALLENGE

Keep statuses as is in this PR

My vote is 2 just due the backwards compatibility. If we can break backwards compatibility, I am fine with (1).

@pedroigor @ahus1 What do you think?

Looking at the docs, I see that this change is targeted for KC22, so it should be backward compatible. Therefore I vote to not rename the options.

Not a blocker for me.

mposolda · 2023-08-29T07:11:12Z

@mposolda Done some more testing with our test environment and it confirmed that this PR solves our problem. Again thanks so much for the work.

@ullgren Thanks for testing and confirming this!

…icated Kerberos principal closes keycloak#22352 keycloak#9422

mposolda · 2023-08-29T08:23:14Z

@ahus1 @pedroigor I've added another commit with the release notes and with some small clarification in the javadoc of CredentialValidationOutput class. In the end, I did not updated any names on CredentialValidationOutput.Status enum. Can you please check it?

I've created #22780 as a follow-up task for the new store.

I've used separate commit, so you can review the changes, but I plan to squash it before merge.

ahus1

LGTM - one small nit-pick around the docs.

docs/documentation/release_notes/topics/22_0_2.adoc

ahus1 · 2023-08-29T09:07:57Z

server-spi/src/main/java/org/keycloak/models/CredentialValidationOutput.java

Looking at the docs, I see that this change is targeted for KC22, so it should be backward compatible. Therefore I vote to not rename the options.

Co-authored-by: Alexander Schwartz <[email protected]>

mposolda · 2023-08-29T09:49:44Z

@ahus1 Thanks for another review! I've applied your comments regarding docs. Also Thanks for the clarification for backwards compatibility.

ghost

Unreported flaky test detected, please review

ghost · 2023-08-29T11:21:09Z

Unreported flaky test detected

If the below flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.forms.RegisterWithUserProfileTest#testRegisterUserSuccess_attributeNotRequiredAndSelectedByScopeCanBeIgnored

Keycloak CI - Forms IT (chrome)

java.lang.IllegalArgumentException: No enum constant org.keycloak.testsuite.pages.AppPage.RequestType.
	at java.base/java.lang.Enum.valueOf(Enum.java:273)
	at org.keycloak.testsuite.pages.AppPage$RequestType.valueOf(AppPage.java:56)
	at org.keycloak.testsuite.pages.AppPage.getRequestType(AppPage.java:49)
	at jdk.internal.reflect.GeneratedMethodAccessor552.invoke(Unknown Source)
...

Report flaky test

org.keycloak.testsuite.forms.VerifyProfileTest#testAttributeRequiredButNotSelectedByScopeIsNotRenderedOnVerificationScreenForcedByAnotherAttribute

Keycloak CI - Forms IT (chrome)

java.lang.IllegalArgumentException: No enum constant org.keycloak.testsuite.pages.AppPage.RequestType.
	at java.base/java.lang.Enum.valueOf(Enum.java:273)
	at org.keycloak.testsuite.pages.AppPage$RequestType.valueOf(AppPage.java:56)
	at org.keycloak.testsuite.pages.AppPage.getRequestType(AppPage.java:49)
	at jdk.internal.reflect.GeneratedMethodAccessor552.invoke(Unknown Source)
...

Report flaky test

…icated Kerberos principal (keycloak#22531) closes keycloak#22352 keycloak#9422 (cherry picked from commit 6f989fc)

mposolda · 2023-08-29T11:37:55Z

@ahus1 @pedroigor Thanks for the review!

mposolda self-assigned this Aug 17, 2023

mposolda force-pushed the 22352-kerberos-fallback branch from 82c0184 to 4ab9609 Compare August 18, 2023 08:36

mposolda marked this pull request as ready for review August 18, 2023 09:20

mposolda requested a review from a team August 18, 2023 09:20

mposolda requested review from a team as code owners August 18, 2023 09:20

ghost added team/core team/cloud-native labels Aug 18, 2023

mposolda requested a review from ahus1 August 18, 2023 09:21

mposolda mentioned this pull request Aug 18, 2023

Make UserStorageManager check all applicable CredentialAuthenticators #22353

Closed

This was linked to issues Aug 18, 2023

map a kerberos provider to one or more ldap provider stores #14665

Closed

Support kerberos realm filter on LDAP provider #9422

Closed

mposolda mentioned this pull request Aug 18, 2023

Use an original domain name of Kerberos Principal in UserModel attrib… #22516

Merged

ahus1 reviewed Aug 18, 2023

View reviewed changes

pedroigor reviewed Aug 21, 2023

View reviewed changes

mposolda added 2 commits August 29, 2023 09:37

Fallback to next LDAP/Kerberos provider when not able to find authent…

6a201ec

…icated Kerberos principal closes keycloak#22352 keycloak#9422

Feedback ...

54ee97b

mposolda force-pushed the 22352-kerberos-fallback branch from 4ab9609 to 54ee97b Compare August 29, 2023 08:15

mposolda mentioned this pull request Aug 29, 2023

Align MapStorageProvider with the changes in LDAP/Kerberos integration #22780

Closed

ahus1 previously approved these changes Aug 29, 2023

View reviewed changes

ghost added the flaky-test label Aug 29, 2023

ghost mentioned this pull request Aug 29, 2023

Flaky test: org.keycloak.testsuite.forms.BrowserFlowTest#testAlternativeNonInteractiveExecutorInSubflow #22644

Closed

This was referenced Aug 29, 2023

Flaky test: org.keycloak.testsuite.forms.ResetPasswordTest#resetPasswordLink #21422

Closed

Flaky test: org.keycloak.testsuite.model.ConcurrentHashMapStorageTest#testStorageSeparation #22645

Closed

Update docs/documentation/release_notes/topics/22_0_2.adoc

65aeff3

Co-authored-by: Alexander Schwartz <[email protected]>

mposolda dismissed ahus1’s stale review via 65aeff3 August 29, 2023 09:48

ahus1 self-requested a review August 29, 2023 10:28

ahus1 approved these changes Aug 29, 2023

View reviewed changes

pedroigor approved these changes Aug 29, 2023

View reviewed changes

ahus1 enabled auto-merge (squash) August 29, 2023 11:17

ahus1 merged commit 6f989fc into keycloak:main Aug 29, 2023

ghost reviewed Aug 29, 2023

View reviewed changes

mposolda added a commit to mposolda/keycloak that referenced this pull request Aug 29, 2023

Fallback to next LDAP/Kerberos provider when not able to find authent…

52af4a9

…icated Kerberos principal (keycloak#22531) closes keycloak#22352 keycloak#9422 (cherry picked from commit 6f989fc)

mposolda mentioned this pull request Aug 29, 2023

Fallback to next LDAP/Kerberos provider when not able to find authent… #22788

Merged

rmartinc mentioned this pull request Oct 19, 2023

LDAP user mapper for username: user appears twice in the GUI #24141

Closed

2 tasks

22352 kerberos fallback #22531

22352 kerberos fallback #22531

Uh oh!

Conversation

mposolda commented Aug 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cypress bot commented Aug 17, 2023

2 flaky tests on run #8515 ↗︎

Uh oh!

ullgren commented Aug 18, 2023

Uh oh!

ahus1 left a comment

Choose a reason for hiding this comment

Uh oh!

ullgren commented Aug 20, 2023

Uh oh!

pedroigor Aug 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mposolda Aug 29, 2023

Choose a reason for hiding this comment

Uh oh!

mposolda Aug 29, 2023

Choose a reason for hiding this comment

Uh oh!

ahus1 Aug 29, 2023

Choose a reason for hiding this comment

Uh oh!

pedroigor Aug 29, 2023

Choose a reason for hiding this comment

Uh oh!

mposolda commented Aug 29, 2023

Uh oh!

mposolda commented Aug 29, 2023

Uh oh!

ahus1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ahus1 Aug 29, 2023

Choose a reason for hiding this comment

Uh oh!

mposolda commented Aug 29, 2023

Uh oh!

ghost left a comment

Choose a reason for hiding this comment

Uh oh!

ghost commented Aug 29, 2023

Unreported flaky test detected

org.keycloak.testsuite.forms.RegisterWithUserProfileTest#testRegisterUserSuccess_attributeNotRequiredAndSelectedByScopeCanBeIgnored

org.keycloak.testsuite.forms.VerifyProfileTest#testAttributeRequiredButNotSelectedByScopeIsNotRenderedOnVerificationScreenForcedByAnotherAttribute

Uh oh!

mposolda commented Aug 29, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mposolda commented Aug 17, 2023 •

edited

Loading

pedroigor Aug 21, 2023 •

edited

Loading