Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@mabartos
Copy link
Contributor

No description provided.

@mabartos
Copy link
Contributor Author

mabartos commented Mar 14, 2023

I'm facing an error related to the fetching "lazy" collection without session:

UPDATE: Workaround described below. Will be resolved in a follow-up tasks.

2023-03-14 18:47:21,116 ERROR [org.keycloak.services.error.KeycloakErrorHandler] (executor-thread-0) Uncaught server error: org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: org.keycloak.models.jpa.entities.RealmEntity.components: could not initialize proxy - no Session
        at org.hibernate.collection.spi.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:635)
        at org.hibernate.collection.spi.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:218)
        at org.hibernate.collection.spi.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:615)
        at org.hibernate.collection.spi.AbstractPersistentCollection.read(AbstractPersistentCollection.java:136)
        at org.hibernate.collection.spi.PersistentSet.iterator(PersistentSet.java:167)
        at java.base/java.util.Spliterators$IteratorSpliterator.estimateSize(Spliterators.java:1865)
        at java.base/java.util.Spliterator.getExactSizeIfKnown(Spliterator.java:414)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:508)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        at java.base/java.util.stream.ReferencePipeline.forEachOrdered(ReferencePipeline.java:601)
        at org.keycloak.storage.AbstractStorageManager.consumeEnabledStorageProvidersWithTimeout(AbstractStorageManager.java:151)
        at org.keycloak.storage.UserStorageManager.preRemove(UserStorageManager.java:505)
        at org.keycloak.models.cache.infinispan.UserCacheSession.preRemove(UserCacheSession.java:875)
        at org.keycloak.models.jpa.JpaRealmProvider.removeRealm(JpaRealmProvider.java:171)
        at org.keycloak.models.cache.infinispan.RealmCacheSession.removeRealm(RealmCacheSession.java:499)
        at org.keycloak.services.managers.RealmManager.removeRealm(RealmManager.java:264)

@keycloak/store Do you know what could cause that?

@mabartos
Copy link
Contributor Author

mabartos commented Mar 14, 2023

I've added some comments above. ATM, it's not possible to specify any global Hibernate ORM properties inside the application.properties file.

AFAIK, for Quarkus 3, we cannot use persistence.xml config files and Quarkus Hibernate ORM properties (inside application.properties) for the default datasource/persistence-unit.

As our JPA configuration is based on legacy persistence.xml files, I kept the current approach and removed the Quarkus Hibernate ORM properties (quarkus.hibernate-orm.*).

It would probably be better to consider configuring our default persistence units via the preferred way suggested by the Quarkus team - via Quarkus properties (in application.properties). It'll be possible to have multiple persistence units specified by persistence.xml files for custom providers created by our community.

@sguilhen
Copy link
Contributor

I remember seeing an error similar to this one when the entities were not initializing their collections in the attribute declaration - i.e. when they had Set<ComponentEntity> components; instead of Set<ComponentEntity> components = new HashSet<>();

But this seems to be something different as it is complaining only about the relationship that is of type LAZY.

@ahus1
Copy link
Contributor

ahus1 commented Mar 16, 2023

@mabartos - I did some debugging today, and found that the trigger for this behavior is em.refresh(realm); in JpaRealmProvider.

As a workaround, update realmRealm() as follows - I was then able to remove a realm when accessing the UI.

    public boolean removeRealm(String id) {
        RealmEntity realm = em.find(RealmEntity.class, id, LockModeType.PESSIMISTIC_WRITE);
        if (realm == null) {
            return false;
        }
        em.refresh(realm);

        // As of Hibernate 6.2.0.CR3, after the em.refresh(entity) the lazily loaded collections in realm don't load anymore, instead there is a
        // LazyInitializationException. As a workaround, fetch the entity again.
        realm = em.find(RealmEntity.class, id, LockModeType.PESSIMISTIC_WRITE);
        if (realm == null) {
            return false;
        }

        /* ... */

It seems the only place in the code where this is called. I'll create a minimal reproducer for the Hibernate team, and ask in the Hibernate chat what they think about this behavior.

In a later step, we can also evaluate if this refresh is still needed, and if we can eliminate it altogether.

UPDATE: If this code snippet works good enough for you, and you choose to commit this code, please create a GitHub issue and add a line comment in the code to track this workaround.

@mabartos
Copy link
Contributor Author

@ahus1 Great! Thank you, I'll try it!

Yes, I'll create a separate issue for that and reference it in the commit related to the lazy loading issue.

@mabartos
Copy link
Contributor Author

@keycloak/store @keycloak/cloud-native
This PR should be ready for review.

@mabartos mabartos marked this pull request as ready for review March 16, 2023 12:39
@mabartos mabartos requested a review from a team March 16, 2023 12:39
@mabartos mabartos requested review from a team as code owners March 16, 2023 12:39
@mabartos mabartos marked this pull request as draft March 16, 2023 12:45
@mabartos mabartos marked this pull request as ready for review March 16, 2023 12:48
Copy link
Contributor

@ahus1 ahus1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for preparing this and making this work. See below for some small changes.

While supporting only JTA for Quarkus is fine, I think this breaks the Undertow setup. When running the KeycloakServer in testsuite/utils, I get an NPE (see below). I assume some of the changes in DefaultJpaConnectionProviderFactory need to be rolled back to make it work again.

Once we support JTA in the Undertow setup and also the model tests, this can be revisited.

I also see a lot of test failures on the GHA run.

Caused by: java.lang.NullPointerException
	at org.hibernate.engine.transaction.internal.jta.JtaStatusHelper.getStatus(JtaStatusHelper.java:59)
	at org.hibernate.engine.transaction.internal.jta.JtaStatusHelper.isActive(JtaStatusHelper.java:101)
	at org.hibernate.engine.transaction.jta.platform.internal.TransactionManagerBasedSynchronizationStrategy.canRegisterSynchronization(TransactionManagerBasedSynchronizationStrategy.java:39)
	at org.hibernate.engine.transaction.jta.platform.internal.AbstractJtaPlatform.canRegisterSynchronization(AbstractJtaPlatform.java:131)
	at org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorImpl.pulse(JtaTransactionCoordinatorImpl.java:150)
	at org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorImpl.<init>(JtaTransactionCoordinatorImpl.java:91)
	at org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorBuilderImpl.buildTransactionCoordinator(JtaTransactionCoordinatorBuilderImpl.java:37)
	at org.hibernate.internal.AbstractSharedSessionContract.<init>(AbstractSharedSessionContract.java:201)
	at org.hibernate.internal.SessionImpl.<init>(SessionImpl.java:218)
	at org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl.openSession(SessionFactoryImpl.java:1273)
	at org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl.openSession(SessionFactoryImpl.java:1161)
	at org.hibernate.internal.SessionFactoryImpl.buildEntityManager(SessionFactoryImpl.java:694)
	at org.hibernate.internal.SessionFactoryImpl.createEntityManager(SessionFactoryImpl.java:717)
	at org.hibernate.internal.SessionFactoryImpl.createEntityManager(SessionFactoryImpl.java:154)
	at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.createEntityManager(DefaultJpaConnectionProviderFactory.java:93)
	at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.addSpecificNamedQueries(DefaultJpaConnectionProviderFactory.java:102)
	at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.lambda$lazyInit$0(DefaultJpaConnectionProviderFactory.java:221)
	at org.keycloak.models.utils.KeycloakModelUtils.suspendJtaTransaction(KeycloakModelUtils.java:879)
	at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.lazyInit(DefaultJpaConnectionProviderFactory.java:143)
	at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.create(DefaultJpaConnectionProviderFactory.java:87)
	at org.keycloak.connections.jpa.DefaultJpaConnectionProviderFactory.create(DefaultJpaConnectionProviderFactory.java:66)
	at org.keycloak.services.DefaultKeycloakSession.getProvider(DefaultKeycloakSession.java:271)
	at org.keycloak.models.jpa.JpaRealmProviderFactory.create(JpaRealmProviderFactory.java:64)
	at org.keycloak.models.jpa.JpaRealmProviderFactory.create(JpaRealmProviderFactory.java:40)

@mabartos
Copy link
Contributor Author

@ahus1 Thanks for your review.

I already have prepared a commit to reverting changes for the DefaultJpaConnectionProviderFactory.

I can reproduce the error for GHA tests, as it's still not possible to remove the realm. ATM, when I run clear distribution, I'm able to remove the realm (with the provider realm). However, for tests, after each test class, all test realms should be removed by calling an endpoint for removing the realm.

The response code is 409 Conflict, which is quite strange. It's probably caused by the Realms' components again and probably even by some cascading issue?

The SQL Error:

16:55:40,156 INFO  [org.keycloak.testsuite.arquillian.AuthServerTestEnricher] removing test realms after test class
2023-03-16 16:56:06,451 WARN  [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (executor-thread-1) SQL Error: 23503, SQLState: 23503
2023-03-16 16:56:06,451 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (executor-thread-1) Referential integrity constraint violation: "FK_COMPONENT_REALM: PUBLIC.COMPONENT FOREIGN KEY(REALM_ID) REFERENCES PUBLIC.REALM(ID) ('test')"; SQL statement:
delete from REALM where ID=? [23503-214]
16:56:06,532 ERROR [org.keycloak.testsuite.actions.AppInitiatedActionUpdateEmailTest] [AppInitiatedActionUpdateEmailTest] null() FAILED

mabartos and others added 6 commits March 17, 2023 09:14
---
Quarkus3 branch sync no. 10 (17.3.2023)
Resolved conflicts:
keycloak/services/src/main/java/org/keycloak/protocol/saml/SamlProtocolUtils.java -	Modified
---
Quarkus3 branch sync no. 9 (10.3.2023)
Resolved conflicts:
keycloak/testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/federation/kerberos/AbstractKerberosSingleRealmTest.java - Modified
keycloak/testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/forms/LoginTest.java - Modified
---
Quarkus3 branch sync no. 8 (3.3.2023)
Resolved conflicts:
keycloak/testsuite/integration-arquillian/tests/base/src/main/java/org/keycloak/testsuite/util/SamlClient.java	Modified - Modified
keycloak/services/src/main/java/org/keycloak/protocol/saml/SamlProtocol.java - Modified
keycloak/examples/providers/authenticator/src/main/java/org/keycloak/examples/authenticator/SecretQuestionAuthenticator.java - Modified
---
Quarkus3 branch sync no. 6 (17.2.2023)
Resolved conflicts:
keycloak/integration/admin-client/src/main/java/org/keycloak/admin/client/resource/ComponentsResource.java - Modified
keycloak/testsuite/utils/src/main/java/org/keycloak/testsuite/KeycloakServer.java - Modified
keycloak/services/src/main/java/org/keycloak/protocol/saml/installation/SamlSPDescriptorClientInstallation.java - Modified
---
Quarkus3 branch sync no. 5 (10.2.2023)
Resolved conflicts:
/keycloak/services/src/main/java/org/keycloak/social/google/GoogleIdentityProvider.java	Modified - Modified
keycloak/services/src/main/java/org/keycloak/social/twitter/TwitterIdentityProvider.java - Modified
---
Quarkus3 branch sync no. 4 (3.2.2023)
Resolved conflicts:
keycloak/quarkus/runtime/src/main/java/org/keycloak/quarkus/runtime/integration/jaxrs/QuarkusKeycloakApplication.java - Modified
---
Quarkus3 branch sync no. 1 (18.1.2023)
Resolved conflicts:
keycloak/testsuite/client/ClientPoliciesTest.java - Deleted
keycloak/testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/client/ClientRegistrationTest.java - Modified
keycloak/model/map-jpa/src/main/java/org/keycloak/models/map/storage/jpa/JpaModelCriteriaBuilder.java - Modified
---
Quarkus3 branch sync no. 5 (10.2.2023)
Resolved conflicts:
keycloak/pom.xml - Modified
---
Quarkus3 branch sync no. 7 (27.2.2023)
Resolved conflicts:
keycloak/model/map-jpa/src/main/java/org/keycloak/models/map/storage/jpa/client/entity/JpaClientEntity.java - Modified
keycloak/model/map-jpa/src/main/java/org/keycloak/models/map/storage/jpa/clientScope/entity/JpaClientScopeEntity.java - Modified
keycloak/pom.xml - Modified
---
Quarkus3 branch sync no. 6 (17.2.2023)
Resolved conflicts:
keycloak/model/map-jpa/src/main/java/org/keycloak/models/map/storage/jpa/JpaMapStorageProviderFactory.java - Modified
---
Quarkus3 branch sync no. 1 (18.1.2023)
Resolved conflicts:
keycloak/model/map-jpa/src/main/java/org/keycloak/models/map/storage/jpa/hibernate/dialect/JsonbPostgreSQL95Dialect.java - Modified
---
Quarkus3 branch sync #1 (18.1.2023)
Resolved conflicts:
keycloak/quarkus/runtime/pom.xml - Modified
keycloak/quarkus/pom.xml - Modified
keycloak/quarkus/deployment/pom.xml - Modified
@mabartos mabartos closed this Mar 22, 2023
@mabartos mabartos reopened this Mar 22, 2023
@mabartos mabartos mentioned this pull request Mar 24, 2023
return message.toLowerCase().contains("duplicate");
};

Predicate<Throwable> throwModelDuplicateEx = throwable ->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ahus1 @vmuzikar
This commit solves the issue of handling exceptions for the JTA tx type.
You can check the original log[1]. There's only a WARN log where the org.hibernate.exception.ConstraintViolationException is thrown, but at the end, we get directly the cause of that (you can see the comment of this method.).

[1] https://gist.github.com/mabartos/e8ea1e3ea945cf0b22748c40c2ce7e63

@mabartos
Copy link
Contributor Author

When I tried to upgrade to Quarkus 3.0.0.Beta1 and Hibernate ORM 6.2.0.CR4, I wasn't able to start server due to this ClassCastException:

Command: ./keycloak-999.0.0-SNAPSHOT/bin/kc.sh --verbose start-dev

ERROR: Unexpected error when starting the server in (development) mode
Error details:
java.lang.RuntimeException: Failed to start quarkus
        at io.quarkus.runner.ApplicationImpl.<clinit>(Unknown Source)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:70)
        at org.keycloak.quarkus.runtime.KeycloakMain.start(KeycloakMain.java:102)
        at org.keycloak.quarkus.runtime.cli.command.AbstractStartCommand.run(AbstractStartCommand.java:37)
        at picocli.CommandLine.executeUserObject(CommandLine.java:2026)
        at picocli.CommandLine.access$1500(CommandLine.java:148)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
        at picocli.CommandLine.execute(CommandLine.java:2170)
        at org.keycloak.quarkus.runtime.cli.Picocli.parseAndRun(Picocli.java:93)
        at org.keycloak.quarkus.runtime.KeycloakMain.main(KeycloakMain.java:88)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at io.quarkus.bootstrap.runner.QuarkusEntryPoint.doRun(QuarkusEntryPoint.java:61)
        at io.quarkus.bootstrap.runner.QuarkusEntryPoint.main(QuarkusEntryPoint.java:32)
Caused by: java.lang.ClassCastException: class org.hibernate.mapping.BasicValue cannot be cast to class org.hibernate.mapping.ToOne (org.hibernate.mapping.BasicValue and org.hibernate.mapping.ToOne are in unnamed module of loader io.quarkus.bootstrap.runner.RunnerClassLoader @63e31ee)
        at org.hibernate.boot.model.internal.BinderHelper.checkMappedByType(BinderHelper.java:1085)
        at org.hibernate.boot.model.internal.CollectionBinder.isReversePropertyInJoin(CollectionBinder.java:1569)
        at org.hibernate.boot.model.internal.CollectionBinder.noAssociationTable(CollectionBinder.java:1579)
        at org.hibernate.boot.model.internal.CollectionBinder.bindStarToManySecondPass(CollectionBinder.java:1544)
        at org.hibernate.boot.model.internal.MapBinder$1.secondPass(MapBinder.java:94)
        at org.hibernate.boot.model.internal.CollectionSecondPass.doSecondPass(CollectionSecondPass.java:45)
        at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.processSecondPasses(InFlightMetadataCollectorImpl.java:1846)
        at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.processSecondPasses(InFlightMetadataCollectorImpl.java:1803)
        at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:324)
        at io.quarkus.hibernate.orm.runtime.boot.FastBootMetadataBuilder.build(FastBootMetadataBuilder.java:404)
        at io.quarkus.hibernate.orm.runtime.PersistenceUnitsHolder.createMetadata(PersistenceUnitsHolder.java:101)
        at io.quarkus.hibernate.orm.runtime.PersistenceUnitsHolder.constructMetadataAdvance(PersistenceUnitsHolder.java:73)
        at io.quarkus.hibernate.orm.runtime.PersistenceUnitsHolder.initializeJpa(PersistenceUnitsHolder.java:40)
        at io.quarkus.hibernate.orm.runtime.HibernateOrmRecorder$1.created(HibernateOrmRecorder.java:78)
        at io.quarkus.arc.runtime.ArcRecorder.initBeanContainer(ArcRecorder.java:75)
        at io.quarkus.deployment.steps.ArcProcessor$generateResources844392269.deploy_0(Unknown Source)
        at io.quarkus.deployment.steps.ArcProcessor$generateResources844392269.deploy(Unknown Source)
        ... 25 more

@ahus1
Copy link
Contributor

ahus1 commented Mar 24, 2023

@mabartos: created a new issue:

There is also a PR with a workaround: mabartos#126

@ahus1
Copy link
Contributor

ahus1 commented Mar 24, 2023

Current status for "identifier of an instance changed"

Investigation for a reproducer continues.

@ahus1
Copy link
Contributor

ahus1 commented Mar 27, 2023

There is now a new branch https://github.com/ahus1/keycloak/tree/quarkus3-props which contains the two workarounds.

I also found a way to make Keycloak start with Quarkus 3.0.0.Beta1 by tidying up a one-to-many relationship: 53b31b2e5818859188b03ef00ccc711c9177e534

@vmuzikar
Copy link
Contributor

@ahus1 That's wonderful news, thanks! Does it mean there's a workaround for all Hibernate issues found?

Also, would it make sense to close this PR and open another one from your branch?

@ahus1
Copy link
Contributor

ahus1 commented Mar 27, 2023

@vmuzikar

Does it mean there's a workaround for all Hibernate issues found?

I'm not so happy about this one: 822ac249347ac01e581e80726e724bc61e2d2245 - it is a workaround for this case "in the middle of the code", and it might pop up in other places. I'd like to get to the bottom of this to figure how it is caused. The other workaround 93cdc4cd2edd7fab9dcc110437534fb2cd7928ec was changing a Hibernate property, which is a more valid workaround IMHO.

At the moment I'm looking at the actions run in my repo to see what is currently failing.

Also, would it make sense to close this PR and open another one from your branch?

Once we know who will take this forward, there will be a new PR. Once there is a new PR, this old PR will be closed.

@ahus1
Copy link
Contributor

ahus1 commented Mar 28, 2023

The work on this topic continues in #19387

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants