Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@denis-yuen
Copy link
Member

@denis-yuen denis-yuen commented Jul 14, 2025

Description

Basic problem is that tool workflow versions are eagerly loaded whereas workflow workflow versions are properly lazily loaded.

See
https://github.com/dockstore/dockstore/blob/1.17.1/dockstore-webservice/src/main/java/io/dockstore/webservice/core/Tool.java#L168
vs
https://github.com/dockstore/dockstore/blob/1.17.1/dockstore-webservice/src/main/java/io/dockstore/webservice/core/Workflow.java#L159

However, https://github.com/dockstore/dockstore/blob/1.17.1/dockstore-webservice/src/main/java/io/dockstore/webservice/core/User.java#L159 essentially cycles through both tools and workflows, which is why the starred tools, starred workflows, and starred notebooks all have roughly the same runtime, more noticeable in aws, despite returning much different results.

An initial idea was to lazy load the tools first, but it breaks too much to just optimize for a bit of an edge case. The smaller optimization is to load only tools for starred tools, only workflows for starred workflows, etc. via JPA.
This could be combined with making tool versions actually lazy later (i.e. not an either/or)

Review Instructions
Load up a recent DB, timings between JPA queries and original should be similar to the following

original
Screenshot from 2025-07-18 11-44-41
with queries
Screenshot from 2025-07-18 11-43-43

Issue
https://ucsc-cgl.atlassian.net/browse/SEAB-7204

Security and Privacy

None

  • Security and Privacy assessed

e.g. Does this change...

  • Any user data we collect, or data location?
  • Access control, authentication or authorization?
  • Encryption features?

Please make sure that you've checked the following before submitting your pull request. Thanks!

  • Check that you pass the basic style checks and unit tests by running mvn clean install
  • Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
  • Follow the existing JPA patterns for queries, using named parameters, to avoid SQL injection
  • If you are changing dependencies, check the Snyk status check or the dashboard to ensure you are not introducing new high/critical vulnerabilities
  • Assume that inputs to the API can be malicious, and sanitize and/or check for Denial of Service type values, e.g., massive sizes
  • Do not serve user-uploaded binary images through the Dockstore API
  • Ensure that endpoints that only allow privileged access enforce that with the @RolesAllowed annotation
  • Do not create cookies, although this may change in the future
  • If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.

@denis-yuen denis-yuen self-assigned this Jul 14, 2025
@denis-yuen denis-yuen changed the title test with lazy tag loading test with jpa loading of starred entries Jul 17, 2025
@denis-yuen denis-yuen changed the title test with jpa loading of starred entries jpa loading of starred entries Jul 18, 2025
@codecov
Copy link

codecov bot commented Jul 18, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.20%. Comparing base (17468a1) to head (e692dd3).
⚠️ Report is 8 commits behind head on develop.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #6130   +/-   ##
==========================================
  Coverage      74.19%   74.20%           
- Complexity      5662     5664    +2     
==========================================
  Files            389      389           
  Lines          20341    20344    +3     
  Branches        2102     2102           
==========================================
+ Hits           15092    15096    +4     
+ Misses          4250     4249    -1     
  Partials         999      999           
Flag Coverage Δ
bitbuckettests 25.93% <27.27%> (+<0.01%) ⬆️
hoverflytests 27.62% <27.27%> (+<0.01%) ⬆️
integrationtests 56.12% <100.00%> (+0.01%) ⬆️
languageparsingtests 10.83% <27.27%> (+<0.01%) ⬆️
localstacktests 21.33% <27.27%> (+<0.01%) ⬆️
toolintegrationtests 29.89% <27.27%> (+<0.01%) ⬆️
unit-tests_and_non-confidential-tests 26.31% <27.27%> (+<0.01%) ⬆️
workflowintegrationtests 37.42% <27.27%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@denis-yuen denis-yuen marked this pull request as ready for review July 18, 2025 17:01
@denis-yuen denis-yuen requested review from a team, kathy-t and svonworl and removed request for a team July 18, 2025 17:02
@denis-yuen denis-yuen requested review from a team, kathy-t and svonworl and removed request for a team, kathy-t and svonworl July 28, 2025 17:57
@denis-yuen denis-yuen removed the request for review from kathy-t July 28, 2025 19:27
@denis-yuen denis-yuen requested review from a team and kathy-t and removed request for a team July 28, 2025 19:58
@sonarqubecloud
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
B Maintainability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@denis-yuen denis-yuen merged commit 945804e into develop Jul 28, 2025
22 of 24 checks passed
@denis-yuen denis-yuen deleted the feature/lazy_tags branch July 28, 2025 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants