Codecov Report

❌ Patch coverage is 60.78431% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.02%. Comparing base (cc46e6f) to head (ac2426a).
⚠️ Report is 4 commits behind head on hotfix/1.18.1.

@@                 Coverage Diff                 @@
##             hotfix/1.18.1    #6174      +/-   ##
===================================================
- Coverage            74.07%   74.02%   -0.05%     
- Complexity            5724     5731       +7     
===================================================
  Files                  397      397              
  Lines                20571    20611      +40     
  Branches              2116     2117       +1     
===================================================
+ Hits                 15238    15258      +20     
- Misses                4326     4345      +19     
- Partials              1007     1008       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

1. Attempting to delete any draft deposit(s?) corresponding to the concept DOI, at the start of the DOI generation process.

Idle thought reading in sequence. How do we think this will react, thinking particularly of the Broad case where they may be deleting and re-creating multiple tags at the same time (will one webservice delete the in-progress draft deposits being created for another one?)

with the caveat that we find drafts via what appears to be an ElasticSearch query on the Zenodo side, which may not always provide up-to-date information (a draft may not yet be indexed, or still be in the index after it is deleted).

On the other hand, maybe this is slow enough?

1. Attempting to delete any draft deposit(s?) corresponding to the concept DOI, at the start of the DOI generation process.

Idle thought reading in sequence. How do we think this will react, thinking particularly of the Broad case where they may be deleting and re-creating multiple tags at the same time (will one webservice delete the in-progress draft deposits being created for another one?)

I think it's ok, the reasoning is something like:

• Currently, we serialize the push processing at the repo level, so, in theory, for a given repo, we should only be generating a single DOI at a given time.
• To create the next version DOI, we only remove drafts associated with the particular concept DOI, so we're not going to accidentally clobber drafts of other workflows.
• The drafts that show up in the query might be stale, and could actually be published or deleted, in which case the deleteDeposit API call fails, the exception is absorbed, and the DOI generation process continues.

Ok, so, I ran some experiments and dabbled with some test code. I made some improvements to the findDraftDeposits function, that make it more reliable and ensure good performance, even if the API starts mixing published deposits into the response for whatever reason.

Here's my conclusion:

I strongly recommend we go with the current solution. Would like to get this into the hotfix, deploy, and assess how it works after a couple of weeks. Can we do that?

My reasoning:

1. The design and behavior of the various APIs suggest that they provide different views of the same ElasticSearch resource (I could be wrong, of course). If that's true, switching the calls doesn't buy us anything, and calling both endpoints burns time.
2. When possible, we should use well-documented endpoints, because undocumented endpoints are more likely to change or disappear.
3. The deleteDeposits endpoint is documented to only delete unpublished deposits, so there's no danger there. As mentioned in point 1, would not be surprising if the deleteDraftRecord endpoint used the exact same machinery on the backend.

When possible, we should use well-documented endpoints, because undocumented endpoints are more likely to change or disappear.

To be clear, this is not an argument in favour of the current approach.

The "new" endpoints are documented by zenodo in openapi, but incompletely without return objects.

The "old" endpoints are purely documented by us in openapi by inspecting their textual documentation and behaviour.

I then extended the openapi description that we use (owned by us in our repository) for the "old" endpoints to cover those two "new" endpoints.

3. The deleteDeposits endpoint is documented to only delete unpublished deposits, so there's no danger there. As mentioned in point 1, would not be surprising if the deleteDraftRecord endpoint used the exact same machinery on the backend.

I'm not sure about the downside of just using the new endpoint here.

When possible, we should use well-documented endpoints, because undocumented endpoints are more likely to change or disappear.

To be clear, this is not an argument in favour of the current approach.

The "new" endpoints are documented by zenodo in openapi, but incompletely without return objects.

The "old" endpoints are purely documented by us in openapi by inspecting their textual documentation and behaviour.

It is indeed an argument in favor of the current approach, both of the "old" endpoints are documented here:
https://developers.zenodo.org/

Quality Gate failed

Failed conditions
54.9% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

I changed the code to use the "new" listUserRecords and deleteDraftRecord API calls.