[API] Add dicom upload #9154

maximemulder · 2024-03-20T20:25:24Z

Brief summary of changes

This is a port of #6161, which contains a description of the changes. Since the old PR has diverged quite a bit from the main branch, and that its author no longer works here, I found it easier to copy the changes on the current main branch and open a new PR rather than to rebase the PR. While porting, I only rewrote the old PR code whenever static analysis or practical stests showed that it was no longer in sync with the main branch (or when my IDE removed trailing spaces).

Testing instructions (if applicable)

Test the four new API routes, described in section 5 of the documentation of the API v0.0.4.

Notes for existing projects

This PR requires to run the following SQL patch SQL/New_patches/2024-03_19-MRIUploadServerProcessRel.sql.

maximemulder · 2024-03-20T20:33:26Z

I must say I am not sure what I am supposed to do regarding the failed tests on files that I did not modify in the PR. Any direction @driusan (or someone else) 🤔.

kongtiaowang · 2024-04-24T18:08:35Z

maximemulder · 2024-04-24T19:56:54Z

Hum, yeah, I already saw that.

I was just suprised to see that the errors are marked as originating from FilesDownloadHandler.php and SiteIDGenerator.php, which are not modified by this PR. I guess I must call these files from elsewhere and that the location reporting is just imprecise.

I'll tackle this tomorrow probably.

maximemulder · 2024-05-22T17:31:12Z

I've looked at the static errors in more details, it appears to be mainly due to an upgrade of phpstan, which changes the typing of string conversions with strval and (string) from mixed -> string to (bool|float|int|resource|string|null) -> string. I still don't know why this PR triggers the error and not other ones (it did trigger on my main branch though), but at least the errors are clear.

However, I find the code that triggers the errors to be quite cryptic, with some lines that seem obviously wrong (at least according to typing). If someone who knows this code better than me could look into it, it would be best. In the meantime, I'm adding this to the next meeting.

laemtl · 2024-05-28T15:33:12Z

I still don't know why this PR triggers the error and not other ones (it did trigger on my main branch though), but at least the errors are clear.

Probably because composer update was run, which triggered an update of the static checker.

maximemulder · 2024-05-28T18:12:02Z

Ok it seems the linting errors are ~~fixed~~, silenced. @laemtl had noticed that the errors were due to a composer update on my end. The errors should still appear in any PR that updates composer and should be fixed eventually. However, this can have its own issue and be another PR.

maximemulder · 2024-08-29T16:27:06Z

@ridz1208 Ok I finally finished updating and re-testing this PR. Generally it seems to work. However, if the pipeline fails during the latter stages, the DICOM archive previously inserted does not have its session initialized. This should be fixed with the new DICOM archive PR, but I don't expect it to be merged before at least a few weeks.

cmadjar

@maximemulder I did a first pass of review. I did not have a chance to actually test the changes as my MCIN sandbox is not letting me logging in. Have to figure out why and it could take some time so figured I would do an initial review.

modules/api/docs/LorisRESTAPI_v0.0.4-dev.md

cmadjar · 2024-09-03T20:10:03Z

modules/api/docs/LorisRESTAPI_v0.0.4-dev.md

+GET /candidates/$CandID/$VisitLabel/dicoms/$Tarname/processes
+```
+
+The response contains all `mri_upload` attempts with the specified `$tarname`. And for


technically, there should only be one UploadID per TarchiveID. So when you overwrite, you should grep the UploadID associated to what is being overwritten instead of inserting a new entry in mri_upload. Especially since the incoming directory content will be overwritten anyway.

I agree that there is only one MRI upload per DICOM archive, but I've seen a few cases in testing (even without this PR) where there are a few failed MRI uploads and one successful for a single file. This is how the batch uploader does it (don't know about other pipelines).

yeah. I think in principle, there should only be one upload per tarchive but unfortunately the batch upload script is not compliant with this set up. However, for the API, I think we should make sure this is the behaviour at minimum, to avoid duplicated entries.

Is this thread resolved?

modules/api/php/endpoints/candidate/visit/dicoms.class.inc

modules/imaging_browser/test/imaging_browserTest.php

php/libraries/DicomTar.class.inc

src/Data/Models/DicomSeriesDTO.php

driusan · 2024-09-04T16:51:58Z

Please remove all the unrelated changes (whitespace, variable name changes, etc). If necessary you can send those in another PR. It is not feasible to review this PR with so many unrelated changed mixed together.

maximemulder · 2024-09-17T20:18:42Z

I replied to all the comments and applied the suggested changes.

I reverted the breaking changes to the API including floats / strings and naming. Back when I wrote them, I thought it was part of the original PR and not something that was already in LORIS.

I made the terminology more consistent by using $VisitLabel everywhere in the DICOM part (section 5) of the API documentation.

I also disabled my trailing spaces auto-removal and removed the associated changes.

Ready for review @cmadjar @driusan

maximemulder · 2024-09-18T14:10:28Z

modules/api/docs/LorisRESTAPI.md

+                {
                    "SeriesDescription" : "MPRAGE_ipat2",
-                    "SeriesNumber" : "2",
+                    "SeriesNumber" : 2,


Regarding this change, the current documentation is inaccurate, I can extract this to a separate PR if wanted.

Example:
https://demo.loris.ca/api/v0.0.4-dev/candidates/587630/V1/dicoms

I need to look into this to see if it's a bug in the documentation or the code. It seems weird that it's only the SeriesNumber that does it and it might be related to PHP (not LORIS) changes in the PDO over time.

It might be faster to split it out.

cmadjar

@maximemulder I re-reviewed quickly your PR. A few additional comments:

In the description of the PR, could you add a "Notes for existing project" with the information of the SQL patches to run?

modules/api/docs/LorisRESTAPI_v0.0.4-dev.md

modules/api/php/endpoints/candidate/visit/dicoms.class.inc

maximemulder · 2025-02-12T23:59:19Z

@driusan Rebased on main again (a new conflict appeared) and reset 0.0.3 documentation to main.

Minor documentation changes of the server process monitor that were present in #9154 and extracted into a new PR.

ridz1208 · 2025-05-22T17:12:47Z

@driusan this is showing as changes requested because of your review.

outdated

This is a port of [aces#6161](aces#6161), which contains a description of the changes. It adds endpoints to the LORIS API for uploading dicom archives

kongtiaowang closed this May 28, 2024

kongtiaowang reopened this May 28, 2024

maximemulder mentioned this pull request Aug 13, 2024

Draft architecture for LORIS API calls within LORIS-MRI Python aces/Loris-MRI#1148

Closed

maximemulder force-pushed the api-dicom-upload branch from fd349f1 to 222b5d4 Compare August 29, 2024 16:22

maximemulder force-pushed the api-dicom-upload branch 8 times, most recently from 4b8a6dc to 9765668 Compare August 29, 2024 18:04

maximemulder requested review from cmadjar and ridz1208 September 3, 2024 15:23

cmadjar requested changes Sep 3, 2024

View reviewed changes

maximemulder force-pushed the api-dicom-upload branch 2 times, most recently from 65869da to 4c63612 Compare September 17, 2024 20:15

maximemulder commented Sep 18, 2024

View reviewed changes

maximemulder force-pushed the api-dicom-upload branch 2 times, most recently from 6f882bc to 8112fc8 Compare September 23, 2024 15:28

cmadjar reviewed Sep 30, 2024

View reviewed changes

modules/api/docs/LorisRESTAPI_v0.0.4-dev.md Outdated Show resolved Hide resolved

modules/api/php/endpoints/candidate/visit/dicoms.class.inc Outdated Show resolved Hide resolved

maximemulder added 14 commits February 12, 2025 23:54

fix process link

e8947d2

file size message

0f73657

re-use old mri_upload

3c52f39

fix lints

dab77ad

add int return type

560091c

typo

3df6596

merge functions

867edad

undo unrelated changes

cdd2203

add swagger schema

20fc9a6

use instead of singleton

cb45756

fix provisionner based on main

4993895

revert seriesnumber type doc change

3dfdc7b

fix rebase fail

b0435fc

reset 0.0.3 doc to main

a6ddf96

maximemulder force-pushed the api-dicom-upload branch from ca8a701 to a6ddf96 Compare February 12, 2025 23:56

driusan pushed a commit that referenced this pull request Feb 20, 2025

[server process monitor] minor doc comment changes (#9457)

108491c

Minor documentation changes of the server process monitor that were present in #9154 and extracted into a new PR.

ridz1208 requested review from driusan and removed request for ridz1208 May 13, 2025 15:45

Merge branch 'main' into api-dicom-upload

abcb31b

github-actions bot added Language: SQL PR or issue that update SQL code Language: PHP PR or issue that update PHP code Module: api PR or issue related to api module Module: server_processes_manager PR or issue related to the server_processes_manger labels May 13, 2025

driusan merged commit 0883fde into aces:main May 23, 2025
20 checks passed

gdevenyi mentioned this pull request May 23, 2025

REST or other such API for data upload? #3022

Closed

CamilleBeau pushed a commit to CamilleBeau/Loris that referenced this pull request Jun 10, 2025

[API] Add dicom upload (aces#9154)

dad7b72

This is a port of [aces#6161](aces#6161), which contains a description of the changes. It adds endpoints to the LORIS API for uploading dicom archives

SKADE2303 pushed a commit to SKADE2303/Loris that referenced this pull request Jun 26, 2025

[API] Add dicom upload (aces#9154)

575f643

This is a port of [aces#6161](aces#6161), which contains a description of the changes. It adds endpoints to the LORIS API for uploading dicom archives

[API] Add dicom upload #9154

[API] Add dicom upload #9154

Uh oh!

Conversation

maximemulder commented Mar 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Brief summary of changes

Testing instructions (if applicable)

Notes for existing projects

Uh oh!

maximemulder commented Mar 20, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kongtiaowang commented Apr 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maximemulder commented Apr 24, 2024

Uh oh!

maximemulder commented May 22, 2024

Uh oh!

laemtl commented May 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maximemulder commented May 28, 2024

Uh oh!

maximemulder commented Aug 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cmadjar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cmadjar Sep 3, 2024

Choose a reason for hiding this comment

Uh oh!

maximemulder Sep 17, 2024

Choose a reason for hiding this comment

Uh oh!

cmadjar Sep 30, 2024

Choose a reason for hiding this comment

Uh oh!

driusan Nov 7, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

driusan commented Sep 4, 2024

Uh oh!

maximemulder commented Sep 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maximemulder Sep 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

driusan Nov 7, 2024

Choose a reason for hiding this comment

Uh oh!

cmadjar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

maximemulder commented Feb 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ridz1208 commented May 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

maximemulder commented Mar 20, 2024 •

edited

Loading

maximemulder commented Mar 20, 2024 •

edited

Loading

kongtiaowang commented Apr 24, 2024 •

edited

Loading

laemtl commented May 28, 2024 •

edited

Loading

maximemulder commented Aug 29, 2024 •

edited

Loading

maximemulder commented Sep 17, 2024 •

edited

Loading

maximemulder Sep 18, 2024 •

edited

Loading

maximemulder commented Feb 12, 2025 •

edited

Loading