Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Move restoration and color data to the combined data directory #3324

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from

Conversation

hmaarrfk
Copy link
Member

@hmaarrfk hmaarrfk commented Aug 5, 2018

It would make it easier to think about what we want to do with all this data:

#3323

(I'm not sure if this will pass on python 3.5 since I used pathlib)

Copy link
Member

@jni jni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! My experience with Pathlib is that it's great until it breaks (some function, even within the standard library, accepts a Path but really they mean a string and they do stringy things to it), but since it works fine here with np.load, this LGTM. Thanks!

@hmaarrfk
Copy link
Member Author

hmaarrfk commented Aug 5, 2018

@jni, so it didn't work right, not with the min requirements. I'll just add a "'string" to the front of the path to cast it as one.

@codecov-io
Copy link

codecov-io commented Aug 5, 2018

Codecov Report

Merging #3324 into master will increase coverage by 0.02%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #3324      +/-   ##
=========================================
+ Coverage   86.77%   86.8%   +0.02%     
=========================================
  Files         339     339              
  Lines       27380   27386       +6     
=========================================
+ Hits        23760   23773      +13     
+ Misses       3620    3613       -7
Impacted Files Coverage Δ
skimage/color/tests/test_colorconv.py 100% <ø> (ø) ⬆️
skimage/restoration/tests/test_restoration.py 96.66% <100%> (ø) ⬆️
skimage/color/tests/test_delta_e.py 100% <100%> (ø) ⬆️
skimage/draw/_random_shapes.py 95.78% <0%> (-2.11%) ⬇️
skimage/filters/thresholding.py 98.14% <0%> (-0.46%) ⬇️
skimage/filters/tests/test_thresholding.py 100% <0%> (ø) ⬆️
skimage/future/graph/rag.py 96.07% <0%> (ø) ⬆️
skimage/segmentation/_felzenszwalb.py 100% <0%> (ø) ⬆️
skimage/restoration/tests/test_denoise.py 99.47% <0%> (ø) ⬆️
skimage/transform/_warps.py 99.45% <0%> (ø) ⬆️
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02c6eb9...f8dd851. Read the comment docs.

@jni
Copy link
Member

jni commented Aug 5, 2018

Oooooh, older NumPy has the same problem I mentioned above. Of course. =)

goddammit

@hmaarrfk hmaarrfk force-pushed the move_restoration_data branch from 944818d to 744af7e Compare August 5, 2018 23:49
@hmaarrfk
Copy link
Member Author

hmaarrfk commented Aug 5, 2018

I feel like this whole string -> path is a going to be a repeat of bytearray -> string
But hopefully it won't create a new version of python :D

@@ -1,15 +1,22 @@
from os.path import abspath, dirname, join as pjoin
from pathlib import Path
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've probably missed a trend... Is os.path.join too mainstream now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I decided I want to backport this, and it might be better to just use os.path.join for an other year.

@soupault soupault added this to the 0.14.1 milestone Aug 8, 2018
@soupault soupault added 🔧 type: Maintenance Refactoring and maintenance of internals status: mrg+1 labels Aug 8, 2018
@hmaarrfk hmaarrfk force-pushed the move_restoration_data branch from 744af7e to c6fa52b Compare August 8, 2018 15:31
@soupault
Copy link
Member

soupault commented Aug 8, 2018

There is also some data in skimage/color/tests/data/. Let me know if you would like to handle those files in this PR or not.

@hmaarrfk hmaarrfk changed the title Move restoration data to the combined data directory Move restoration and color data to the combined data directory Aug 8, 2018
@hmaarrfk
Copy link
Member Author

hmaarrfk commented Aug 8, 2018

@soupault I moved that data, but it was a really small file. The data in restoration was quite large.

@soupault
Copy link
Member

soupault commented Aug 8, 2018

@hmaarrfk I was mentioning the ones from color/tests/data, not color/tests - https://github.com/scikit-image/scikit-image/tree/master/skimage/color/tests/data 🙂 .

@hmaarrfk
Copy link
Member Author

hmaarrfk commented Aug 8, 2018

🎠

@soupault
Copy link
Member

soupault commented Aug 9, 2018

@hmaarrfk Hmm, sorry, I've just noticed that those latter files are quite small, and we don't introduce too much sense moving them... Although, the consistency is a good thing, in any case. I think, I still prefer the PR in its current state. Thanks!
@jni are you OK with the changes?

@hmaarrfk
Copy link
Member Author

hmaarrfk commented Aug 9, 2018

Thats ok. Depending on @jni's comments, I can revert them quite easily. Undoing the revert would be harder ;)

@jni
Copy link
Member

jni commented Aug 12, 2018

I agree with @soupault, I like the consistency of having all data files under one tree. However I wouldn't mind a third @scikit-image/core chiming in.

@hmaarrfk hmaarrfk force-pushed the move_restoration_data branch from 9e8c0ff to 0ddb13d Compare August 12, 2018 15:47
@sciunto
Copy link
Member

sciunto commented Aug 13, 2018

It's not a bad thing to combine.

I think it is interesting maintain the distinction between the data that can be used for a broad range of things (what the current data is) and the data used for specific tests (that the common users will never care about).

In the taste of keeping things separate a little, wouldn't be interesting to do not put theses data in tests/ (that I associate with the tests for data/) and move them in a directory named like "for_unittests" (or anything better).

@hmaarrfk
Copy link
Member Author

hmaarrfk commented Aug 13, 2018 via email

@sciunto
Copy link
Member

sciunto commented Aug 13, 2018

For my point number 1, yes :) That point was just to underline the importance to keep things tidy. About my point number 2, I would say no. Usually, skimage/SUBMODULE/tests contains the tests of SUBMODULE, not the data for another part. Here, skimade/data/tests contains the tests for our front-end data. Therefore, I suggested to put the files somewhere else, like skimage/data/for_unittests (but the name is not excellent...). Is that clear?

@hmaarrfk
Copy link
Member Author

hmaarrfk commented Aug 13, 2018 via email

@soupault soupault removed this from the 0.14.1 milestone Aug 20, 2018
@soupault soupault added this to the 0.15 milestone Aug 20, 2018
@emmanuelle
Copy link
Member

@hmaarrfk are you considering moving the files to another directory inside data (like suggested by @sciunto ?)? [I'm going through the merge + 1 PRs and had the impression that this one was ready, but the suggestion of @sciunto seems like an improvement]

@hmaarrfk
Copy link
Member Author

hmaarrfk commented Sep 2, 2018

lets see what happens with the cis.

@sciunto
Copy link
Member

sciunto commented Sep 2, 2018

it works :)

@hmaarrfk
Copy link
Member Author

hmaarrfk commented Sep 2, 2018

Can we just leave this as is for a while? Maybe we can think of a better way in a bit, I'm fine with having this one wait until we are ready to pull the trigger on a real solution for splitting off the data package.

I can touchup this PR if it needs to in th near future.

@sciunto
Copy link
Member

sciunto commented May 2, 2020

@hmaarrfk Should we close this PR? Turn it to an issue?

@hmaarrfk
Copy link
Member Author

hmaarrfk commented May 2, 2020

I think it can just be closed. Pooch dealt with this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💪 Work in progress 🔧 type: Maintenance Refactoring and maintenance of internals
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants