Prettier Parallel logs if the number of tasks is known - #1608

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.76%. Comparing base (e02400a) to head (e5033ce).
Report is 46 commits behind head on main.

@@            Coverage Diff             @@
##             main    #1608      +/-   ##
==========================================
+ Coverage   95.24%   95.76%   +0.52%     
==========================================
  Files          45       46       +1     
  Lines        7719     7799      +80     
==========================================
+ Hits         7352     7469     +117     
+ Misses        367      330      -37     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Just one question, why not directly display Done XXX out of YYY when it is known?

Huh that's a good point 😅

Yes I agree that using a list is not the pattern we want to promote.

But there are some instances where we can implement both lazy inputs and a len, so why not improve the reporting for those?
For instance, using tqdm to wrap an iterator gives:

from tqdm import tqdm
it = tqdm(range(100), total=100)
len(it)  # 100

while not materializing the tasks. The same is possible with torch.Dataset objects.
So I am in favor of making this small change to improve the output, while still not advocating for using list of tasks.

A last nitpick. Also, if you could add a test and an entry in the change log , that would be great. Once again, thanks for the contrib!

Where should I add an entry in the changelog?

A last nitpick. Also, if you could add a test and an entry in the change log , that would be great. Once again, thanks for the contrib!

And I'm not sure how integrate the test in testing.py correctly. For the test logic itself, I'm thinking of something like

vert_pos = {}
for line in PROCESS_OUTPUT_LINES:
    if "|" in line:
        vert_pos.add(line.find("|"))
assert len(vert_pos) in [0, 1]

to make sure the | always appears at the same position.

• For the changelog: https://github.com/joblib/joblib/blob/main/CHANGES.rst
• We are relying on pytest for the tests, so adding a function with a name test_* in the joblib/test/test_parallel.py file will add a new test. To check the displayed text after a call to Parallel, you can add capsys as an argument to the new test entry and take inspiration from this test.

Note that if you are not familiar with pytest, I can add a test by myself so feel free to let me know.

A last nitpick. Also, if you could add a test and an entry in the change log , that would be great. Once again, thanks for the contrib!

• Changelog -> acb2d27
• Test -> 6e32193