Just a minor comment @maxulysse regarding the description of the function.

So this wont actually fix the problem I was eluding to i.e. the function itself isnt findable when used in a custom.config with the -c parameter. It looks like an improvement on the code though so be good to add this in 👍

Ohhh I see, sorry, I didn't catch that.
Then yes, that should be included in any custom config using it.
At least, it's smaller than the current check_max().
Or we could improve Nextflow so that one function from one config can be used within a specified config file with -c.

I agree that it's the only issue:
For the moment, our solution in sarek is:
https://github.com/nf-core/sarek/blob/64d9c10bcb1bcee3d2b41198ea8c7e59fe512b7a/conf/base.config#L14

Is that any better than https://github.com/nf-core/rnaseq/blob/37f260d360e59df7166cfd60e2b3c9a3999adf75/conf/base.config#L30 ..? 😉

Not for the memory, but definitively better for time and cpus: https://github.com/nf-core/rnaseq/blob/37f260d360e59df7166cfd60e2b3c9a3999adf75/conf/base.config#L35

But "5h" for time also won't work..? And "4" cpus also..

According to tests we did with @alneberg we only had issues with memory, nothing with time or cpus.
But we should definitively iron that out before 1.8

check_max needs to be re-written anyway and the max_* parameters need to be renamed to something like check_max_* because they are often misunderstood.

If anyone fancies demonstrating that this new function works with command line options like --max_time 4h (or --check_max_time 4h?) then I'm happy 👍

I'm still worried about edge cases though, like people specifying memory in bytes as an integer and other weird stuff. I generally prefer the explicitness of the existing code we have personally.

This is the current Sarek:

https://github.com/nf-core/sarek/blob/77073c50fc36abaa03f01227bc3b9a3146a71a8e/nextflow.config#L195-L209

https://github.com/nf-core/sarek/blob/77073c50fc36abaa03f01227bc3b9a3146a71a8e/conf/base.config#L13-L15

So I'll try Phil suggestion on Sarek to see if it works.

It appears to be working as planned:

nextflow run . -profile docker,test --max_time 10sec

Error executing process > 'Get_software_versions'

Caused by:
  Process exceeded running time limit (10s)

ok nice 👍 Then if there is general agreement that this code is better than what was there before, then I'm happy to be outvoted 😄

Would be good to update the parameter names though 👍 (will need updates in schema and possibly also linting?)

Codecov Report

Merging #408 into dev will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##              dev     #408   +/-   ##
=======================================
  Coverage   68.14%   68.14%           
=======================================
  Files          11       11           
  Lines        1987     1987           
=======================================
  Hits         1354     1354           
  Misses        633      633           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b3630b6...595e5df. Read the comment docs.

@drpatelh so you want to change the params name in a separate PR?

I am not entirely sure about check_resource as a name 😏 Maybe set_max_resource would be better because we are setting the max resources? Dont know, but would be good if the parameter names we choose are the same as this function somehow? e.g. set_max_cpu? Suggestions?

I agree for set_max_resource. It sounds like a good name.
And then set_max_cpus, set_max_memory and set_max_time are good ideas

But then, it probably makes sense to do all that in just this one PR...

not sure about set_max_resource actually.
This function is not setting the max_resource, but actually setting the minimum between the set_max_resource and the requested resource.
Maybe just set_resource then?

Ok, so this issue has been rumbling on for years now. I'm going to try to summarise the decisions so that we can wrap it up.. Please correct me on any of this @maxulysse

As I see it, we have discussed three distinct things:

• Refactoring the thresholding function code
• Renaming the thresholding function code
• Renaming the --max_cpus etc params (can be a separate PR, kind of unrelated)

The two renaming points are fairly minor and I don't think that there is any controversy there really. The first point is what this PR is really about and the reason it hasn't been merged yet.

As I understand it, the core change in the refactoring is that the new function code guesses the resource type instead of requiring it to be passed as a parameter. This means that we can change the way that the function is called:

- time = { check_max( 5.h * task.attempt, 'time' ) }
+ time = { check_max( 5.h * task.attempt ) }

My main concern with this at the beginning was that the process of guessing might be difficult, leading to loss of flexibility when providing command line parameters. However @maxulysse has done a good job of testing a bunch of these and showing that they still work.

I have an outstanding concern that there will be edge cases where people could provide memory or time without units (bytes, seconds) and that will be interpreted as cpus. But it's an edge case at best and probably unlikely to happen much.

My only remaining problem with this change is that it requires updating all configs in all pipelines for what I view as a very minor improvement in code succinctness. Basically, we gain a few keyboard characters when writing configs, but we force all pipeline developers to manually edit all of their configs as well as incurring extra sync overhead. For me, the benefits do not outweigh the costs.

However, the change is not big, and most of my initial fears were unfounded. @maxulysse seems super keen on this change, so I'm happy to be overruled and accept this change (hence my positive review above). I am aware that I can be annoyingly stubborn on minor things like this, and I'm quite happy to let it go - @maxulysse has earned this through matching my stubbornness 😅

Hopefully this is a fair summary of where we are. I will leave it to everyone else to decide how to continue.

Phil

I think it's a good summary.

I think it's a good opportunity to change the function name as it was a bad description of what it was actually doing anyway.

Don't really mind about the params, and I do think it's a big change, which can break things up.

So can keep that part in another PR.

Maybe keep both functions check_max() and set_resource() for now, that way nothing is break, but add a comment saying that it's deprecated, and remove it on the following tools release?

As I see it, we have discussed three distinct things:

* Refactoring the thresholding function code

* Renaming the thresholding function code

* Renaming the `--max_cpus` etc params (can be a separate PR, kind of unrelated)

The two renaming points are fairly minor and I don't think that there is any controversy there really. The first point is what this PR is really about and the reason it hasn't been merged yet.

Entirely agree on this - renaming these is something different and I think its also helpful if then we have less users misinterpreting the parameter(s) involved across all pipelines.

I have an outstanding concern that there will be edge cases where people could provide memory or time without units (bytes, seconds) and that will be interpreted as cpus. But it's an edge case at best and probably unlikely to happen much.

Same here. I just feel a bit bad about a function that "hides" what it is changing, instead of making it explicitly clear what is changed/modified by the code here. It might be totally unjustified, but saving 9 letters once per pipeline doesn't justify changing the method across all pipelines in my opinion. This has also the potential for side-effects that we currently don't see/estimate/expect, but might happen. Good example for something like this was that config here: nf-core/configs#162, which worked fine with the exception of the case that Phil discovered. Could be similar here and the current code works a.) fine and b.) is very well tested in reality ;-)

OK, so for the 3 points:

Refactoring the thresholding function code
Renaming the thresholding function code

As the first one involve removing one argument to the function, I think it's a good opportunity to rename it to an actually more meaningful name.
Also, I think that that way, people making the switch will actually change function and argument.

Renaming the --max_cpus etc params (can be a separate PR, kind of unrelated)
Agreed that it's unrelated, or even unnecessary actually.

I hadn't thought about the fact that renaming the check_max() function even if its unedited will also mean that every config file will also have to be updated 🤦

Ok then my vote would be to just change the --max_cpu etc parameter names and be done with it..

I'm not super keen on having two functions side by side, I think we should decide which style we want and be done with it.

I'm not sure about changing the params name.
Or actually, I can improve the function so that they do correspond finally to what they actually mean.

For example if I set up --max_cpus 2, then maximum 2 cpus are to be used.
Same for memory and time.

What do you think?

I'm not sure about changing the params name.
Or actually, I can improve the function so that they do correspond finally to what they actually mean.

For example if I set up --max_cpus 2, then maximum 2 cpus are to be used.
Same for memory and time.

What do you think?

Forget I said anything, this is what the function is actually doing.
🤦
So for me, params have the right names.

if I do --max_cpus 2, I'm sure that the maximum amount of cpus used is 2.
Makes sense, this is what we have for --max_multiqc_email_size

I'm giving up on that PR ;-)