Thanks to visit codestin.com
Credit goes to github.com

Skip to content

all(range()...)) is needlessley slow #81312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Topper-123 mannequin opened this issue Jun 2, 2019 · 6 comments
Closed

all(range()...)) is needlessley slow #81312

Topper-123 mannequin opened this issue Jun 2, 2019 · 6 comments
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage

Comments

@Topper-123
Copy link
Mannequin

Topper-123 mannequin commented Jun 2, 2019

BPO 37131
Nosy @stevendaprano, @serhiy-storchaka, @topper-123

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-06-02.11:59:57.213>
created_at = <Date 2019-06-02.09:39:48.990>
labels = ['interpreter-core', '3.7', 'invalid', 'performance']
title = 'all(range()...)) is needlessley slow'
updated_at = <Date 2019-06-02.11:59:57.212>
user = 'https://github.com/Topper-123'

bugs.python.org fields:

activity = <Date 2019-06-02.11:59:57.212>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2019-06-02.11:59:57.213>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2019-06-02.09:39:48.990>
creator = 'Petersen'
dependencies = []
files = []
hgrepos = []
issue_num = 37131
keywords = []
message_count = 6.0
messages = ['344263', '344264', '344265', '344267', '344268', '344282']
nosy_count = 3.0
nosy_names = ['steven.daprano', 'serhiy.storchaka', 'Petersen']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue37131'
versions = ['Python 3.7']

@Topper-123
Copy link
Mannequin Author

Topper-123 mannequin commented Jun 2, 2019

Checking if a range's items are ll truthy can be done y checking if 0 the range contains 0, however currently Python iterates over the range, making the operation slower than needed.

    >>> rng = range(1, 1_000_000)
    >>> timeit all(rng)
    19.9 ms ± 599 µs per loop

If the all function could special case range, this could be like made fast like this:

    if isinstance(obj, range):
        if 0 in obj:
            return False
        return True

@Topper-123 Topper-123 mannequin added 3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage labels Jun 2, 2019
@stevendaprano
Copy link
Member

Why should all() special case range? Apart from showing off benchmarks, what's the point?

Should any() also special case it? How about other lazy sequences and computed iterables, such as itertools.count objects, itertools.cycle objects, itertools.repeat objects, etc? How many special cases do we want?

Personally, I don't *necessarily* oppose special-casing types, but it breaks the independence of types, and couples the all() function and the range() function. To be worth doing, we should get some practical benefit out of it. It isn't clear what that benefit is.

Don't get me wrong, its not like I *want* all(range(1, 10**10)) to be slow. But nor do I want to couple the all() function to the range function and complicate the implementation if there is no corresponding benefit. After all, testing with all() is fundamentally an O(N) worst case operation so anything better than that is a nice surprise.

If there were special dunders __all__ and __any__ it would be easy to encapsulate this behaviour inside the range objects themselves, and neither any() nor all() would need to know anything about range objects.

@Topper-123
Copy link
Mannequin Author

Topper-123 mannequin commented Jun 2, 2019

Why should all() special case range? Apart from showing off benchmarks, what's the point?

Mostly to avoid silly mistakes, and the overhead of doing it would be very small, so a very small trade-off.

Should any() also special case it?

No, any()takes at most two iterations to check truthiness in a range(), so that's wouldn't be needed.

> How about other lazy sequences and computed iterables, such as itertools.count objects, itertools.cycle objects, itertools.repeat objects, etc? How many special cases do we want?

No, I wouldn't propose that. The argument would be that those are iterators/generators while range() is special-cased built-in sequence with known properties.

@Topper-123
Copy link
Mannequin Author

Topper-123 mannequin commented Jun 2, 2019

> If there were special dunders __all__ and __any__ it would be easy to encapsulate this behaviour inside the range objects themselves, and neither any() nor all() would need to know anything about range objects.

This sounds very interesting, and more general. It would be useful for e.g. numpy arrays, where all(arr) currently is slower than arr.all(). Don't know it there's a reason it wasn't implemented this way originally?

@stevendaprano
Copy link
Member

Why should all() special case range? Apart from showing off benchmarks, what's the point?

Mostly to avoid silly mistakes

What sort of silly mistakes?

and the overhead of doing it would be very small, so a very small trade-off.

Its not just the overhead, its the coupling between two chunks of code
which should be independent.

@serhiy-storchaka
Copy link
Member

You can implement it yourself.

_all = all
def all(obj):
    if isinstance(obj, range):
        return 0 not in obj
    return _all(obj)

I do not see reasons to do this in the stdlib.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.7 (EOL) end of life interpreter-core (Objects, Python, Grammar, and Parser dirs) performance Performance or resource usage
Projects
None yet
Development

No branches or pull requests

2 participants