A few high level comments:

• Title and description are unclear, is it about "list of lists" specifically, something a little broader (like tuple of lists/tuples too), or any kind of sequence?
• Your current text says you want to raise an error for explicit dtype=object as well in some cases. This seems odd, explicit dtype=object should preserve current behavior I'd think.
• Related to the previous point: np.ragged_array_object is not a good idea, just make dtype=object not raise.
• Please add some rationale for why to special-case this type of input, rather than making all object array creation raise unless it's explicitly requested.

np.ragged_array_object is not a good idea, just make dtype=object not raise.

I believe it is a good idea, because it would have a non-optional array_depth argument. This would solve the following problem:

a = np.ragged_array_object([[1, 2], [3, 4]], array_depth=1)
b = np.ragged_array_object([[1, 2], [3, 4, 5]], array_depth=1)
assert a.ndim == b.ndim == 1

Today there is no easy way to construct a.

The name is awful, but I think to be useful at all, a depth argument is needed.

This would solve the following problem:

Try to look at this from an end user perspective. Why is this a problem worth solving? Why would you want such behavior? In every case I can think of, there are much better alternatives.

Another way to say this: if you would design such a feature without knowing about the current issue that gave rise to this NEP, would you ever propose np.ragged_array_object for inclusion in NumPy?

Why would you want such behavior?

We're all agreed I think that no one wants the behavior of np.array([[1, 2], [1]])

My claim is that additionally no one wants the behavior of np.array([[1, 2], [1]], dtype=object). To illustrate that, consider:

student_lists = np.array([class1.students, class2.students], dtype=object)
assert student_lists.ndim == 1

This code works just fine unless by some coincidence the two classes have the same number of students. Because of this, I strongly think np.array([[1, 2], [1]], dtype=object) should also be an error.

What remains is the question "what is the correct way to write the above?". Today, the best I can come up with is the wasteful:

def ragged_array_object(seq, depth):
    arr = np.array(seq, dtype=object)  # this relies on the broken behavior!
    assert arr.ndim >= depth
    arr2 = np.empty(arr.shape[:depth], dtype=object)
    arr2[...] = arr
    return arr2

Perhaps forcing the user to do this workaround is ok (edit: forcing the user to use a workaround also breaks this workaround), but removing np.array([[1, 2], [1]], dtype=object) would be a lot more palatable if we provided an easy replacement.

would you ever propose np.ragged_array_object for inclusion in NumPy?

Something similar to it, yes - I've repeatedly found myself wanting np.ragged_array_object(some_obj, depth=0).

We're all agreed I think that no one wants the behavior of np.array([[1, 2], [1]])

yes, I think so too

My claim is that additionally no one wants the behavior of np.array([[1, 2], [1]], dtype=object).

Perhaps, but not because of some minor inconsistency. more because of object arrays being very hard to work with anyway, and hence not being very useful.

What remains is the question "what is the correct way to write the above?".

That is not what remains, that's what I'm trying to make clear with my questions above. What remains is: are you (the user) really trying to work with ragged arrays here? If so, how good do you want that experience to be? NumPy is limited and is unlikely to ever get nice, well-thought-out support for ragged arrays. NumPy can try to make it less likely for users that don't want ragged arrays to not shoot themselves in the foot (hence, raising an error by default). For users that do want ragged arrays, they're probably better off using Arrow (I think, not an expert) or XND (which has proper ragged array support), or staying with list of lists.

Perhaps forcing the user to do this workaround is ok, but removing np.array([[1, 2], [1]], dtype=object) would be a lot more palatable if we provided an easy replacement.

I think just documenting some workaround is much better, for the unlikely case where a user really wants these kind of object arrays (she probably doesn't!).

My 2 cents about ragged arrays: I am not sure I like a special function for it, but I think usage out there is large enough that we cannot just ignore it. The long way to write it is:

arr = np.empty(correct_shape, dtype=object)
arr[...] = values

is fairly short, but also not very discoverable, plus you need to know the exact shape instead of just the number of dimensions which could be enough.

The next way would be to add a specific ndim=3 kwarg, so that we can stop shape discovery when we hit the correct dimension (we already do this, just the ndim on the python level is always 32).

The last way would require an additional hook for dtypes (which we can always add later if we do new dtypes; and which actually would somewhat make sense also to solve the "tuple is scalar" issue for void coercion). And that is provide a way ask the DType "Is this a scalar or a sequence" during coercion.

We could then write np.array([...], dtype=PyObject[np.ndarray]) (assuming the inside sequences are arrays in this case). I have not looked into that yet, because it seems to me that it is easier to just special case tuples+void right now.

We may have to decide here (or at least mention) if we want to do the same thing for the two other cases which create object arrays possibly surprisingly):

1. Failure to promote: np.array([np.array((1,2), "i,i"), 1]) (note that the conversion to string or string+floats is not as such a failure to promote right now, because we actually define it specifically)

2. The dtype of large python integers can fluctuate randomly (especially on windows).

The 2. part is likely a different issue (or at least is much easier addressed on its own). The first one may have overlap here depending on where we want to fix this. Note that if we are not in a hurry, I am hoping to rewrite the whole thing (I have actually done so, but it needs cleaning up) for new dtypes. Although, that does not matter too much w.r.t. starting this now in the old code already.

Updated to put off discussion of depth until later, reformatted to use a Usage and Impact section.

Copying some valuable comments in case they get lost in the rewrites

... I would suggest to do whatever is easier (I don't know what that is). If just ragged arrays are easier, perhaps give that as a rationale. If we have to jump through extra hoops to do just ragged arrays (IIRC we failed once before?), then why bother (i.e., fail if no dtype is specified, mattip)?

Since you call out ndarray here - will this start failing?

outer = np.array([None, None])
outer[0] = outer[1] = np.array([1, 2, 3])
np.array(outer).shape  # today: (2,)
np.array([outer]).shape  # today: (1, 2,)

outer_ragged = np.array([None, None])
outer_ragged[0] = np.array([1, 2, 3])
outer_ragged[1] = np.array([1, 2, 3, 4])
# will both of these emit warnings?
np.array(outer_ragged).shape  # today: (2,)
np.array([outer_ragged]).shape  # today: (1, 2,)

Examples of things that need decisions but should probably still work:

np.array([[[Decimal(1)]]])

The previous failure was trying to change the error message of array(<lists-of-lists>, dtype=int) to something that indicated ragged arrays.

As for the examples, maybe we should get a PR going, mark it WIP and see how painful this all is and what we can and cannot detect.

xref gh-14794

I think this is ready for the mailing list? From NEP 000

Once the PR is in place, the NEP should be announced on the mailing list for discussion. Discussion about implementation details will take place on the pull request, but once editorial issues are solved, the PR should be merged, even if with draft status. The mailing list e-mail will contain the NEP upto the section titled “Backward compatibility”,

I think this is ready for the mailing list?

agreed, it reads well. a few final comments made

Note that this comment still needs addressing:

Since you call out ndarray here - will this start failing?

My impression is that the answer is no (at least, based on your PR), and that the wording needs tweaking there. Let's add a test to the PR and see what the behavior is?

Let's add a test to the PR and see what the behavior is?

Added here, they all pass with no DeprecationWarning

they all pass with no DeprecationWarning

Right - so we need to change either the implementation or the NEP, because currently they disagree. My feeling would be to just remain silent on np.ndarrays being considered sequence objects in the NEP, which is the easiest path - if nothing else, the case that we're trying to stop users being bitten by is for nested lists of regular python objects, not so much lists of arrays.

Are you referring to the outer_ragged example? I added a footnote hopefully clarifying the algorithm.

There were no repsonses on the mailing list. I assume that means no-one opposes the NEP. Can we merge the draft status?

Yep, merged! Thanks @mattip and @eric-wieser