This seems a tricky issue to solve, but yes we should try to do so while improving pandas interoperability

This snippet gets the job done:

categoricalOrderedFeatures={
    'Alley': ['_UNAVAILABLE','Grvl', 'Pave'],
    'ExterCond': ['Po','Fa','TA','Gd','Ex'],
    'LotShape': ['IR3','IR2','IR1','Reg']
}
...

# Incrementally add encoded ordered categorical features
for feature in categoricalOrderedFeatures.keys():
    X[feature]=-1 # initialize target feature
    i=0
    for category in houses[feature].unique().categories:
        
        # Get row indexes for this category from source DataFrame
        indexes = houses.index[houses[feature] == category]
        
        # Imputation
        X.loc[indexes,feature] = i
        
        # move along
        i=i+1

I was not aware of OrdinalEncoder(), I'll have to try it.

But my point with this bug report is that LabelEncoder or even OrdinalEncoder should respect order of CategoricalDtype(order=True) features automatically because order is important. And not just increment integers as data is processed.

Again, note how order semantics is embedded in data of CategoricalDtype(order=True):

print(houses['quality'].unique())
[poor, fair, typical, good, excellent]
Categories (4, object): [poor < fair < typical < good < excellent]

About the snippet, sorry for the concise code. It takes the row indexes from the source DF (houses) and use it to know where to write the correct integer (i) in the target DF (X).

So what people used LabelEncoder for (to encode features) in the past, we have now added OrdinalEncoder (or OneHotEncoder if the final goal was to have dummy variables). And, in contrast to LabelEncoder, OrdinalEncoder has a categories keyword that at least allows a user to solve this manually:

OrdinalEncoder(categories=[houses['quality'].cat.categories])

That said, I agree that scikit-learn should ideally handle pandas categorical dtypes automatically (making use of the categorical dtype information, now we simply discard that and convert to object dtype to then factorize the values again).

I think we decided to leave that for later, and I think it is also too late for 0.20 to get that in. But the annoying thing is that, if we add this properly, this will actually change the behaviour in a future release in case you are already using categorical dtypes. And I also don't see an easy way to do such a change with a deprecation cycle ..

you would just have a FutureWarning that says "In the future, the order of CategoricalDtype(ordered=True) will be respected. To retain current behaviour please specify the categories for ordered pandas Categoricals".

The main problem would be excess noise.

But you've given your opinion that you don't think we could get this right in a hurry.

Yes, I am indeed worried about excessive noise / annoying warning.
It would be easy to add such a FutureWarning, but that further delays introducing the change without an easy way for the user to already get that behaviour + silence the warning (passing the categories manually like I did in the example above can get quite cumbersome if you have multiple columns, in combination with a ColumnTransformer, ..)

untagging as this is actually kinda complex and tied to many other things

Closing this in favor of #14953, which has some more recent discussion.