Set Table primary key when copied indices are attached - #20362
Open
taldcroft wants to merge 1 commit into
Open
Conversation
A table built from columns that already carry an index, such as a column-name slice or Table([indexed_col]), kept the index but had no primary key, so loc/iloc failed with a TypeError. Make the first copied index primary in _init_from_cols, and keep the parent's primary key on a column slice when that index survived. Co-Authored-By: Claude Fable 5.1 <[email protected]>
Contributor
|
Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.
|
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A
Tablebuilt from columns that already carry an index, most commonly a column-name slice liket[["a", "b"]], keeps the index but has no primary key. Every consumer ofprimary_keythen fails on it:.loc,.iloc,.loc_indices,repr(t.loc), andwrite(..., write_indices=True). The slice looks fully indexed (t2.indicesis populated), so the failure is surprising and the error message points nowhere useful.The cause is that
primary_keyis set only byadd_index(), whileTable.__init__attaches copied indices to the new columns without ever setting it. This PR sets the primary key where those copied indices are attached, following the same "first index is primary" rule asadd_index(), and additionally keeps the parent's primary key on a column slice when that index survived the slice. Row slices (t[1:],t[mask]) were never affected, since_new_from_slicealready copiesprimary_key.This is the
astropy.tablehalf of the fix for #20297; theastropy.timeserieshalf is #20361. The two are independent and either one alone fixes theaggregate_downsampleexample in that issue.Related to #20297. Supersedes the
astropy.tablepart of #20354.AI disclosure
This PR includes AI-generated content using Claude Fable 5.1. I have carefully reviewed the content and fully understand the changes in
astropy/table/table.pyandastropy/table/tests/test_index.py.Details
Click to expand
Set
primary_keywhenever a newTablereceives copied indices1. How a table gets indices without a primary key
col_copy(col, copy_indices=True)deep-copies a column'sSlicedIndexobjects and re-points them at the new column, soTable([t["a"]])andt[["a", "b"]](which goes through the same constructor) produce columns with live, valid indices.Table._init_from_colsalready walks these to deduplicate index objects shared between columns of a multi-column index, but nothing setsself.primary_key, whichTable.__init__initialised toNone. Onlyadd_index()sets it, and only when the table has no indices yet.TableLoc._get_index_id_and_itemusesself.table.primary_keyas the index id, soNonereachesTableIndices.__getitem__, which falls through tolist.__getitem__(None).2. Implementation
Table._init_from_cols, after the existing deduplication loop that fillsindex_dict(keyed by index id in column order):The first index in column order becomes primary, matching
add_index(). This coversTable([indexed_col]),QTable(table_with_indices), and any other constructor path that copies indices. When the input is aTable,__init__has already copieddata.primary_key, so this only fills in a missing key and never overrides one.Table.__getitem__, in the list-of-column-names branch:A reordered slice such as
t[["b", "a"]]on a table with indices on both columns would otherwise get("b",)from the rule above; this keeps the parent's("a",). The membership test against the slice's actual indices means a stale key (one whose index was dropped by the slice, or one left behind byremove_indices, see below) is never propagated.None in [...]isFalse, so an unindexed parent is unaffected.3. Things deliberately left alone
t.add_index(["a", "b"]); t[["a", "c"]]) now becomes the primary key..loc[1]on it raisesIndexError: tuple index out of range, but the unslicedt.loc[1]raises the identical error for a composite index, so this is consistent rather than a regression. Filtering such indices out would be a separate policy change.Table.remove_indicespromises in its docstring to reassign the primary key when the primary index is removed but never does, leaving a stale key. That is pre-existing and not touched here; the__getitem__guard above just avoids spreading it.Table(t, copy_indices=False)already gives a table with aprimary_keybut no indices. Unchanged.4. Tests
test_index.py::test_primary_key_from_copied_indices(fails onmain, passes here) checks:t[["a", "b"]]has one index andprimary_key == ("a",)with working.locand.iloc;t[["b"]]has no index and no key;Table([t["a"]])has a working.locandrepr(t4.loc)no longer raises; with a second index onb,t[["b", "a"]]keeps("a",),t[["b"]]gets("b",), andTable([t["b"], t["a"]])gets("b",).python -m pytest astropy/table/tests/test_index.py astropy/table/tests/test_df.py docs/table/indexing.rston this branch:710 passed, 56 skipped. The fullastropy/table,astropy/io/miscand ECSV suites were run with this change applied together with #20361 (6949 passed, 89 skipped, 23 xfailed), not separately.pre-commit run --fileson all changed files passes.5. Backwards compatibility
No API change, but one visible behavior change:
to_pandas()(andto_df()) use a single-column primary key as theDataFrameindex by default, sot[["a", "b"]].to_pandas()now returns a frame indexed byawith onlybas a column, where onmainastayed a regular column with aRangeIndex. That is what the unslicedt.to_pandas()already did, andto_pandas(index=False)restores the old output. Tables without indices, row slices, and tables built throughadd_index()are bit-for-bit unchanged.Changelog fragment:
docs/changes/table/20362.bugfix.rst.🤖 Generated with Claude Code