Codestin Search App

Ryan Waskewich (rwask) · 2026-02-10T18:55:18Z

fix(file-based): override primary_key in PermissionsFileBasedStream to avoid invalid parser-defined key

Summary

When permissions transfer mode is enabled on file-based connectors (e.g. SharePoint Enterprise), the PermissionsFileBasedStream uses a completely different schema (with fields like id, file_path, publicly_accessible, allowed_identity_remote_ids) than the standard content schema. However, it inherited DefaultFileBasedStream.primary_key, which falls back to the parser-defined primary key — "document_key" for UnstructuredParser. Since document_key doesn't exist in the permissions schema, the destination rejects the catalog:

ConfigErrorException: A primary key column does not exist in the schema: document_key

Changes:

PermissionsFileBasedStream: Overrides primary_key to return only self.config.primary_key (user-configured PK or None), skipping the parser-defined fallback that returns a key belonging to the content schema.
FileBasedStreamFacade: Delegates primary_key to the underlying legacy stream instead of reimplementing the fallback logic. This ensures permissions streams wrapped in the concurrent facade also get the correct behavior. For DefaultFileBasedStream, this is functionally equivalent to the previous logic.

Review & Testing Checklist for Human

Verify behavioral equivalence of the facade change for non-permissions streams: FileBasedStreamFacade.primary_key previously did config.primary_key or parser.get_parser_defined_primary_key() inline; now it delegates to self._legacy_stream.primary_key. Confirm that DefaultFileBasedStream.primary_key has identical logic (it does at time of writing — line 104-108), and that no other subclasses override primary_key in a way that would change behavior unexpectedly through the facade.
Verify None primary key doesn't break downstream: When no user-configured PK exists, PermissionsFileBasedStream.primary_key now returns None. Confirm this is handled correctly in catalog generation (sourceDefinedPrimaryKey becomes empty) and that destinations handle the absence of a source-defined PK gracefully (user must configure one for dedup).
Test plan: Ideally test with a file-based connector in permissions mode (e.g. SharePoint Enterprise with permissions enabled) to confirm (a) discovery no longer advertises document_key as PK, and (b) syncs succeed when an appropriate PK from the permissions schema is configured.

Notes

Link to Devin run
Requested by: Ryan Waskewich (@rwask)

Summary by CodeRabbit

Refactor
- Simplified primary key resolution in file-based streams to use a single source of truth, eliminating complex fallback logic.
- Exposed primary key configuration as a public property for improved accessibility and consistency across file-based stream implementations.

…o avoid invalid parser-defined key Co-Authored-By: Ryan Waskewich <[email protected]>

devin-ai-integration · 2026-02-10T18:55:23Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment and CI monitoring

github-actions · 2026-02-10T18:55:34Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1770749470-fix-permissions-stream-primary-key#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1770749470-fix-permissions-stream-primary-key

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

/autofix - Fixes most formatting and linting issues
/poetry-lock - Updates poetry.lock file
/test - Runs connector tests with the updated CDK
/prerelease - Triggers a prerelease publish with default arguments
/poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
/poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

CDK API Reference

📝 Edit this welcome message.

github-actions · 2026-02-10T19:03:18Z

PyTest Results (Fast)

3 856 tests ±0 3 844 ✅ ±0 6m 46s ⏱️ +6s
1 suites ±0 12 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 315c4d8. ± Comparison against base commit 201bdb8.

devin-ai-integration · 2026-02-10T19:05:21Z

Aldo Gonzalez (@aldogonzalez8) Maxime Carbonneau-Leclerc (@maxi297) — Requesting your review on this fix. You both authored the permissions stream code that's affected here.

Summary: PermissionsFileBasedStream inherits DefaultFileBasedStream.primary_key, which falls back to the parser-defined PK (document_key from UnstructuredParser). But the permissions schema doesn't contain document_key, causing destinations to reject the catalog with ConfigErrorException: A primary key column does not exist in the schema: document_key.

The fix overrides primary_key in PermissionsFileBasedStream to return only self.config.primary_key (no parser fallback), and updates FileBasedStreamFacade to delegate to the underlying stream's primary_key property.

cc Ryan Waskewich (@rwask) who reported this issue.

github-actions · 2026-02-10T19:07:20Z

PyTest Results (Full)

3 859 tests ±0 3 847 ✅ ±0 10m 54s ⏱️ -1s
1 suites ±0 12 💤 ±0
1 files ±0 0 ❌ ±0

Results for commit 315c4d8. ± Comparison against base commit 201bdb8.

airbyte_cdk/sources/file_based/stream/concurrent/adapters.py

Aldo Gonzalez (aldogonzalez8)

APPROVED

coderabbitai · 2026-02-11T16:39:43Z

📝 Walkthrough

Walkthrough

Two file-based stream classes are updated to improve primary_key handling: one simplifies resolution logic to use the legacy stream's primary_key directly, eliminating fallback logic; the other exposes configuration's primary_key as a public property with proper type annotation.

Changes

Cohort / File(s)	Summary
Primary Key Property Updates `airbyte_cdk/sources/file_based/stream/concurrent/adapters.py`, `airbyte_cdk/sources/file_based/stream/permissions_file_based_stream.py`	Refactored primary_key resolution in FileBasedStreamFacade to return legacy stream's value directly instead of combining config and parser-defined keys. Added public primary_key property to PermissionsFileBasedStream with PrimaryKeyType annotation exposing config's primary_key value.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: adding a primary_key override in PermissionsFileBasedStream to prevent invalid parser-defined keys from being used.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch devin/1770749470-fix-permissions-stream-primary-key

No actionable comments were generated in the recent review. 🎉

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

fix(file-based): override primary_key in PermissionsFileBasedStream t…

315c4d8

…o avoid invalid parser-defined key Co-Authored-By: Ryan Waskewich <[email protected]>

devin-ai-integration bot assigned Ryan Waskewich (rwask) Feb 10, 2026

devin-ai-integration bot requested a review from Aldo Gonzalez (aldogonzalez8) February 10, 2026 19:04

Ryan Waskewich (rwask) added the airbyte-python-cdk label Feb 10, 2026

Aldo Gonzalez (aldogonzalez8) reviewed Feb 10, 2026

View reviewed changes

airbyte_cdk/sources/file_based/stream/concurrent/adapters.py Show resolved Hide resolved

Aldo Gonzalez (aldogonzalez8) approved these changes Feb 11, 2026

View reviewed changes

coderabbitai bot approved these changes Feb 11, 2026

View reviewed changes

Ryan Waskewich (rwask) marked this pull request as ready for review February 11, 2026 16:36

Ryan Waskewich (rwask) merged commit 796bb34 into main Feb 11, 2026
31 of 32 checks passed

Ryan Waskewich (rwask) deleted the devin/1770749470-fix-permissions-stream-primary-key branch February 11, 2026 16:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(file-based): override primary_key in PermissionsFileBasedStream to avoid invalid parser-defined key#903

fix(file-based): override primary_key in PermissionsFileBasedStream to avoid invalid parser-defined key#903
Ryan Waskewich (rwask) merged 1 commit intomainfrom
devin/1770749470-fix-permissions-stream-primary-key

Ryan Waskewich (rwask) commented Feb 10, 2026 •

edited by coderabbitai bot

Loading

Uh oh!

devin-ai-integration bot commented Feb 10, 2026

Uh oh!

github-actions bot commented Feb 10, 2026

Testing This CDK Version

PR Slash Commands

Helpful Resources

Uh oh!

github-actions bot commented Feb 10, 2026

Uh oh!

devin-ai-integration bot commented Feb 10, 2026

Uh oh!

github-actions bot commented Feb 10, 2026

Uh oh!

Uh oh!

Aldo Gonzalez (aldogonzalez8) left a comment

Uh oh!

coderabbitai bot commented Feb 11, 2026

Walkthrough

Changes

Estimated code review effort

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Ryan Waskewich (rwask) commented Feb 10, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

fix(file-based): override primary_key in PermissionsFileBasedStream to avoid invalid parser-defined key

Summary

Review & Testing Checklist for Human

Notes

Summary by CodeRabbit

Uh oh!

devin-ai-integration bot commented Feb 10, 2026

🤖 Devin AI Engineer

Uh oh!

github-actions bot commented Feb 10, 2026

👋 Greetings, Airbyte Team Member!

Testing This CDK Version

PR Slash Commands

Helpful Resources

Uh oh!

github-actions bot commented Feb 10, 2026

PyTest Results (Fast)

Uh oh!

devin-ai-integration bot commented Feb 10, 2026

Uh oh!

github-actions bot commented Feb 10, 2026

PyTest Results (Full)

Uh oh!

Uh oh!

Aldo Gonzalez (aldogonzalez8) left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot commented Feb 11, 2026

Walkthrough

Changes

Estimated code review effort

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Ryan Waskewich (rwask) commented Feb 10, 2026 •

edited by coderabbitai bot

Loading