Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Model building and change tracking changes to avoid loading vector properties#37829

Open
AndriySvyryd wants to merge 2 commits intomainfrom
Issue37278
Open

Model building and change tracking changes to avoid loading vector properties#37829
AndriySvyryd wants to merge 2 commits intomainfrom
Issue37278

Conversation

@AndriySvyryd
Copy link
Member

@AndriySvyryd AndriySvyryd commented Mar 3, 2026

Fixes #37278
Part of #37277

Adds infrastructure for partial property loading — the ability to mark entity properties as "not auto-loaded" so they can be excluded from queries and skipped during change tracking. This is the foundation for lazy-loading individual properties (e.g., large BLOBs, vectors) without loading the entire entity.

After this Query should provide sentinel values for not loaded properties.

Cosmos support not implemented.

Vector-specific model building API will be added as part of #36350

Model building Fluent API, explicit loading API and query overrides (Include) will be added in #1387

@AndriySvyryd AndriySvyryd requested a review from a team as a code owner March 3, 2026 02:18
Copilot AI review requested due to automatic review settings March 3, 2026 02:18
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds core infrastructure to support partial (non-auto) property loading, allowing properties to be marked as not automatically loaded and then treated as “not loaded” by change tracking/update generation (initially aimed at vector properties).

Changes:

  • Introduces IsAutoLoaded metadata on properties (mutable/convention/runtime) plus conventions/validation for disallowed scenarios (keys/FKs/concurrency/discriminator, JSON-mapped, Cosmos).
  • Adds change-tracking/update support for “not loaded” scalar properties (PropertyEntry.IsLoaded, IUpdateEntry.IsLoaded, new internal state flag) and adjusts update SQL generation to skip unloaded properties.
  • Adds SQL Server convention to mark vector properties as not auto-loaded by default, plus extensive test/baseline updates.

Reviewed changes

Copilot reviewed 59 out of 62 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
test/EFCore.Tests/Metadata/Internal/PropertyTest.cs Adds unit tests for Property.IsAutoLoaded default and toggling.
test/EFCore.Tests/Metadata/Internal/InternalPropertyBuilderTest.cs Tests configuration-source precedence for IsAutoLoaded.
test/EFCore.Tests/Metadata/Conventions/ConventionDispatcherTest.cs Verifies new OnPropertyAutoLoadChanged convention dispatch behavior.
test/EFCore.Tests/Infrastructure/ModelValidatorTest.cs Adds validation tests for disallowed IsAutoLoaded = false scenarios (key/FK/etc.).
test/EFCore.Tests/ExceptionTest.cs Updates fake IUpdateEntry implementation for new interface member.
test/EFCore.Tests/ChangeTracking/PropertyEntryTest.cs Adds change-tracking tests around unloaded properties and sentinels.
test/EFCore.Tests/ChangeTracking/Internal/StateDataTest.cs Extends internal state-flag manipulation tests for new property flag.
test/EFCore.Sqlite.FunctionalTests/Scaffolding/Baselines/No_NativeAOT/ManyTypesEntityType.cs Updates compiled-model baseline to emit autoLoaded: false.
test/EFCore.Sqlite.FunctionalTests/Scaffolding/Baselines/BigModel_with_JSON_columns/ManyTypesEntityType.cs Same baseline update for big model (JSON columns variant).
test/EFCore.Sqlite.FunctionalTests/Scaffolding/Baselines/BigModel/ManyTypesEntityType.cs Same baseline update for big model.
test/EFCore.SqlServer.Tests/Metadata/Conventions/SqlServerAutoLoadConventionTest.cs New tests for SQL Server vector autoload convention behavior.
test/EFCore.SqlServer.Tests/Infrastructure/SqlServerModelValidatorTest.cs Adjusts expected validation error around JSON/vector/autoload interaction.
test/EFCore.SqlServer.FunctionalTests/Update/NonSharedModelUpdatesSqlServerTest.cs Adds provider-specific SQL baselines validating unloaded columns are skipped on UPDATE.
test/EFCore.SqlServer.FunctionalTests/Scaffolding/Baselines/No_NativeAOT/ManyTypesEntityType.cs Updates compiled-model baseline to emit autoLoaded: false.
test/EFCore.SqlServer.FunctionalTests/Scaffolding/Baselines/BigModel_with_JSON_columns/ManyTypesEntityType.cs Same baseline update for big model (JSON columns variant).
test/EFCore.SqlServer.FunctionalTests/Scaffolding/Baselines/BigModel/ManyTypesEntityType.cs Same baseline update for big model.
test/EFCore.Specification.Tests/Scaffolding/CompiledModelTestBase.cs Configures a property as not auto-loaded and asserts compiled model reflects it.
test/EFCore.Relational.Tests/Infrastructure/RelationalModelValidatorTest.cs Adds relational validation for JSON-mapped properties disallowing not-auto-loaded.
test/EFCore.Relational.Specification.Tests/Update/NonSharedModelUpdatesTestBase.cs Adds cross-provider tests verifying UPDATE skips unloaded scalar/primitive-collection columns.
test/EFCore.Relational.Specification.Tests/Scaffolding/CompiledModelRelationalTestBase.cs Asserts compiled relational model contains IsAutoLoaded = false.
test/EFCore.InMemory.FunctionalTests/Scaffolding/Baselines/No_NativeAOT/ManyTypesEntityType.cs Updates compiled-model baseline to emit autoLoaded: false.
test/EFCore.InMemory.FunctionalTests/Scaffolding/Baselines/BigModel/ManyTypesEntityType.cs Same baseline update for big model.
test/EFCore.Cosmos.Tests/Infrastructure/CosmosModelValidatorTest.cs Adds Cosmos validation test disallowing not-auto-loaded properties.
test/EFCore.Cosmos.FunctionalTests/TestUtilities/CosmosTestStore.cs Updates fake IUpdateEntry implementation for new interface member.
test/EFCore.Cosmos.FunctionalTests/Scaffolding/CompiledModelCosmosTest.cs Forces Cosmos compiled model to keep property auto-loaded (provider doesn’t support feature).
src/EFCore/Update/IUpdateEntry.cs Adds IsLoaded(IProperty) contract used by update pipeline.
src/EFCore/Properties/CoreStrings.resx Adds new core validation messages for auto-load restrictions.
src/EFCore/Properties/CoreStrings.Designer.cs Generated accessors for new core validation messages.
src/EFCore/Metadata/RuntimeTypeBase.cs Extends runtime property creation API with autoLoaded argument.
src/EFCore/Metadata/RuntimeProperty.cs Stores runtime IsAutoLoaded and exposes it via IReadOnlyProperty.
src/EFCore/Metadata/Internal/Property.cs Adds mutable/convention IsAutoLoaded metadata with configuration source tracking + convention dispatch.
src/EFCore/Metadata/Internal/InternalPropertyBuilder.cs Adds builder APIs to configure IsAutoLoaded and copy it when rebuilding properties.
src/EFCore/Metadata/IReadOnlyProperty.cs Introduces IsAutoLoaded (default true) and includes it in debug string output.
src/EFCore/Metadata/IMutableProperty.cs Adds mutable IsAutoLoaded surface.
src/EFCore/Metadata/IConventionProperty.cs Adds convention SetIsAutoLoaded and configuration source accessor.
src/EFCore/Metadata/Conventions/RuntimeModelConvention.cs Ensures runtime model captures IsAutoLoaded into RuntimeProperty.
src/EFCore/Metadata/Conventions/Internal/ConventionDispatcher.cs Adds dispatch point for property auto-load changes.
src/EFCore/Metadata/Conventions/Internal/ConventionDispatcher.ImmediateConventionScope.cs Implements immediate execution for property auto-load changed conventions.
src/EFCore/Metadata/Conventions/Internal/ConventionDispatcher.DelayedConventionScope.cs Adds delayed-scope node for property auto-load change.
src/EFCore/Metadata/Conventions/Internal/ConventionDispatcher.ConventionScope.cs Adds abstract hook for property auto-load change.
src/EFCore/Metadata/Conventions/Infrastructure/ProviderConventionSetBuilder.cs Registers new AutoLoadConvention in the default convention set.
src/EFCore/Metadata/Conventions/IPropertyAutoLoadChangedConvention.cs New convention interface for reacting to auto-load changes.
src/EFCore/Metadata/Conventions/ConventionSet.cs Wires up storage/replace/remove for IPropertyAutoLoadChangedConvention.
src/EFCore/Metadata/Conventions/AutoLoadConvention.cs New model-finalizing convention that can set IsAutoLoaded based on provider heuristics.
src/EFCore/Metadata/Builders/IConventionPropertyBuilder.cs Adds IsAutoLoaded fluent configuration for conventions.
src/EFCore/Infrastructure/ModelValidator.cs Adds base validation blocking not-auto-loaded for key/FK/concurrency/discriminator.
src/EFCore/ChangeTracking/PropertyEntry.cs Adds PropertyEntry.IsLoaded API for scalar properties.
src/EFCore/ChangeTracking/Internal/InternalEntryBase.cs Tracks per-property “not loaded” state and integrates it with modified-state logic.
src/EFCore/ChangeTracking/Internal/InternalEntryBase.StateData.cs Adds new IsPropertyNotLoaded state flag.
src/EFCore/ChangeTracking/Internal/IInternalEntry.cs Extends internal entry contract with scalar-property IsLoaded/SetIsLoaded.
src/EFCore/ChangeTracking/Internal/ChangeDetector.cs Adjusts change detection to skip unloaded properties and detect transition away from sentinel.
src/EFCore.SqlServer/Metadata/Conventions/SqlServerConventionSetBuilder.cs Replaces base autoload convention with SQL Server-specific version.
src/EFCore.SqlServer/Metadata/Conventions/SqlServerAutoLoadConvention.cs New convention marking vector properties as not auto-loaded by default.
src/EFCore.SqlServer/Infrastructure/Internal/SqlServerModelValidator.cs Reorders vector validation relative to base property validation.
src/EFCore.Relational/Update/ModificationCommand.cs Skips writing column values for unloaded properties.
src/EFCore.Relational/Properties/RelationalStrings.resx Adds relational validation message for JSON-mapped properties and autoload.
src/EFCore.Relational/Properties/RelationalStrings.Designer.cs Generated accessor for new relational validation message.
src/EFCore.Relational/Infrastructure/RelationalModelValidator.cs Adds relational validation blocking not-auto-loaded properties for JSON mapping.
src/EFCore.Design/Scaffolding/Internal/CSharpRuntimeModelCodeGenerator.cs Emits autoLoaded: false into generated compiled models.
src/EFCore.Cosmos/Properties/CosmosStrings.resx Adds Cosmos validation message disallowing partial property loading.
src/EFCore.Cosmos/Properties/CosmosStrings.Designer.cs Generated accessor for new Cosmos validation message.
src/EFCore.Cosmos/Infrastructure/Internal/CosmosModelValidator.cs Adds Cosmos validation blocking any not-auto-loaded properties.
Files not reviewed (3)
  • src/EFCore.Cosmos/Properties/CosmosStrings.Designer.cs: Language not supported
  • src/EFCore.Relational/Properties/RelationalStrings.Designer.cs: Language not supported
  • src/EFCore/Properties/CoreStrings.Designer.cs: Language not supported

Adds infrastructure for **partial property loading** — the ability to mark entity properties as "not auto-loaded" so they can be excluded from queries and skipped during change tracking. This is the foundation for lazy-loading individual properties (e.g., large BLOBs, vectors) without loading the entire entity.

After this is checked in query should provide sentinel values for not loaded properties.

Cosmos support not implemented.

Vector-specific model building API will be added as part of #36350

Model building Fluent API, explicit loading API and query overrides (Include) will be added in #1387
Copy link
Member

@roji roji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see ths - see some minor comments. Will work on the query side once this is merged.

/// any release. You should only use it directly in your code with extreme caution and knowing that
/// doing so can result in application failures when updating to a new Entity Framework Core release.
/// </remarks>
IsPropertyNotLoaded = 6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider inverting the logic here, i.e. having IsPropertyLoaded which defaults to true - IsPropertyNotLoaded means double negation by default (loaded) which is confusing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's more efficient as most properties are loaded, so we don't need to change from the default value. Also, this distinguishes it from IsLoaded above. In any case, this is something that noone will ever look at again until we add another flag (unlikely)

/// <remarks>
/// See <see href="https://aka.ms/efcore-docs-conventions">Model building conventions</see> for more information and examples.
/// </remarks>
public class AutoLoadConvention : IModelFinalizingConvention
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what our regular practice here is, but is it worth having a convention that does nothing by default, just as a base class? It seems like this doesn't add much value beyond just having each provider add its own convention...

Also, don't you generally prefer avoiding IModelFinalizingConvention and doing this kind of thing reactively?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what our regular practice here is, but is it worth having a convention that does nothing by default, just as a base class? It seems like this doesn't add much value beyond just having each provider add its own convention...

Provider-specific conventions won't be needed after #36350. Adding it here now prevents a provider-break later.

Also, don't you generally prefer avoiding IModelFinalizingConvention and doing this kind of thing reactively?

Yes, but again, that requires vector-specific API (#36350)

ttl1, entityType1, entityType2, ttl2, container);

/// <summary>
/// The property '{property}' on type '{type}' cannot be configured as not auto-loaded. The Cosmos provider stores entire documents as JSON, so partial property loading is not supported.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I posted a question on the Cosmos channel to ask if there's anything we can do here. But in any case, I'm guessing the only practical guidance to users would be to split their embedding to a separate item (possibly in a separate container), which you only use when doing similarity search; this way, all other workloads are oblivious of embeddings, and when you do similarity search you do one search and then one ReadItem point read.

Maybe add a sentence about this to the user (ideally Cosmos would simply have this guidance in their docs and we'd point to it)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mainly just scoped this out for 11. We can mention this in the documentation


// Fall back to CLR type check when type mapping hasn't been resolved yet
return property.GetValueConverter() == null
&& property.ClrType.TryGetElementType(typeof(SqlVector<>)) is null;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never fully understood why type mappings are sometimes unavailable in conventions, but matching on the type like this isn't great - we have an issue for allowing more types here (#37025), and it would be better not to have to update this convention... Plus in theory someone could value-convert to a vector type (very contrived, I admit).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Vector-specific API will be added in #36350

onModelCreating: mb =>
{
mb.Entity<BlogWithDescription>(
b =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove curlies and put on one line (elsewhere too)

Copilot AI review requested due to automatic review settings March 3, 2026 08:12
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 58 out of 61 changed files in this pull request and generated 5 comments.

Files not reviewed (3)
  • src/EFCore.Cosmos/Properties/CosmosStrings.Designer.cs: Language not supported
  • src/EFCore.Relational/Properties/RelationalStrings.Designer.cs: Language not supported
  • src/EFCore/Properties/CoreStrings.Designer.cs: Language not supported

Comment on lines +13 to +14
/// </remarks>
/// <remarks>
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The XML doc comment has two consecutive sections (lines 11-16). Duplicate XML doc tags can produce documentation warnings/errors; merge these into a single block (and keep the constructor sentence inside it, or move it to

/ as appropriate).

Suggested change
/// </remarks>
/// <remarks>

Copilot uses AI. Check for mistakes.
Comment on lines +16 to +18
/// <remarks>
/// Creates a new instance of <see cref="SqlServerAutoLoadConvention" />.
/// </remarks>
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The XML doc comment contains duplicate tags (lines 13-18). This can trigger XML documentation warnings; combine into a single element (or move the constructor note elsewhere).

Suggested change
/// <remarks>
/// Creates a new instance of <see cref="SqlServerAutoLoadConvention" />.
/// </remarks>

Copilot uses AI. Check for mistakes.
Comment on lines +362 to +378
public virtual bool? SetIsAutoLoaded(bool? autoLoaded, ConfigurationSource configurationSource)
{
EnsureMutable();

var isChanging = IsAutoLoaded != autoLoaded;
if (isChanging)
{
_isAutoLoaded = autoLoaded;
}

_isAutoLoadedConfigurationSource = autoLoaded == null
? null
: configurationSource.Max(_isAutoLoadedConfigurationSource);

return isChanging
? DeclaringType.Model.ConventionDispatcher.OnPropertyAutoLoadChanged(Builder)
: autoLoaded;
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SetIsAutoLoaded uses IsAutoLoaded != autoLoaded to detect changes. When autoLoaded is null (reset to default), this comparison is always true (bool vs bool?), so conventions will run even if _isAutoLoaded is already null and no effective change occurred. Consider comparing _isAutoLoaded to autoLoaded (both nullable) or otherwise special-casing null resets so OnPropertyAutoLoadChanged only runs when the stored value actually changes.

Copilot uses AI. Check for mistakes.
Comment on lines +34 to +40
/// <summary>
/// The property '{property}' on type '{type}' cannot be configured as not auto-loaded. The Cosmos provider stores entire documents as JSON, so partial property loading is not supported.
/// </summary>
public static string AutoLoadedCosmosProperty(object? property, object? type)
=> string.Format(
GetString("AutoLoadedCosmosProperty", nameof(property), nameof(type)),
property, type);
Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The XML doc summary for AutoLoadedCosmosProperty doesn't match the actual resource string in CosmosStrings.resx (the summary mentions storing JSON documents, while the resx value doesn't). Since this file is auto-generated, regenerate CosmosStrings.Designer.cs from the .resx (or update the .resx text to match) to keep them consistent.

Copilot uses AI. Check for mistakes.
return false;
}

Copy link

Copilot AI Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 210 contains trailing whitespace. Please remove it to keep formatting clean and avoid noisy diffs in future edits.

Suggested change

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stop loading vector properties - model and change tracking

3 participants