Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Dependency resolution during install can be slow #3451

New issue
New issue
Closed
#3471
Closed
Dependency resolution during install can be slow#3451
#3471
Assignees
josh-cooley
Labels
5 - ReleasedThe issue has been resolved, and released to the public for consumption.EnhancementIssues that introduce new functionality to the project, or enhances/updates existing functionality.
Milestone
2.4.0
@josh-cooley

Description

@josh-cooley
josh-cooley
opened on May 24, 2024

Checklist

  • I confirm there are no unresolved issues reported on the Chocolatey Status page.
  • I have verified this is the correct repository for opening this issue.
  • I have verified no other issues exist related to my problem.
  • I have verified this is not an issue for a specific package.
  • I have verified this issue is not security related.
  • I confirm I am using official, and not unofficial, or modified, Chocolatey products.

What You Are Seeing?

choco upgrade my-package command takes a very long time resolving dependencies. The verbose output shows that package resolution is occurring repeatedly for the same packages. Debugging the issue shows it processing multiple versions of the same dependency in NugetCommon.GetPackageDependencies and NugetCommon.HandleDependencies

Example dependencies

graph TD;
    A-->B;
    A-->C;
    B-->D;
    C-->D;
Loading

The dependency for B->C is similar to [1.0.1000, 2.0.0)
The dependency for C->D is similar to [1.0.0, 2.0.0)
All versions between 1.0.0 and 1.0.1000 are resolved and dependencies for D are then resolved for each instance.

Downgrading to 1.4 allows this to work.

What is Expected?

Expect resolution to the narrowest constrained dependencies. If 1.0.1000 or newer is required, no version less than 1.0.1000 should be considered.

How Did You Get This To Happen?

  1. I ran choco upgrade my-package -y
  2. Waited 30 min.
  3. Upgrade did not complete, download did not begin.

System Details

  • Operating System: Windows 10
  • Windows PowerShell version: 7.4.2
  • Chocolatey CLI Version: 2.2
  • Chocolatey Licensed Extension version: N/A
  • Chocolatey License type: Open Source
  • Terminal/Emulator: Windows Terminal

Installed Packages

7zip 21.7
7zip.install 21.7
azure-cli 2.53.0.20231015
cadinstall.extension 1.0.1
chocolatey 1.4.0
chocolatey-compatibility.extension 1.0.0
chocolatey-core.extension 1.4.0
chocolatey-dotnetfx.extension 1.0.1
chocolatey-windowsupdate.extension 1.0.4
dotnet-6.0-aspnetruntime 6.0.28
dotnet-6.0-runtime 6.0.28
dotnet-8.0-aspnetruntime 8.0.0
dotnet-8.0-runtime 8.0.0
DotNet4.6.1 4.6.01055.20170308
dotnet4.8 4.8.3761.0
erlang 25.1.2
filezilla 3.58.0
git 2.41.0
git.install 2.41.0
handle 5.0
ilspy 7.2.1
KB2919355 1.0.20160915
KB2919442 1.0.20160915
KB2999226 1.0.20181019
KB3033929 1.0.5
KB3035131 1.0.3
netfx-4.7.2 4.7.2.0
notepadplusplus 8.4.8
notepadplusplus.install 8.4.8
nuget.commandline 6.8.0
pandoc 3.1.6.2
perfview 3.1.7
procexp 17.2
procmon 3.83
rabbitmq 3.12.2
sql-server-2019 15.0.2000.20210324
sql-server-management-studio 15.0.18386.0
sqlsysclrtypes 11.2.5058.3
Tracker 10.2409.0.137
vcredist140 14.38.33130
windirstat 1.1.2.20161210
winmerge 2.16.24

Output Log

2024-05-23 16:16:25,736 61996 [INFO ] - Chocolatey v2.2.2
2024-05-23 16:16:25,741 61996 [DEBUG] - Chocolatey is running on Windows v 10.0.19045.0
2024-05-23 16:16:25,743 61996 [DEBUG] - Attempting to delete file "C:/ProgramData/chocolatey/choco.exe.old".
2024-05-23 16:16:25,743 61996 [DEBUG] - Attempting to delete file "C:\ProgramData\chocolatey\choco.exe.old".
2024-05-23 16:16:25,748 61996 [DEBUG] - Command line: "C:\ProgramData\chocolatey\choco.exe" upgrade product.db-tool -y
2024-05-23 16:16:25,749 61996 [DEBUG] - Received arguments: upgrade product.db-tool -y
2024-05-23 16:16:25,774 61996 [DEBUG] - RemovePendingPackagesTask is now ready and waiting for PreRunMessage.
2024-05-23 16:16:25,779 61996 [DEBUG] - Sending message 'PreRunMessage' out if there are subscribers...
2024-05-23 16:16:25,784 61996 [DEBUG] - [Pending] Removing all pending packages that should not be considered installed...
2024-05-23 16:16:25,814 61996 [DEBUG] - Performing validation checks.
2024-05-23 16:16:25,816 61996 [DEBUG] - Global Configuration Validation Checks:
2024-05-23 16:16:25,817 61996 [DEBUG] -  - Package Exit Code / Exit On Reboot = Checked
2024-05-23 16:16:25,819 61996 [DEBUG] - System State Validation Checks:
2024-05-23 16:16:25,821 61996 [DEBUG] -  Reboot Requirement Checks:
2024-05-23 16:16:25,822 61996 [DEBUG] -  - Pending Computer Rename = Checked
2024-05-23 16:16:25,823 61996 [DEBUG] -  - Pending Component Based Servicing = Flagged
2024-05-23 16:16:25,825 61996 [DEBUG] - Cache Folder Lockdown Checks:
2024-05-23 16:16:25,825 61996 [DEBUG] -  - Elevated State = Checked
2024-05-23 16:16:25,826 61996 [DEBUG] -  - Folder Exists = Checked
2024-05-23 16:16:25,828 61996 [DEBUG] -  - Folder lockdown = Checked
2024-05-23 16:16:25,830 61996 [INFO ] - 3 validations performed. 2 success(es), 1 warning(s), and 0 error(s).
2024-05-23 16:16:25,831 61996 [INFO ] - 
2024-05-23 16:16:25,832 61996 [WARN ] - Validation Warnings:
2024-05-23 16:16:25,833 61996 [WARN ] -  - A pending system reboot request has been detected, however, this is
   being ignored due to the current Chocolatey configuration.  If you
   want to halt when this occurs, then either set the global feature
   using:
     choco feature enable --name="exitOnRebootDetected"
   or pass the option --exit-when-reboot-detected.

2024-05-23 16:16:25,840 61996 [DEBUG] - The source 'https://chocolatey.org/api/v2/;https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/' evaluated to a 'normal' source type
2024-05-23 16:16:25,841 61996 [DEBUG] - 
NOTE: Hiding sensitive configuration data! Please double and triple
 check to be sure no sensitive data is shown, especially if copying
 output to a gist for review.
2024-05-23 16:16:25,846 61996 [DEBUG] - Configuration: CommandName='upgrade'|
CacheLocation='C:\Users\myname\AppData\Local\Temp\chocolatey'|
CommandExecutionTimeoutSeconds='2700'|WebRequestTimeoutSeconds='30'|
Sources='https://chocolatey.org/api/v2/;https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/'|

SourceType='normal'|ShowOnlineHelp='False'|Debug='False'|
Verbose='False'|Trace='False'|Force='False'|Noop='False'|
HelpRequested='False'|UnsuccessfulParsing='False'|RegularOutput='True'|
QuietOutput='False'|PromptForConfirmation='False'|
DisableCompatibilityChecks='False'|AcceptLicense='True'|
AllowUnofficialBuild='False'|Input='MaintainCadDb'|AllVersions='False'|
SkipPackageInstallProvider='False'|SkipHookScripts='False'|
PackageNames='MaintainCadDb'|Prerelease='False'|ForceX86='False'|
OverrideArguments='False'|NotSilent='False'|
ApplyPackageParametersToDependencies='False'|
ApplyInstallArgumentsToDependencies='False'|IgnoreDependencies='False'|
CacheExpirationInMinutes='30'|AllowDowngrade='False'|
ForceDependencies='False'|PinPackage='False'|
Information.PlatformType='Windows'|
Information.PlatformVersion='10.0.19045.0'|
Information.PlatformName='Windows 10'|
Information.ChocolateyVersion='2.2.2.0'|
Information.ChocolateyProductVersion='2.2.2'|
Information.FullName='choco, Version=2.2.2.0, Culture=neutral, PublicKeyToken=79d02ea9cad655eb'|

Information.Is64BitOperatingSystem='True'|
Information.Is64BitProcess='True'|Information.IsInteractive='True'|
Information.UserName='myname'|Information.UserDomainName='INGRNET'|
Information.IsUserAdministrator='True'|
Information.IsUserSystemAccount='False'|
Information.IsUserRemoteDesktop='False'|
Information.IsUserRemote='False'|Information.IsProcessElevated='True'|
Information.IsLicensedVersion='False'|
Information.IsLicensedAssemblyLoaded='False'|
Information.LicenseType='Foss'|
Information.CurrentDirectory='C:\Users\myname'|
Features.AutoUninstaller='True'|Features.ChecksumFiles='True'|
Features.AllowEmptyChecksums='False'|
Features.AllowEmptyChecksumsSecure='True'|
Features.FailOnAutoUninstaller='False'|
Features.FailOnStandardError='False'|Features.UsePowerShellHost='True'|
Features.LogEnvironmentValues='False'|Features.LogWithoutColor='False'|
Features.VirusCheck='False'|
Features.FailOnInvalidOrMissingLicense='False'|
Features.IgnoreInvalidOptionsSwitches='True'|
Features.UsePackageExitCodes='True'|
Features.UseEnhancedExitCodes='False'|
Features.UseFipsCompliantChecksums='False'|
Features.ShowNonElevatedWarnings='True'|
Features.ShowDownloadProgress='True'|
Features.StopOnFirstPackageFailure='False'|
Features.UseRememberedArgumentsForUpgrades='False'|
Features.IgnoreUnfoundPackagesOnUpgradeOutdated='False'|
Features.SkipPackageUpgradesWhenNotInstalled='False'|
Features.RemovePackageInformationOnUninstall='False'|
Features.ExitOnRebootDetected='False'|
Features.LogValidationResultsOnWarnings='True'|
Features.UsePackageRepositoryOptimizations='True'|
ListCommand.LocalOnly='False'|ListCommand.IdOnly='False'|
ListCommand.IncludeRegistryPrograms='False'|ListCommand.PageSize='25'|
ListCommand.Exact='False'|ListCommand.ByIdOnly='False'|
ListCommand.ByTagOnly='False'|ListCommand.IdStartsWith='False'|
ListCommand.OrderByPopularity='False'|ListCommand.ApprovedOnly='False'|
ListCommand.DownloadCacheAvailable='False'|
ListCommand.NotBroken='False'|
ListCommand.IncludeVersionOverrides='False'|
ListCommand.ExplicitPageSize='False'|
ListCommand.ExplicitSource='False'|
UpgradeCommand.FailOnUnfound='False'|
UpgradeCommand.FailOnNotInstalled='False'|
UpgradeCommand.NotifyOnlyAvailableUpgrades='False'|
UpgradeCommand.ExcludePrerelease='False'|
NewCommand.AutomaticPackage='False'|
NewCommand.UseOriginalTemplate='False'|SourceCommand.Command='unknown'|
SourceCommand.Priority='0'|SourceCommand.BypassProxy='False'|
SourceCommand.AllowSelfService='False'|
SourceCommand.VisibleToAdminsOnly='False'|
FeatureCommand.Command='unknown'|ConfigCommand.Command='Unknown'|
ApiKeyCommand.Command='Unknown'|PinCommand.Command='Unknown'|
OutdatedCommand.IgnorePinned='False'|
ExportCommand.IncludeVersionNumbers='False'|Proxy.BypassOnLocal='True'|
TemplateCommand.Command='unknown'|CacheCommand.Command='Unknown'|
CacheCommand.RemoveExpiredItemsOnly='False'|
2024-05-23 16:16:25,847 61996 [DEBUG] - _ Chocolatey:ChocolateyUpgradeCommand - Normal Run Mode _

followed by many pages of this

2024-05-23 16:17:03,781 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.localization'&semVerLevel=2.0.0
2024-05-23 16:17:03,782 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.localization'&semVerLevel=2.0.0
2024-05-23 16:17:04,070 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.config'&semVerLevel=2.0.0
2024-05-23 16:17:04,072 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.config'&semVerLevel=2.0.0
2024-05-23 16:17:04,224 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.upgrade-tool'&semVerLevel=2.0.0
2024-05-23 16:17:04,225 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.upgrade-tool'&semVerLevel=2.0.0
2024-05-23 16:17:04,529 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.localization'&semVerLevel=2.0.0
2024-05-23 16:17:04,530 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.localization'&semVerLevel=2.0.0
2024-05-23 16:17:04,852 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.config'&semVerLevel=2.0.0
2024-05-23 16:17:04,854 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.config'&semVerLevel=2.0.0
2024-05-23 16:17:05,012 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.localization'&semVerLevel=2.0.0
2024-05-23 16:17:05,013 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.localization'&semVerLevel=2.0.0
2024-05-23 16:17:05,294 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.config'&semVerLevel=2.0.0
2024-05-23 16:17:05,295 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.config'&semVerLevel=2.0.0
2024-05-23 16:17:05,422 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.upgrade-tool'&semVerLevel=2.0.0
2024-05-23 16:17:05,424 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.upgrade-tool'&semVerLevel=2.0.0
2024-05-23 16:17:05,752 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.localization'&semVerLevel=2.0.0
2024-05-23 16:17:05,754 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.localization'&semVerLevel=2.0.0
2024-05-23 16:17:06,055 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.config'&semVerLevel=2.0.0
2024-05-23 16:17:06,056 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.config'&semVerLevel=2.0.0
2024-05-23 16:17:06,232 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.upgrade-tool'&semVerLevel=2.0.0
2024-05-23 16:17:06,234 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?id='product.upgrade-tool'&semVerLevel=2.0.0
2024-05-23 16:17:06,564 61996 [INFO ] - [NuGet]   CACHE https://community.chocolatey.org/api/v2/FindPackagesById()?id='product.localization'&semVerLevel=2.0.0
2024-05-23 16:17:06,565 61996 [INFO ] - [NuGet]   CACHE https://pkgs.dev.azure.com/product-company/_packaging/CadChocolateyPackages/nuget/v2/FindPackagesById()?

Additional Context

NugetCommon.HandleDependencies has a dependencyCache parameter. The cache is checked for exact matches with dependencyCache.Contains(dependency). Changing this to check the VersionRange with IsSubSetOrEqualTo allowed the upgrade to complete and target the narrowest range in the dependencies. The check looks like:
dependencyCache.Any(d => d.Id == dependency.Id && d.VersionRange.IsSubSetOrEqualTo(dependency.VersionRange));
Instead of the Contains call.

Metadata

Metadata

Assignees

Labels

5 - ReleasedThe issue has been resolved, and released to the public for consumption.EnhancementIssues that introduce new functionality to the project, or enhances/updates existing functionality.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions