Source Tracking Beta

What’s new?

New versions of force:source:push/pull/status and force:source:tracking:clear/reset in https://github.com/salesforcecli/plugin-source under the source:beta subtopic

A new open-source library for source tracking at https://github.com/forcedotcom/source-tracking that provides direct interaction with the local source tracking files via git.

Bug fixes and feature requests

How do I use them?

Unlike previous beta tests of new commands, we’re trying something different. We’re shipping them in everybody’s CLI.

To use them, just add the beta subtopic, like sfdx force:source:beta:pull instead of sfdx force:source:pull

The regular commands (without the beta) remain unchanged.

Don’t mix the versions

The tracking files are different and stored in slightly different places. If you have a project using the old commands, you can’t use the new ones on that project until you either 1) clear the tracking or 2) start with a new org

Both the beta and normal versions of these commands check the tracking files to keep you from accidentally using the wrong ones.

What else is changing?

Performance

This is using the same library deploy/retrieve library that VSCode and source:deploy/retrieve/delete use. It’s open source, so you can run a debugger OR check if your metadata type is missing or contribute one that is. Now, all supported tooling is using the same library, which deploys/retrieves faster than the previous version.

Behavior of multiple package directories

If your sfdx-project.json has multiple packageDirectories, force:source:beta:push will (by default) merge them all into one deployment, which for most users will be much faster.

Thanks to those who raised #1269 and @kjonesffdc proposed an elegant solution: the new pushPackageDirectoriesSequentially property in sfdx-project.json will make each package directory a separate deployment, in the order specified in sfdx-project.json.

"packageDirectories": [
    {
      "path": "force-app",
      "default": true
    },
    {
      "path": "my-app"
    },
    {
      "path": "foo-bar/app"
    }
],
"pushPackageDirectoriesSequentially": true,

If you're using an IDE that validates your project.json schema, you'll get a warning about this property not existing until that schema gets updated forcedotcom/schemas#42

Hooks have breaking changes

The [pre|post][deploy|retrieve] hooks will mirror those of source:deploy|retrieve (different from the old push/pull). Learn way more on the blog

Instant project changes

The new commands always check your forceignore and project.json packageDirectories so changes in those files are immediately picked up.

Source tracking files and location

Now stored in .sfdx/orgs/[orgId] instead of by username
The sourcePathInfos files no longer exist. They’ve been replaced by git-style repository files

Using git to interact with local tracking files

So, yeah, there’s a git repo to track local changes. Manually manipulating your tracking files shouldn’t be part of your normal workflow, but for the adventurous, you can now use git commands starting with git --git-dir .sfdx/orgs/[orgId]/localSourceTracking to do things like

1. ignore a local change without impacting the org on your next push
   1. git --git-dir .sfdx/orgs/[orgId]/localSourceTracking add [somefile]
   2. git --git-dir .sfdx/orgs/[orgId]/localSourceTracking commit -m 'manual update'
2. see changes to source tracking files from push/pull actions
   1. git --git-dir .sfdx/orgs/[orgId]/localSourceTracking log

Simplified flags and ignored on force:source:status

--all was redundant, so it’s gone. It you want both local and remote, just say sfdx force:source:beta:status
You can specify —remote or --local but not both, because that’s also the same as sfdx force:source:beta:status

The output of force:source:status shows all changes but indicates which ones, if any, are covered by your .forceignore file (and therefore won’t be part of push/pull).

These currently only happen when there's a local copy of the file...you won't see it marked as ignored if it's in the Remote Add state, even though it won't be pulled.

Quiet mode for push

Adding —quiet to force:source:beta:push will result in no list of successful components (you’ll still see any errors)

Fixed issues

#1387
#1264
#1132
#451
#1295
#119
#537
#1308
#582
Pushes with large deletes (~100+ files) don’t result in 431 errors
Fixed conflict detection on CustomLabels

Feature requests delivered

Source status shows what’s currently ignored #119
Quiet mode for push #129

Known issues

1. We're working on a way to mark remote changes that will be ignored when the files don't exist locally

What’s next?

• All the bugs y'all report during the beta
• Ways to get source tracking "fixed" when it gets in a bad state (ex: timeout or dropped connection during a push)
• We want deploy/retrieve to be able to interact with source tracking files. Imagine using force:source:retrieve to retrieve a few specific files, instead of pulling all the changes, and having that update instead of break your tracking.