agent: fix a flakey test case

Adding instructions for migrating data after storage mapping changes

Added "Migrating your existing data to the new storage mapping" section.

docs: document path parameters in source-http-ingest

flowctl: build using clang instead of gcc

Switches to using clang instead of GCC in order to work around a
compilation error in one of our dependencies, which is due to a bug in
GCC 9. See:
rustls/rustls#1967

dekaf: Introduce transient error retry limit

crates/agent: authorization API uses DB snapshots

Update the API to fetch on-demand snapshots from the DB of
authorization state, bounding both the frequency with which snapshots
may be taken and also for how long a stale snapshot may be used.

A request will either succeed against a current snapshot (happy
fast path), or will fail. If it fails, the caller is asked to retry
after a given delay, and meanwhile the API will have typically
started to refresh the current snapshot.

If the request continues to fail against a snapshot which as taken
_after_ the issued-at time of the request token, only then is the error
considered permanently failed.

This means that requests are cheap, evaluted only by the agent-api,
but we *also* don't have caching artifacts that could result in false
errors when tasks or collections are being created or changed. Instead, we
incur a minor delay while a sufficiently-recent snapshot is taken.

A final change is that we now prefix match on shard ID templates and
journal name templates, rather than munging both into a more-approximate
prefix match over catalog names.