fix: set final_lsn for subxacts before cleanup

histogram reworks

First published logical decoding on standby

I've prepared a working initial, somewhat raw implementation for logical decoding on physical standbys. Since it's a 20-patch series at the moment you can find the latest version here:

 [x]

It incorporates the following changes:

* Timeline following for logical slots, so they can start decoding on the correct timeline and follow timeline switches (with tests); originally [3]

* Add --endpos to pg_recvlogical, with tests; originally [3]

* Splitting of xmin and catalog_xmin on hot standby feedback, so logical slots on a replica only hold down catalog_xmin on the master, not the xmin for user tables (with tests). Minimises upstream bloat; originally [4]

* Suppress export of snapshot when starting logical decoding on replica. Since we cannot allocate an xid, we cannot export snapshots on standby. Decoding clients can still do their initial setup via a slot on the master then switch over, do it via physical copy, etc.

* Require hot_standby_feedback to be enabled when starting logical decoding on a standby.

* Drop any replication slots from a database when redo'ing database drop, so we don't leave dangling slots on the replica (with tests).

* Make the walsender respect SIGUSR1 and exit via RecoveryConflictInterrupt() when it gets PROCSIG_RECOVERY_CONFLICT_DATABASE (with tests); see [6]

* PostgresNode.pm enhancements for the tests

* New test coverage for logical decoding on standby

The following open items remain:

* The method used to make the walsender respect conflict with recovery interrupts may not be entirely safe, see walsender procsignal_sigusr1_handler thread [5];

* We probably terminate walsenders running inside an output plugin with a virtual xact whose xmin is below the upstream's global xmin, even though its catalog xmin is fine, in ResolveRecoveryConflictWithSnapshot(...). Haven't been able to test this. Need to only terminate them when the conflict affects relations accessible in logical decoding, which likely needs the upstream to send more info in WAL;

* logical decoding timeline following needs tests for cascading physical replication where an intermediate node is promoted per timeline following thread [3];

* walsender may need to maintain ThisTimeLineID in more places per decoding timeline following v10 thread [3];

* it may be desirable to refactor the walsender to deliver cleaner logical decoding timeline following per the decoding timeline following v10 thread[3]

also:

* Nothing stops the user from disabling hot_standby_feedback on the standby or dropping and re-creating the physical slot on the master, causing needed catalog tuples to get vacuumed away. Since it's not going to be safe to check slot shmem state from the hot_standby_feedback verify hook and we let hot_standby_feedback change at runtime this is going to be hard to fix comprehensively, so we need to cope with what happens when feedback fails, but:

* We don't yet detect when upstream's catalog_xmin increases past our needed catalog_xmin and needed catalog tuples are vacuumed away by the upstream. So we don't invalidate the slot or terminate any active decoding sessions using the slot. Active decoding sessions often won't have a vtxid to use with ResolveRecoveryConflictWithVirtualXIDs(), transaction cancel is not going to be sufficient, and anyway it'll cancel too aggressively since it doesn't know it's safe to apply changes that affect only (non-user-catalog) heap tables without conflict with decoding sessions.

... so this is definitely NOT ready for commit. It does, however, make logical decoding work on standby.

Since it doesn't look practical to ensure there's never been a gap in hot standby feedback or detect such a gap directly, I'm currently looking at ways to reliably detect when the upstream has removed tuples we need and error out. That means we need a way to tell when upstream's catalog_xmin has advanced, which we don't currently have from xlogs. Checkpoint's oldestXID is insufficient since advance could've happened since last checkpoint.

This series supercedes:

 * Timeline following for logical slots
  [1] https://www.postgresql.org/message-id/flat/CAMsr+YH-C1-X_+s=2nzAPnR0wwqJa-rUmVHSYyZaNSn93MUBMQ@mail.gmail.com

 *  WIP: Failover Slots
  [2] https://www.postgresql.org/message-id/CAMsr+YFqtf6ecDVmJSLpC_G8T6KoNpKZZ_XgksODwPN+f=evqg@mail.gmail.com

and incorporates the patches in:

 * Logical decoding timeline following take II
  [3] https://www.postgresql.org/message-id/flat/CAMsr+YEQB3DbxmCUTTTX4RZy8J2uGrmb5+_ar_joFZNXa81Fug@mail.gmail.com

* Send catalog_xmin separately in hot standby feedback
  [4] https://www.postgresql.org/message-id/CAMsr+YFi-LV7S8ehnwUiZnb=1h_14PwQ25d-vyUNq-f5S5r=Zg@mail.gmail.com

*  Use procsignal_sigusr1_handler and RecoveryConflictInterrupt() from walsender?
   [5] https://www.postgresql.org/message-id/CAMsr+YFb3R-t5O0jPGvz9_nsAt2GwwZiLSnYu3=X6mR9RnrbEw@mail.gmail.com

Also relevant:

 * Use procsignal_sigusr1_handler and RecoveryConflictInterrupt() in walsender
  [6] https://www.postgresql.org/message-id/CAMsr+YFb3R-t5O0jPGvz9_nsAt2GwwZiLSnYu3=X6mR9RnrbEw@mail.gmail.com

Add new pg_replication_slot_set_failover(slotname,isfailover)

Add pg_replication_slot_set_failover(slotname name, failover bool)
function to toggle failover mode of existing replication slot.

Update the regression tests to cover it.

Add new pg_replication_slot_set_failover(slotname,isfailover)

Add pg_replication_slot_set_failover(slotname name, failover bool)
function to toggle failover mode of existing replication slot.

Update the regression tests to cover it.

Allow replication slots to follow failover

Originally replication slots were unique to a single node and weren't
recorded in WAL or replicated. A logical decoding client couldn't follow
a physical standby failover and promotion because the promoted replica
didn't have the original master's slots. The replica may not have
retained all required WAL and there was no way to create a new logical
slot and rewind it back to the point the logical client had replayed to.

Failover slots lift this limitation by replicating slots consistently to
physical standbys, keeping them up to date and using them in WAL
retention calculations. This allows a logical decoding client to follow
a physical failover and promotion without losing its place in the change
stream.

A failover slot may only be created on a master server, as it must be
able to write WAL. This limitation may be lifted later.

pg_basebackup is also modified to copy the contents of pg_replslot.
Non-failover slots will now be removed during backend startup instead
of being omitted from the copy.

This patch does not add any user interface for failover slots. There's
no way to create them from SQL or from the walsender. That and the
documentation for failover slots are in the next patch in the series
so that this patch is entirely focused on the implementation.

Craig Ringer, based on a prototype by Simon Riggs

Retain extra WAL for failover slots in base backups

Change the return value of pg_start_backup(), the BASE_BACKUP walsender
command, etc to report the minimum WAL required by any failover slot if
this is a lower LSN than the redo position so that base backups contain
the WAL required for slots to work.

Add a new backup label entry 'MIN FAILOVER SLOT LSN' that, if present,
indicates the minimum LSN needed by any failover slot that is present in
the base backup. Backup tools should check for this entry and ensure
they retain all xlogs including and after that point.

Add the UI and for failover slots

Expose failover slots to the user.

Add a new 'failover' argument to pg_create_logical_replication_slot and
pg_create_physical_replication_slot . Accept a new FAILOVER keyword
argument in CREATE_REPLICATION_SLOT on the walsender protocol.

Document failover slots

Add 'failover' to pg_replication_slots

Introduce TAP recovery tests for failover slots

Update decoding_failover tests for failover slots

Fixups after rebase

Backward-compatible version of 9.6 failover slots

Stable ordering of tests

Fix a minor oversight in failover slots tests

Update failover slots to be on-disk compatible with unpatched PostgreSQL

Add new pg_replication_slot_set_failover(slotname,isfailover)

Add pg_replication_slot_set_failover(slotname name, failover bool)
function to toggle failover mode of existing replication slot.

Update the regression tests to cover it.

After merge fixups

fix physical slots

Permit logical decoding on standby with a warning

Failover slots with logical decoding timeline following, ondisk compa…

…tible, with a refactoring backported from 9.6 for cleanup, see #45625

Backport of failover slots for 9.5, on-disk compatible, see #45625

Add new pg_replication_slot_set_failover(slotname,isfailover)

Add pg_replication_slot_set_failover(slotname name, failover bool)
function to toggle failover mode of existing replication slot.

Update the regression tests to cover it.