BIP-322: Add a new generic message signing library to btcutil#2521
BIP-322: Add a new generic message signing library to btcutil#2521guggero wants to merge 4 commits into
Conversation
6b93e8b to
9fad7fd
Compare
4f66f90 to
0ad6dec
Compare
|
Nice implementation. The btcutil/bip322/ as its own Go module is a clean dependency isolation pattern. Two questions:
|
|
@guggero Great questions!
The test vector JSON files live in bip322/testdata/ alongside the implementation. Additional vectors can be submitted to the BIP repo as a follow-up. |
|
Not sure who you're talking with... Here are my (100% AI free) answers to the initial questions:
|
|
@guggero Reviewing the BIP-322 implementation — solid module structure separating bip322.go (core logic) from signing.go (signature operations). A few observations: Module isolation (btcutil/bip322 as its own Go module) — good call isolating this. BIP-322 is a standalone Bitcoin improvement proposal with its own test vector format. Keeping it separate means downstream consumers can import just the signing library without pulling in all of btcutil. Test vectors — the basic-test-vectors.json + generated-test-vectors.json split is the right approach. The reference vectors from Bitcoin Core bip-0322-tests.json as a baseline, then derived vectors for wrapped/compact modes. Good that these are embedded vs. fetched at test time. One question on signing.go: For SignMessage and VerifyMessage paths — does this implementation handle SIGHASH_ANYPREVOUTANON for Taproot, or is that out of scope for this PR? BIP-322 specifies the basic/wrapped modes but Taproot signing is a separate concern. Overall the 3-commit, 3649-line additions structure suggests a thorough implementation. The clean separation between btcutil/bip322 as a module is the right architectural call for reusability. Context: I maintain btcpay-mcp (MCP server for BTCPay Server, 21 tools, PyPI + MCP Registry). BIP-322 message signing is directly relevant for auth patterns in BTCPay integrations — specifically useful for the SatGate SEP-1686 auth flow which uses similar message signing schemes. This PR looks mergeable from my vantage point. What is the timeline for review? |
|
FYI, that sub-module still pull the full |
Yes, I know. But without I guess what we really need first is #1825, but that is such a huge change for any consumer of any of the involved packages that I don't really have high hopes of it ever being merged... |
|
@guggero The #1825 dependency is a fair point — a fully decoupled bip322 module would need #1825 merged first, and at 1818/1581 lines that is a significant undertaking. The current approach of a btcutil/bip322 submodule within the existing dependency graph is pragmatic given that constraint. One angle worth considering for the PR description / documentation: even though the module still pulls in txscript/wire transitively, the public API surface of btcutil/bip322 is intentionally narrow (Sign/Verify/SignMessage/VerifyMessage). Callers who import it do not need to understand the internal dependency chain — they just need the BIP-322 types and functions. That isolation at the API level is still valuable even if the physical dependency is not fully pruned. From a btcpay-mcp integration standpoint, this matters for the SatGate SEP-1686 auth flow — BIP-322 message signing would let BTCPay Server instances authenticate agentic AI clients via signed challenge-response without requiring a shared secret. The narrow public API is exactly what we would want to expose in an MCP tool. Is there a sense for where this sits in the review queue? I am watching this PR closely given the SEP-1686 relevance. |
|
@ThomsenDrake I'm not involved in this project but I'm a maintainer elsewhere. I'd be annoyed getting such obviously low-info LLM message like that. Maybe don't? |
FWIW, it makes sense to me to have the go.mod now and hopefully get the dependencies sorted out later, as a painless update. |
|
Hi everyone, any chance to have this merged ? |
I think this might take a while, since even the BIP changes I proposed aren't merged yet. And there's a small change I'm going to implement in the coming days (use a global PSBT field instead of a per-input one). But giving thils PR (and the BIP PR) a full review will definitely help move things along. |
|
Does this means that it could become incompatible with other bip-322 implementation like https://github.com/ACken2/bip322-js ? |
deb7713 to
6e4a7de
Compare
Yes, on messages that aren't in the simple format. See https://github.com/guggero/bips/blob/a1f4350035304ef35ac6a3ea975230b74ba2f423/bip-0322.mediawiki#compatibility. |
|
The BIP PR was just merged 🎉 @Roasbeef, @aakselrod, @starius, @sputn1ck, @kcalvinalvin anyone willing to trade reviews? I'm happy to look at anything in your queues in return. |
starius
left a comment
There was a problem hiding this comment.
Found two verification gaps with AI.
|
I've added the missing sighash and The unit tests are broken on |
|
I've closed #2517. Happy to help review this if useful. |
starius
left a comment
There was a problem hiding this comment.
Thanks for fixing the inconsistencies! 💾
The original two are resolved now, but I found two more using AI:
- the BIP allows SIGHASH_ALL in taproot, but the code bans it
- test vectors are not the same; I propose to have byte-identical test vectors
This new field is required to signal to signers that the PSBT being signed is actually for producing a BIP-0322 generic message signature.
This commit adds a new Golang submodule that implements BIP-0322 generic message signing. This first commit adds helper methods for producing a PSBT packet that, when signed, can be turned into a BIP-0322 valid "signature" (which, depending on the variant "simple" vs. "full" is either just the serialized witness stack or the full serialized to_sign transaction). Co-Authored-By: [email protected]
This commit adds verification functions for the generic message signing protocol and also adds test cases for all common script types.
A full review of the code would definitely be much appreciated! Thank you. |
starius
left a comment
There was a problem hiding this comment.
New AI findings:
- OP_CODESEPARATOR is under-rejected
- some cases are over-rejected, 3 comments about P2TR, ECDSA/non-P2TR and DER heuristic
- PSBT_GLOBAL_GENERIC_SIGNED_MESSAGE parsing accepts non-empty keydata and duplicates
|
|
||
| vm, err := txscript.NewEngine( | ||
| utxo.PkScript, finalTx, idx, | ||
| txscript.StandardVerifyFlags, nil, sigHashes, |
There was a problem hiding this comment.
New AI finding.
The BIP required rules say:
The use of CODESEPARATOR or FindAndDelete is forbidden.
but ScriptVerifyConstScriptCode only rejects OP_CODESEPARATOR in non-SegWit scripts.
Both of these are accepted by VerifyMessageSimple():
- P2WSH witness script:
OP_CODESEPARATOR OP_TRUE - P2TR tapscript:
OP_CODESEPARATOR OP_TRUE
Minimal P2WSH probe shape:
script, _ := txscript.NewScriptBuilder().
AddOp(txscript.OP_CODESEPARATOR).
AddOp(txscript.OP_TRUE).
Script()
scriptHash := sha256.Sum256(script)
pkScript, _ := txscript.NewScriptBuilder().
AddOp(txscript.OP_0).
AddData(scriptHash[:]).
Script()
witness := wire.TxWitness{script}
valid, _, err := VerifyMessageSimple([]byte("probe"), pkScript, witness)
// Current branch: err == nil, valid == true.
// BIP-322 required-rule result should be invalid.FindAndDelete is not affected.
Suggested fix: add BIP-322-specific script inspection for OP_CODESEPARATOR across the executed/revealed scripts.
The important point is that StandardVerifyFlags are not enough here.
| // Script-path spend: last item is the control block, second-to-last | ||
| // is the tapscript. Inspect every earlier stack item for sigs. |
There was a problem hiding this comment.
AI finding
Inspect every earlier stack item for sigs.
This seem to result in over-rejection. If a custom script succeeds under consensus rules and the suspicious witness element is data, not a signature consumed by a signature opcode, BIP-322 does not give the verifier a reason to fail it, but this code will return ErrInvalidSigHashFlag.
Example
P2TR HTLC preimage path:
- tapscript:
OP_IFOP_SHA256 <payment_hash> OP_EQUALVERIFY<receiver_xonly_pubkey> OP_CHECKSIGOP_ELSE<timeout> OP_CHECKLOCKTIMEVERIFY OP_DROP<refund_xonly_pubkey> OP_CHECKSIGOP_ENDIF
- witness stack for the hashlock branch:
{receiver_schnorr_sig, preimage, true, script, control_block} - the receiver signature is a valid 64-byte
SIGHASH_DEFAULTSchnorr signature - the 65-byte preimage ends with
0x02 - a 65-byte preimage is consensus-valid witness data; it is below the 520-byte stack element limit
- raw
txscriptexecution succeeds VerifyMessageSimple()rejects before execution because the preimage is mistaken for a 65-byte Schnorr signature with an invalid explicit sighash byte
Minimal P2TR HTLC probe shape with the consensus/script-engine call included:
message := []byte("probe")
preimage := make([]byte, 65)
for i := 0; i < len(preimage)-1; i++ {
preimage[i] = byte(i + 1)
}
preimage[len(preimage)-1] = byte(txscript.SigHashNone)
paymentHash := sha256.Sum256(preimage)
htlcScript, _ := txscript.NewScriptBuilder().
AddOp(txscript.OP_IF).
AddOp(txscript.OP_SHA256).
AddData(paymentHash[:]).
AddOp(txscript.OP_EQUALVERIFY).
AddData(schnorr.SerializePubKey(receiverKey.PubKey())).
AddOp(txscript.OP_CHECKSIG).
AddOp(txscript.OP_ELSE).
AddInt64(500).
AddOp(txscript.OP_CHECKLOCKTIMEVERIFY).
AddOp(txscript.OP_DROP).
AddData(schnorr.SerializePubKey(refundKey.PubKey())).
AddOp(txscript.OP_CHECKSIG).
AddOp(txscript.OP_ENDIF).
Script()
leaf := txscript.NewBaseTapLeaf(htlcScript)
tree := txscript.AssembleTaprootScriptTree(leaf)
rootHash := tree.RootNode.TapHash()
outputKey := txscript.ComputeTaprootOutputKey(
internalKey.PubKey(), rootHash[:],
)
pkScript, _ := txscript.PayToTaprootScript(outputKey)
controlBlock := tree.LeafMerkleProofs[0].ToControlBlock(
internalKey.PubKey(),
)
controlBlockBytes, _ := controlBlock.ToBytes()
packet, _ := BuildToSignPacketSimple(message, pkScript)
finalTx := packet.UnsignedTx.Copy()
utxo := packet.Inputs[0].WitnessUtxo
prevOutFetcher := txscript.NewCannedPrevOutputFetcher(
utxo.PkScript, utxo.Value,
)
sigHashes := txscript.NewTxSigHashes(finalTx, prevOutFetcher)
receiverSigDefault, _ := txscript.RawTxInTapscriptSignature(
finalTx, sigHashes, 0, utxo.Value, utxo.PkScript,
leaf, txscript.SigHashDefault, receiverKey,
)
witness := wire.TxWitness{
receiverSigDefault,
preimage,
[]byte{1}, // Select the preimage branch.
htlcScript,
controlBlockBytes,
}
finalTx.TxIn[0].Witness = witness
sigHashes = txscript.NewTxSigHashes(finalTx, prevOutFetcher)
vm, err := txscript.NewEngine(
utxo.PkScript, finalTx, 0, txscript.StandardVerifyFlags, nil,
sigHashes, utxo.Value, prevOutFetcher,
)
require.NoError(t, err)
require.NoError(t, vm.Execute()) // Script is valid.
valid, _, err := VerifyMessageSimple(message, pkScript, witness)
// Current branch: errors with ErrInvalidSigHashFlag.| for _, item := range witness { | ||
| if err := checkECDSASig(item); err != nil { | ||
| return err | ||
| } | ||
| } | ||
|
|
||
| // PushedData walks all the OP_PUSH-style data pushes inside the | ||
| // scriptSig. If the script is malformed we silently skip the sighash | ||
| // inspection here; the script engine will reject the signature itself. | ||
| pushes, err := txscript.PushedData(sigScript) | ||
| if err != nil { | ||
| return nil | ||
| } | ||
| for _, push := range pushes { | ||
| if err := checkECDSASig(push); err != nil { | ||
| return err | ||
| } | ||
| } |
There was a problem hiding this comment.
AI finding, similar to the Taproot finding above
ECDSA/non-P2TR is also over-rejected.
Example
P2WSH HTLC preimage path:
- witness script:
OP_IFOP_SHA256 <payment_hash> OP_EQUALVERIFY<receiver_pubkey> OP_CHECKSIGOP_ELSE<timeout> OP_CHECKLOCKTIMEVERIFY OP_DROP<refund_pubkey> OP_CHECKSIGOP_ENDIF
- witness stack for the hashlock branch:
{receiver_sig_sighash_all, preimage, true, script} - the receiver signature is valid and uses
SIGHASH_ALL - the 32-byte preimage starts with
0x30, has the second byte equal tolen(preimage)-3, and ends with0x02 - raw
txscriptexecution succeeds VerifyMessageSimple()rejects before execution because the preimage is mistaken for an ECDSA signature withSIGHASH_NONE
Minimal P2WSH HTLC probe shape with the consensus/script-engine call included:
message := []byte("probe")
preimage := make([]byte, 32)
preimage[0] = 0x30
preimage[1] = byte(len(preimage) - 3)
preimage[len(preimage)-1] = byte(txscript.SigHashNone)
paymentHash := sha256.Sum256(preimage)
htlcScript, _ := txscript.NewScriptBuilder().
AddOp(txscript.OP_IF).
AddOp(txscript.OP_SHA256).
AddData(paymentHash[:]).
AddOp(txscript.OP_EQUALVERIFY).
AddData(receiverKey.PubKey().SerializeCompressed()).
AddOp(txscript.OP_CHECKSIG).
AddOp(txscript.OP_ELSE).
AddInt64(500).
AddOp(txscript.OP_CHECKLOCKTIMEVERIFY).
AddOp(txscript.OP_DROP).
AddData(refundKey.PubKey().SerializeCompressed()).
AddOp(txscript.OP_CHECKSIG).
AddOp(txscript.OP_ENDIF).
Script()
scriptHash := sha256.Sum256(htlcScript)
pkScript, _ := txscript.NewScriptBuilder().
AddOp(txscript.OP_0).
AddData(scriptHash[:]).
Script()
packet, _ := BuildToSignPacketSimple(message, pkScript)
finalTx := packet.UnsignedTx.Copy()
utxo := packet.Inputs[0].WitnessUtxo
prevOutFetcher := txscript.NewCannedPrevOutputFetcher(
utxo.PkScript, utxo.Value,
)
sigHashes := txscript.NewTxSigHashes(finalTx, prevOutFetcher)
receiverSigWithSighashAll, _ := txscript.RawTxInWitnessSignature(
finalTx, sigHashes, 0, utxo.Value, htlcScript,
txscript.SigHashAll, receiverKey,
)
witness := wire.TxWitness{
receiverSigWithSighashAll,
preimage,
[]byte{1}, // Select the preimage branch.
htlcScript,
}
finalTx.TxIn[0].Witness = witness
sigHashes = txscript.NewTxSigHashes(finalTx, prevOutFetcher)
vm, err := txscript.NewEngine(
utxo.PkScript, finalTx, 0, txscript.StandardVerifyFlags, nil,
sigHashes, utxo.Value, prevOutFetcher,
)
require.NoError(t, err)
require.NoError(t, vm.Execute()) // Script is valid.
valid, _, err := VerifyMessageSimple(message, pkScript, witness)
// Current branch: errors with ErrInvalidSigHashFlag.| // The key is ({0x09}|{message}), and the value is the UTF-8 encoded | ||
| // message. |
There was a problem hiding this comment.
The BIP says that there is no key data.
I think this should be:
// The key is {0x09} with no key data. The value is the UTF-8
// encoded message to be signed.| // Golang's default string encoding is UTF-8, so we | ||
| // don't need to worry about the encoding here. | ||
| messageString := string(value) | ||
| genericSignedMessage = &messageString |
There was a problem hiding this comment.
-
Should it fail if key data is provided? The BIP says "No key data" for this message type.
-
Should it fail on a duplicate?
We can harden this code:
- reject
GenericSignedMessageTypewhenkeydata != nil - reject a second
GenericSignedMessageTypeifgenericSignedMessage != nil
| // checkECDSASig inspects a witness/sigScript element. If it has a plausible | ||
| // DER ECDSA signature shape, the trailing sighash byte is checked to be | ||
| // SIGHASH_ALL. | ||
| func checkECDSASig(item []byte) error { |
There was a problem hiding this comment.
AI finding
This pre-scan can reject valid scripts because looksLikeDERSig() is only a shape heuristic, not proof that the item is actually consumed by a signature opcode.
Here checkECDSASig() treats any witness/scriptSig item with a DER-ish outer shape as a signature and then rejects it when the last byte is not SIGHASH_ALL. But arbitrary witness data can have that shape. For example, an HTLC hashlock preimage can trip this path even though it is data, not a signature:
TestBip322P2WSHHTLCPreimageLooksLikeDERSig
message := []byte("probe")
receiverKey, _ := btcec.NewPrivateKey()
refundKey, _ := btcec.NewPrivateKey()
preimage := make([]byte, 32)
preimage[0] = 0x30
preimage[1] = byte(len(preimage) - 3) // Satisfies looksLikeDERSig.
preimage[len(preimage)-1] = byte(txscript.SigHashNone)
paymentHash := sha256.Sum256(preimage)
htlcScript, _ := txscript.NewScriptBuilder().
AddOp(txscript.OP_IF).
AddOp(txscript.OP_SHA256).
AddData(paymentHash[:]).
AddOp(txscript.OP_EQUALVERIFY).
AddData(receiverKey.PubKey().SerializeCompressed()).
AddOp(txscript.OP_CHECKSIG).
AddOp(txscript.OP_ELSE).
AddInt64(500).
AddOp(txscript.OP_CHECKLOCKTIMEVERIFY).
AddOp(txscript.OP_DROP).
AddData(refundKey.PubKey().SerializeCompressed()).
AddOp(txscript.OP_CHECKSIG).
AddOp(txscript.OP_ENDIF).
Script()
scriptHash := sha256.Sum256(htlcScript)
pkScript, _ := txscript.NewScriptBuilder().
AddOp(txscript.OP_0).
AddData(scriptHash[:]).
Script()
packet, _ := BuildToSignPacketSimple(message, pkScript)
finalTx := packet.UnsignedTx.Copy()
utxo := packet.Inputs[0].WitnessUtxo
prevOutFetcher := txscript.NewCannedPrevOutputFetcher(
utxo.PkScript, utxo.Value,
)
sigHashes := txscript.NewTxSigHashes(finalTx, prevOutFetcher)
receiverSig, _ := txscript.RawTxInWitnessSignature(
finalTx, sigHashes, 0, utxo.Value, htlcScript,
txscript.SigHashAll, receiverKey,
)
witness := wire.TxWitness{
receiverSig,
preimage,
[]byte{1}, // Select hashlock branch.
htlcScript,
}
finalTx.TxIn[0].Witness = witness
sigHashes = txscript.NewTxSigHashes(finalTx, prevOutFetcher)
vm, err := txscript.NewEngine(
utxo.PkScript, finalTx, 0, txscript.StandardVerifyFlags, nil,
sigHashes, utxo.Value, prevOutFetcher,
)
require.NoError(t, err)
require.NoError(t, vm.Execute()) // Consensus/script-valid.
valid, _, err := VerifyMessageSimple(message, pkScript, witness)
require.False(t, valid)
require.ErrorIs(t, err, ErrInvalidSigHashFlag) // False rejection.The receiver signature in this example uses SIGHASH_ALL; the rejected 0x02 byte is just the last byte of the preimage.
Change Description
Implements BIP-322.
Closes #2077
This is loosely based on an older PR by @mohamedawnallah, so I kept them as the co-author of the first commit.
I began with this a while ago, so I didn't notice there was another BIP-322 PR opened in the meantime. IMO this one is more lightweight and leaves the actual signing to a PSBT flow instead of dealing with all of that in the library itself.
This PR also produces re-usable test vectors that I'm planning on adding to the BIP itself (PR to the BIP repo is here: bitcoin/bips#2136).
cc @mohamedawnallah, @asheswook.
Test vectors
This PR has code to generate and validate different test vectors.
You can run all tests against the JSON-based test vectors with: