CHIP 2021-01 PMv3: Version 3 Transaction Format

You know, I think another discussion made this finally “click” with me just now!

While pondering this, I realized that CashTokens is exactly where you’d arrive at if you want to solve those problems I describe! Let me see if I got it right:

  1. You implement all CashToken interfaces in the actual contract, hash it, and it becomes the genesis. Nobody knows it’s a CashToken genesis because they only see the P2SH hash in the UTXO at this point and it could be any contract. Only those with whom the author shared the contract via a side-channel could know.
  2. You then make another TX spending it, where the full contract is written to the blockchain in the input script and revealed to the world.
  3. Through a covenant, you enforce the outputs to carry that same token contract, and it can be hash-compressed in the following TXes.
  4. The new P2SH hash hashes the previous hash+token contract, that’s why it’s always different but that only proves the input script == output script, right?
  5. The HashedWitness is then needed to prove that input script == previous TX output script
  6. By induction, any CashToken output can then prove its lineage back to genesis
  7. Anyone can verify that he’s got a token from just the receiving TX and the contract. I’m unsure where he obtains the contract, from the genesis TX? In all TXes that follow it’s compressed into a hash yeah?

Script stuff still scares me but the nice thing is: I don’t need to understand it to understand how CashTokens work!

One detail, where do you store the token state? It must live outside the part that’s enforced to be the same in this covenant chain. So I guess it’s somewhere in the input script and that part doesn’t need to satisfy the in==out but must satisfy CashToken semantics which are verified with the fixed part, something like that yeah?

Hey everyone, just wanted to share an update on PMv3:

I’ve spent a lot of time testing these ideas over the past few months, and I stumbled upon a modification to the PMv3 draft that both 1) fixes a malleability vector and 2) opens a path for signature aggregation.

Background

In the first draft of PMv3, I had originally discounted 3rd-party malleability as a significant problem. In the Bitcoin Cash world, we have both covenants and reasonably secure zero-confirmation transactions: malleability is at most an inconvenience. (We’re not trying to migrate most user activity onto chains of signature-less transactions.) However, when @rnbrady identified the arbitrary detaching/re-attaching above, I realized malleability needs to be directly addressed for detached proofs (formerly “Hashed Witness”), since 3rd parties could actually disrupt important activity (beyond just fiddling with the TXID before confirmation).

In working on solutions, I spent a lot of time thinking about optimizing contracts, covenants, and transactions in general. I realized there are several closely related problems that an ideal solution should cover:

  • Malleability makes contracts less efficient and harder to validate – most non-trivial contracts must carefully validate all unlocking bytecode data to prevent vulnerabilities introduced by malleation, and this validation both bloats the contract and makes it harder to review for security. (For example, most covenants which use OP_SPLIT are vulnerable to a sort of “padding” attack which is not intuitive to first-time contract authors.)

  • The primary blocker to deduplication in transactions is unlocking bytecode malleability – because unlocking bytecode typically contains signatures (and signatures can’t sign themselves), unlocking bytecode is excluded from transaction signing serialization (“sighash”) algorithms. This is also the reason why unlocking bytecode must contain only push operations – the result of any non-push-including unlocking bytecode is a “viable malleation” for that unlocking bytecode. But if unlocking bytecode is signed, transaction introspection operations offer an opportunity to further reduce transaction sizes via deduplication. In a sense, if non-push operations could be used in unlocking bytecode, transactions would effectively have safe, efficient, zero-cost decompression via introspection opcodes.

  • Signature aggregation could save >20% of network bandwidth/storage – we know signature aggregation could save a lot of space and possibly improve transaction validation performance, but there’s no good place to put signatures which are shared between inputs. While we don’t want to bloat PMv3 with signature aggregation, a good v3 transaction format should not ignore this problem.

There are several possible solutions for each of these, but there is one particular solution I think is very elegant – it’s now specified in the CHIP:

TL;DR:

  • “Hashed Witnesses” have been renamed to Detached Proofs.

  • Detached Signatures are now separated from Detached Proofs, and they both comprehensively solve third-party malleability and enable signature aggregation (some immediately, some via a future sighash upgrade like the one proposed by Chris Pacia).

  • The PMv3 specification is now a proper CHIP (CasH Improvement Proposal): CHIP-2021-01-PMv3, and includes a lot more supporting detail outside of just the technical specification.

I’ve incorporated the answers to many of the questions from this thread, so thank you again to everyone who has contributed here.

I’d appreciate any feedback, reviews, or questions you have, either in this thread or in the GitHub issues. Thanks!

2 Likes

Ah, sorry to keep you waiting for an answer here – your numbered steps are very close – maybe Richard’s summary will help to make 4 through 7 clearer:

In the latest CHIP revision, I’ve tried to make things a bit clearer too, but I may try to add an appendix of sorts to walk through this particular “cross-contract interface” in detail.

Yes! Each tokens’ “genesis” transaction hash It’s stored in the top-level “corporation” covenant, which holds the full set in a merkle tree. So CashTokens can be moved around independently, then eventually checked-back-in to the top-level covenant. If you haven’t read it yet, see the description in the CashTokens demo.

That may help to get a sense for why detached proofs are so useful – they allow contract designers to build interfaces between different contracts (without resorting to the “global state” model of systems like Ethereum).

In the case of CashTokens, I’ve written a simple top-level covenant to keep track of 4, homogenous child “token” covenants. But this same strategy is useful for breaking even very large contracts up into smaller subcontracts.

In general, with PMv3 we can implement new token schemes and other high-level contract protocols entirely at the wallet/application layer, without network-wide upgrades or other coordination.

1 Like

Wow, this addresses so many of my blocking questions that I will be able to give another go understanding what Jason created. Thanks so much Richard!

2 Likes

@bitjson looking at the chip fresh after talking yesterday, there is one thing I notice that I didn’t mention before - I think a contrast exercise would be helpful for both high level general understanding and high level technical understanding. What I mean is taking a contract or use case and then showing where the dividing line is: Currently we can only do X. With pmv3 then Y becomes possible. The CHIP as I read it now basically skips straight to Y so it is harder to understand the scope of the gain.

1 Like

Good idea, thanks! I’m just now realizing that this revision doesn’t even link to the CashToken demo anymore.

I’ll work on adding that appendix directly to the spec. Probably want 2 examples:

  • basic parent transaction inspection - the covenant checks that it’s been given an un-hashed copy of a parent transaction for one of its inputs (validated by comparing the hash with an outpoint TX hash), then inspects some simple parent property like transaction version.
  • inductive proof covenants - the covenant inspects two transactions back to confirm that its grandparent used the expected covenant and its parent spent that covenant successfully, proving that the grandparent either is or descends from an original “mint” transaction of the same covenant. (This one is what @rnbrady outlines above.)

As a followup and to make things more concrete I have the following suggestion for your proposal.

I would like to split the proposal into two parts with the aim to simplify activation and to make sure that we have a smaller change that I believe can reach consensus next May.
The actual changes in the transaction format would then become this;

  • 2022-05. Transaction v3 introduced, where needed the input-script is replaced with a 32-byte hash and appended to the transaction is the list of detached proofs.
    When it comes to the transaction format, this is the only change. This actually does unlock your many improvements and inductive proofs. It allows Script to look backwards in time. It enables tokens and it enables most of your cool examples scripts. Other scripting improvements are a separate CHIP.

  • 2023: Transaction v4 introduced for variable-int sizes and various other ideas (tbd).
    Variable int sizes is one of the ideas you separately came up with which was previously part of FlexTrans (BIP 134), there were some other neat ideas that would be useful to combine in such a transaction format change.

The direct result of seperating your one CHIP is that parsers that read current (v2) style transactions will very likely not need to be modified to accept v3 transactions. Afterall, you just add some more data to the end. Even TXID calculation is going to be unchanged.
This is going to make deployment much simpler and since the May 2022 upgrade already has a large number of proposals I think it makes sense to keep it simple.

Changing the integer-format (the var-int idea) in transactions is simply a much more invasive change, 100% of the existing transaction-parsing code will need to get support that and for that it really makes sense to wait until after Restrict Transaction Version has activated which is specifically meant to make this kind of deployments simpler. Hence the suggestion to push it forward.

So, the separation of those more invasive transaction changes out of the detached signature change would be nice to move to the upgrade of 2023 to give ourselves as well as the wider ecosystem plenty of time to get it right.