CHIP 2021-01 Allow Transactions to be smaller in size

CHIPs Standards Track
,
#1

Problem statement

Since protocol upgrade 1542300000 (November 15th 2018) there is a protocol rule which states that:

Transactions that are smaller than 100 bytes shall be considered invalid.

The reasoning for this is valid, this avoids an attack against a known Bitcoin merkle-tree weakness. (source: Leaf-Node weakness in Bitcoin Merkle Tree Design).

The downside of picking the value 100 bytes is that a default coinbase transaction is smaller than that. The coinbase is special in that it doesn’t have any signatures since it lacks inputs. This means that people (including mining software) creating a coinbase may well hit this limit without realizing and this simply makes life harder than it has to be.

More advanced usage of Bitcoin Cash does show there are quite a lot of valid transactions that can be created which are smaller than 100 bytes, only P2PKH and similar grow in size quite fast, but script certainly allows many other types. These kind of usecases being made impossible was the unintended sideeffect of the introduction of the 100 byte minimum rule.

We suggest to change this and simply refuse transactions of exactly 64 bytes instead. This is enough to avoid the attack explained in the linked article.

Evaluation of solution

In the linked research we find this summary (emphasis mine):

To summarize, the most space-efficient solution without a soft-fork would be hybrid: show inclusion proof of the rightmost tx id branch if the tree is not full, else the coinbase (showing mid-state and length) if length is greater than 64 bytes, or else show the full coinbase transaction.

The author suggests that the coinbase should be larger than 64 bytes. There is no explanation I could find why the BCH developers in 2018 picked 100 instead, which is indeed larger than 64 bytes. But as we saw above, this causes issues for normal usage.

A normal tx can not be smaller than 60 bytes (click for more)

The simplest tx has these values and we count the size in the first column:

Absolute Offs Size Description
0 4 Version
4 1 Input count
5 32 Input 0 Tx hash
37 4 Input 0 Txout index
41 1 Input 0 script length
42 0 Empty Script
42 4 nSequence
46 1 Output count
47 8 value
55 1 Output 0 script length
56 0 Output 0 scriptPubKey
56 4 lock_time
60

Total = 60 bytes with a completely empty transaction.

With regards to coinbases: the existence of BIP34, which forces a serialization of the block-height into the coinbase output script, implies that the minimum size at current height isn’t 60 bytes, BIP34 actually increases this by 4 bytes. The extranonce adds 9 bytes, and a nonempty output script (23 bytes for P2SH) brings a generally used coinbase up to 96 bytes, or 98 for P2PKH.

Following the suggestion from the problem statement to make a transaction any size other than 64 bytes we can see that this has no effect on normal mining where transactions are almost always larger. This affects only the simplest of transactions, and only very specifically crafted ones, and thus have a minimal impact on future users while still making the attack impossible due the the fact that only 64 byte transactions are described to have this issue.

No rationale was given for forbidding transactions lower than 100 bytes, or indeed lower than 64 bytes, as such it seems useful to re-allow them and avoid valid and non-harmful transactions to be forbidden. Future growth of the network is maximized by minimizing the number of rules.

As such we conclude that forbidding only transactions of exactly 64 bytes is the simplest way to protect the network from this issue with minimal impact both technically as well as a minimum harm to future opportunities for users.

Stakeholders

Who do we need to consult for opinion, or at minimum inform of this change.

The full node implementations are the ones currently validating this rule, they reject blocks with a transaction below the 100 bytes threshold. As such they need to implement this change.

SPV wallets can not create transactions below the 100 bytes limit anyway, as such they are not expected to have been affected by the introduction back in 2018, they will not be affected by the change.

Mining software already made for BCH currently forces the coinbase to be at minimum 100 bytes, while they do not need to be updated (their current behavior will stay compatible) it would be useful to inform them this rule is no longer so strict. This also helps people migrate BTC software as the rule is less strict there than it is on BCH.

Proposal

The current network has a rule of 100 bytes per transaction minimum. This rule will be revoked.

In its place we require that any transaction in a block can not be exactly 64 bytes.

5 Likes
Cash Improvement Proposals (CHIPs and Proto CHIPs) List
#2

Thanks for reminding about this issue, @tom!

I would suggest changing this sentence in the opening paragraphs:

We suggest to simply refuse transactions of exactly 64 bytes instead.

to

We suggest to simply refuse transactions smaller than 65 bytes instead.

That’s in line with what you propose further down anyway.

This is to be changed in a future protocol upgrade to be set to a minimum of 65 bytes.

#3

I agree with freetrader it’s unclear whether you want to permit e.g. 63-byte txs. Also:

This rule, in combination the existing BIP34, effectively means we can drop the minimum transaction size as 65 bytes is the minimum size possible for a valid coinbase.

This is confusing - the original vulnerability presents a 64-byte non-coinbase tx. So the rule cannot be dropped even if no coinbase tx of 64 bytes can be constructed.

An explanation of both the smallest possible coinbase tx and the smallest possible non-coinbase tx (ignoring minimum tx size) would add some clarification.

1 Like
#4

It’s worth noting that there have been tens of thousands of 62 byte transactions. They were done for a good cause, which was to clean up UTXO set: https://blockchair.com/bitcoin-cash/transactions?s=&q=size(62)#

Furthermore:

5 Likes
#5

Thanks guys, I definitely got myself confused halfway through writing and researching the proposal and switched between minimum and “not 64” a bit.

I think from these excellent replies it will make most sense to change this to be a proposal to forbid exactly 64 bytes. I’ll update the initial post (notice you can check diffs under the pencil icon of a post).

#6

Note that I worked with a mining pool recently that was installing BCHN and they were / are primarily a BTC pool (no surprise there). They forgot that BCH requires 100 byte coinbase tx and ended up mining something like >100 dud blocks (on testnet, thankfully!). They were about to blame BCHN for this before realizing the 100 byte rule.

So the pain is real.

I think >= 65 bytes is simple and solves the problem. 100 is just too high for coinbase tx. You have to actively pad your scriptsig in many cases. Forgetting to do so on some pool software is not uncommon.

6 Likes
#7

Another factoid just to be clear on it: a modern barebones-but-practical coinbase might have (in addition to the 60 byte tx overhead): BIP34 (+4 bytes), extranonce (+9 bytes), and nonempty output script (+23 bytes for P2SH). That works out to 96 bytes, or 98 for P2PKH.

The only reason we didn’t often see these prior to the upgrade was that miners tend to stuff extra messages / stuff in scriptSig, and sometimes they have multiple outputs. However as Calin notes if you are doing a common P2PKH coinbase (98 bytes) then the 100 byte rule means you are forced to pad.

Stats on <100 byte coinbases: https://blockchair.com/bitcoin-cash/transactions?s=&q=size(..99),is_coinbase(true)# . Note how in early days we had no BIP34 and smaller extranonces so coinbases were often even smaller (smallest ever was 91 bytes).

4 Likes
#8

Thanks guys, I updated the initial post to clarify the proposal and background and I stole the best ideas and points from your replies, hope you don’t mind :slight_smile:

#9

Instead of allowing anything unequal to 64, I would prefer simply changing the minimum to 65. This way, the range of valid transaction sizes remains a continuous interval without gaps, reducing footgun potential.

1 Like
#10

I don’t really have a strong opinion either way.

As I wrote before, the idea that people can create (if they really know what they are doing) less than 64 byte transactions, then why not allow them?
As “normal” people never end up creating transactions less than 64 bytes, the issue is really not something that is visible to, well, normal people.

This is why I followed the idea that its better to allow as many types of transactions as possible and move the usability issue to the wallets or the TransactionBuilder class etc.

1 Like
#11

Mark just give an example of why a 62 byte transaction was good. Is there a reason why would you want to disallow that?

1 Like
#12

Yeah just to continue this, I honestly don’t see any serious problem with making the rule “>64” vs. “!=64”. I tend to lean for the latter just because it is more precise.

  • The 62-byte transactions could have easily combined together by the kind miner doing the UTXO cleanup. They only needed 42 bytes per cleaned-up input. It’s a very rare situation and not a hard obstacle.
  • There are only 17 possible 63 byte standard transactions (using different 1-byte P2SH scripts: OP_1 to OP_16, and OP_1NEGATE), and honestly I can’t think of any actual use for these.
  • Both of these small txes, which output to OP_RETURN, can be padded with a few bytes after the OP_RETURN to be >64.
  • No wallet is affected by this, not even by the 100 byte limit, since they will always spend signed inputs.
  • A fun aside but not relevant for us: if we had segwit then the smallest practical tx size would be 61 bytes (burning to fees by spending from a bech32 address, with OP_RETURN output – the witness-stripped legacy tx, as inserted in merkle tree, would be 61 bytes, even though the full tx would be >64 vbytes).
3 Likes
#13

In my opinon, the current “stakeholders” section is more specifically a “costs” section.

For stakeholders, I think it is necessary to reach out and find out who would actually want to make transactions that might be impacted here. Why do they make them? Would this cause any problems for them? Would this create any value for them?

1 Like
#14

The answer is in the problem statement: primarily the coinbase transactions made by miners. See also Calin’s reply.

#15

I agree, except I think it’s pools more than miners. I want to make sure that @tom reaches out to make sure at least one or two of their voices are included here.

2 Likes
#16

Devil’s advocate: So far the only real usecase I can see is the one presented by @cculianu : Some miners transitioning from BTC with new pool software may have have to additionally check for > 100 bytes. Established pools, I assume, already made the necessary adjustments.

UTXO cleanup, while a nice act in the distant past, does not seem to present new significant benefits.

On the other hand, changing consensus, even for a small change like this, incurs non-zero costs across full nodes. Furthermore, setting a precedence where consensus changes are not taken lightly and require at least a significant usecase to be deployed, I think, has benefit in itself.

I’ll suggest that we should at least find someone who is impacted by the 100-byte limit right now , and wish to have it relaxed going forward (instead of “I wish the limit was never imposed”), before we proceed.

2 Likes
#17

My reason is philosophical rather than technical.

I don’t like this thinking. It is similar to, but not quite the same as, “I can’t think of a use case therefore one must not exist”.

As a principle, I think we should in general try to be as inclusive as possible. We know that something is wrong with specifically 64 bytes, so we disallow that to prevent harm to other users. We do not have data to show that something is wrong with other byte amounts so we allow them.

2 Likes
#18

It’s likely beneficial to be as inclusive as possible when picking two possible changes with the same costs. Comparison between change and no change, though, should not follow the same line of thought, for changing the rules has an inherent cost over not changing.

In other words, if today we’re in a situation where “we need to restrict to something tomorrow, do we restrict to >100 or <>64?”, then <>64 is likely a reasonable choice given what we know. We are, however, in a situation where “status quo is restrict to >100, do we do nothing, or change it again to <>64?”; I’d argue that the latter has a higher bar in terms of justification.

#19

The benefit of the change is indeed small, but so is the cost. I think the small benefit exceeds the small cost.

I think the best way to settle this is for the proponents to simply start implementing this into the full nodes we have.

1 Like
#20

On Stakeholders:

  • people that build (with) libraries, or any software that interacts with transactions will hit this usecase often in the context of testing.
    As a direct example: in Flowee the Hub there is a test-chain of 115 blocks (regtest) which is loaded during black-box testing in order to do API tests, including double spend proof tests. A chain like this is made purely to test specific features and the chain that was generated before the 100 bytes rule was introduced indeed has some < 100 bytes coinbase transactions.
    Developers (including me) that build mock chains to test their unrelated features, also in the future, will be thankful for one less requirement to have to fulfull.
    Developers of software completely unrelated to mining may test their software and end up with test transactions that a full node rejects, quite unexpectedly, due to these size issues.

  • Miners trying to reuse their BTCs software that has an 80 bytes (IIRC) limit will like the avoidance of yet another restriction.

From my point of the view the benefit is quite substantial, the reason for this is simply based on the fact that we have no documentation for this or most of the consensus rules. An unexpected consensus rule that is tighter than elsewhere has the potential to be super expensive because a block may be rejected by the network. When you find the issue on testnet that is fine, but with a magic threshold there is a good chance that this will only be found a week after going into production.

For all we know half the mining software on BCH doesn’t actually implement this rule because normal usage never hits it. But a simple user config change may make a block hit this limit with the result of a miner losing his block reward.

And, I do agree the cost is low. But we can certainly help to make it lower and I do agree with the suggestion for the product owner (PO) to facilitate that.
I have done the implementation for the DSProof and worked with all C++ implementations to get it in there, I would have no problem working with the wider ecosystem to change the 100 bytes rule according to what we end up with as new rule.

2 Likes