Non-standard transactions & out of band miner submission

Note that exploiting standardness vs consensus gaps does have other edge cases and impacts.

New problematic issue with exploiting the gap between standardness relay policies & consensus has emerged.

It’s become trendy on BTC for miners to include (non standard) sub 1 sat / vByte transactions in large batches (driven by Ordinals/Inscriptions demand, and a regular lack of competing “regular” transactions at 1 sat+ rates). Note: Bitcoin Core is currently 85 ish% of the network, Knots has grown to about 16%, although in this case I don’t believe there is a policy distinction (yet, it may be imminent). Regardless, because these transactions are outside Bitcoin Core policy, lots of node operators (including miners presumably) have to backfill these unheard transactions after a new block filled with them is published. This has massively reduced the benefit of compact blocks, as requesting maybe 10kb of new transactions has in some cases shot as high as 0.8 MB.

The Bitcoin Core plan is to lower the min relay fee for standard transactions, which is probably a good response, albeit 1) it’s kind of avoiding the greater issue that BTC is not only failing to drive the needed “oil tanker” fees to sustain security budget in the long run on a tiny blocksize it’s actually moving in the opposite direction with increasingly lower fees and 2) it’s upset the Knots filter guys who are becoming increasingly aware of the hijacking now that their desired filter changes to block Inscriptions are being continually ignored while filter changes to facilitate Inscriptions are being fasttracked.

In any case, the upshot for us is that this seems to be another data point that shrinking the gap between policy & consensus is a good idea to avoid this fragmentation of mempools encouraged by varying relay policies and/or incentives for miners to include non-standard/widely unheard transactions.

Edit: One line summary that this is a BIG impact

In June, we were requesting less than 10 kB worth of transactions per block on average for about 40-50% of blocks. Now, we are requesting close to 0.8 MB worth of transactions per block on average for about 70% of blocks.

3 Likes

This is a misunderstanding of the situation. The implication seems to be that if all policy rules become consensus rules that this issue would not exist.
Maybe so, but minimum relay fee being consensus is the natural result of that thought process. And that can’t be the intention.

Furthermore, relay rules are not limited in any way to fee level. The inclusion of certain op-returns has been a debate on BTC and is a relay rule too.

In Bitcoin Cash I expect a lot of innovation to happen on the side of the mining software where they pick and choose what to include. For instance higher days-destroyed may allow lower fees.

So, this is a misinterpretation of the issue.
But more importantly the solution has been known for some years, it’s just that nobody has implemented it yet. See;

Here a post from 5 years ago detailing it;

Specifically (bold mine):

Things are continuing to fall apart on BTC. The “Knots” guys have cottoned on to & accepted the Hijacking (apparently if the company is called “Chaincode” then that’s a real corporate takeover funded by bankers but if it’s called “Blockstream” it’s a conspiracy theory lol), but they still have loads of propaganda & are in a very confused state about everything while they sort through it.

As part of that, the latest panic is that Bitcoin Core’s upcoming change in v30 (coming next month in October) raises filters to allow 100 000 byte OP_RETURNs by default (instead of 80 as now, 40 on Knots & 220 on BCH). This matches the 100 000 consensus limit on both chains.

The panic is that this opens the floodgates to people posting child porn into the blockchain, which they are then required to process and validate while holding it in mempool. See this 1 hour video. There is of course already such material obscured in public keys and so forth since 2011 or earlier - and arbitrary data up to 100kb OP_RETURNs could already be got in around relay policy today (albeit with some culpability trackable to the miner who included it).

Nevertheless, it does once again raise interesting questions if the BCH community should consider lowering the OP_RETURN consensus limits from 100 000 bytes? Should consensus be the same as the current standard relay of 220 bytes? Or is there a better number lower than 100 000 it should be set at? What is the need for having consensus allowing such large OP_RETURNs? Is anyone using it? Can we close this issue with “undesirable” uploads to BCH (at least to the point of having increased “plausible deniability” that we’re operating a p2p cash system and not a free-for-all global hosting network of such material) before it becomes an issue for us?

3 Likes

Note this concern about such material is not theoretical. As per usual, BSV has already led the way by abandoning all reason on this topic and then immediately receiving the consequences of their actions. Source

We can learn from our BTC friends on this subject. Here are some links to existing discussion on their side (thanks @StackingSaunter on X):

Search results for 'Spam' - Delving Bitcoin

https://groups.google.com/g/bitcoindev/c/d6ZO7gXGYbQ/m/GRKe8kq-EgAJ?pli=1

1 Like

I’ll just voice my opinion here that I am generally in support for closing the gap between standardness and consensus.

My understanding is that there is only a limited number of differences remaining, and while addressing them is certaintly not a high priority, I would welcome anyone who champions a CHIP to address any or all of those differences.

Ideally I would prefer each difference to be addressed by a separate CHIP, and I consider progress towards the overall goal of closing the gap to be valuable even if we never reach a state where all differences are entirely eliminated.

4 Likes

The discussion on filtering on BTC has literally nothing to do with BCH or with standardness rules, so not sure why that is being added here. It feels political. “Look at this bad effect, lets change this completely different thing because that would safe us!”.

The truth is that even if 100% of consensus and standardness are in agreement, miners can and should still be able to pick which transactions they include. That is required for an open market to work.
For instance a consensus rule on which fee is to be paid would not only be impractical, it would lead to bad effects economically and centralization and raising costs as a result. Look price fixing up as a economic / government topic if you want to know why this is. (also covered in “economics in one lesson” from Henry Hazlitt).

Filtering transactions is basically just free market at work, miners pick what they mine for reasons of their own. If you are against this, you are against free markets and you should likely not use Bitcoin Cash at all.

I prefer keeping fees and other aspects that the node can adjust out of the consensus rules. Perhaps we could establish a best practice for the node policy, but each miner should have the freedom to choose what they include in their transactions, and the free market should ultimately decide. Node can restrict further, but the protocol should leave everything open

It could be worth it to try and revisit the opreturn standardness topic in 2026

currently we have a very strict standardness limit of 220 bytes which prevents some useful items (like storing P2SH contract params for an AnyHedge contract) but no special consensus enforced rules (which can also be problematic like we saw on BTC)

and as Jason has mentioned on the P2S topic, according to the original rational with a correct calculation, the limit should arguably have been higher:

I think the mximum tx size topic and minimum fee topic are probably less important for the coming year

3 Likes

I don’t even know what the complete set of policy (and/or consensus) rules is, it would be nice to get a full broken down list so that then we could start researching them one by one.

But as always, a bit of a /someone problem.

Some smallish tweaks to policy or baking them into consensus or something could make good candidates for the next upgrade given that besides 1 minute blocks there isn’t much on the table at the moment.

Checking whether Standardness Quirks of BTC still apply

Motivation

The new “BinoHash” idea for BTC is surfacing very unexpected quirks about non standard transactions again. It is a very hacky way to enable a narrow BitVM-related covenant use case, and it relies on old Bitcoin technical debt around FindAndDelete during signature hashing.

At the end of the podcast At the end of the podcast “BitVM3 & The Garbled Circuits Revolution | ROBIN LINUS & LIAM EAGEN”, the BinoHash non-standardness mechanics are laid out

Findings

Verified against the official BCH upgrade specs (with help from GPT and Claude):

1. BCH removed FindAndDelete from the active signing path in 2017.
The BCH replay-protected sighash spec (UAHF 2017) is explicit: “Contrary to the original algorithm, this one does not use FindAndDelete to remove the signature from the script.” As a consequence, “it is not possible to create a valid signature within redeemScript or scriptPubkey as the signature would be part of the digest.” (spec)

The exact BTC-style trick surface that BinoHash relies on is simply not available on BCH’s modern signing path.

2. The old FindAndDelete code path is not reachable for valid modern BCH transactions.
The old behavior only applies when SIGHASH_FORKID is not set — but since UAHF activation, transactions without this flag are rejected by consensus. The code exists in the codebase but cannot be exercised by any valid modern BCH transaction. The 2019 Schnorr multisig spec makes this even more explicit: “The findAndDelete operation only applies to old transactions prior to August 2017, and does not impact current transactions, not even in legacy mode.” (spec)

3. BCH’s sighash is structurally different, not just a flag change.
The UAHF digest replaced the signing algorithm with a different structure adapted from BIP143. It commits to the value of the output being spent, uses reusable precomputed hashes (hashPrevouts, hashSequence, hashOutputs), and includes a 4-byte sighash type with an embedded fork ID for replay protection. This isn’t a patch on top of the old algorithm — it’s a replacement. (spec)

4. OP_CODESEPARATOR is the one inherited script-shaping mechanism that survives.
While FindAndDelete is gone, OP_CODESEPARATOR still affects which portion of the script gets committed in the sighash. If present, everything up to and including the last executed OP_CODESEPARATOR is stripped from the scriptCode before hashing. Worth noting for completeness, though it’s unrelated to the BinoHash trick. (spec)

5. BCH repurposed OP_CHECKMULTISIG's historical dummy element rather than just carrying it as dead weight.
The old off-by-one bug that required an extra unused stack element was turned into a mode switch in the November 2019 upgrade: a null dummy element triggers legacy ECDSA mode, a non-null dummy triggers Schnorr mode with a checkbits bitfield for efficient signature-to-pubkey matching. Legacy baggage turned into useful functionality. (documentation.cash)

Conclusion

The BinoHash trick on BTC is a good example of what can happen when old consensus quirks remain available as live surfaces.

The findings above confirm that BCH’s 2017 sighash overhaul structurally removed this trick surface from the active signing path, and the 2019 Schnorr multisig spec went further by explicitly stating that FindAndDelete does not affect current transactions, even in legacy mode. While BCH still has inherited historical oddities worth cataloging, this particular BTC rabbit hole does not appear to carry over to valid modern BCH transactions — a useful distinction to keep in mind when comparing BTC standardness issues to current BCH standardness questions.

3 Likes

Common BCH W.

Small tweaks made years ago in a common sense way unsurprisingly later turn out to have unanticipated benefits as protection against undiscovered exploits.

This is both extremely logical and a surprisingly frequent occurrence.

Great example of why we should look to close any remaining gaps on standardness!