Non-standard transactions & out of band miner submission

Consensus
#3

Alright, here I go with some opinions. I’ll qualify this immediately by saying I’m sometimes wrong, I am not a Bitcoin historian and I don’t have as deep an appreciation of why things are the way they are, as some others in the Bitcoin Cash community.

I’ll do your qestions in reverse:

Why is there a separation between consensus rules & node relay policies

IMO: the relay policies offer a protective layer with two functions:

  1. originally, to restrict the mainstream use cases to those transactions which is better understood in terms of performance impact on the network, i.e. more safe to scale
  2. layered defense - policy changes can often be adapted quickly and independently by node operators to mitigate new threats. Changes within policy scope do not cause chain splits. Whereas consensus is consensus - changing it is harder and requires wide coordination.

Is there a “good reason” for non-standard transactions?

I would see it in being able to push the envelope in terms of new use cases that can be allowed to develop while being challenged in the real economic environment - something which may not be revealed on testnets despite best intentions. Sort of an entry gate into later becoming accepted into standardness once they’ve proven themselves to be non-problematic. In practice I’m not sure we can say that there are good enough checks and balances in case some non-standard transactions turn out to cause problems.

But maybe someone else has a better reason than I do for non-standard. Of course the dichotomy arises directly from having the distinction between what policy permits and what consensus permits. Non-standard being just the term for that which is allowed by consensus but not by relay policy. So this boils down to “why allow such a bifurcation between relay policy and consensus in the first place”, which is again your first question, so … somewhat circular.

Should we also consider fixing non standard issues by changes straight into consensus? Do we need to, or have we already?

We can consider, but I don’t know that the need is very great, indeed if we have to ask this question. I do hear some people argue in certain specific cases, such as the discrepancy between max standard tx size vs. max consensus tx size, that this poses difficulties for some protocols and an alignment would be good.

Alignment would generally reduce complexity in code bases, and I would consider that a large benefit but it has to be done very thoughtfully of course. One doesn’t want to haphazardly relax rules that might serve a protective function, without being quite certain that exploitation is impossible or costly enough to deter.

And any alignment of policy <-> consensus comes with the cost of changing many codebases that have baked these rules in either way.

Is this something the BCH community should try to be ahead of the game on?

We should be ahead in all areas, so my answer is YES and I love the initiative to table this for discussion.

However, I think it’s less urgent and a much broader and long-winded topic than looking at the VM limits (CHIP) which has an analytical approach already and is more well-defined in terms of the benefits. I don’t think we in BCH have a burning issue here, unlike Core and the BSV crowd. For now, our challenge is to not fuck things up rather than unfucking things, and I think by filtering changes through the CHIP process carefully enough, we might be able to keep it that way :slight_smile:

4 Likes
#4

Thanks for opening the topic @BitcoinCashPodcast. Copying my thoughts from BCHN Telegram for linking:

I wrote about where I’d like to see standardness further relaxed here:

and some of the issues we need to fix first (VM limits CHIP gets us pretty close). Most notable is that current standardness forces covenants and non-interactive multiparty contracts to waste at least 20-32 bytes per TX input.

I’d say it’s hard to summarize into “standardness good” or “standardness bad”; right now I think standardness is an excellent Fence, I wrote more about it here: Standard Vs. Non-Standard VMs

Over time I can see standardness fading away as we figure out more about contract design requirements and pick away at rough areas. I can also see standardness being a permanent and important part of upgrading the network with less downstream technical debt. I doubt I’ll be certain about either way any time soon.

For standardness issues like transaction fee rates and dust limit, I really like the ABLA approach of taking something with soft consensus and firming it up into a long-term reliable consensus rule. It seems like most of standardness could be replaced that way with enough research.

I’ll just add that BCH has done extremely well on stewarding the VM design and consensus parameters; it’s hard to even summarize all the technical details that BCH “got right” vs. various other bitcoin splits. The P2P cash community obviously had game theory missteps leading up to the split and loss of BTC, but it’s a nice silver lining that the ecosystem is more technically advanced and harder to kill now than we might have been without a remnant phase.

A VM example that comes to mind:

5 Likes
#5

Thank you for opening this thread, it is indeed a topic that every now and then re-surfaces and todays discussion on telegram gave me the impression that even though people approach it from different directions the ideas and “what next” kind of thinking are remarkably similar.

The first thing to point out is that ‘standardness’ is a collection of rules. They cover a wide range of topics and it is not really useful to discuss them as a whole.

I echo Freetrader’s general points. The reason they are in standard instead of in consensus is because that makes changing them decentralized, which means that changes do not need a coordination event. No protocol upgrade needed. I think that is a good thing, especially as I still hope we can slow down the protocol changes to a trickle over the next decade or so.

In a large number of cases I’d say that what is a standard rule today is because of some problem elsewhere. We have seen miners create transactions of well over 100kb in size in order to consolidate outputs. We have a dust limit in the standard rules too. We also have an op_return limit in the standardness rules.

Those 3 together are there because there is no economic balance, no tools or policies present that allows these items to be decided by “the market”. The issue is best explained by going back to this blogpost by Gavin Andresen from 2016 One-dollar lulz

What the blogpost shows is that there had to be a max blocksize because it was too cheap to create a block, meaning there was no incentive against creating blocks others might reject.
The balance only started working when the reward for a single block went up to a significant amount.

Back to those 3 standardness rules: tx-size, dust & op-returns.
I postulate that we can provide the tools to allow the market to set the cost for every single transaction. It can be based on things like op-return size, it can be based on amount of dust transactions being consolidated into only a couple of outputs and naturally the fee that needs to be paid. And in such a world the standardness rules are no longer needed.
You don’t need a dust limit if it becomes nearly free to find old dusting outputs to consolidate.
You don’t need an op-return limit if the cost of creating such a transaction is many-fold the cost of an economic transaction.

Here too I’d like to refer to freetrader again, as I fully agree with that:

1 Like
#6

Here’s an additional case we might want to consider regarding Output Standardness (P2PKH, P2SH, P2SH32). It can be worked around, but is inelegant and adds size/op-code overhead:

1 Like
#7

So, to start, this is technically a different topic. Yes, they are both standard-rules, but they are not really all that much related.

That said, I’d be supportive of removing the standard rules requiring “only known templates in output scripts” completely. The p2sh wrap makes all things possible anyway, so it’s not really limiting people and your example is a good point why having that check actually hurts.

However, it probably should wait until after the VM-limits CHIP has activated because we likely don’t want to just allow any-length output scripts, and the VM-limits chip could be a great place to synchronize the introductiion of limits.

2 Likes
#8

A fresh round of contention about filter limits & OP_RETURN in BTC land has once again highlighted the problems of consensus/standardness disparity.

We’re already closing some of the gap with P2S CHIP. What about the rest? Is there room for a follow-up CHIP that closes the gap completely? VM Limits is about to ship, so that is no longer in question.

2 Likes
#9

image
image966×1178 136 KB

Seems to be some interest here. CC: @bitcoincashautist @MathieuG @Jonathan_Silverblood

1 Like
#10

Yeah I would like to see an effort made for a ‘great standardness cleanup’

the P2S CHIP removes the output standardness and the ‘input bytecode length’ standardness

This leaves opreturn standardness, dust-amounts standardness and tx size standardness
If we were to align the relay rules with the consensus rules, then the question because which one to keep

the dust relay standardness rules don’t even seem standardized across implementations, see the
standardness-docs.

converge on consensus

  • no max opreturn specific rule, except for max data item?
  • increasing the standard max tx size 10x (from 100kb to 1mb)
  • no dust limit?

converge on relay

  • opreturn max being consensus enforced at 220 but allowing multiple opreturns
  • lower the consensus by factor 10 (from 1mb to 100kb)
  • all implementations converging on the BCHN dust rules?

I’m not sure it will be easy to pick which to converge on or whether we need to consider on a case-by-case basis.

Note that (custom) nodes can still chose to not relay transactions according to any arbitrary criteria, so there is no guarantee that you’ll ever be able to change everyones relay policy (as we see now with Knots) but I think there likely won’t be any BCH node implementation which intentionally wants to introduce custom default network relay rules.

2 Likes
#11

we don’t have to; I think what matters here is that the default is maximally permissive and then node operators / miners can further restrict it if they want

as opposed to status quo of being restrictive and then some operators may relax it and cause headaches for everyone else

I would not include min. fee and dust limit here, that would be dangerous to fully relax because right now there are no consensus limits on those; we may even want to research a good way to set some consensus limits there

2 Likes
#12

Agreed on this. The alternative would definitely would cause more headaches.

#13

I found this a really interesting breakdown as well with regards to the actual reasons for non-standardness transactions here

It’s being used to justify hijacking BTC even further, so caution required, but it’s good thoughts.

Feels like none of them apply to BCH though tbh.

  1. DoS defence is covered by VM Limits.
  2. Incentive alignment isn’t needed in BCH, we don’t have to “policy nudge wallet devs” cos it’s not full of RBF and some nonsense “fee market”. BCH wallet devs just make wallets that work lol.
  3. Upgrade safety also not an issue, because we have the CHIP process for that and can hard fork.

image
image1856×818 158 KB

#14

what problems are people having (on Bitcoin Cash!!) that removing standardness rules solves?

These rules that are left are there for a reason, so to remove one or more, some stronger reason must exist that makes the whole a positive change.

I haven’t really seen anything more than a feelng of cleanlyness.

#15

Prevention is better than cure. We’re in a privileged & fantastic place to NOT be suffering the kind of drama & meltdown that BTC is, as well as having a smaller community more easily able to be aligned (FOR NOW). The gap between consensus & relay policies seems to just be a permanent location for hijackers to target in order to create drama and brinkmanship. If we seal those gaps, we fix the problem before it ever arises.

Note also that increasing BCH success is drawing attention from BTC defectors, converts & saboteurs. “Nobody in BCH wants to fight over policy stuff” - yes, right NOW, but there are very good odds that will become increasingly controversial as we grow, particularly from a pool of people that ARE of very strong opinions on that topic.

Does the potential for issues justify the effort & work needed? Well, that’s what we’re doing in this thread & with a potential CHIP proposal. No different than figuring out ABLA before the blocks are all full. At the very least, it would be nice to have a well-documented layout of what is and isn’t consensus and policy and the specific reasons for each. Even as involved in Bitcoin as I am, I’m not even fully clear & that’s the first step of the process - getting to clarity on the current situation and then secondarily evaluating the risks & potential improvements.

1 Like
#16

hey,
now, not because I’m disagreeing or something. I’m just not following the logic here.

What kind of things can hijackers do with regards to the differences between standardness and consensus?

For instance, transaction size. Maybe we should guard point of sale systems against huge transactions, making the difference between standard and consensus irrelevant.
But at the same time, double spend proofs would still kick in whenever anyone wants to abuse this, making this attack die before it begins.

Can you explain the thing that you worry about?

#17

Create enormous amounts of ecosystem stress & chaos arguing over different node implementations, which nodes allow which policy settings, why one group of devs is changing the defaults to this or to that etc. If you haven’t been following all the BTC drama, you can read up on the links on slide 4 here

What I’m concerned about at the moment is not really to do with technical problems, because as we both know everything is currently working fine. It’s all to do with social contention - which is far harder to solve once it gets rolling with a significant amount of non-technical people getting into the debate (which they inevitably do).

#18

Hahaha, that is funny. There is literally zero risk of that in either BTC or BCH.

Look, the miners are extremely risk averse and have for years made it super clear that they ALL want to run the same mining software. So if there is a mistake in one, they will not get rejected by other miners because they run the same.
Ask the Verde guys about this, they made infrastructure to test a block template on multiple node implemenations. It basically got ignored. It wasn’t seen as useful by miners they talked to, because they don’t expect other full nodes to be used by the miners.
Even on BTC there is like one miner with a tiny sub-percentage of hashpower that is politically picking “the other” full node.

So this is a completely empty threat from the other teams.

Does it create drama? Sure, nobody doubts that. But it is completely without merit and that means that it makes no sense to worry about it. Drama based on lies will not go away by technical fixes.

#19

This is the BTC Core writeup of how they think about policy vs consensus. Maybe worth looking into, with the caveat that chains are constantly diverging and they might have propaganda or incorrect assumptions in there too of course.

1 Like
#20

Things are heating up in the mining scene with a new initiative by several pools to join MARA in accepting non-standard transactions!

The three mining pools, representing over 36% of Bitcoin’s total computing power (hashrate), have agreed to support non-standard transactions (NSTs) — a critical piece of BitVM’s challenge-response mechanism, the firms said. Their support removes a key bottleneck to BitVM deployment and brings the system closer to widespread use.

The case for essentially bringing standardness & consensus into alignment growing stronger by the day it seems.

4 Likes
#21

Note that exploiting standardness vs consensus gaps does have other edge cases and impacts.

#22

New problematic issue with exploiting the gap between standardness relay policies & consensus has emerged.

It’s become trendy on BTC for miners to include (non standard) sub 1 sat / vByte transactions in large batches (driven by Ordinals/Inscriptions demand, and a regular lack of competing “regular” transactions at 1 sat+ rates). Note: Bitcoin Core is currently 85 ish% of the network, Knots has grown to about 16%, although in this case I don’t believe there is a policy distinction (yet, it may be imminent). Regardless, because these transactions are outside Bitcoin Core policy, lots of node operators (including miners presumably) have to backfill these unheard transactions after a new block filled with them is published. This has massively reduced the benefit of compact blocks, as requesting maybe 10kb of new transactions has in some cases shot as high as 0.8 MB.

The Bitcoin Core plan is to lower the min relay fee for standard transactions, which is probably a good response, albeit 1) it’s kind of avoiding the greater issue that BTC is not only failing to drive the needed “oil tanker” fees to sustain security budget in the long run on a tiny blocksize it’s actually moving in the opposite direction with increasingly lower fees and 2) it’s upset the Knots filter guys who are becoming increasingly aware of the hijacking now that their desired filter changes to block Inscriptions are being continually ignored while filter changes to facilitate Inscriptions are being fasttracked.

In any case, the upshot for us is that this seems to be another data point that shrinking the gap between policy & consensus is a good idea to avoid this fragmentation of mempools encouraged by varying relay policies and/or incentives for miners to include non-standard/widely unheard transactions.

Edit: One line summary that this is a BIG impact

In June, we were requesting less than 10 kB worth of transactions per block on average for about 40-50% of blocks. Now, we are requesting close to 0.8 MB worth of transactions per block on average for about 70% of blocks.

image
image1016×2258 280 KB

image
image1093×584 123 KB

3 Likes