CHIP 2021-05 Targeted Virtual Machine Limits

It has a rationale. Please read the spec section: GitHub - bitjson/bch-vm-limits: By fixing poorly-targeted limits, we can make Bitcoin Cash contracts more powerful (without increasing validation costs). – in particular expand out the " Selection of 130,000 Byte Stack Memory Usage Limit " bullet point.

In summary: 130KB is the limit we implicitly have now. The new explicit limit just preserves status quo.

2 Likes

I worry now that with 10KB pushes, it might be possible for miner-only txns to contain data blobs in the scriptSig. We have 1650 byte limit as a relay rule for scriptSig – but what do people think about moving that limit to consensus perhaps as part of a separate CHIP?

1 Like

Without having done the in depth research probably required, I am in favour of making consensus rules in alignment with relay rules, I’m still not entirely sure why they’re different. Maybe there is a good reason, so Chesterton’s Fence says not to fuck that up, but it seems like an area the BCH community should be looking into anyway. Of course, we’re all unreasonably busy.

2 Likes

My two cents here: fwiw I am 10000% in favor of just tightening consensus rules to 100% match relay rules now. The fact that the two differ seems to me like the original Bitcoin devs (before they were captured)… were deprecating some things and they intended to remove them. FWIW I think it would make life a lot easier on everybody if consensus tightened to precisely match relay. No surprises. No possibility also of perverse incentives. This is because consensus rules are so liberal they may end up allowing for… shenanigans. Better to plug that hole. My two cents.

3 Likes

I agree.

I am still waiting for somebody to give a good reason “why not”.

Until I hear some strong arguments against, I will remain in support of this.

3 Likes

Two reasons.

First, it is a pipe dream. You can take all todays relay rules and make them consensus rules. But tomorrow someone may introduce new relay rules. You’re back to square one.
Or, in other words, if your intention is to equalize a decentralized system, you’re doing it wrong.

Second reason is that the standard rules (here named relay rules for some reason) are tightening of the consensus rules mostly beyond what is reasonable in a proper free market system. To be clear, we don’t actually yet have a free market system. Which is why those tightened rules make sense today.

A set of properties are currently not possible to be adjusted by the free market. There isn’t enough volume for one, but more importantly there is no software that exists that allows those properties to be taken into account when doing Bitcoiny things (like mining).

Make those tools and then make the properties like op-return size, script-type, script-size etc etc able to be limited by the free market, and you don’t need them as standard rules anymore, and certainly not as consensus rules.

I don’t understand, I think you meant the reverse.

Take all the consensus rules and copy them to the relay rules (source relay rules from consensus rules).

Isn’t it?

Then you achieve 100% coherency in the decentralized system plus also you increase consistency too, since the system is more “obvious” and transparent about everything.

You seem to think that “standard” rules (you call relay rules) are somehow set by a central party and not somehow decentralized.

The reality is that any miner or merchant can add new rules to their node on which they reject transactions.

To repeat:

if your intention is to equalize a decentralized system, you’re doing it wrong.

No you’re not. The entire reason for a blockchain is to have decentralised agreement on an “equal” set of transactions.

Some things can be user adjustable sure. But some things can’t. We’re just discussing what falls into each category.

Maybe today there is quite a lot of flexibility on relay rules. In future, perhaps the “default” is to have a much less flexible set, everything being more likely consensus.

Does that mean we should or could somehow BAN people doing stuff on their own node? No, of course not. Does it mean we could benefit from having more sensible defaults so that the likelihood of discrepancies is smaller? Yes, absolutely.

Same as ABLA. Do you NEED to use ABLA, could you instead just manually tweak your own blocksize to be above the limit? Yes. Is that what most people are likely to do? No, they’ll just use ABLA & that’s brilliant.

Having better sensible defaults is not the same as attempting to ban or deny some kind of configurability, and having sensible defaults is a really really good idea (and trying to prevent progress on better defaults by trying to conflate them with an attempt to ban alternatives is not).

2 Likes

Yup, anyone can patch their node to make their own node’s relay rules tighter than consensus.

Can we really expect that people would start experimenting with relaxing relay rules on their own? AFAIK the only precedent of this happened with Inscriptions/Ordinals where some pools created a way to get some non-standard TXs mined.

Those are what you call consensus rules. Relay rules are more like: “I won’t bother with this TX for now, even though it is valid. I’ll get it later if it gets mined.”

This. If the default is that relay rules are much stricter than consensus, then some pools relaxing it on their own could put others at a disadvantage, because everyone else would have to download a bunch of “unseen” TXs once the more lax pool would announce their block.

Q re. impact on DSPs: if some edge node sees a TX violating its relay rules and trying to double-spend a TX which was relay-compliant, will it still generate and propagate a DSP?

If the default is that relay rules are == consensus by default, then whomever patches their node to make relay more strict would put themselves at a disadvantage (because he’d have to later download a bunch of TXs all at once on block announcement).

You might want to refer to the earlier post instead where this was addressed as “second reason”:

I don’t know, the only reason I pointed this out is because the GOAL of equalizing a decentralized system is not really tenable. It should not be a goal just like you don’t have as a goal to create a language that all people talk. The moment you think you succeeded, someone WILL change something. So, I repeat, if your GOAL is to equalize a decentralized system (a group of random people) you’re doing it wrong. Instead, embrace the chaos and make sure you’re capable of dealing with failures.
This is like making sure all cars drive the same speed, they won’t. Not as long as you have people driving them.

Oh, I get your point now.

Sure, what you are saying might happen, but it most probably won’t happen.

Miners are the most following part of BTC and BCH ecosystem. It has been the rule for the last 8 years. I am not seeing this changing.

So, what will happen is miners will follow whatever rules BCHN makes by default, like they follow the fee inclusion rules right now. Unless we break stuff and make life hard for them.

So, historically and psychologically/socially your argument is unfortunately completely incorrect.

It is usually a good decision to increase consistency in a decentralized system, it makes it easier for system participants to stay in sync. Assuming they will follow, which they most probably will.

1 Like

:+1:

This is why increasing consistency and transparency is the way. Having an incosistent or not transparent decentralised system is a quick way to create chaos and trouble for ourselves in the future, like in the example you gave.

Sourcing the TX relay rules from TX fee inclusion rules is one of the ways to create more transparency and consistency.

Generation of DSProofs is done by a full node that already has one in its mempool, and a second is received.
All nodes will propagate DSProofs where at minimum one of the transactions are in the mempool.

So, to answer your question, all nodes will propagate the dsproof in your scenario.

1 Like

On BCHN Slack, I asked what’s the status of this, and Calin said:

i’m still waiting on Jason for final spec… but we have benchmarked some things and so far he’s pleased that stuff that is slow for him in libauth is slow is BCHN and fast in libauth is fast in BCHN so he’s confident he can use libauth’s relative performance to come up with the spec.

@bitjson could you give us an update, let’s bring this over the finish line?

3 Likes

I will be doing a Podcast episode with @bitjson this Friday to discuss in detail, I believe he plans to time it somewhat with a release of an updated version, so you can look forward to that.

4 Likes

Hey everyone, thanks for your patience while I was working on this over the past few months.

I’m still working on cleaning up a PR to the CHIP, I’ll hopefully have something up shortly. There was some related discussion in Telegram recently, so just wanted to post some notes here too:

  • My 2021 benchmarks considered hashing to be much more relatively-costly than other operations, so I had misjudged the impact of increasing the stack item length limit on the cost of other operations. Hashing is still generally the worst, but there’s another class of operations that realistically need to be limited too.
  • However: it turns out that simply limiting cumulative bytes pushed to the stack is a nearly perfect way to measure both computation cost and memory usage. I’m just calling this “operation cost” for now.
  • O(n^2) arithmetic operations (MUL, DIV, MOD) also need to be limited: it works well to simply add a.length * b.length to their “cost”.
  • Hashing I still think is wise to limit individually, but should also be added to cost (right now I’m using the chunk size of 64 * hash digest iterations)
  • And then to avoid transactions being able to max out both the operation cost and sigchecks, we’ll want to add to the operation cost appropriately when we increment sigchecks. (I’m still benchmarking this, but simply adding the length of the signing serialization seems very promising, and even resolves the OP_CODESEPARATOR concerns).

I’m still working on the precise multiplier(s) for operation cost (the goal is to derive from just beyond the worst case of what is currently possible, maybe even reigning some things in via standardness), but it will simply be “N pushed bytes per spending transaction byte”.

I also now agree with @tom’s earlier ideas that we don’t need the 130KB limit or even to track stack usage at all. Limiting the density of pushing to the stack is both simpler and more than sufficient for limiting both maximum overall memory usage as well as compute usage.

I’ll post a link here as soon as I can get a PR to the CHIP open. For now you can review the benchmark work for:

5 Likes

Also want to share logs from a recent benchmark run here (and also attaching a formatted image of just the BCH_2023 VM results for convenience):

The benchmarks use Libauth’s bch_vmb_tests (BCH Virtual Machine Bytecode) file format described in more detail here. We mark some tests as benchmarks by including the string [benchmark] in their description field.

One of the benchmarks (trxhzt) is also marked with the string [baseline]; that is the most “average” possible transaction, including 2 P2PKH inputs (one Schnorr signature, one ECDSA signature), and 2 P2PKH outputs, useful to illustrate the expected baseline “density” of computational cost in transaction verification (validation time / spending transaction byte length).

In general, our goal is to tune limits such that valid contracts do not significantly exceed this “relative time per byte” (RT/byte) density. There are currently other standard transactions with higher RT/byte values (see 1-of-3 bare legacy multisig, first slot, but note that the Libauth results reflect Libauth’s lack of signature or sighash cache; BCHN’s results here are much closer to baseline).

Libauth’s implementation of both the VM and benchmarking suite print out quite a bit more information than any other implementations need: single-threaded validations-per-second on the current machine (Hz), Relative Time (Rel. Time), Operation Cost (Op. Cost), Hash Digest Iterations (Hash D. Iter.), Pushed Bytes, Max Memory (Max Mem.), Arithmetic Cost (Arith. Cost), Bitwise Cost (Bit. Cost), and SigChecks (according to the 2020 upgrade spec). Each of these also has a column where they are divided by transaction byte length to provide relative densities. These columns are useful for understanding what is happening and designing new benchmarks, but other node implementations only need to determine RT/byte to validate their performance across all kinds of contracts.

You’ll also see some outdated benchmarks from previous iterations of the proposal (before we began constraining compute/memory by density of pushed bytes) – I’ll delete those from the final set soon, but wanted to first publish them and share the results to make it easier for others to validate that counting pushed bytes is a more reliable approach.

3 Likes

And thanks again to @BitcoinCashPodcast for having me on to talk about this work yesterday. We covered some more detail and some of the historical context for the current limits here:

There is more benchmarking to do, but we now have a solid enough set of tests to make confident statements about worst-case performance. Next I’d like to work with other node implementations to get benchmarking implemented and verify that this latest approach is the right direction.

When we’ve figured out the important constants and locked things down a bit more, I plan to continue building out benchmarks until we have a full combinatorial set of tests covering all opcodes (and every evaluation mode of the more complex operations) that can become part of continuous integration testing for all BCH implementations.

7 Likes

Hi all,

The CHIP is updated! This reflects all the research since February, feedback and questions are appreciated:

A TL;DR of the whole CHIP if you’re looking at a Satoshi-derived C++ implementation like BCHN:

  1. MAX_SCRIPT_ELEMENT_SIZE increases from 520 to 10000.
  2. nOpCount becomes nOpCost (primarily measures stack-pushed bytes)
  3. A new static inline pushstack to match popstack; pushstack increments nOpCost by the item length.
  4. if (opcode > OP_16 && ++nOpCount > MAX_OPS_PER_SCRIPT) { ... } becomes nOpCost += 100; (no conditional, covers push ops too).
  5. case OP_AND/OP_OR/OP_XOR: adds a nOpCost += vch1.size();
  6. case OP_MUL/OP_DIV/OP_MOD: adds a nOpCost += a.size() * b.size();
  7. Hashing operations add 1 + ((message_length + 8) / 64) to nHashDigestIterations, and nOpCost += 192 * iterations;.
  8. Same for signing operations (count iterations only for the top-level preimage, not hashPrevouts/hashUtxos/hashSequence/hashOutputs), plus nOpCost += 26000 * sigchecks; (and nOpCount += nKeysCount; removed)
  9. SigChecks limits remain unchanged; nHashDigestIterations and nOpCost also get similar density checks.
  10. Adds if (vfExec.size() > 100) { return set_error(...

Compared to v2.0.0 of the proposal:

  • There’s no need to check total stack memory usage after each operation, limiting pushed bytes is both simpler and more effective.
  • The limits are no longer static per-input, they’re all density-based like SigChecks. (Of course, they were always density based, but now we set them explicitly rather than as an implied result of worst-case transaction packing. :sweat_smile:)

And that’s it.

The constants are all derived from existing limits and (re)calculated as densities. In essence, this carries forward the performance testing done in the 2020 SigChecks spec, and also reduces worst-case hashing costs by an order of magnitude, setting a lower standard limit at the “asymptotic maximum density of hashing operations in plausibly non-malicious, standard transactions” (from Rationale: Selection of Hashing Limit).

So the elevator pitch is now: “Retargets limits to make Bitcoin Cash contracts more useful and reduce compute requirements for nodes.”

6 Likes