Iâll add that OP_EVAL is a contract implementation detail, itâs purpose is compression. We can already do MAST-like constructions, we can already run âfunctionsâ (words) multiple times by duplicating the code, VM limits are already solved with or without OP_EVAL, etc.
OP_EVAL just allows contracts to be expressed in fewer bytes (often added by a compiler stage rather than by deliberate design of the contract author).
Thanks for all the great comments @bitcoincashautist â just want to note (because I ed the post) that the analogy still isnât accurate: I see a great variety of uses for â3rd party pluginsâ across multiple interpretations of that phrase.
Instead, Iâm saying that OP_EXEC does not add any âflexibilityâ or âsecurityâ in any possible case.
Stack âisolationâ fundamentally misunderstands contract development. Itâs an âobviousâ solution at the top of a DunningâKruger peak.
I really want to encourage people to get past reasoning by analogy about OP_EVAL and/or OP_EXEC:
Thereâs no security-for-efficiency âtradeoffâ
Bytecode isnât âuntrustedâ or âtrustedâ
This is not a question of âarchitectureâ or âphilosophyâ
There is no âcallerâ nor âcalleeâ
Itâs not a âRISC vs. CISCâ thing
âhigher-levelâ and âlower-levelâ are nonsensical in this context
OP_EXEC is not âisolationâ nor a âsandboxâ
OP_EXEC doesnât add âflexibilityâ, etc.
If you care to understand the topic, drop these mental crutches. Otherwise, hereâs a correct analogy:
The claim here is interesting in that what we are doing on bitcoincashresearch is designing a plaform for random people we have never met to start and do âcontract developmentâ.
That is to say, it makes zero sense to claim some future people will or will not do something. If given the chance, people will do something. I have the entire history of (computer) security to back that up.
Jason then goes and writes:
Bytecode isnât âuntrustedâ or âtrustedâ
which doesnât even attempt to disprove or otherwise debate the actual specific technical detailed post I wrote here. It just states the opposite like it is true.
I do agree with various other points he makes there, though.
There is no âcallerâ nor âcalleeâ
Itâs not a âRISC vs. CISCâ thing
âhigher-levelâ and âlower-levelâ are nonsensical in this context
Itâs just interesting how Jason is pushing for an idea that has been rejected by the bitcoin community a decade ago. Not even touching the ideas that could fix it, because that would mean debating those ideas. Instead we see him undercutting that possible debate by repeatedly making elusive statements about how basic security would not be needed in bitcoin cash because⌠something fuzzy.
Anyway, this debate seems not to go anywhere where it may improve the proposal.
What are you even trying to debate that heâs supposedly âundercuttingâ? Some alternative you randomly came up with that solves non-existent problems?
Youâve been doing the âundercuttingâ by writing word salads and trying to make parallels that donât make sense in this context and then whining and making ad-hominems at Jason whoâs been very patient here and tolerant of your passive aggression.
We all learned the pattern from previous CHIP cyles: you being wrong and everyone having to spend insane amount of energy and time showing exactly how youâre wrong, all the while tolerating your annoying condescenting tone coming from the peak of Dunning-Kruger hill. I donât blame Jason for trying to avoid such timesink here.
I donât know if youâre malicious or incompetent, Iâm leaning towards incompetence, but outcome is the same: itâs a drag.
Your posts here show how clueless you are about how smart contract stuff works and yet you posture as some authority because of some âhistoryâ and expect people to give you the attention and authority you think you deserve. Ironic above you tried to call out peopleâs egos all the while your own ego desperately trying to be relevant.
Sure, need to contemplate it some more since itâs an unknown unknowns kind of thing. OP_EVAL is really powerful, especially with recursion, and I start to wonder if the loops CHIP is needed or that would be syntactical sugar.
Makes sense, please let me know if you come across items we need to address in the CHIP
You can technically achieve some compression of iteration logic with OP_EVAL, but loops are generally much more efficient, e.g. merkle tree validation or aggregations like âsum all input valuesâ.
For a good direct comparison, see the VMB tests for each CHIP (loops, eval).
You can see that OP_ACTIVEBYTECODE + internal conditionals allow us to emulate loops with recursive OP_EVAL, but it requires some overhead (larger transaction sizes), is often harder to review, and cannot iterate beyond 100 without some sort of âtrampoline patternâ to limit control stack depth. (So OP_EVAL is even worse for >100 iterations, e.g. summing all inputs in large transactions.) Depending on the length of the iterated code, the overall cost of pushing evaluated code can also accumulate very quickly and make otherwise-reasonable things impractical (esp. aggregations from all inputs/outputs).
On the other hand, only OP_EVAL can most efficiently âcompressâ any shape of contract, esp. common functions called in disparate locations. While you can technically wrap your whole contract in a giant loop + (ab)use conditional logic to activate and deactivate various code snippets at the right moment(s), that setup is going to produce quite a bit more overhead than simply calling functions in the right locations.
Summary
OP_EVAL and loops each have their strengths, and theyâre often better together.
Loops are ideal for iteration, and OP_EVAL is ideal for compression of all other code sequences longer than 3 bytes.
Iâll also add that logistically, itâs quite a bit easier to implement both Loops and OP_EVAL at the same time: each requires modification to the control stack + carefully testing and benchmarking the VMâs control flow behavior(s) for performance regressions. (Though thanks to the VM Limits upgrade, we already have a very strong benchmarking suite for this purpose.)
Caught up to all the great discussion here and in the âbrainstorming EVALâ thread.
Categories of Potential Use-cases
I appreciate the overview of categories for potential use-cases for OP_EVAL @bitjson!
Contract compression via reusable functions ( common case)
MAST-like constructions
Post-funding assigned instructions ( security theater/use-case covered by CashTokens )
User-provided pure functions ( contrived example )
From the CHIP itâs clear that OP_EVAL is proposed mainly for the âreusable functionsâ use-case, but itâs great to now have an analysis the other potential uses for the opcode also. Especially because MAST-like constructions was amongst the rationale in historic OP_EVAL discussions.
Comparison of OP_EVAL and OP_EXEC
The 'long evaluation comparing OP_EVAL to optimized OP_EXECâ makes it clear that the cases OP_EXEC claims to optimize for are niche use-cases, and when we investigate closely, the âstack isolation featureâ doesnât bring any real benefits to the smart contract designs.
I fully agree with Jasonâs arguments and his conclusions written down in this thread:
Other Evaluated Alternatives
Iâm glad to see there also comparison/evaluation of the old proposals further in the thread!
One alternative I havenât seen mentioned in this thread is that we can emulate OP_EVAL, as I know @bitcoincashautist described in his article. It would be good to explicit mention the benefits of ânative evalâ vs âemulated evalâ somewhere also.
Emulated eval allows mast-like constructions but DOESNâT allow compression, because each âcallâ needs its own input to replicate & execute the code + the overhead of introspection opcodes gluing everything together.
For CashScript we created an issue for reusable functions in Jul, 2023. The issue we were thinking about at the time was to create easy emulation for muldiv which wouldnât overflow in the intermediate result.
Our toy-example (which doesnât contain the real logic) was to allow a .cash library file like the following:
library Math {
function muldiv(int x, int y, int z) returns (int) {
return x * y / z;
}
}
which users could then use in their contract like
import { muldiv } from "Math.cash";
contract Example() {
function test(int x, int y, int z) {
int result = muldiv(x, y, z);
...
}
}
CashScript, a modular programming language
Having callable functions with return types adds a lot of extra scope to CashScript. Weâd now need to figure out import functionality/syntax, function dependency graphs, possibly also dependency versioning etc.
With this functionality CashScript would become a âmodular programming languageâ, which is in line with industry standard but definitely an important expansion from the single-file contracts we have today.
Itâs definitely be a challenge but it would make a âproperâ fully featured programming language from CashScript
User-defined functions
To have complex zero-knowledge programs, we need user-defined functions. OP_EVAL can be a compiler detail to some extent that it is optimizing the resulting code, but for developer experience it is essential users can define reusable function APIs.
As Jason wrote in the âbrainstorming evalâ thread, weâd expect developers to reuse functions in their code, which uses other functions in their code which uses other functions. The smart contract code to implement zero-knowledge proof verification should be readable as if it was written in JS. This logic is not blockchain specific, it is just math/cryptography.
What happens with the result of this function, how it affects the desired transaction shape, would of course be using BCH-specific logic using transaction introspection.
Function APIs
Jason described the problems with OP_EXEC lambda/function evaluation behavior:
I definitely agree with point 1 and 3, but Iâm not sure I understand/agree with 2
letâs discuss how I envision CashScript would use âfunction APIsâ
instead of:
Instead of modifying the existing deep stack items, which might be at variable depths each time the reusable function is called, we just stack juggle the params to the top before executing the function and then leave the function results at the top of the stack. We wouldnât modify deep stack elements.
This is the coding paradigm that CashScript reusable functions would have, similar to functional programming, functions have inputs and outputs and donât have any side-effects of mutating/consuming the existing stack besides the function parameters.
If we would not provide the params up front, the alternative is to try to stack juggle them inside the âevalâ reusable function logic, where there is no guarantee where in the stack things are. functions would not really be re-usable at all.
Stack protection doesnât add any security, but I also donât see CashScript usage modifying the original stack from inside a function, just consuming params on top of the stack & adding back results to the stack
Counting arguments and clean_stack rule
In CashScript we also donât have to check for the number of unlocking arguments provided by a spender, the âclean stackâ rule makes sure that the program would not work if there is more or less arguments provided than what is expected by the program. In the same way we donât have to count the number of unlocking arguments we also donât have to count the number of (user) arguments provided to a reusable function.
I do believe that without any clean_stack rule, these considerations would be different.
That makes sense in the generic case. Compiling code to use OP_EVAL with deep stack elements would be an optimization. Consider the following code where foo(int) is only called in these two places.
int a = 10;
[...]
if(x == 1) {
foo(a);
[...]
} else if (x == 2) {
foo(a);
[...]
} else {
[...]
}
Since the stack element a would be in the same location for both invocations of foo() the bytecode executed by OP_EVAL could very well be compiled to pick it from deep in the stack.
One thing I canât put my finger on is the OP_ACTIVEBYTECODE behavior inside code executed within OP_EVAL. The old specification of it is:
Push the full bytecode currently being evaluated to the stack. For Pay-to-Script-Hash (P2SH) evaluations, this is the redeem bytecode of the Unspent Transaction Output (UTXO) being spent; for all other evaluations, this is the locking bytecode of the UTXO being spent.
What is the reason for changing this? If we kept that behavior and wanted a recursive OP_EVAL the bytecode to be evaluated could be OP_DUP´d at the top of the stack before calling OP_EVAL.
It just feels like an asymmetry that the full stack is available but not the full active bytecode (unless it is pushed to the stack before evaluation).
All of the value in both loops and OP_EVAL can be summarized as âcompressionâ. Our VM can technically compute anything already, but saving KBs or MBs from transaction sizes makes many more use cases practical.
Clarifying review of OP_EXEC
All good, #2 isnât talking about internal âfunction APIsâ but rather a hypothetical kind of secondary, external contract API. (A fuzzy idea that people are reasoning-by-analogy from OP_EXEC, not realizing that CashTokens are an already superior alternative available today.)
Deep stack juggling (usually) considered harmful
Perfect, this is an optimal compilation result for Forth-like syntaxes.
The stack is for piping arguments through multiple words/functions without the overhead of naming and plumbing all outputs to the next inputs. (And this is the reason stack-based languages are better suited for interactive development than static text.)
If your compiled programs do much deep-stack juggling, theyâre probably not optimally factored.
Itâs long been frowned upon to dig deeply in the stack from a word/function definition, for many reasons. From Thinking Forth (1984):
Some folks like the words PICK and ROLL. They use these words to access elements from any level on the stack. We donât recommend them. For one thing, PICK and ROLL encourage the programmer to think of the stack as an array, which it is not. If you have so many elements on the stack that you need PICK and ROLL, those elements should be in an array instead.
Second, they encourage the programmer to refer to arguments that have been left on the stack by higher-level, calling definitions without being explicitly passed as arguments. This makes the definition dependent on other definitions. Thatâs unstructuredâand dangerous.
Finally, the position of an element on the stack depends on whatâs above it, and the number of things above it can change constantly. For instance, if you have an address at the fourth stack position down, you can write 4 PICK @ to fetch its contents. But you must write ( n) 5 PICK ! because with ânâ on the stack, the address is now in the fifth position. Code like this is hard to read and harder to modify.
Of course, the aim of this text is different from BCH VM design: this author is thinking in Forth for application design, while the BCH VM simply uses Forth-like syntax to simplify VM implementation and minimize the byte length of contracts.
Our PICK and ROLL are more useful because we donât have arrays or other data structures (and even if we had them, optimally-compiled bytecode would still use PICK, ROLL, and/or use the alt stack instead).
Regardless, CashScript users can simply think in CashScript, so all else being equal, consensus upgrades should focus on enabling smaller compiled results to be produced by CashScript, Libauthâs compiler (for Libauthâs CashAssembly), etc.
CashScript doesnât necessarily need syntax for OP_EVAL
Yes, as @Jonas noted, some optimizations enabled by OP_EVAL require visibility over the entire contract code.
Iâll go a step further: all optimizations enabled by OP_EVAL are best applied near the end of the compilation pipeline, i.e. CashScriptâs function behavior doesnât need to be tied to OP_EVALâs semantics at all.
In fact, if CashScript were to attempt to produce âcode with OP_EVALâ as a result of function compilations in early compiler stages, the final compiled bytecode will be stuck at a local maximum in optimizing contract byte length.
The ideal behavior is for CashScript to ignore that OP_EVAL exists in early compiler stages, producing a much longer raw bytecode with many repeated sequences of bytes, then feed that long sequence into a deterministic âOP_EVAL optimizationâ function â i.e. optimize(rawBytecode) -> optimizedBytecode (Libauthâs compiler will have one or more of these, so CashScript should be able to simply choose and import one).
Note that unlike earlier CashScript stages, this final stage is capable of taking advantage of the VMâs concatenative semantics: there may be sections of 4+ byte repeats in many contracts which the contract author and earlier compiler stages did not anticipate, and this final compiler stage is in the best place to identify and factor those out, arrange them optimally to minimize stack manipulation, then OVER/DUP/CAT/PICK/ROLL/EVAL to minimize total bytecode length.
As with other compiled languages, contract authors are likely to try to âwork with the optimizerâ to produce better and better outputs, but itâs useful to note that the language itself doesnât necessarily need any syntax for this, and itâs probably better to wait a while for the ecosystem to experiment before trying to add any sort of syntax for âoptimization hintingâ.
So my recommendation: donât try to make functions compile into OP_EVAL. Just make functions work like macros and leave OP_EVAL to later compilation stages. Youâll end up with better-optimized bytecode.
Yes, and in this case you donât want the function to be compiled in at all! If you call it in two places with the same input, the compiler should just move it up and encode the call once, without the OP_EVAL if more efficient (and if possible, precompute, e.g. with CashAssemblyâs $() internal evaluation syntax).
What about OP_EVALed OP_ACTIVEBYTECODE?
Thanks for the question!
First I want to note that itâs not a change from the current behavior.
Right now, OP_ACTIVEBYTECODE returns the âactive bytecodeâ truncated at the last executed code separator. In P2S, this is the locking bytecode, but in P2SH itâs the redeem bytecode â without the âparentâ locking bytecode. Currently the CHIP matches this behavior exactly: within an OP_EVAL, the âactive bytecodeâ is the eval-ed bytecode without any âparentâ bytecode (again, truncated at the last executed code separator).
For more background on why OP_ACTIVEBYTECODE behaves as it does, see:
Should OP_ACTIVEBYTECODE encode the whole âcall stackâ?
On the other hand, ignoring OP_ACTIVEBYTECODE's P2SH behavior, itâs easy to assume that OP_ACTIVEBYTECODE should behave differently inside OP_EVAL, e.g. by stepping through the control stack to concatenate the encoding of every âparentâ active bytecode (presumably separated by OP_CODESEPARATOR) before finally appending the currently-active bytecode.
In this case, OP_ACTIVEBYTECODE would cover the whole âcall stackâ rather than just the currently-active bytecode.
To compare this alternative against the CHIPâs specified behavior, we need to review both items impacted by âactive bytecodeâ (A.K.A. coveredBytecode or scriptCode): 1) OP_ACTIVEBYTECODE, and 2) transaction signature checking (OP_CHECKSIG[VERIFY] and OP_CHECKMULTISIG[VERIFY], but not OP_CHECKDATASIG[VERIFY]).
Also note the Ongoing Value of OP_CODESEPARATOR Operation is to save bytes in specific constructions (allowing one public key to sign for different code paths in a single contract without risking a counterparty maleateing the transaction to follow an unintended code path).
Results:
OP_EVALed OP_ACTIVEBYTECODE covering full "call stack"
Greater consensus implementation complexity: this requires a new consensus-critical encoding for the âcall stackâ, more code to specify the full encoding behavior, and creates more surface area for performance regressions in specific VM implementations (if call stack encoding is unusually slow in a particular implementation, it could be a denial-of-service concern).
OP_ACTIVEBYTECODE is less useful: it carries baggage from all âparentâ function calls, likely separated with OP_CODESEPARATOR. Contracts must almost certainly seek-through and OP_SPLIT the encoded output to get to anything within it. In practice, this will never be more efficient than simply OP_PICKing whatever parent/active bytecode youâre after.
Transaction signature checking: reduced flexibility, increased complexity and contract lengths:
No added security, reduced flexibility: A single public key can sign for multiple code paths regardless of whether or not the whole âcall stackâ is encoded. In both cases, signatures using different code paths must execute a differing OP_CODESEPARATOR to prevent the signature from being misused in an unexpected code path. At best, the call stack behavior has saved a byte by serving as an âimplicitly-calledâ OP_CODESEPARATOR. On the other hand, itâs also not possible for contracts to avoid this behavior, so in some cases workarounds could waste many more bytes creating a solution that idempotently accepts the same signature in multiple places (perfectly efficient with the current OP_EVAL CHIP).
Increased complexity of signing implementations: encoding the call stack has no impact on the âsingle key, multiple pathsâ use case for OP_CODESEPARATOR, but it has a huge impact on the practicality of signing implementations. Now, instead of being able to sign the more predictable, static bytecode which contains the intended signature checking operation, signing implementations are forced to understand or be hinted with the expected shape of the call stack at each signature checking location. In practice, I expect many contracts would instead chose to prefix all signature checking evaluations with an OP_CODESEPARATOR, wasting a byte to skip dealing with this call stack nonsense.
OP_EVALed OP_ACTIVEBYTECODE covering only the active bytecode
Simple consensus implementation: matching the current behavior for P2SH.
OP_ACTIVEBYTECODE is useful: since OP_ACTIVEBYTECODE is always available, compilers can simplify code both before and within the OP_EVAL call, occasionally saving a few bytes vs. OP_DUP some_stack_juggling OP_EVAL + additional stack juggling within the OP_EVAL.
Simplified signing implementations, shorter contracts: signers donât need to encode the expected call stack, and can typically encode some statically-analyzable bytecode. making signing implementations much simpler, especially for offline and/or hardware wallets. Signature checks are idempotent: if they pass in one place, they can be made to pass it in multiple places at zero cost (by wrapping them in a âfunctionâ and calling that function in each location). Signatures from the same public key can also be easily made to not pass when checked in multiple locations, also at zero cost (because every function evaluation instantiates its own âactive bytecodeâ, many contracts wonât even have to add the extra 1-byte OP_CODESEPARATOR to get the âsingle key, multiple pathsâ behavior).
Summary: OP_EVALed OP_ACTIVEBYTECODE
The OP_EVAL CHIPâs handling of active bytecode matches the existing P2SH behavior and simplifies consensus implementations, simplifies signing implementations (esp. offline and/or hardware wallets), and reduces contract sizes.
ed the post) that the analogy still isnât accurate: I see a great variety of uses for â3rd party pluginsâ across multiple interpretations of that phrase.
