Maybe you can include the observations there, that the risk can be lowered massively by staggering the tx-format versioning upgrade and the actual tx-format change upgrade.
1 Like
Added a subsection:
Versioning The FORMAT As a Feature
There would be value in paying off the technical debt by versioning the FORMAT, so any future upgrades that would prefer to change it would not have to take the whole burden of changing the format.
Old software doesn’t expect the version field to have anything to do with the format of the TX it’s being fed.
First step would be locking the version field using consensus, so it could later be used to signal any future consensus rules that would come after the version lock.
Just the version lock wouldn’t break old software so only new, version-aware, software would benefit from that.
It would still help with general preparedness for a future, breaking, change.
From then onward, we could use part of the version field as an upgrade counter and match it 1-to-1 with applicable consensus specification.
As a consequence:
- Any
versionvalue seen before the version lock would relate many-to-many with “prehistorical” consensus specifications, therefore be indeterminate.
However, it would narrow it down to pre-lock era. - Post-lock, the counter part of the
versionfield would be 1-to-1 with newer consensus specifications.
We’d have other bits free to allow us to have multiple kinds of TXes exist part of the same consensus specification, which would make the wholeversionfield relate many-to-1 with consensus specifications.
Alongside the counter, we could use a flag to signal whether an upgrade is breaking or non-breaking, e.g. when we increment the counter for a new HF, if it’s non-breaking then we don’t toggle the flag, and if it is, then we toggle the flag.
Old version-aware software would know the old state of the flag so could know whether it can safely process counter+1 TXes even if it knows it can’t fully understand them.
In that scenario, upgraded software could know what rules apply to the TX even without knowing the block height, but it still needs to be upgraded so that it could match the version with applicable consensus rules.
As the consensus upgrade counter part of the version field would be the most important and long-lived feature, the 4-byte* uint should be split into 16 bits for the counter, 1 bit for the breaking flag, and 8 bits for the type of transaction.
The remainder* would be reserved for future use.
(*) Because the version field was never locked, and if we want to enable context-free transaction parsing, we’d have to use only the not-seen-prior-to-lock numbers to encode the newVersion so the new version would have a range smaller than 4 bytes.
1 Like
Good section, and an interesting proposition. I left some comments on the MR.
1 Like
It’s unfortunate that we didn’t use the chance to lock the version field but I think it doesn’t have to mean a potential future change would be set back for one upgrade cycle, because it’s possible to roll it out simultaneously and deal with the “attack” of new numbers appearing before activation.
The node would have to build a little database of seen TX versions and keep updating it till’ activation height, and the newVersion post-activation would be deterministically computed based on the database.
If most miners are mining v1/v2 then TX-es having non-standard numbers would be significantly rate-limited, so whatever “attack” would be rate-limited and rendered pointless by the newly-seen-version-aware activation code.
At some point post activation, the code could be shed and the database hard-coded.
To illustrate, imagine the version field is a 4-bit number (0 to 15), but we have mined some “pre-lock” blocks so numbers 0, 1, 2, 3, 5, 6, and 12 are unusable. We’d have only 10 numbers available for the cleanVersion.
We could assign the pre-lock TXes a single version, like cleanVersion==2 and define a function using some simple number skip algorithm:
version |
cleanVersion = dirty2clean(version) |
|---|---|
| 0 | 0 |
| 1 | 0 |
| 2 | 0 |
| 3 | 0 |
| 4 | 1 |
| 5 | 0 |
| 6 | 0 |
| 7 | 2 |
| 8 | 3 |
| 9 | 4 |
| 10 | 5 |
| 11 | 6 |
| 12 | 0 |
| 13 | 7 |
| 14 | 8 |
| 15 | 9 |
It’s not a bijection, so we’d have to pick a number for converting back, e.g. clean2dirty(0) == 0.
Implemented, and edited the post above, too.
The timing issue is still not addressed in your idea. Your conclusion seems overly optimistic.
The timing issie is where deployment requires all libraries, apps, javascript files and wallets to get replaced with the latest code at the time of the protocol upgrade. A nightmare of deployment.
I would say we want to give the ecosystem months or years time to update the software, test it and deploy it. And that is only possible by staggering the version number restriction protocol upgrade and the transaction-format change protocol upgrade.
1 Like
You speak like its the past tense, the protocol upgrade is still many months in the future.
It would show maturity of the leadership of BCH to either include the (trivial to code and test) this upgrade that already has majority support. Or to accept that, and plan for, the tx-format change to be activatd no earlier than 2024.
Take responsibilty for the decisions made, in short.
If you code them to use the newVersion accessible through a new node API, then they can prepare in advance. Whatever version is added between software release and HF activation would not affect the newVersion value, it would only affect the upper bound of the available range. There’s still the problem of those who’d want to read the version directly from the raw TX, they’d need to keep their list of used up versions updated, and continue to be aware of block height context until the HF activation. So yeah, not impossible but still ugly, and I was overly optimistic.
I know. It would be great to have it locked in 2022. Even if we don’t change the format, context-free parsing is still valuable. Post upgrade, version could relate 1-to-1 to the applicable consensus specification.
Because the nature of the change makes it a soft-fork, maybe it could still be added if people see the value? Even if it missed the November window… with this kind of change, it’s enough that miners upgrade. Edit: SFs are ugly… my thinking was flawed.
We got the data (till’ block 720189) now!
And the data tells us that 100% of the miners are currently mining v1/v2, sticky defaults?
1 Like
@tom issue tracker is disabled on your repo, there was some talk on telegram regarding moving this forward and @Jonathan_Silverblood wanted to submit an issue when he discovered he can not.
The last commit was 9 months ago, so, we need to know, do you intend to continue to work on this CHIP?
1 Like
For reference, I think that it would be good to add a clear list of identified stakeholders, make a technical section which shows all possible outcomes and behaviour and I’ve be happy to provide a statement of my own if suitable.
The statement by andrew can further be strengthened now that BU have voted for the change in BUIP170.
In a discussion touching on this subject with regards to native introspection, some research was done on how much abuse there has been over time for the transaction version which would be helpful to have in the chip as well.
@bitcoincashautist or @bitjson can probably provide more detailed information on the usage.
Both Jonathan and Jason have been given full access to the repo, please feel free to do anything with it you want, I know you guys have the same goals I have so no need to ask permission first.
Hope that helps!
3 Likes
Just wanted to leave a comment here that it’s unfortunate we didn’t get this into the May 2022 upgrade. But I strongly support this as a new consensus rule moving forward for any future upgrade. Bump +1
1 Like
I support this change as a BCHN developer and Knuth lead dev.
1 Like
I’ve done a bunch of changes and set up an MR, would appreciate review so I won’t mess stuff up:
2 Likes
Is this ready for implementation?
It looks to be in a good state to me, so I would be really sad if it doesn’t make in the next cycle. If anyone knows a reason for why this shouldn’t be accepted as it stands today, please raise your concerns so we can fix them in good time.
4 Likes
My view: Yes
Sorry, must add more text to satisfy 20-char minimum 
3 Likes
Added some more, expanded some sections, mainly costs&risks, and summarized costs/benefits over classes of stakeholders. Motivation&benefits section could use some more love, though. Let’s get this activated in this cycle 
The effect of this change is to prevent new transaction versions being added via soft fork.
I suggest instead that the rule is that
a) Only miners enforce the rule
b) Software “MUST” have a parameter that sets the valid transaction version
If a new transaction ID is defined, then miners just have to update their configuration file, e.g. max_transaction_version=3.
Non-miners are unaffected, since they accept all transaction versions anyway.
I guess it matters less with Bitcoin Cash, since hard fork upgrades are the norm anyway.
1 Like
Uhg, we want to avoid funky SF mechanics, especially since the objective here is to tie the TX version to consensus specification (changing which is a HF), so when we do future upgrades (not necessarily of the format) we can bump the version, and it’ll be guaranteed by consensus rules that TXes with that version were adhering to that set of consensus rules. Right now, you need block height information to know which rules are applicable for a TX. Context-free parsing is the general benefit here.