Staging CHIPs on Testnet

CHIPs
#1

Continuing a discussion from BCH Devs & Builders with @emergent_reasons, @cculianu, @fpelliccioni, and @Jonas:

Should we deploy Cash Improvement Proposals (CHIPs) to testnet before they are deployed on mainnet?

This would:

  • Ensure all proposals are thoroughly and publicly reviewed, integrated, and tested (including deployment procedures) in an environment where most large ecosystem actors are likely to encounter any eventual issues they will need to address in their own software. Testnet is nearly as adversarial as mainnet, so defects are more likely to be identified prior to mainnet deployment.
  • Enable user experimentation with new features – experimenting with new features can require significant time and resources. On testnet, developers can often use existing infrastructure to test their applications, reducing costs and enabling wider integration tests. New features can often even be tested by end users without significant technical expertise.
  • Strengthen testnet infrastructure – BCH itself is cheap to send, and these days, many developers simply test their contracts on mainnet. Testnet support has fallen behind: many block explorers no longer support BCH testnet, testnet faucets are commonly empty, and few wallets maintain testnet support. By giving developers a strong reason to use testnet, we ensure testnet infrastructure gets used and maintained.
  • Improve business certainty around upgrades – Testnet deployment would offer a strong Schelling point for any particular upgrade: if it’s not already live on testnet, it’s not going to suddenly happen on mainnet. Testnet forks are also relatively low-impact and less likely to be complicated by “fork markets” and other possible conflicts of interest. If meaningful technical disagreement exists, that disagreement will be very visible long before it can hurt end users, giving businesses plenty of planning time. (Coupled with sufficient prediction markets, this could practically eliminate upgrade-related business uncertainty.)
  • Stronger mainnet launches – With improved certainty of deployment and easier access to testing infrastructure, developers are more likely to commit resources to development which uses the new upgrade before it is deployed on mainnet. By the time an upgrade is deployed to mainnet, more downstream applications are likely to be production-ready, making network upgrades more exciting and valuable for users and businesses.

To do this, we would probably want each CHIP to specify two different activation times in its Deployment section, one for testnet (either November or May), and one for mainnet (typically May).

Some questions:

6 Likes
CHIP 2022-02 CashTokens: Token Primitives for Bitcoin Cash
#2

One reason I’ve been thinking about this: I believe the math and introspection CHIPs will almost certainly be ready to “lock-in” in this November and go live in May 2022. But some of the other CHIPs should probably wait, if only to set a careful precedent.

I agreed with @emergent_reasons here:

Ya, I’m also thinking it would be best for the May 2022 upgrade to only include the math and introspection CHIPs. It would be really great to set the precedent that BCH consensus moves carefully with >1 year of review.

The math and introspection CHIPs will solve the most urgent issues with the VM, unblocking a lot of really valuable use cases. The other consensus CHIPs I’ve proposed - targeted limits, bounded loops, and PMv3 - are very important, but can stand to have a longer time horizon.

Limits and loops would be immediately useful, but aren’t actually blocking any ready-to-deploy use cases, they would just make those use cases more efficient. And PMv3 could be immediately useful to improve the efficiency and UX of a few use cases, but big things like SLP 2.0, decentralized exchanges, and prediction markets will probably take another 6-12 months of development to have the first production-ready implementations.

One idea I’ve been thinking about: we can get almost the same value by coordinating deployment of limits, loops, and PMv3 to the primary testnet in May 2022 (the testnet used by most block explorers). That would give the other proposals a full year to be thoroughly tested while allowing developers to start working on real, complex applications using them in a reliable, almost-production environment.

Thoughts?

It would be a great precedent to set that future consensus changes are staged on the primary testnet for 6-12 months before mainnet deployment. And I think we’re at a sweet spot development-wise: the math and introspection CHIPs really reduce the current pain for BCH contract developers. If we got the other proposals on testnet in May 2022, there aren’t a lot of other big pain points which developers are going to be eager to “fast-track”, and we can afford to be much more patient with e.g. future crypto APIs, zero-knowledge proofs, MAST, multi-key input aggregation, etc.

We’ll have plenty of time to deliberate over the other CHIPs before May (when they would hit testnet), so the primary focus for now would be getting math and introspection on testnet in November.

3 Likes
#3

Re: which testnet:

I’d like to propose that node implementations and businesses begin migrating to testnet4 in November as their “canonical” testnet.

This isn’t easy, but it would pay significant dividends. Testnet4 is tuned for low resource consumption and good block difficulty resilience, making it really excellent for developers to test functionality on new block explorers, applications, and wallets. It’s also very young, so the 50 tBCH rewards are very helpful to fill new faucets and developer tools. (Trivial, but we shouldn’t underestimate how much fun meaningful numbers are for new developers.)

Looking around, I count 4 working BCH testnet3 block explorers right now:

Notably, bitcoin.com no longer has a BCH testnet explorer because they migrated to blockchair, which doesn’t yet support any BCH testnets: https://blockchair.com/explorers?from=bitcoin.com

There are two wallets which support BCH testnet3 as far as I know:

  • BitPay
  • Electron Cash

And looking around, every BCH testnet3 faucet I can find is drained. (Which isn’t surprising when you note that at time of writing – testnet3 block 1,464,335 – the block reward is only 0.78125 tBCH.)

I think a coordinated switch to testnet4 would really revitalize BCH’s testnet infrastructure. With just a few working testnet4 block explorers and faucets, we’d be well ahead of testnet3’s existing infrastructure.

If most node implementations committed to making testnet4 their “default” testnet, and we can get a few of the above services to switch over, we’d have a very nice test ecosystem for developers to work on contracts using the new CHIPs before they ship on mainnet. :rocket:

3 Likes
#4

If it were desired, it would be trivial to change testnet4 to always have a 50 BCH block reward preventing this issue from occurring again. This would not require coordination from any implementations. They may make the change at their leisure as long as it is done before the first halving date on testnet4 (otherwise a testnet fork might occur).

2 Likes
#5

Thinking about it, the underlying issue is that testnet is so far ahead of mainnet due to abuse of past difficulty adjustment algorithms. Testnet is ~14 years ahead of mainnet (the 7th halving from 0.78125 BCH to 0.390625 BCH at block 1,470,000 should happen in ~2036), so the inflation rate of mainnet is currently closer to testnet4 than to testnet3. :laughing:

In the long run, I think it would be ideal if testnet4 maintained the same inflation schedule as mainnet, but offset to be earlier in its history. Then the numbers used on testnet should generally be sensible when compared to real BCH, with slightly higher available circulation to make faucet funding easier. (Otherwise, if BCH is dominant by ~2034, testnet4 would have a supply of 31.5M tBCH and a daily inflation rate of >$300M in today’s tUSD. It would probably be filled with comically large transactions such that faked currency conversions in user interfaces would all look like real-estate sized transactions.)

So I think the underlying issue has already been solved by the new difficulty adjustment algorithm, we just need to reset our canonical testnet because testnet3 is hopelessly far ahead. Without any changes, testnet4 will probably maintain ~8x mainnet’s inflation rate. I think that will be just enough to encourage testnet miner/faucet “generosity”, but not enough to ruin test conversion rate UIs. :ok_hand:

2 Likes
#6

Update: for the May 2022 upgrade cycle, we ended up with a long-running fork of testnet4 starting in December and active until after May 15 (when mainnet, testnet3, and testnet4 upgraded). There were also a few shorter-lived activation tests coordinated between implementations.

For the May 2023 Upgrade, it would be great to have a long-running testnet available when CHIPs are locked-in on November 15.

I was initially thinking we should set a precedent of upgrading testnet4 in November (and mainnet + all other networks in May), but @im_uname convinced me that it’s safer to leave testnet4 consensus in sync with mainnet and create/grow a separate network.

So a proposal to solve this topic:

We create chipnet, a permanent fork of testnet4, and activate the May 2023 upgrade on chipnet at 12 UTC on November 15th, 2022.

The goal of chipnet would be to serve as a staging ground for CHIPs. Upgrades should activate on chipnet 6 months before mainnet. Chipnet should allow projects and companies to test all infrastructure well in advance of upgrade day + other benefits mentioned above.

I’ll commit to mining, running a DNS seed, and including chipnet support in Chaingraph, Libauth, and other projects. I’ll start reaching out to other infrastructure providers to see about adding support as well.

If chipnet proves useful over the course of this upgrade cycle, we can work to make it an accessible test network in all implementations.

6 Likes
#7

I agree!
I’m going to be writing about a new protocol change proposal (what we call CHIP today) model for years to come.

#8

Since this has at least been approved in BCHN – should I go ahead and just add a -chipnet command-line argument now? I guess -chipnet uses:

  • the same seeds as testnet4 initially
  • the same genesis block as testnet4
  • just has its upgrade9 activationtime set to Nov. 15 2022. And at that point we fork them by sending some txns that are mutually incompatible on each of them…

After a while we can update -chipnet to use its own DNS seeds… but since it doesn’t exist yet – the initial version of it at least in BCHN will use -testnet4 seeds.

How does that sound @bitjson @im_uname @freetrader and others?

2 Likes
#9

Update: I’d been thinking we’d fork from testnet4 on November 15th, but @im_uname and @cculianu had a better idea – fork from testnet4 earlier, then simply activate the upgrade on November 15th. That means we can prepare some infrastructure for the new network before upgrade day, and we don’t have to deal with any chaos from the split at the same time as the upgrade.

A -chipnet flag would be fantastic!

I just set up node to mine on the network @im_uname forked from testnet4 yesterday, and now I’m running a patched seeder at chipnet.bitjson.com. :rocket:

(For future readers: note that DNS seed URLs respond back with a random set of IP addresses on each request – occasionally people also host websites at their IP address, but typically you’ll get an error if you try to “visit” a DNS seed in a web browser. Use dig instead: dig chipnet.bitjson.com.)

1 Like
#10

For anyone else who would like to set up a chipnet node right away, here are the CashToken testnet instructions updated for chipnet:

To connect a BCHN node:

  1. Either build the test branch yourself (Upgrade9 (May 2023) - Everything plus fixups (!1600) · Merge requests · Bitcoin Cash Node / Bitcoin Cash Node · GitLab) or use Calin’s build here (Index of /downloads/BitcoinCash/testing/mr-1600_mr-1603)

  2. (Optional) If you have a testnet4 node running already, you can copy the whole data directory (e.g. name the copy chipnet). You’ll need to update your configuration with the chipnet ports, e.g. bitcoin.conf:

txindex=1
whitelist=127.0.0.1
rpcuser=localuser
rpcpassword=hunter2
testnet4=1
[test4]
port=48333
rpcport=48332

Be sure to open/forward the port on your router or firewall so other nodes can connect to you.

  1. Run the CashTokens-enabled BCHN client using this iteration’s activation time: -upgrade9activationtime=1668513600 (that is Nov 15th 2022 12:00:00 UTC)

  2. Using RPC or the debug UI window, force your node to follow this testnet fork: first invalidateblock 00000000ae25e85d9e22cd6c8d72c2f5d4b0222289d801b7f633aeae3f8c6367, then: reconsiderblock 00000000040ba9641ba98a37b2e5ceead38e4e2930ac8f145c8094f94c708727. If you’re still having trouble connecting, try manually adding a peer: addnode chipnet.imaginary.cash:48333 add or use dig chipnet.bitjson.com to find other IP addresses.

1 Like
#11

A binary is available now that has -chipnet natively so you won’t need to do the above.

Get it here: Index of /downloads/BitcoinCash/testing/mr-1600_mr-1603

Use -chipnet (or chipnet=1) in the conf file and also you need to name the conf file section [chip].

It should “find” the right nodes to talk to … without needing to do the invalidateblock and all of the above since it now has the proper checkpoints and the proper activationtime, etc.

1 Like
#12

There is a new Fulcrum release which now also knows about chipnet and supports it: Release Fulcrum 1.8.2 · cculianu/Fulcrum · GitHub

1 Like
#13

Fantastic, thank you @cculianu!

I also sent a PR to add chipnet to bch-rpc-explorer: https://github.com/sickpig/bch-rpc-explorer/pull/182

3 Likes
#14

I like the chipnet initiative a lot! It’s a very good idea that the netnet is already forked at present but only upgraded November 15th.

To support this effort made a PR to add “chipnet” as a network option to the Cashscript SDK and clearly differentiate between testnet3 and testnet4. Previous upgrade cycle Cashscript was upgraded early to add a “staging” network option which connected to the May 2022 upgrade testnet. https://github.com/rkalis/cashscript/pull/133

Looking forward to being able to use the new Cashtoken native-introspection opcodes with Cashscript on chipnet!

3 Likes
#15

Just leaving this here as a resource – the May 2023 upgrade activated on Chipnet at block #121,957: [CHIP] - Block #121,957, 0000000056087dee73fb66178ca70da89dfd0be098b1a63cf6fe93934cd04c78

I also created a larger patch to add full CashToken support to bch-rpc-explorer here: https://github.com/sickpig/bch-rpc-explorer/pull/184

It’s already live at https://chipnet.chaingraph.cash/, which I will be maintaining indefinitely as a Chipnet block explorer.

2 Likes
#16

Just want to add a recent observation here for future reference:

It’s fantastic to now have testnet4 and chipnet descend from the same genesis block, share network magic, etc. because their co-existence helps developers test handling of past splits.

So block explorers and indexers that support multiple bitcoin forks (e.g. BTC, BSV, XEC) can test their split visualization UI on testnet, indexers can test that they don’t duplicate shared blocks, wallets can test that they don’t fall apart in the presence of two valid-looking chain tips, and nodes can test that they gracefully handle the existence of similar networks (as is the case on BCH mainnet today, where BSV and XEC nodes regularly connect to BCH nodes and can even sync early chain history from each other).

1 Like