The "node pool" (similar to "mining pool" but for everything else)

Random
#1

With ordered Merkle tree from 2018 CTOR upgrade the mempool can be sharded by transaction hash, and a node can split into an arbitrary number of shards that produce a block in parallel. A shard “owns” its transaction hash range, any other shard that wants to use an “unspent output” simply requests the right to do so, and first to ask gets to spend it. The shards of a node can be operated by different people who all together form a “team”, the node as a “pool” or “team” of people. The shards or “sub-nodes” can be in geographically different locations. Nodes can propagate mempool and blocks filtered by transaction hash range, so that propagation becomes via “sub-nodes” (shards) directly, removing any bandwidth bottleneck. This type of parallelization has no computation, storage or bandwidth bottlenecks. The “node pool” coordination to build Merkle root and request UTXO adds latency cost of maybe half a second?

Game theory for “node pool” is the same as for a centralized node, if the node (“node pool”) produces invalid blocks those get rejected by other nodes and the node does not get paid. Nodes validate blocks produced by other nodes so that they mine on the highest difficulty chain.

With “node pool” a block can be very large. Consider 1024 shards each with 32 MB “sub-block”, you have 32 GB blocks and 100k transactions per second. Each sub-node simply has similar burden as a current Bitcoin Cash node would have.

I think this is a really interesting and promising concept. I have not seen it discussed much. I wanted to open ability to discuss it here. I apologize for my previous post on this diverting into a bad idea of “singular transaction trie”, it took away from what I actually wanted to discuss.

1 Like
#2

In computer science there is a nice true-ism.

It it 100 times as expensive to use main-memory over the in-CPU memory.
It is 100 times as expensive to use disk over main-memory.
It is 100 times as expensive to use (close-by) networking over using disk.

So, based on this first-principles concept that’s been largely unchanged for decades, sharding simply makes no sense. The ONLY sharding you might benefit from is the useless idea of having multiple chains that need confirmations to switch between. I’d like to think we’ll never need such a system.
(as an aside, this kind of multi chain system with an ‘exchange’ is likely how multi-planetary crypto would work).

When I researched scaling as part of Flowee, I reached the conclusion that the only bottleneck in real life will be the UTXO database. Because that is where we register money that is spent. And as such that is where you check to avoid a double spend. It is a bottle neck since you need to verify two “shards” are not spending the same money at the same time.

For this reason the UTXO database (part of Flowee) I wrote 5 years ago is massively multi-threaded and prefers to keep data in memory over writing it to disk. All this helps with a full node design that uses its many cores to validate transactions in parallel. While still maintaining the certainty that no double spend is possible.

#3

My intuition is people neglect “node-as-pool” more because of the assumption that the miner attestation is trustless, which it is not, it is trust-based. The financial reward does not make it trustless (and ability for anyone to audit and prove invalid blocks does not make the miner attestation itself trustless, it is trust-based entirely). And that ideological bias will then affect reasoning. Bitcoin Cash propagates block over network (see your “true-ism”). Sharding as I describe sub-nodes will filter mempool and sub-blocks by their transaction hash range. All propagation is sub-node to sub-node. With N sub-nodes you reduce the expensive network transfer by N times. With 1024 sub-nodes, you cut it by 1024 times. The added time (data transferred) for input or output checks during block validation or block production per sub-node is less than a sub-block. With 32 GB block and 1024 sub-nodes you end up transferring maybe 50 MB per sub-node, a 700x reduction in the expensive networking from your “true-ism”.

#4

You realize that if a user spends a UTXO it will extremely likely not end up in a transaction in the exact same shard, right? Even more, if you spend a lot of inputs, the transaction may end up spending utxo’s from hundreds of your “shards”.

So when you claim there is no cost, you’re wrong. The cost is asking all the sub nodes for details.

And because you’re still worried about double spending, this asking of information from another node (maybe on another continent) is slowing down the adding of that transaction to the mempool.

Not to mention that this information you requre to avoid double spending is also then trusted data, you’re no longer validating those UTXOs, but you’re trusting a stranger to not lie about a utxo being unspent.

Hope that gives a bit of insight into the basic concept of sharding being irrelevant because you can’t shard the UTXO database in any way that actually helps scaling.

1 Like
#5

There is a true-ism in computer science that network 100 times more expensive than storage, storage 100 times main-memory, main-memory 100 times in-CPU memory. With the architecture I describe (“internal” parallelization by social delegation that is decentralized geographically) you reduce cost on networking step in your “true-ism” by N/2 times where N is number of shards you use. This means it was reduced by cost similar to reducing 50N at storage or 5000N at main-memory or 500000N at in-CPU memory.

To organize by trust is by definition not “trusting strangers”. Whereas the attestation of the miner in Nakamoto consensus is to trust strangers not to lie, with incentives to encourage honesty, mathematically provable misbehavior from whoever audits, and competition among what are assumed to be an honest majority, the same game theory also what the “node-as-pool” would operate under.

#6

I think this is the biggest misunderstaning you have on how Bitcoin Cash works.

The conclusion of the whitepaper from Satoshi explains this briefly, but you may want to read the whole thing. It’s only 8 pages, so no excuse not to :slight_smile:

In short, the mining software creates the “attestation”, but the miner validates it and puts a LOT of money towards extending the chain with this attestation, turning it into what we in lingo call a block. The money is expressed in proof of work that is required to make it a valid block.
Other miners then validate the block again and ONLY adopt the block into the chain after they find it to be correct.

This eliminates the trust in a single miner, it removes the need to trust any miners at all because you have a choice. (See economics discussions on this topic by Murray Rothbard at all). The ability to sell your coins and leave the ecosystem hurts the miners income. Thus the incentive exists to be honest. Or, simpler: dishonest miners go bankrupt, dishonest chains stop being used.

Long story short, miners are not “trusted entities” in the bitcoin cash ecosystem. They are just opportunistic actors that can join and leave at will, getting paid for the honest work they do. Not getting paid for dishonest work.

1 Like
#7

To scale blockchain you need to understand the parts individually, as you need to scale each part. The consensus mechanism (that alternates validator from a very large pool, an honest majority) is one mechanism. The attestation by the validator (their signature, in proof-of-work the proof-of-work does both signature and selecting next validator) is another part. As the latter is based on trust (the financial reward does not make it trustless), you distribute that mechanism among an honest majority. Such is not needed in “trustless attestation” (“encrypted computation that cannot lie” or something similar) as there the attestation to correctness is mathematically provable directly. It does not rely on trust anymore then (but censorship resistance and fork-selection still require it). But this is not the paradigm for any Bitcoin fork (or Ethereum or anything else in that generation). There, the attestation is entirely based on trust, and can therefore only be scaled by trust. The reason people miss this is because they have the false belief that because the sum of the parts approximates “trustless”, all parts must be trustless. And, since Bitcoin in 2008 did not use ordered Merkle tree, any attempt at scaling would have been futile anyway, it was not until BCH CTOR upgrade in 2018 that it became technically possible. What I say is easily verifiable, anyone can look at how Gavin Wood and Vitalik Buterin have tried to scale by “random sampling from validator pool with stateless validation” which would only make sense in a “trustless validation” paradigm. They have the fundamentals wrong. As do you Tom. Here, and also in the Bitcoin Cash Telegram group where we had some correspondence since your last reply.