Tailstorm: A Secure and Fair Blockchain for Cash Transactions

#45

Not required, but they can and this is favored - because there’s benefit in choosing to extend the longest subchain rather than referencing the previous FB.
The penalty system incentivizes mining on top of best known subtip in order to avoid the penalty. It also incentivizes against subblock witholding because then nobody will see your blocks and they’ll extend the other branch while you mine in secret, and if the public branch wins - you get hit by the penalty if your branch will be shorter.

Why, if the mempool policy stays the same, how would you even obtain a conflicting TX? If anything, it’s likely that subs will include duplicates, like the scenario im_uname laid out:

a quick top-of-my-head example:

block A1-A2 is the longest subchain.

block B1 competes with A1 and has a transaction T1 that doesn’t conflict with A1, yet doesn’t exist in A1-A2 either. it’s a branch, uncle, whatever.

I now want to mine A3. Can I mine T2 that spends from outputs of T1, if block B1 “contributes” transactions? I assume I’ll need to include T1 first so my block is internally consistent. If I include T1, did block B1 actually “contribute” anything?

To which I answered:

I see, good Q, yeah you’d have to include T1, or you’d have to extend the subchain that already has it

Yup. Not rejecting is the secret sauce of Tailstorm, it integrates blocks that would otherwise be orphaned: takes in their PoW contribution + any non-conflicting TXs, and there’s PoW-based dispute resolution for conflicts.
Re. time, they can lie, sure, but it has no effect here. Accumulated PoW settles the dispute just like now, not the reported time. In case of equal PoW (same competing subchain length) it’s a coin toss based on subhashA < subhashB.

No, because if you observe your TX is included in a fast-advancing subchain, you can have some confidence that it won’t be replaced by another subchain - just like now you can have confidence that a 2conf won’t be reorged by another tip.

Miners are incentivized to announce their subblocks ASAP, so let’s say K=60, and there are subtips:

    1. 30 blocks
    1. 15 blocks
    1. 14 blocks

the network needs just 1 more SB in order to announce a FB, and then everyone’s expected to start mining on top of it, else they risk their extra SB getting reorged.

If you have a TX in 1., then you can be pretty sure it will win over whichever tip produces the last SB.

If you have a TX in 2. or 3., there’s uncertainty, if 3. mines the last SB and it has a conflict then it’s a coin-toss. If 2. mines the last SB then it wins over any TX in 3. but NOT against 1.

If 2 SBs arrive simultaneously, the FB picks one of them, and shortly after some SBs in the next epoch start appearing so the network can have confidence which FB won.

You can’t mine a FB unless there’s a pick of SBs available that would accumulate (1-K) PoW.
At that moment, mining a SB would imply overproducing PoW and you risk the SB getting reorged by some other miner mining a FB and not picking your SB. There’s incentive to go for a FB as soon as enough SBs have been announced.

You could still go for an empty SB or FB, why not? Not sure it breaks header-first, I’m not ready to claim either yes/no at this moment.

You’d see higher number of branching / 10min, but not orphans / 10min. That’s the secret sauce. The would-be-orphans get merged, their miners get paid (and only get hit by 1/K penalty), so latency has less impact on their profit, so big pools get less of an advantage over small pools.

If small pools are trying to stay on main branch in order to avoid penalty, sometimes they’d manage, sometimes they wouldn’t, and then you’d have something like this:

-- F -- o -- o -- o --o -- o -- o -- o -- o -- o -- o -- o -- F -- o -- o --
        \ o                \ o       \ o                 \ x

When they’re not late - they just extend the best tip. When they’re late, they branch off the best tip, but their subs later get picked up by the FB. Only if they’re late to the last sub would their sub get orphaned.
Their loss here would be (3 * 1/K) + 1/K = K.

Compare that to just reducing block time, you’d have this:

-- o -- o -- o -- o --o -- o -- o -- o -- o -- o -- o -- o -- o -- o -- o --
        \ x                \ x       \ x                 \ x

Their loss here would be 4K.

#46

Ah, so you’re saying that the double spend problem is solved at the mempool level already today. :slight_smile:

You have between 5 and 10 seconds, a block comes in every 16 seconds? “Fast approaching” is not the term I’d use. Because that requires you to stand still for a minute or so. Which is actually what you were trying to avoid.

Your answer also doesn’t actually address the point I made. Two subblocks having conflicting transactions. Completely legal to do, provided they are not in the same sub-tree.

The statement that you can have confidence that it won’t get replaced is misguided. That misses the entire concept of WORK. The reason people are doubting bch is because we’re the < 1% coin. And it costs peanuts to reorg us.
A subblock costs much less to reorg compared to the full block. You can’t assume they won’t be.

This sounds very much like reorganizing chairs on the deck of the Titanic. All your assumptions of security are based on miners not wanting to make some other miner lose their subblock and thus payout. But your assumptions actually use the COST of a full block, which it is not. At all. It is like 1/50th or so.

The rest of your answers are all over the place. Either you’re not at all following my explanation, or you chose to not understand. Either way, re-read above point if you want to understand the issues. I’m not here to convince you.

I liked this self-undermining point most:

So, in my scenario of you not being certain that getting included in a SB means it will actually get mined in a FB was me thinking another SB could conflict with it.
Instead you’re now saying that all SBs can be completely ignored and the FB can just skip all transactions that were in the SBs that lead to it!

Nice.

1 Like
#47

Incentives are such that everyone’d want to extend the main subchain. If you broadcast a TX and it gets 2 confs in the main chain, chances are it would continue to get mined by honest miners.

Dishonest miner would need to branch an older subblock if he’s to have a shot at getting a conflict be accepted instead of your TX, and he’d be losing money by trying to win the race vs main chain because rewards of lesser branches are penalized.

Even if race is equal and there’s uncertainty until FB decides a winner and chain advances to next epoch - then observing both TXs works like superpowered DSP - one that can cover any kind of TX.

But it costs, and the cost would grow with subchain lenght just like it now grows with confs. Attempting to doublespend a 0-conf TX costs nothing.

Yes, legal, but it costs the lesser branch’s miner. If he’s intentionally creating a branch just to include a conflict, he’ll take a 1/K hit on revenue of that sub. Edit: Yeah, if he’s overtly trying to replace it the total cost of attempt can be just (N/K)*1/K for the epoch. This way, everyone would be alerted of the attempt & race.
But as you’ll see below, if he’s trying to do it covertly, then he stands to lose N/K.

I’m not, but if the amount is not worth the cost of branching, then it’s unlikely. Same logic now applies for reorgs, right? With subs you get more granularity in accumulation of cost-to-replace.

It’s 1/50 when the sub is at the tip, but if your TX is buried at the beginning of that subchain, the miner who’d want to replace it in the FB would have to pop some N subblocks from the tip and replace them with his, and during that time network could advance to next epoch he’d lose all those extra subs rather than just 1.

I was imprecise, when I said empty FB I wanted to say that it doesn’t contribute any additional TX on top of whatever TX the subblocks it references already included.
The FB is is still confirming whatever TXs are in the subblocks it references.
If you need just 1 more sub to reach required number of subs to produce a FB, you can mine an empty SB and then a FB and advance the chain.
You can reference the subs without validating the TXs in them, but you won’t know the UTXO state until you do the validation, so you’d have to keep adding empty subs until you catch up with validation to avoid risk of putting in a TX which would invalidate your block.
So, I think it doesn’t break header-first mining, but sure, we must examine the edge cases more carefully to better understand.

1 Like
#48

Someone pointed out that Tailstorm is reminiscent of Ethereum’s uncle blocks. There are however some key differences, noting them here for future reference:

  • Uncles get paid for their PoW contribution (Tailstorm pays uncles, too) but reward decays with their distance from the nephew block that references them, down to 0 after distance of 7.
  • Cousins can’t be merged, i.e. any block mined on top of an uncle is reorged (Tailstorm can merge the whole uncle’s tree).
  • Uncles can be merged every block (whereas Tailstorm can merge them every Kth subblock, and only if they belong to the same epoch).
  • Nephews get paid a flat reward for referencing uncles.
  • The main branch gets only the PoW contribution from merging uncles, but no TXs are merged (impossible to merge TXs on account-based architecture like Ethereum’s).

Jianyu Niu, Chen Feng, “Selfish Mining in Ethereum” (2019) demonstrated that Ethereum’s PoW consensus system is more vulnerable to selfish mining than Bitcoin. It’s not surprising considering there’s incentive to intentionally uncle weaker miner’s blocks in order to reduce their revenue and increase our own (nephew reward).

1 Like
#49

Thinking about this more, it depends on how we arrange data structures, but there will be trade-offs.

With the above idea full blocks would look like and link up just like regular blocks do now, but they’d all have apparent difficulty reduced to 1/K (if you’re just looking at FBs and ignoring subblock headers).

This is a problem for header-first mining: you could mine subblocks header-first but not the full blocks, because how would you compute merkle root when it would have to cover all the TXs contributed by the subblocks?

Or, we make FBs merkle root cover just its own TXs (so that it can be an empty block). But then we break the view for non-upgraded software: they wouldn’t see subblock TXs unless they ask for subblocks, too.

Aargh!

#50

Your imprecision happened to just allow you disagree with my statement that a miner needs to validate and have all transactions in order to make an FB.

You correcting this means that the original points of creating a lot more orphans as explained by me above is back on the table.

I think you misunderstand header-first mining. Not a single miner today pushes out blocks that have unverified transactions contained in them.

If you depend on that, you’re doing it wrong.

#51

I had it wrong. FB can’t be done header first with the structure I was considering above because you’d at least need the list of subblock TXIDs to compute the FB’s merkle root. Maybe it could be done with an alternative structure, but then you have more breakage elsewhere (similar breakage to changing block time).

Not quite, this still holds:

There’d be orphan risk just around the FB, but subblocks can still get mined header-first so they still have that advantage.

1 Like
#52

Then you agree.

Where today there are no orphans, TailStorm introduces a very large increase of orphans as I explained in message 44. (this being msg 52).

1 Like
#53

Agree that FB can not be mined header-first with the structure considered here. If that’s a deal breaker then we could investigate alternative ways to implement Tailstorm and see where that leads us.

That’s an overstatement, because:

  • Subblocks can still be mined header-first, so (K-1)/K of the time you won’t be affected.
  • When it’s time to start mining FB (last subblock announced), you don’t have to download and validate ALL referenced subblocks at that moment - because you ought to already have done the validation on most of them, except for the one that just got announced, and you can do a lighter first pass on it just to get on with the mining, and then complete the validation in parallel.

Compared to header-first, you’ll be slightly delayed only on the FB and only for the time it takes you to:

  • download the last subblock needed to accumulate enough PoW (K-1)
  • copy outputs from subbases into your coinbase
  • compare spent UTXOs in it vs those spent in other subblocks of the current epoch, update list of TXIDs and conflict TXIDs
  • compute FB’s merkle root (covering only the referenced subs non-conflicting TXIDs + your FB coinbase) and start mining the FB

and this delay will only happen every Kth subblock, and risk losing 1/K of the epoch’s reward. In parallel you can do: complete full validation (TX consensus rules + update UTXO state). When that completes, you can start adding any new TXs to FB’s block template.

We can argue whether this is significant or not, but do you agree the above is accurate?

#54

You seem to be in love with arguing your way through technical problems. This is not a marital issue, this is a technical problem where it really doesn’t matter what I or you think about it. It matters how it works in the real world.

I have explained to you with a very simple analysis that it will behave massively worse than status-quo. You can continue arguing that it’s not that bad, and maybe you expect normal people (as you’re a self-described autist) to relent if you want it so much.

Facts remain, all I’ve said in post 44 stands and is true.

We go from virtually no orphans to removing header-first (on FBs) and thus introducing delays that are not 100ms but likely counted in full seconds with small blocks. But much longer with 1GB blocks. Which will create lots of orphans. QED.

You also missed the second point; when there is a need for an exact number of subblocks and more are not allowed to be used in a a full block, then you get orphans on subblocks too. Especially when there are lots of them per 10 minutes. (also explained in comment 44).

You know me, I’d like to be accommodation and I try to stay on focus. But most people would have given up when you didn’t try to solve the issues, instead just try to explain the flaws away.
Your choice, you can take the oldie’s comments about the issues you’ll face on implementations or not. In which case it will show up on actual implementation which may be a year down the road and you’d be going back to the drawing table.

Personally I expect this to be a fatal flaw that completely dismisses TailStorm, the basic concept is too complex to allow header-first mining. Therefor you’ll have more orphans which means it is a net negative.

#55

1 GB blocks would mean 17 MB subblocks (K=60), and it is those last 17 MB + computing merkle root on all TXIDs (about 160 MB big list of TXIDs) that would be delaying your start of mining the FB. Let’s say it is as you say - a deal breaker.

I’m in the “How can we make this work?” mode of thinking, and you’re in the “This is why it won’t work.” mode. You win this round. :smiley:

It doesn’t dismiss Tailstorm, it dismisses this particular approach to implementing it.

Why even try implementing it if what you point out is a deal breaker.
It’s back to drawing table now, to find a way where header-first mining works on all blocks, and see if other deal breakers would appear as a consequence of accommodating header-first mining.

#56

Just for reference, the no-PoW-in-FB variant would look something like this:

rect3
rect31185×710 231 KB

Impact:

  • still not possible to do header-first on FBs, because FBs merkle root computation would be blocking start of next epoch
  • breaks non-upgraded SPV clients, because FB header chain security gets reduced to 0. SPV clients would have to learn to obtain coinbase TXs from FBs, extract subblock headers from FB coinbase and verify Tailstorm PoW.

Note that non-upgraded SPV security reduction is present in previous yes-PoW-in-FB, but it goes down to 1/K rather than 0.

#57
  1. Breaks the promise of only having around 4MB of headers a year.
1 Like
#58

We could unblock header-first mining on “full” blocks, but it’d ugly and would totally break legacy SPV and blockchain view of non-upgraded software:

rect3
rect31185×710 181 KB

and it also breaks blockchain parsing for non-upgraded software, there’d be missing transactions if they’d just follow the headers and TXs covered by their merkle roots (while ignoring out-of-chain subblocks & their headers in main chain’s OP_RETURN).
I think it’s the only way to unblock full block header-first mining because merkle root here is local to the full block so state can be resolved later, while mining an empty block template.
OK this sucks, it unblocks header-first but makes everything else so much worse.

But when chaining full blocks through the best subchain we can do this:

rect3
rect31185×710 186 KB

Notice the difference? If we require all non-conflicting out-of-chain TXs to be “inside” the FBs merkle root then the header chain is enough to prove all TXs via legacy SPV, and here it can prove almost full PoW (it can’t accumulate PoW from out-of-chain subblocks, unless upgraded to request and parse coinbase TXs & embedded subchain headers).

I have an idea to improve on this, but it’ll take me a day or two to find a nice way to present it.

1 Like
#59

So, rather than collecting uncled subblocks all in 1 summary block, we can instead wrap each in an individual wrapper, a shim block, which is allowed to have difficulty 0 if its coinbase has auxiliary PoW, and merkle root covers only non-conflicting TXs from the referenced uncle.

rect3
rect31158×1385 212 KB

This method would flatten the subtrees into a chain, and avoid having to compute merkle root from non-local TXs:

rect52
rect52741×420 28.6 KB

1 Like
#60

This could be stylistic, but in classic blockchain there are no sub blocks (S1-5). What is classic blockchain referencing?

Would punishing work pretty much the same? Is it possible to punish in such a scheme? I imagine so since reward only paid out once summary/merge/whatever block is completely mined.

Does this discourage selfish-mining? Granted, I think that’s less a concern, but as long as we can ensure that smaller miners can mine and insert ignored transactions by larger miners (if such an attack were to ever occur), then this checks that box for me!

1 Like
#61

It’s the same structure, a chain of blocks, I call it “sub” here just to better map it to the version where subs are “inside” the summary. Here they’re “outside” as if just forming a lower block time chain, and the summary is a block just like them except that it serves as epoch marker and it is restricted from having siblings. IDK what would work better, to have PoW and TXs in the marker, or to have it be an empty 0-PoW with just coinbase in it, deterministically generated from blocks between itself and the previous marker.

Yes and yes, question is how to best implement this in a way that doesn’t break non-upgraded software. My intuition is that it can be implemented in a nice and clean way. We want this to look just like the network did nothing else but reduce block time.

Depends on the penalty scheme, not so much on how we structure the epoch’s data. This is still Tailstorm mechanics. It improves things for those who’d otherwise have their blocks orphaned, but in absence of block reward you’d still have selfish mining. The fee splitting idea might improve that aspect, I don’t know, my intuition is that it would, but intuition can be wrong so we’d have to analyze this somehow.

Sure, say we’re 1 block short of having enough “subs” for a summary, and some uncle block just got announced. Imagine you’re a 30% miner, do you ignore the new sub and try to extend the “main” chain? No. Because you don’t know what other miners are doing. If you spend hash on that, while they include the sub and spend hash on summary, they can reorg you and you lose 1/K reward.

1 Like
#62

There was an open design question on how to handle block reward settlement. I think this would do it:

rect3
rect31158×728 153 KB

With this we:

  • maintain normal blockchain progression (1 PoW per 1 height),
  • maintain legacy SPV compatibility,
  • maintain assumevalid blockchain parsing compatibility.
1 Like
#63

So, there are essentially 2 ways to implement Tailstorm while preserving blockchain structure:

  • The “inner” way, where all of the epoch’s subblocks (best chain + would-be-orphans) are mashed together “inside” the summary block,
  • The “outer” way, where faster blocks are chained together, and would-be-orphans are merged as individual blocks on top of best chain to reach the settlement block.

Both would maintain parsing and assumevalid compatibility with non-upgraded software (e.g. an indexer like Fulcrum could build its indexes as it does now, while completely ignoring Tailstorm), but some things would seem off:

  • The “inner” way would look like as if (K-1)/K hashrate just left the chain and DAA instantaneously adjusted (HF).
  • The “outer” way would look like both target difficulty and average block time got reduced by a factor of 1/K, and with some blocks here and there having the min. difficulty of 1 rather than 1/K.

The immediate benefit of “inner” variant would be reduction of 1-conf (10 minute target) variance:

image
image707×253 28.3 KB

and services would have to opt-in to recognize any subblock confirmations.

With the “outer” variant, benefits would be as if the network changed block time to shorter blocks.

1 Like
#64

the other day I edited the sketch so it shows the longest chain of subblocks, dropping it here so I don’t lose it :slight_smile:

image_2024-10-19_11-04-44
image_2024-10-19_11-04-441080×625 62.3 KB

2 Likes