CHIP - Unconfirmed Transaction Chain Limit

CHIPs Standards Track
,
#17

I think this is the root of our miscommunication. This discussion is ONLY about the unconfirmed transaction chaining limit. I am not advocating BCH begins promising “fire and forget guaranteed”. The other stakeholders are also not making this claim as a part of this document. Please help me find how I can better articulate this distinction so that others do not also assume the same.

Does this clarification change your opinion regarding the unconfirmed chaining limit? I assume and hope that it will, but if not, then let’s please focus on the technical reasons why a limit should be imposed and what that limit should be, because at the current point in time I still don’t see anyone providing evidence supporting its continued existence.

#18

I believe I am that person. I’m representing (to the best of my ability) the people that have endorsed this request. I am also taking care to point out when I’m speaking on behalf of others or when I am speaking to my own personal opinion. Although I’m sure there will be mistakes.

2 Likes
#19

Your anecdotes specifically refer to the problem where you, verde, had a problem where you did a fire-and-forget which failed. And you seem to conclude that the limit should be changed. The reality is that this is too simple.

The other stakeholders have had exactly the same kind of stories where they claim a single interger will change their product from non working to working. Again, that is simply not true and too simple.

I think I fuly agree that this is the core of the discussion. But the one that failed to communicate this is me. I apologize and I will be more clear.

I want to make clear that solving the problems stakeholders have stated will take more than changing one number. Changing one number will be much like BSV changing to a huge block-size. That change doesn’t accomplish scaling. It just does for a litlte while, until it doesn’t.

Removing the limit doesn’t make stakeholders’ usecases more reliable for very long either.

Edit: We can make this a simple technical statement without any usecases of a number that should change. Nothing more, nothing less.

As long as the chip is about solving usecases that assume fire-and-forget, then you should include not just this magic number, you should actually solve the usecase not just for this year, but also for next year. So it doesn’t come back to bite you or the other stakeholders.

2 Likes
#20

This is unlimited for all intends and purposes, and if other nodes can achieve this as well, it’d be a great solution.

2 Likes
#21

I agree with that. It would be a great step forward to removing a silly limit in the software.

2 Likes
#22

Ah, gotcha. Yeah, that’s a fair criticism. To summarize your point to make sure I understand it: even if the limit is removed, all of the problems described in the anecdote don’t just vanish. Transactions can fail for a plethora of other reasons. I agree with this; application code doesn’t become completely streamlined only with the removal of the limit. It does reduce the likeliness that the edge cases are encountered though, and more importantly: it’s one less thing to worry about among a sea of others.

In the end, I think it’s a criticism of the document though, right? Not the limit itself. I’ll add a paragraph to the document in a near-future revision that explicitly calls out that BCH still won’t be “fire and forget” with removal of the limit so it’s clear for historic reasons and for the other stakeholders.

I totally agree. Application developers still need to handle the edgecases–of which there are many. I’m just hoping we can make it ever so slightly easier for them.

I think this is super valuable feedback for the release of the CHIP. Thanks, Tom. I want to include anecdotes such that the need is communicated well, but at the same time we definitely need to be clear that this change doesn’t promise things that aren’t true. I’ll revise this weekend or early next week. Would appreciate your feedback on that once it’s done.

2 Likes
#23

Thanks for raising this CHIP.

BCHN is still evaluating whether we can only raise (and by how much), or whether we can eliminate this limit entirely for May.

At this point I don’t have numbers yet on the performance of the options we’re investigating, but we should have those quite soon and we’ll report back here once we’ve got them.

2 Likes
#24

For reference, General Protocols have also encountered problems with regards to the 50-unconfirmed chain limit. We don’t create these transactions ourselves, but we have had submissions sent to us that we were unable to broadcast due to the limit.

#25

My reasoning for a two-step approach is that we get some real-world testing between 0 (current state) and 1 (fully removed). We might be really confident right now, but I feel it’s prudent to get more real-world data.

Ideally, the 25->50 raise would’ve given us data, but the change was so small that I just can’t say I learned anything from it, other than that “50 is not enough”.

As for the 5000 number, I feel that if there’s anything we might’ve missed, maybe it’s visible at 5k? I’m not sold on any specific number, but I don’t want a case where it’s just a small minor bump and we still don’t learn what stresses there can come from it.

This tweet kindof shows why I feel the way I feel about this: https://twitter.com/brenankeller/status/1068615953989087232

As for removing CPFP, I think you’re absolutely right in that we shouldn’t let that deter from the focused discussion and it’s up to each node to offer or not. Generally speaking I like clean solutions, and sometimes that means cutting features no one, or only a few, actually uses in order to get to a better state for everyone else.

#26

Link: GP Statement on CHIP “Request Update to the Unconfirmed-Transaction Chain Limit”

Text from summary:

GP is supportive of the premise of the CHIP. In order for GP to have confidence that the CHIP is worth investing resources into, we would require it to have a more explicit owner who takes on some accountability for the eventual outcome, and a more complete description of the costs, risks and benefits of the various options. As one example, CPFP is objectively still used, and misaligned changes could lead to fragmented mempools, degraded 0-conf reliability, and ultimately damage BCH utility and value. More generally, since this is not a consensus change, it may also be a good idea to add discussion about what safe cooperation looks like. In an effort to be constructive, GP created this template as an example of how to fit these concerns into a CHIP.

GP looks forward to providing more specific feedback and support as the CHIP evolves.

#27

In BCHN, we figured out that part of the reason we need a limit is because of performance issues associated with the counting needed to check the limit. Fully removing the limit could thus reduce code complexity and improve performance compared to simply raising the limit. So unless there is a reason to keep the limit, we should remove it. As noted before, even if there is no limit, in practice you still have the mempool memory limit - but that does not have performance issues because the node has much more freedom on what to do if that limit is hit.

1 Like
#28

The only reason I can offer is what I’ve given above - but I’m developing a full node myself so my views are from a more generalized perspective. It’s also just an opinion, and not a conviction or faith, so if everyone lands on the same page and says “lets’ remove the limit entirely”, then I 'd be happy about that outcome as well.

#29

I think its’ not good enough. If I fail to send given the UTXO’s I’ve selected, and I have other UTXOs I can use, then I shouldn’t be expected to wait. The problem is I can’t go around and try to brute-force the UTXO selection by broadcasting over and over.

Particulary so since this isn’t concensus, and one node might accept while another rejects.

I would like to be able to know the policy of the node I’m broadcasting too as part of any errors (too-long-mempool, utxo [outpoint] has X ancestors while we only allow Y), and better access to understanding my own UTXOs (how many ancestors does this UTXO have).

#30

@freetrader @BigBlockIfTrue have you been able to conclude whether BCHN can remove the limit without adverse effect, or am I misunderstanding your take?

#31

I don’t know if @BigBlockIfTrue has concluded something, but work on CPFP removal is still in progress. There are some open questions related to the keeping of counts (which we wanted to retain if possible even if we remove the CPFP feature), and the status is that we don’t know feasible the retention of that counting is. If we can’t count accurately, how can we maintain a limit? - this is one open question. The other is the performance under CPFP fully removed - we have yet to assess whether it would allow us to remove the limit entirely or how much we could raise.

Figures mentioned in this thread are obviously higher than the x10 increase for which we were conservatively aiming so far.

Even if we get positive results for the path of complete CPFP removal, I’d prefer is we also assess how much performance we can get out of it while keeping CPFP.

If push comes to shove, we could probably remove CPFP in BCHN first, and re-introduce it if there is an actual need for it, but it’s not my preferred route.

#32

Was asked to add my comments here after making them on Telegram. I think they echo some already made but here goes:

Cache management is one of the hardest things to get right in “computer science” - and that’s ultimately what the Mempool is for our decentralized ledger. Right now we’re depending on undefined behavior (as in its not part of the “official” specification) where nodes have created an arbitrary limit as to how deep it can go. Worse yet, different nodes can have different limits. This gives us an excuse to ignore the actual realities of physical resource limits of whatever hardware the node happens to run on and likely results in an underutilization of its potential in most cases. Best to recognize that there are physical limits that will be hit and enact official behavioral guidance policies that will inform transaction submitters on what conditions might make their txs more likely to get prioritized. But absolutely make clear that a pure “fire & forget” wallet model in a decentralized trustless system is utterly irresponsible.

So I think we need to remove this limit but that’s only the first step in a much more complex and longer effort to make clear what is defined and undefined behavior for the network side of the protocol. The goal should be to clearly define the boundaries of what can be counted on and good heuristics to help devs understand the conditions that might make their transactions get kicked out and have to be submitted again. I admit our own initial BCH development assumed a fire and forget model as there wasn’t any documentation that implied otherwise. Perhaps a good specification of what a well behaved wallet model should be would be in order? Anyway - I’m glad to see the efforts BCHN are making to get this first step done and wish them luck.

2 Likes
#33

I think we can say now with high confidence that we will be able to remove the unconfirmed tx chain limit from BCHN completely in May.

The time before May seems too short for us to fully explore keeping CPFP around in the way that BU did in their client. So BCHN client will most likely NOT support CPFP after May, at least for a while. If there is some real demand for CPFP that would merit additional complexity, we would consider re-instating it at some later time.

8 Likes
#34

I’ve updated the post to match the proposed CHIP format/structure and updated this CHIP’s version to 1.1. These changes are reviewable at bitcoin-cash-chips/unconfirmed-transaction-chain-limit.md at master · SoftwareVerde/bitcoin-cash-chips · GitHub .

3 Likes
#35

Nice update! GP has accordingly updated its statement.

TLDR: :rocket:

#36

Without intending to be rude, I want to draw attention to the fact that the Technical Description section of this CHIP is essentially garbage at this time (unused template stuff and a link to another document without any context). I really want to urge the CHIP authors to improve this, quickly. At the very minimum, this section should specify the exact activation time and mechanism of the change.

2 Likes