Lower the default relay fee, create a fee estimation algorithm

Technical
#42

BCH devs have at least two mandates:

  1. Listen to users, facilitate as many value-adding usecases as plausible through protocol and software improvements.

  2. Make sure the network remains reliable across the board while doing #1.

Note that #1 and #2 are often in conflict. Take the fee thing for example, no matter how low the fees are, unless they’re literally zero there will always be “why not lower” calls, and they can’t possibly all be satisfied before unrealistic policies start cutting into reliability of other things we care about (say, 0-conf). Compromises that invite hate and disappointment will have to happen constantly - the alternative is the community and chain falling apart by everyone taking their balls and going home, reducing network effect to zero. I’ll always heavily side with #2 when such a conflict arises, others might think differently, and that’s totally okay - different people have different risk appetites.

Re: Letting the fees loose and free-for-all it by having unreasonable defaults: Trust me, I’d love to do that, it’ll at the very least take the burden off developers - now everyone will have someone else to lobby. Unfortunately we can’t do that, because highly fragmented mempool policies across the network is very bad for 0-conf. Maybe it can get better if the miners will all get together in a smoke-filled room to coordinate fee levels, but that’ll still be bad for whomever not invited to that room. Won’t we want ways to formally/permissionlessly vote and coordinate fee policies, so nodes can automatically adjust? Hey I sure do, but that’ll take time, and likely still invite hate anyway because not everyone has faith in the soon-to-be-decimated chinese miners. Maybe there are better ways than the two I listed above. I don’t know what they are, but one thing I’m sure is none of them will please everyone, not even if we restrict the target audience to “low fees at any cost” folks.

In the end there probably aren’t even “good” solutions, only bad ones and kinda plausible compromises. Let’s identify what a latter looks like and don’t rush while doing it, shall we?

2 Likes
#43

Off-topic, but this “don’t rush” …

show more

…stuff is what literally cuts my trust in BCH (and its ability to make any decisions)… Scaling debate all over again. Didn’t we go through all of this already? Shouldn’t we learn to make decisions instead of putting everything into “no rush” mode? It will just solve itself in 18 months! Somebody will solve it somehow.

Decisions move us forward (change the block size, change the faulty algorithm, change it again, kick out lazy lead dev), “no rush”, “let’s play it safe” is BTC approach.

To be clear, I don’t have a good solution neither to the fee problem, nor to the governance problem. But I really don’t want to get stuck in “let’s maybe think about it” mode while looking at $100 BCH fees. (no rush, though)

#44

I mean, on the other hand we have been doing “make bold decisions, don’t ask, just do it, fork if we disagree” for most the past few years. I’d argue the results aren’t very encouraging and there’s always room to fall further.

There’s a likely a… drumroll… reasonable middle ground :scream: in between “we shall never hardfork” and “just do it bro”, but people gotta agree it’s worth doing and defending (this part was severely lacking during BTC scaling wars) instead of going one extreme versus the other. If we just “be bold” and go it alone every single time there’s a disagreement - be it IFP or CTOR - instead of having compromises, when do we get to the actual adoption part that has been reset several times already, each time with less fanfare?

2 Likes
#45

That was a lot of meta and I can see the arguments of both parties. And honestly I think there is a lot more overlap than either party (likely) thinks.

First of all, the first planned upgrade is almost a year in the future. There are literally a dozens of topics on this forum discussing things for activation at least that far in the future. Some even furhter. This topic can certainly be planned for the future too. I never got the impession it was meant for shorter term activation.

Discussing solutions on how to do things well, how to solve problems and what timeline is safe, those are all done on this site too. Discussions and allowing people to openly worry about something, those are also a really big part of keeping the chain together and avoiding people taking their balls and going home. So, fine Im_uname, we don’t have to activate anything this quarter :smiley: . But we should be able to discuss this and consider solutions, which include realistic timelines.

One very important part about the timing is this nice quote:

I presented an ecnomically sound solution that allows the market to find a price. No smoke filled rooms needed. BUT, it indeed would be bad for zero-conf in the current network.

Now, here we have two choices.

  1. The topic is discussed and we can find a solution that people can work on. Which in this case includes a requirement that the solution is not going to be rolled out until double-spend-proofs are much more universally used by merchants and wallets.
  2. or we stiffle discussion with the direct effect that some people get upset and leave our chain.

I believe we need to have trust that the smart people part of this movement will do the right thing.

1 Like
#46

Adding one more thing: having the ability to “make decisions” without resorting to easy outs like dictatorships or never making any decision whatsoever is not trivial.

BTC took the easy way out: Just never make difficult decisions. That got them to where they are - network effect preservation can be really powerful even in the face of stagnation and gross degradation of usability. The fact that ransomware criminals still choose BTC is a testament to how powerful it is. We do have a different mission and cannot possibly choose this path. So…

ETH and a number of other coins took the other easy way out: Just have a central body decide everything, be it people in a specific weekly call, one guy, one company, or I-swear-these-premined-tokens-are-totally-not-all-in-the-hands-of-three-VCs tokenholders. That might look very attractive as humans have thousands of years of history saying central authorities work, at least for a while. ABC and BSV both went down this path. But is this what we really want? What’s the point of permissionless p2p cash if one body tells you exactly what to do, and the whole place rises and falls with them?

I like to think BCH is traveling down a path less traveled - we’re pushing the boundaries of how to make reasonable decisions without crowning anyone or anything. To convince as many people as possible, avoid landmines, keep the place from falling apart, yet still service the world as your money. That’s hard, but also worth doing IMO - for the alternatives are much worse given our brief mission statement on the subtitle of Satoshi’s whitepaper.

And while we’re doing that, attempts to figure out the reasonable thing to do will always look like the “stagnance” of #1 or “devs fucking with things” of #2, to people who like #2’s “efficiency” or #1’s “immutability”. It’ll be easy to take potshots at these straightforward points, while the nuances of the third approach will always be much harder to defend. I don’t have good solutions to that, only plausible solutions we can try, that some of us are indeed trying. But the more people joining the effort instead of taking potshots from either end, the higher our odds for success, so I’ll urge people to join where they can.

5 Likes
#47

I can’t resist. I feel the need to chime in here. FWIW I’m into the following 2 ideas:

  • bring back coin days destroyed. This was the bitcoin I grew up with back in the day and that I knew and loved.
  • be bold and set the default relay fee lower than 1000 sats/kB and let the market sort it out.

That being said I don’t want people to split over this so I’m ok with whatever ends up materializing (or not) in this regard. Right now we’re looking at a mini (or major) bear market anyway so… it’s not like the world is on fire yet with regards to fees…

3 Likes
#48

Earlier I replied using my developer’s perspective. This reply comes from my perspective as a startup owner who uses SLP tokens in the services that we build and provide.

As a startup owner who uses SLP tokens, the min tx fee relay impacts our business. While BCH price is low, this is not a problem, we only pay about 800 sats per transaction ( which includes the dust limit ). But, if Bitcoin Cash price increased 10x this would become a very expensive business cost for us and our user base. I like that the BCH community is proactive when it comes to scaling. The blocks being 32 MB which leaves a lot of room for growth. But with min tx fee relay at 1 sat / byte this can become a problem extremely fast. If BCH rises 10x in price which could happen in a very short period of time ( Just look at DOGE coin ).

Thank you to everyone who is partaking in this conversation about min tx fee relay, cause while it may seem trivial it does have a big impact on businesses and scaling just like block size.

2 Likes
#49

Also I think it is important to note that when price does increase 10x many new developers, businesses, and startups will take a serious look at Bitcoin Cash. If BCH can still maintain sub penny transaction fees even with a high price per coin, that will make BCH look very appealing to all the new devs and businesses that are considering to use BCH to build their products and services on top of.

2 Likes
#50

Going off what @im_uname wrote earlier

Some form of algorithm that reads from aggregate fees of past n blocks. It’s easy to see how that’ll work in a congested chain - minimum inclusion fees are fairly useful as proxy to demand, due to users bidding against each other (otoh the UX, as we know, will be absolutely horrible). On a mostly non-congested chain like BCH I’m not sure how that can be done though - in “peacetime” we’ll have the overwhelming majority of tx be at the exact prevailing fee floor, so reading from that aggregate of fees doesn’t yield much useful information. We can’t have the floor constantly lower itself from prevailing fee, since that inevitably ends in a jungle of different policies as miners ditch the unreasonable adjustment algorithm. Again, 0-conf is far more important than crossing some arbitrary line denominated in a constantly devaluing fiat.

I’m going to throw a hybrid notion in the room. Based on, long term, we need to be able to rely on miners acting in the best interests of the currency anyway?

The proposal might look like letting miners vote on algorithmically suggested decreases + increases of the minfee.

We could have a mechanism based on the past N blocks, suggesting a small adjustment up or down, which is visible to all via node info (like EB), but does not take effect unless enough hashrate has voted for it to take effect. After it has been moved, it cannot be moved again for a little while (maybe 1-2 weeks - expressed in a number of blocks) to allow the algorithm to evaluate and propose a next action (which could be up, down or no change).

Or we could simply make the adjustments +1%, -1% or 0% and put a supermajority threshold on activation of a change.

Mining fees and relay fees would be coupled (equal) under such a system, ensuring that propagation is not impacting by differing policy.

The applicable fee level at any time would be something that full nodes can calculate from the blockchain data (based on past fees + hashrate votes in the blocks).

Full nodes would need to publish this, and other apps would need to trust that. This seems to be the downside, possibly the Achilles heel of this? - it would seem susceptible to sybil attacks unless we can find a way to guard against that, maybe with some validated commitment.

Another problem with it is that due to blocks validating at different times, there would need to be some kind of tolerance / buffer built into it, or everytime it adjusts, some txs might be kicked out of the mempool even though they were acceptable “until just now”. Can’t think of a very elegant fix for it right now. Might be the second Achilles’ heel of such a proposal.

1 Like
#51

Instead of combining past fees + hashrate votes, with a careful enough voting process with appropriate limiters, a case could be made for a simple hashrate vote.

1 Like
#52

I still got the feeling that with any kind of automatic - or vote based - adjustment, we will need to build fuzziness into the minimum fee rate. Or suffer ugly transition effects.

#53

Indeed they should - perhaps some sort of “don’t evict when in transit” rule when a new rate activates.

Also in the short term, we can make the number not automatically adjustable above 1sat/byte (unless manually adjusted, of course) to prevent abuses.

1 Like
#54

Or perhaps simpler:
Every miner signals in coinbase or such a proposed fee, expressed in sat/kb, and every 2016-ish block the avarage (or some fancy-pants algorithm to avoid abuse or keeping transitions smooth) of all votes are used as a fee for the next two week cycle.
Further tweaks on this calculated value could be applied to coindays, consolidations or fan-outs etc.

Yes. Don’t evict transactions that has already been admitted to the mempool.

2 Likes
#55

I gave the above thoughts some more time and I think this is a really viable solution.
To be more precise.

  1. Always allow fees >= 1000 sat/kb. That fee was OK when BCH was down under $200 USD/BCH and should be fine going forward and this shouldn’t break any existing wallets fee estimation.
  2. Each miner has it’s own preference of a minimum fee of X sat/kb. In each coinbase transaction there is an OP_RETURN output with some info specifying X sat/kb.
  3. At block_heigth % 2016 == 0 (every two weeks) the median of each fee preferences is used as the minimum for a cycle.
  4. Never evict old transactions from the mempool due to a higher fee activates.

SPV clients must download and parse every block header (should be done today) and coinbase transaction including merkle proof. A little less then 1 Mb / 2 weeks which is on par with 1 minute of a track on Spotify.

The main weakness for 0-conf occurs if the fee will transition upwards, say from 100 sat/kb to 500 sat/kb, which can be anticipated days before the next cycle begins. An attacker sends one transaction with the low fee very close to when the last block in the old cycle is found and then sends a new conflicting transaction shortly after. This is roughly the same risk as sending two conflicting transactions at the same time today (“fast respend”) and DSP is one mitigation technique. Also, merchants wallets knows in advance when this weakness can occur and can take some additional seconds to wait for a DSP in this case that happens once every two weeks.

There is also a risk of a malicious miner that bumps the fee to 1000 sat/kb by 51% attacking BCH and mine 1008 blocks and thus cause a DoS attack requiring very high fees if the value of BCH has mooned to an extrodinary high price. I don’t think the high fees would be the biggest issue if the chain was 51% attacked in this case…

Of course the specifics above (2016 blocks, median of all preferences) are just there to an example to the idea.

1 Like
#56

This is really a fee messaging system, and miners are free to ignore it and still refuse any TX above such minimum. A system to penalize them for disrespecting it would make the scheme more convoluted and complex and still impossible to enforce because mining is anonymous. Making miners authenticate further increases complexity and changes the design paradigm of Bitcoin… So it all depends on goodwill of participants in whatever coordination system.

1 Like
#57

I don’t know if this was directed at me or more broadly towards the entire topic.
We already have a fee messaging system today (someone sets 1000 sat/kb as the default value) and miners are free to ignore that right now. I haven’t proposed a penalizing method nor mining authentication since I don’t think there is a need for it. There might be a need for a better fee messaging system going forward and that’s what I think we are discussing.

1 Like
#58

Since we’re having this conversation, and in brainstorming mode, I’d like to point out something Mr Toomim mentioned a while ago about fees, I hope he can elaborate more himself sometime. I’ll try to characterise the problem from the pov of miners, as I understand it.

With each additional transaction a miner is willing to include in a block, they increase their risk of orphaning. The bigger the block is, the longer it takes to propagate, so a smaller block found at the same time could win the orphan race. So there is a real tradeoff in including each next transaction, up to a threshold which is that miners risk tolerance.

The propagation time is relative to blocksize, but most of all is relative to checksig operations, which are definitely the most expensive (and time consuming). This is a cost payed for by all relaying nodes, and not just miners, so it makes sense to keep at least some fee as an anti dos feature.

I also like the idea on coindays-destroyed for prioritizing transactions somehow. I hear that NANO, after the recent spam spree, have implemented something based on coin age, and it has been so far ok.

#59

I’d say that the propagation time is first and foremost relative to the mempool consistency between nodes. If the mempool of node B have a superset of transactions as the mempool of node A then block propagation from A to B does ideally not include any additional checksig operations – they were already done when the transaction was included in the mempool at some previous time. That’s one of the reasons that the mempool policies should be uniform over the network and miners are even incentivized to have it that way. That’s also one of the reasons for adjusting the fees by some parts of the network setting the parameter independently is a bad idea. Another reason is the 0-conf breakage…

2 Likes
#60

My 2 cents:

This problem is actually only difficult if you try to find a perfect solution. But it is actually not necessary (or perhaps even possible) to create a perfect fee calculation solution on the first try.

Even creating an algorithm that calculates fees with 80% correct precision 80% of the time will be good enough for now, but we can simply upgrade it later.

For example, a solution based on fetching BCH/fiat price data from centralized services and taking an average out of it would work, at least temporarily - before it would start getting abused.

So a good way to play this could be to create a quick and working-for-now (but not dirty) solution that will fix the problem for the next year or so and use that time to create a “perfect” or “near-perfect” solution that will satisfy the most long-term.

1 Like
#61

Also, again: I agree with @mtrycz @cculianu and several other multiple people in this thread that ask for returning Coin Days Destroyed to the codebase.

I didn’t even know Core removed it.

They never had any valid reason to remove it. They probably wanted to break on-chain transactions to make profit on their sidechain and layer2 solutions.

1 Like