Rolling minimum fee mempool policy with decay

Protocol
#11

Dumping to disk sounds tricky, insofar as you want to keep the tx linkages (spent TXOs, and new TXOs) in memory still. I guess that you just need multiple bloom filters.

I guess you might actually want cuckoo filters since element removal is an important property.

Anyway, it’s interesting to note the mempool flooding on ABC right now. Obviously exacerbated by lack of people mining blocks, but informative nonetheless. (And note that practically, ‘full mempool’ on ABC is only ~90-100 MB of tx data, for the txes being used. The 300 MB limit is, as I mentioned, including data structure overheads.)

Perhaps, but I would love it if we had mempool set reconciliation that works to heal any source of mempool mismatch.

#12

(Here is an article describing the real-life full-mempool problem and mempool inconsistencies, on BTC: https://b10c.me/blog/001-the-300mb-default-maxmempool-problem/)

#13

Time to get this going again!

1 Like
#14

Bumping this as it’s been brought up recently.

My 2c: wallets should get smarter about their fees.

I think too many people default to min. fee and got used to having the TX be mined in the next block. This is only possible because our network is underutilized and mempool is mostly empty. This is not something the network can guarantee as it would require infinite capacity.

we can’t expect all users to think about fees too much… one day someone will spend 1 BCH to blast 100MB worth of TXs, and a bunch of users who defaulted to 1sat / byte will be like “bch is sloooow reeee, my tx had to wait for 5 blocks!!”

and then some users/wallets may be like “lol, that’s what you get for not thinking ahead, OUR users were not affected”

2 Likes
#15

I have a dream… Ok, recurring dream. Hmm, checking. Yeah, earliest blogpost I found is Sept 2017 where I introduced these ideas. But they have been refined ever since. The way-to-complicated post nr 3 above shows some progress but not much eloquence on explaining it.

The ideas are themselves not exactly complex, but they are quite different and thus they need a lot of supporting descriptions. So I sat down when this came up on Telegram again and started writing that. I’m sure I missed some concepts, please feel free to ask so I can improve it.

Again, the concepts are not specifically hard, they are just different from what Bitcoin Core introduced, I tried to learn from their mistakes and do it better. They added on and it has become a mess with RBF and more. Instead I think the step back and looking at this slightly differently will help.

I feel that these are the ideas that will help our child leave the nest and start living on their own.

1 Like
#16

Love the writeup. Quick thought, though. Why would wallets getting smarter themselves be an issue? As long as they put the min fee necessary to be in the next block, they should be mostly fine.

On the contrary, using an algorithm to score/weight tx priority, we have little way of predicting future types of transactions, so we don’t know for certain if this might need to be weighted differently in the future. Also now with CashTokens, this gets significantly more complicated. Imagine a financial institution settling stocks built on Cash Tokens. They will pay a higher fee to guarantee they are in the next block regardless. But they might engage in some really weird transaction format, or they could be swapping the same CTs 1,000 times in a day. And that could all be very valid economic activity.

Scoring transactions, I think, might be taking it too far and introducing another potential complication. Maybe putting together a filter that will look at recursive transactions on a P2SH (similar to what I did in tests), but if filtering for economic or anti-attack purposes, people will always refine their attack. I think the economic hurdle is big enough that we don’t need to worry significantly.

1 Like
#17

A transaction that increases the value of BCH is marked as an “economic transaction”. This is much more narrow than the same term used in finance. And I expect that to have caused confusion here.

The point is that a miner that gets paid in BCH is supposedly glad to sponsor a tranaction that directly increases the value of BCH, but not one that just moves tokens around which likely are only very indirectly connected to the value of the coin.

It is an unpopular opinion, I understand, but really nothing anyone should worry about because it is not a huge difference. A cashtoken transaction is in actual fact not buying something in exchange BitcoinCash, it is thus not an economic transaction in the sense that the simple fact that people voluntarily did that trade, the bitcoincash total value increased.
As a result it should be a slightly higher fee BECAUSE todays low-fees are sponsored by miners that want more economic activity and higher price.

No, it is actually a required part of a healthy coin. You keep the balance as described in;
https://twitter.com/FloweeTheHub/status/1664722936744271872

A choice quote:

Everyday trades are the trunk of our tree, the more there are the bigger and sturdier the tree-trunk and thus our entire tree. On a big tree there is no problem with some side-branches sticking out quite far.
We just have to keep the balance, should the side-branch get bigger than the trunk, the whole thing will collapse.

To not keep the balance is what we see in BTC and in BSV.

BSV has almost no actual economic transactions, all are fake. Impossible to filter on, but nobody will disagree they are fake. And the tree has collapsed more than once.

BTC has ordinals and those are not economic transactions. They can have it because the btc value doesn’t stem from utility anyway, but it becomes clear when every time people make a big deal of yet another block without any transactions and then their ACTUAL economic activity has to pay more fees just to get mined…

Not prioritizing your actual economic activity is not healthy. I think BCH did well by keeping fees discounted, but as the worry today is about people abusing that discount, I’ll challange anyone to solve it better.

Simply making actual economic activity (what people refer to as “wallets” above) pay more without any other adjustments is not good enough IMO.

#18

Let me be more clear about the whole idea that’s still floating around of “cashtokens should get the same fee as other transactions”.

We, the kind of central planners, made the miners give a fee discount with the 1 sat/byte.

The idea is to stop us central planners do cental planning. Which means we give the power to set fees to the miners.

What do you think will happen if we don’t create any tools? What will happen is that ALL fees will rise. Obviously. Can’t demand from miners to give away stuff for free.

Alternatively, my suggestion, we can produce tools to help differentiate between economic transactions and non-economic transactions. If we agree we need to stop central controlling the fees and the miners can set it, then they can use those tools to NOT rise the prices on economic transactions.

Thereby helping the most people.

1 Like
#19

A cashtoken transaction is in actual fact not buying something in exchange BitcoinCash

Every BCH transaction is buying something with BCH: it’s buying hashes from a miner.

Fee is the ultimate “score” of TXs economic worthiness. Value is subjective and the fee is proof that the TX has at least that much economic value to whomever made the TX.

Why should miners discriminate their fee-paying customers?

1 Like
#20

Yes, but that is a tiny tiny revenue stream on the whole. A rounding error.

Me paying travalla a couple hundred euros or the transaction paying less than a cent in fees. Both the same transaction.

Which do you think will move the BitcoinCash price?

Because money gains value from usage. People that use your infrastructure without creating value are thus to be charged more.

Only if you haven’t figured out that USING bitcoincash (the money, not the chain) makes the value of the coin go up can you possibly disagree with that…

#21

If we don’t create any tools, obviously miners will create them.

But the tools miners create will be

  • Hidden
  • Not open source
  • Different for each pool
  • Not standarized

Which is generally bad, it tends to create chaos in the ecosystem when every miner uses completely different set of tools to determine his way of getting profit.

For a solution, I particularly like your idea with 2 separate mempools (1: “master” mempool for relaying and 2: “slave” mempool for including in blocks) and some algorithm that decides which transactions get moved from 1 to 2. It would make sure that even if a transaction does not get mined, it gets relayed to another miner that may want to mine it.

An idea is, the rules for moving transactions between 1 and 2 could be decided by some commonly used scripting language like LUA maybe and be a separate “engine” of sorts, so

  • It gets easier for miners to fine-tune their inclusion policies
  • It makes it easier for developers to create few standard types of rules and distribute them as LUA OR
  • It makes it easier for developers to make few sets of “default” choices for miners depending on what transaction pricing schemes they find desirable
1 Like
#22

For now, but at scale and in aggregate it would not be. At 100 MB blocks those would all add up to 1 BCH which ought to be enough for security budget especially since at that level of utilization the price could be $100k.

Ok, but why would Travala accept BCH if they couldn’t just as easily split it and use it to pay some of their smaller bills? It’s all connected, you try to pick out this or that case as more valuable you’re guaranteed to be wrong because you’re just 1 mind and market is a hivemind made of millions.

Infrastructure cost of any 1 TX is a tiny tiny cost on the whole. A rounding error. See what I did there? :smiley:

1 Like
#23

100MB blocks filled with economic-activity transactions are sure as [censored] going to be moving more value than the fees. They’d better, otherwise the people will move elsewhere.

I feel like you’re not understanding the difference between BCH the chain and BCH the token.

The token is the one that gains value with usage. The chain is just supporting that.

See BSV, immense usage of the chain, if you’d follow your logic it would be immensely expensive.
Instead, BSV the token has no real world usage, therefore the whole is not valued high.

On BCH you can have 20MB of memo (the messaging service) transactions and 1MB p2pkh transactions (that are spending older coin).

The ONLY value of the whole comes from the latter.

Edit; lets go into detail WHY:
Lets assume that those memo messages are paying massive fees and thus miners like mining them. Dropping all economic transactions.
But those miners get paid in BCH the token. They can’t actually benefit from the massive fees if there is no utility to BCH the token, nobody will pay much for that. See BSV again.

We’re not to be confused with VISA that charges a percentage of the transferred cost, they are a middle-man that makes money. They gain value that way.
The possible confusion is two fold:

  1. BitcoinCash gives away the infrastructure that VISA charges for for next to nothing. The fees.
  2. BCH-the token, gains value from usage but VISA doesn’t have such a coin. It just supports fiat currencies.

Not really… If bored; read some Rothbard.

#24

PS.

This just got to me guys:

If we create a standarized sets of LUA code that determines the TX fee for usage by miners, these rules could also potentially be used by wallet creators too, ensuring the ecosystem stays in sync and everybody is one the same page.

This would be great for synchronicity, achieving “fee consensus” and removing many errors and misunderstandings.

What’s more, miners too could publish their fee policies globally as LUA to make it clear to wallet creators what they expect.

In turn, wallet devs can use these publicly accessible sets of rules to determine their TX sending fee policies.

1 Like
#25

I think y’all are overthinking this. No rube-goldberg policy can solve this problem:

And the problem is there because we can’t have infinite mempool and infinite blocksize. It can always happen that it starts overflowing and people need to adjust:

  • Wallets should get smarter about their fees, like start using the Electrum get_fee_histogram API to alert user in case mempool situation is not as what he’s been used to.
  • What @tom has been saying: wallets should take some responsibility and track the “last mile delivery” of a TX, don’t just broadcast and forget. Keep watching until it gets 1-conf, and re-broadcast if/when needed.
  • Recipients can do the same: cache inbound TXs and rebroadcast them if they see the TX is not getting confirmed when it should’ve been.

Also, isn’t min. fee supposed to be like training wheels anyway? Once blocks get big enough for orphan rates to start mattering, miners will implement their own fee floors.

I touched on this in ABLA CHIP:

When block capacity is underutilized then the opportunity cost of mining “spam” is less than when blocks are more utilized. We will here define “spam” as transactions of extremely low economic value, but how do we establish value when value is subjective? The fee paid is a good indicator of economic value: if someone is willing to pay 1 satoshi / byte in fees, then it is a proof that the transaction has economic value (to that someone, subjective value) greater than the fee paid. Consider the current state of the network: the limit is 32 MB while only a few 100 kBs are actually used. The current network relay minimum fee is 1 satoshi / byte, but some mining pool could ignore it and allow someone to fill the rest with 31.8 MB of 0 fee or heavily discounted transactions. The pool would only have increased reorg risk, while the entire network would have to bear the cost of processing these transactions.

The limit can therefore also be thought of as minimum hardware requirements, or minimum cost of participation in the network. Some selfish pool could increase the cost of participation for everyone - much before the network reaches “escape velocity” of utilization, and in doing so reduce chances of network success by increasing adoption friction.

What do we mean by “escape velocity”? It would be a situation where many businesses would be running their own infrastructure in order to securely participate in a high-value blockchain network, and it would just be a business cost easily offset by their earnings. If the network would succeed to attract, say, 20 MB worth of economic utility, then it is expected that a larger number of network participants would have enough economic capacity to bear the infrastructure costs. Also, if there was consistent demand of 1 satoshi / byte transactions, e.g. such that they would be enough to fill 20 MB blocks, then there would only be room for 12 MB worth of “spam”, and a pool choosing 0 fee over 1 satoshi / byte transactions would have an opportunity cost in addition to reorg risk.

1 Like
#26

It is good that nobody was talking about a full mempool problem, then.

The rest of us were talking about full block problems. (you can fit a lot of blocks in one mempool)

Why are you writing weird off-topic long posts (3 today alone) that are frankly all over the place and you ignore answers to your points.

In other words, putting fingers in ears and shouting LA LA LA.

WTF man. I was starting to like you.

#27

First you say we are overthinking this, and then you say:

Basically, none of what you said has any conflict whatsoever with what I have said.

Both can be done.

But I am unsure if what you are saying that “wallets should take some responsibility” is not much more troublesome than what I propose.

So, you’re asking all wallets in a decentralized ecosystem with no central planning to do “this” and “this”, “somehow”. You’re saying “wallet creators, be wiser”. This is too generic and ambiguous.

Well I am not sure this is going to work out better than just having a set of public LUA policies of fee inclusion. Wallets can just download them and use them “as is”.

The problem here is that every wallet will figure out on his own what works - meaning it will be chaos.

Having a set of rules standarized and publicized will create more clarity and less chaos.

1 Like
#28

Wallets can just query the already available get_fee_histogram API and use median “as is” as the default fee.

There’s already a set of rules standardized and publicized:

  • (1) 1 sat / byte is the fee floor, nothing below will get accepted by most (if not all) nodes
  • (2) 1st seen rule
  • (3) dynamic fee floor & mempool eviction, if mempool grows bigger than few blocks worth - admittedly this one wasn’t publicized well, I learned about it the other day, around the same time as @_minisatoshi who wanted to open a topic about it, and then I found this existing one and directed him to it

Instead of coming up with even more rules and overhauling the whole system (what problem would that solve?), how about first figuring out the implications of current rules? Maybe all that’s needed is more standardization and publicization of existing rules, and a tweak here or there.

Many people don’t know about (3), so we’re doing the publicizing now.

Ideas for some tweaks to existing rules:

  • Tweak to (1): reduce min. fee to 0.5 sat / byte now, and commit to halving the min. fee with block rewards halvings, so next halving we’d reduce to 0.25 sat / byte.
  • Tweak to (3): not really a policy change but rather implementation change: when TXs are evicted from mempool, dump them to disk and store for later. When dynamic fee decays, try to load them back into mempool (if they’re still valid) and rebroadcast. This would help users whose TX got evicted during whatever rush hour.

Implication of dynamic fee floor is that it’d be possible to attempt double spend of low fee TXs by:

  1. Broadcasting some TX with min. fee
  2. Flooding the mempool to get it evicted
  3. Broadcast an alternative version of the TX, which would not be rejected by rule (2) because the 1st TX will have been evicted already

Note that the attempt would still cost the attacker at least 0.32 BCH / attempt, so min. fee 0-conf could still be OK for payments smaller than that. Strategies relying on DSPs could take fee into account when evaluating risk.

You can talk about adding more rules, taking into consideration things like coindays or number of UTXOs destroyed / created, but wouldn’t that only make it harder for everyone to evaluate risk and know what to expect in case of some burst of TXs filling the mempool?

It’s literally the topic here, recall the opening post:

OK I’m sorry for calling it rube-goldberg, no need to get all touchy about it.

1 Like
#29

This isn’t entirely accurate. There was no minimum fee beforehand. We put that minimum fee in. Calling it a discount isn’t really accurate. And I doubt you’ll find any (at least OG) miner that would prefer few tx with high fee to many tx with low fee.

Overall – I don’t think we should be implementing any real filters on transactions. At most, obvious 0 economic activity transactions (like respending the same UTXOs over and over and over thousands of times per block) could be filtered, but again, leave that to the miners. They are more than capable of deciding if they want to filter that out themselves. We don’t need to do it for them.

My point, here, is that there is no way to predict what an economic transaction is versus not. Yes we can make very accurate guesses right now, but no way can we see the future with certainty.

As I explained in my other post about exploring the future of tx fees replacing the block reward, this can be SIGNIFICANT. Who cares about right now? Any additional fees to miners are good. Even if they are small.

If we think this is an issue, then Bitcoin has largely failed as an idea, imo. Bitcoin runs primarily off of incentives. It was devs with powerful backers that fucked up BTC. But in its natural state, Bitcoin and all parties are held accountable and will do what is in their best economic interest. Who cares if these tools are different. Miners can choose to do as they please, as always has been the point.

1 Like
#30

This reads like we’re actually fully in agreement. But you seem to have some confusions.

Nobody is filtering. No filtering is proposed.

The entire idea is to create tools for the miners, decentralized decision making and free market choice. Nobody is filtering, and miners are the ones that decide how they want to sell their blockspace. They decide. “We’re” not doing any filtering or ranking or anything.

I hope that clears up the page I posted, you might want to take a look at it again. The entire premise (as detailed in the first part) is to move stuff over to the free market what is today really centrally planned (due to lack of tooling, I might add).