On BCH-Native Wallets and App Stacks

[emergent_reasons]

On BCH-Native Wallets and App Stacks

The Bitcoin Cash ecosystem is at an interesting place in history. We are about to graduate from simple “send/receive” and start to realize the full power of Bitcoin Script / Virtual Machine that was there from the beginning.

Everybody has different priorities on what needs to happen next, but one specific outcome is that we will certainly end up with one or more native bch wallets plus one or more native app stacks that can that can handle every possible bch transaction. The switch from “send/receive” to bch-native is a simple concept with deep complexities.

Translated: we have a mountain of work to do.

A Suggestion about How to Approach the Situation

Standards for BCH-native wallet and apps are going to emerge one way or another. I suggest that:

1. Nobody has “the answer” at this point - we still need a lot of experimentation.
2. The long term goal is truly excellent, massive-scale, world-class solutions. Something to keep in mind as we build up from the foundation.
3. Design by committee doesn’t work, and design in complete isolation doesn’t work. The incentives of the BCH network give us alignment that we can leverage to find a sweet spot in that spectrum.
4. A conscious effort to communicate along the way increases low-key friction in the form of early constructive debate, and critically reduces the potential for acrimonious sunk-cost contention later on.
5. Having a somewhat cohesive state of things will make BCH ecosystem look more attractive to entrepreneurs and builders considering jumping in.

The current “BCH Devs and Builders” group may be good for this discussion. I’ve also made a focused group “Native BCH Wallets and Apps”" in case that turns out to be a better way.

1 BCH for Design Documents to Get Started

To get things started and to show that this is more than spouting ideas, I am offering a minor incentive of 1 BCH (requires pre-approval) for publication of reasonably detailed design documents from current projects.

Minimum Requirements:

• Pre-approval from me (for 1 BCH - obviously anyone can post whatever they want). You can contact me in private wherever. For example on telegram or in messages here.
• Medium term (not current) architecture diagram of how you think a BCH-native ecosystem should work. Wallets, app stacks, libraries, web apps, mobile apps, browsers, mobile devices, etc.
• How should a BCH-native wallet handle arbitrary utxo and contract state?
• How should a BCH-native wallet handle arbitrary utxo and contract state backup and recovery?
• How should a BCH-native wallet handle exclusive or shared ownership of utxos?
• In terms of “BCH native stuff”, where is your project today vs. where you are going?
• What lessons have you learned, what should other projects do / not do based on your experience so far?
• What issues do you predict are not visible today that will be important in the future?
• What do you need that you wish already existed?
• Are there any specific things you are working on that might become a standard or that you wish we already had a standard for?

A note regarding CHIP Process: As described in the CHIP process, it is intended for hard-line decisions that must be made one way or another, not for softer, open-ended issues like this. Anyone can make a CHIP for whatever they want, but I do not support any effort into making a CHIP out of this topic.

[emergent_reasons]

(Apologies - original list was missing the important issue of state management. Added that to the list.)

[bitcoincashautist]

I’ll go ahead and drop some notes here
I think wallets should differentiate between on-chain and off-chain events.

Imagine this flow, this is initiated by someone else dropping an NFT to your address, an on-chain event triggers it:

1. Your address just received an NFT, what does it do?
2. Your wallet queries BCMR registries and learns that the NFT can be used in multiple TX templates
3. Your wallet fetches the templates and offers you choices of where to use the NFT

What if you’re the initiator, then you can be the one broadcasting:

1. You want to create a template instance where others can access it, like list your NFT for sale, how to do?
2. You spend from some DAO to put your NFT into a contract - this broadcasts a on-chain event
3. People subscribed to the DAO template can learn of a new NFT offered for sale

What does it mean to be “subscribed”? In case of static DAO address, it’s just like p2pkh wallets do - using Electrum to stay updated and parse new TXes and populate the local view depending on TX contents.

But you could also put the NFT into a secret P2SH trade offer, but then who’ll know about it? In that case you broadcast an off-chain event by some means, like the communication between BCH BULL & GP’s auto-settlement back-end. That one’s private, but we could have public relays, too! I think Nostr P2P network can somehow fit in here to fill this gap.

More on “templates”… for me, learning to work with BitAuthIDE was very educational and helped me build my mental model of how stuff works, I encourage anyone to learn the ropes of it, start from simple P2PKH spend from a single UTXO, and then try design a simple covenant that will be dependent on an NFT spend in the same TX. When you use the IDE to make a ready-to-broadcast TX you have to do a lot of things manually, but doing it like that will give you an accurate mental model of what goes where and how it all works under the hood. Then, when you start relying on higher level stuff for automation you’ll know exactly what it’s supposed to be doing.

With that in mind, IMO the IDE’s templates approach is awesome, it’s flexible enough to produce any kind of interaction, we can differentiate between a contract “template” and a contract “instance”. A contract template becomes an instance by filling in the variables and creating the UTXO with the particular setup. Then, spending the UTXO means executing a “scenario” (which gets built into a ready-to-send valid TX).

“entities” will have their specific data which they need to have in order to spend from the UTXO, and each UTXO can have multiple spending paths. Some spending path may require only 1 entity to fill in the blanks, some spending paths may require multiple entities, and some spending paths may require multiple UTXOs to be spent together in the same TX and in a specific way. Entities may provide variables that are read as input script (to unlock an existing instance) or they may provide variables that are used as instance configuration (think hard-coded values in redeem scripts). A contract instance may require creation of child contract instances according to a template - we call these covenants.

I saw someone on Twitter doing things with “lambda functions” and it resembles our blockchain model very much - each UTXO is a lambda function, and when combined in a TX they make a program that transmutes a set of BCH to another set of BCH tied to new lambdas.

So, a template in the IDE tells you what you need to provide to satisfy this or that scenario. But how do you find the pieces (UTXOs) that can satisfy it? That part is not done by the IDE, I did it manually using explorers & Electron Cash, but figuring out working with IDE can inform how to design tools for finding UTXOs.

The template already defines which kind of UTXOs can satisfy a scenario, but it doesn’t define the methods to find those UTXOs, and contract designers also influence this - by designing their contract’s pieces to be more easily findable, e.g. I like to extract state to NFTs and have the P2SH have a static address, that makes it easy to find UTXOs by looking up a static address. You don’t even need an NFT, you can use a sibling input, too! Also, if you hold an UTXO, how do you know which templates/scenarios it may be used to access? Obviously we’ll need some external registries - they may be public, secret, wallet-local, etc.
Then there’s the kind of UTXOs which are just offloading some bigger computation to sibling outputs (like what I did to do uint256 math, or implement a bin2hex function - needed 4 inputs for that one), you’d not find them directly, you’d find them by finding the “main” UTXO and just offseting the output index.

Jason had the great idea with Chaingraph - that of bytecode patterns, and I think I improved on it, see here. With that, we could build a blockchain-external index (like this one) of all contract templates that have been spent from at least once. Then apps could simply refer to a “fingerprint” (hash of the pattern) when building their internal DB of what kind of templates they could access. Contract creators could publish the “fingerprint” + pattern ahead of deploying the contract, and also specify which variable slots in the pattern must be fixed to be considered a valid contract instance, and which are user-determined.

[emergent_reasons]

Good points.

@bitcoincashautist

1. Your wallet fetches the templates and offers you choices of where to use the NFT

This is a great example of why security concerns around template content, usage and access have to be extremely strict. Spam someone with a token, shitty template involves a contract that says it does but actually just sends as many utxos as it can get to some address.

@bitcoincashautist

we could have public relays, too

Also for recovery, although offline is possible, I suspect the eventual go-to solution will be public but encrypted storage of recovery data, maybe through a paid service?

@bitcoincashautist

the IDE’s templates approach is awesome, it’s flexible enough to produce any kind of interaction, we can differentiate between a contract “template” and a contract “instance”.

I agree! Just to note that the templates are not tied to the Bitauth IDE. I think they and their constituent parts will end up being the underlying primitive for all transaction builders.

@bitcoincashautist

I saw someone on Twitter doing things with “lambda functions” and it resembles our blockchain model very much

Yes! “lambda” is more commonly associated with “anonymous functions”, but along those lines, what you are talking about is in the direction of functional programming, which is I think the best description of the UTXO model in a “smart transactions” context.

@bitcoincashautist

but it doesn’t define the methods to find those UTXOs ,

This, state management, is what I predict will be one of, if not the hardest problem we face, especially when combined with security requirements. I have heard some say they think it won’t be that difficult. I look forward to seeing what everyone comes up with.

@bitcoincashautist

Jason had the great idea with Chaingraph - that of bytecode patterns, and I think I improved on it, see here. With that, we could build a blockchain-external index (like this one) of all contract templates that have been spent from at least once. Then apps could simply refer to a “fingerprint” (hash of the pattern) when building their internal DB of what kind of templates they could access.

In general I agree with you, but I think the bytecode fingerprint will be insufficient to describe a template. The entirety of the template will need to be included, at the very least for security purposes. I think your technique should work for the sub-fingerprint of the script itself.

Really appreciate all these thoughts. They bring up several of the open-ended issues that we are going to need to sort out.

[tom]

@bitcoincashautist

Your wallet fetches the templates and offers you choices of where to use the NFT

Why would an end user want to download some untrusted data from the Internet which can most likely be used to cheat that person out of things they value?

I’m ok with Templates in general, but it would be essential to think defensive about them. This is real money, end users expect the makers of their wallet do be the defensive thinker.

[tom]

@emergent_reasons

• How should a BCH-native wallet handle arbitrary utxo and contract state?
• How should a BCH-native wallet handle arbitrary utxo and contract state backup and recovery?

Not sure what you are asking. The simple and direct answer to this is that the wallet stores it in some data structure, just like it stores previously found (and verified) transactions and their value.

Is that a good enough answer, or is the question different than I read it?

@emergent_reasons

What issues do you predict are not visible today that will be important in the future?

I expect a lot of losses of funds due to p2sh data being lost. I expect lock-in into wallet ecosystems due to the same data that is hard to store.

So… one step back. We moved from the old bitcoin-core wallet-type which was a list of randomly created private keys and we reached the HD seedphrase style wallet. The rationale is clear and simple: you either need to do a backup every month, or you need to do the backup only once at start.

WIth p2sh style transactions there is no way for a wallet to figure out how to unlock it (or even find the transaction on chain) without some extra data that basically re-introduces the problem we fixed with HD wallets.

My suggestion is that for many users it is preferable to actually send the entire locking script to be mined on-chain. (nobody will remove p2sh on bch, obviously).
But that means we need to change the ‘is-standard’ rule on full nodes to allow that.

@emergent_reasons

What do you need that you wish already existed?

Some indexer / server that supplies double-spent-proof (beta) state of a random in-mempool transaction. It would take into account unconfirmed parents as well, contrary to the p2p network which only looks at the one level. I’ve been hoping Fulcrum adds it, but nobody has taken an interest so far.

@emergent_reasons

Are there any specific things you are working on that might become a standard or that you wish we already had a standard for?

• dev/CHIP wallet backup - pay - Codeberg.org

• A new payment protocol that avoids splitting a transaction into fields. This stops depending on the ‘address’ concept to request payment. Instead the reciever hands a fully assembled output that it wants the sender to fund. Solves a lot of problems.

[tom]

@emergent_reasons

I agree! Just to note that the templates are not tied to the Bitauth IDE. I think they and their constituent parts will end up being the underlying primitive for all transaction builders.

indeed,

the concept of templates comes from the Satoshi client, or maybe the Bitcoin Core times.

That said, I have never seen a succinct explanation of what bitauth thinks are templates. So maybe it does add some neat ideas. IDK.

[2qx]

From the perspective of a Blockchain Application Developer…

Medium term (not current) architecture diagram of how you think a BCH-native ecosystem should work. Wallets, app stacks, libraries, web apps, mobile apps, browsers, mobile devices, etc.

Ideally, the data architecture would be flatter, and more connected,

Users and nodes would pass data in the standard format, both amongst each other and down.

(node) ⇄ (node)
  ⇅         ⇅
(user) ⇄ (user)

Users should be able get items in the current UTXO set from nodes (and any data they may need), and be able to validate that data quickly. They should be able to easily store that data in a fast key-store index, and share locally with other users. The transport and storage should be pluggable.

The user should get some kind of data with a zk-snark to prove that it’s valid for the current chain they’re following.

A pull:

(request) content identifier
(response) data, proof

A push:

(request) data
(response) content identifier

A query

(request) query
(response) content identifiers

Currently, we have two mining node implementations, and a array of (essentially) indexers with an array of distinct and special features. Each ecosystem that grows up around those features may be cut-off by the maintainer abandoning maintenance of their software. Indexers are also a target for capture and regulation.

If the user is getting data from an indexer they could easily get from any node, and if users could provide provable data to each other, it makes the indexing services less of a target or risk.

How should a BCH-native wallet handle arbitrary utxo and contract state?
How should a BCH-native wallet handle arbitrary utxo and contract state backup and recovery?

The contract state can be handled by a protocol spec that defines interpretation of utxos. Perhaps working backward from what scenarios are possible is better than what we think can be done now.

Utxos can be cached locally and synced from the nodes or peers, and validated.

Not sure if this would work, but ideally, an app developer should get some simple caching storage module (IndexedDB,elasticsearch,postgres, etc) that keeps a copy of the utxo state locally using a protocol available from both nodes and indexers. Similar to the way js-ipfs just gives devs a local service to request from that stores data for them.

Sending a transaction should update the local state, and other cache invalidation is the chore of the storage/network intermediary.

How should a BCH-native wallet handle exclusive or shared ownership of utxos?

It might be better to encapsulate or generate protocol specifications from op_codes, and allow the specification to handle everything possible with op_codes in that manner.

So a contract could transpile to some protocol spec stub, and the developer could annotate what the inputs, tokens or outputs mean. And the different scenarios and states from there.

So if there are multiple signatures required, or no signatures required, that would fall out of the parameters and unlocking bytecode.

In terms of “BCH native stuff”, where is your project today vs. where you are going?

Unspent Phi is set of contracts, using introspection, to approximate the functions of traditional financial instruments. They’re all single party, atomic, without oracles, and have no withdraw feature.

The application layer is handled by custom logic per contract type, ideally that would be generic. It relies on CashScript, but a transition to AuthenticationTemplates would be preferable.

A custom caching storage/network provider that interlopes requests to the network and caches locally was built. There are some quirks and features specific to the implementation.

What lessons have you learned, what should other projects do / not do based on your experience so far?

A protocol template that wallets can use to identify and build transactions would eliminate a lot of redundant custom logic and make protocols easier to implement.

An unexpected amount of time and logic was still spent estimating fees.

What issues do you predict are not visible today that will be important in the future?

The high technical bar for UTXO apps has acted as a shield to keep some bad actors out of the space. When there are a lot of people building things, a party writing a contract to rug people is a much more likely possibility.

It would be nice if a user got a warning like, “hey this signature can rug funds in this time window, using this call” automatically from a protocol template.

What do you need that you wish already existed?

Pat created the CashScript to AuthenticationTemplate bridge. A caching NetworkProvider in js that uses indexedDB is almost done and reduces chatter a bit and is faster than going over the network repeatedly.

Are there any specific things you are working on that might become a standard or that you wish we already had a standard for?

The discussion for standardizing the integration of protocol with wallets is certainly a step toward making protocols “pluggable”. For simple contracts the logic isn’t that horrible. But if there are more risks around the execution of the contract, and more complexity, it might be a bigger maintenance issue for apps.

While such a protocol would be a much bigger thing than I’d like to own, I’d certainly look forward to implementing the protocol for contracts that exist, and providing feedback or implementing some component under direction.

[emergent_reasons]

Great answers. We didn’t discuss approval of this, but it qualifies for my list. If you are interested, please DM me a bch address.

@2qx

How should a BCH-native wallet handle exclusive or shared ownership of utxos?

On this one though, I honestly have no idea what you are saying in your answer. My question is aimed more toward wallets/apps and what the user sees. In the simplistic world of send/receive, it’s easy to have a concept of a total balance. But in contract land, let’s say, with an escrow - is that money part of a balance? Is it some separate balance? Should there be two? Multiple?

The point being that the answer is not trivially obvious and conversely trivially obvious issues in the past such as “wallet balance” need further thought.

[2qx]

Towards a specification for how an app or wallet would interpret a protocol…

If we come to the problem with our current understanding of use cases, and design a protocol specification around the union of AnyHedge and Unspent Phi, we’d box in someone who conceives of a completely new usecase, using script, outside that.

I think this question relates to “owns” the utxos, which could be taken as the entity authorized to spend or unlock funds. But also, where funds can be sent, so the entity or address that locks it.

So a developer could write a Will contract, and feed it into a application protocol stubber that returns three scenarios.

• The principal may authorize funds be returned to themself as an entity
• Heirs may authorize funds be sent to some estate entity after some window.
• Or anyone may send funds to some charity entity, without authorization if the heirs don’t claim it.

So the protocol developer would get a list of three scenarios, three entities and two authorizing parties from the BitcoinScript, CashScript, or AuthenticationTemplate. They’d have to contextualize the entities and authenticators by marking up the stub to describe who those entities are.

But, there could be a situation where someone writes a bunch of contracts without entities that authorize or sign anything. And there could be contracts that don’t send funds to locking entities.

---

To the more practical question of showing a balance with the above template. If the user is the only entity that can receive some funds in a currently valid scenario, that may be reflected in their balance.

But in the situation where someone writes a Will and forgets about it, after some time it may be spendable to the principal, heirs or some charity entities. If contract was made public, and the balance was calculated for those entities, there would have to be a huge asterisk next to it, indicating it’s actually up for grabs.

Similarly in the case where someone writes a ruggable contract, there would have to be a huge asterisk shown to the user indicating that there was another entity that could own the funds. A wallet could check that with inference from the unlocking bytecode if the rug wasn’t in a sidecar or something.

[MathieuG]

Wanted to share some real progress on the native wallet stack: Cashonize supports the pre-alpha version of cashconnect. I know @jimtendo is working hard on finalizing the API for a stable v1 release, at which point other wallets can also feel confident adopting the protocol through the standardized library.

see the announcement on X:

"From the screenshots it becomes really clear what the benefits of CashConnect are:
– wallet permissions
– smart contract templates
– detailed transaction info
"

[_minisatoshi]

This is big. Thank you for all the work!

[emergent_reasons]

This really is real progress. Love it so much.

Later steps will move responsibility for input selection and construction from the app to the wallet, where it should be.

[jimtendo]

The CC pre-alpha does already do UTXO-selection on the Wallet-side in the sense that it if it’s provided a transaction like:

{
  // ...
  outputs: [
    {
      lockingBytecode: "someAddress",
      valueSatoshis: 12345
    },
    {
      lockingBytecode: "someAddress"
      valueSatoshis: 12345
      token: {
        // NOTE: Token category must be whitelisted upon session negotiation
        category: "abcde..."
        amount: 12345
      }
    }
  ]
}

… it will select UTXOs to meet those outputs (and also automatically add change outputs). Inputs can be manually specified too, but they can only be either: a) NFTs Whitelisted (under allowed tokens) during session negotiation or b) not belonging to the Wallet.

But the problem with the above is that the Transaction Shape itself in the pre-alpha is determined by the Dapp at runtime (not in the template). This means that a malicious Dapp could easily sneak in another output there that pays to itself without the user noticing.

This defeats the security assurances that the templates are meant to provide, so signTransaction will be removed in V1. Instead, the Transaction Shapes are moved to the template under the concept of Actions and will be invoked using an executeAction method which will be something like the following:

const actionResult = await wallet.executeAction(
  SOME_TEMPLATE,
  {
    // The action to invoke
    action: 'someActionDefinedInTheTemplate',
    // The data to pass into this action.
    data: {
	  oraclePublicKey: '0x02d09db08af1ff4e8453919cc866a4be427d7bfe18f2c05e5444c196fcf6fd2818',
	  amount: 100_000_000,
	  // ... etc
    }
  }
);

This also simplifies the UX in that the user can view/verify the data passed to the action as opposed to having to inspect each individual transaction (and the data passed to each individual transaction).