WebRTC/WebSocket-based P2P networking for BCH

Infrastructure
#1

The original bitcoin software is written in C++, is a standalone application and uses low level ‘sockets’ to build its peer-to-peer network over the Internet.

Then we have the other world where apps run in a webbrowser, are using the Internet only via methods webbrowsers support: http and web-sockets.

Those two worlds hardly ever interact and what I want to focus on is thinking about decentralization in a world of the web.

The situation today is pretty dire. Any web-app or browser-plugin is not capable to connect to full nodes via the peer-to-peer network because the browser simply won’t allow web-sockets to connect to normal Internet sockets.

What we mostly see today is web-apps that connect to an indexing server in order to sync with the blockchain. Most of those apps even have the indexing server specified by the server-operator. Is this decentralized? Not only is the webserver that hosts the app a central point of failure, the indexing server is another central point of failure.

An obvious solution is to make such web-apps connect to a lot of indexing servers, just like our p2p network has lots of servers.
At the moment we are simply looking at people adding some 15% more disk space usage next to their existing full node.
But as explained here, the cost of such indexers will continue to rise and it won’t simply be the additional disk space as our base of users grows. As usage grows, the machine to run such an indexer will start to cost more, costs growing much faster than Moore’s law. Which again leads to centralization as the cost of running such indexers means more will not do so.

Bridge the gap

If we are interested in bringing the decentralized and permissionless development to the web as well, there is a need to bridge the gap between the two worlds outlined in the start of this article.

It feels very important to make sure that the two worlds actually connect. The existing peer-to-peer network that full nodes use is the core of our decentraization today, reusing that means we strenghen the current decentralized network. Remember basic networking theory; the network grows in value relative to the amount of possible connections. Having one peer to peer network used by full nodes as well as by the web will thus have a massively more value than two seperate ones.

Yet, the problem is that the web uses ‘web-sockets’ while the full nodes use raw sockets. Conceptually the same, but incompatible. So, websites can’t use the existing p2p network.

But what if we made full nodes create the peer to peer mesh over both raw sockets as well as allowing people to connect on web-sockets from their browser?

Adding web-sockets to software like BCHN would be easy enough, afterall it already has a web-server (using libevent) for the RPC/JSON interface. The biggest issue is likely encountered by users deploying the full node: web-sockets use a priviledged port and thus needs configuration to make it work. On Linux with systemd that is easy enough, maybe someone with knowledge of Windows can comment on how hard that is.

So, if web-browsers can connect to full nodes directly and talk with some nice JavaScript library the native peer-to-peer language, what would that give us?

A web-wallet, deployed via ipfs / on many servers, that stores all its private keys locally. It can then become an SPV client (using merkleblock messages) without any central servers, getting blockchain data from the big amorphous p2p network.

Just providing this option in full nodes won’t magically make web-apps decentralized. But the reality is that web-app developers today basically don’t really have any options. I end up saying to devs that they can go to commercial suppliers of APIs, but naturally open source devs find that less appealing.

What do web-devs think about this? Would you use a library (assuming someone smarter than me writes one) that makes your wallet update via the p2p network?

6 Likes
#2

Super exited about this, thanks for starting a topic!

I responded on telegram, just copying it here:

and:

1 Like
#3

Aside: maybe it would be easier to find this topic if you name it to something like WebRTC-based P2P networking for BCH?

2 Likes
#4

Thanks for your reply Jason.

I think we are a good combination of people posting here as the two worlds of development leave a very low number of people that actually know both well enough. I certainly am not an expert on the web. I only really heard about webrtc from a distance.

Happy to see enthusiasm for doing some experiments on what could work for our entire ecosystem.

2 Likes
#5

To my understanding, although the IPFS folks have been asking for years, it seems like w3c is reluctant to allow WebRTC connections outside of the main window context of a browser. So it can’t be used in the service worker of a PWA, or SharedWorker running an SPV wallet/indexeddb process in the background.

So in an ideal world, this wallet might end up looking like this?:

  • Shared Worker - High-Level Wallet API… spawns:
  • Worker 1 - existing SPV Wallet logic (wasm)
  • Worker 2 - Abstracted wallet storage, using IndexedDB, et al.
  • Worker 3 - Abstracted wallet networking, using Websockets / WebRTC (via hack)

I don’t think there is any storage limitation for IndexedDB, however, two large maintainers of browsers (Google/Apple) could become hostile toward PWAs, if they competed with app revenue. And the third major browser (Firefox) can be somewhat hostile toward IndexedDB usage in any privacy related context.

#6

Where does QUIC fit into the comparisons here. It seems to have gained some traction and I understand it is being implemented on XEC. Pros?, cons?

#7

Good to know! (relevant issue)

I need to look into it more, but I expect that means early wallets may not be able to e.g. trustlessly display local push notifications when a payment is received; the user would need to deliberately navigate to the app and allow it to connect.

These are web standards though – a good first step to making it happen would be to have an existing ecosystem of mobile wallets and users that want WebRTC in service workers. The issues look hard but solvable, and existing user bases could add a lot more momentum.

That’s a good point. In this case, our requirements are more marginal: we just need these alternative wallet apps to be able to work, not necessarily to provide the highest-quality experience. Both vendors are likely to continue adding features that are only accessible to native apps to ensure PWAs remain sub-par for most use cases (NFC, watch integrations, other hardware access, etc.), but that doesn’t hinder us.

So our interests aren’t at odds with these vendors – our goal is to ensure a secure, functional wallet ecosystem can exist outside of app ecosystems, reducing the incentive to use app ecosystems as a censorship vector (e.g. a government banning app stores from distributing cryptocurrency-related apps). Our work would actually relieve pressure from these companies, not threaten their business.

Both WebSocket and QUIC are more general-purpose transport protocols. WebSockets are widely available in browsers (practically universal now), and HTTP/3 (A.K.A. HTTP-over-QUIC) is widespread among modern browsers (~75% of global usage).

Neither WebSocket or QUIC really address the use case here: building a P2P network to which web applications can connect. For that we need WebRTC, which is already practically universal and widely used for P2P video and audio calls between browsers.

Note that WebRTC is quite a bit more complex and application-specific, and we probably need WebSocket or HTTP for the WebRTC signaling/initial bootstrapping of new clients on the network.

This will be no small task and likely will require several iterations, but it could unlock massively more P2P bandwidth (users can contribute to the network by simply using a web page) and strengthen censorship resistance (it’s hard to pick out Bitcoin Cash WebRTC connections vs. video/audio calls, and wallets can be distributed as PWAs rather than relying on app stores).

#8

I haven’t used this (and it doesn’t look like a “great” solution), but this might mitigate some of the problems (e.g. we may be able to sync block headers in the background at set intervals) for PWA Apps.

Unsure if there are constraints in terms of whether WebRTC connections would actually be allowed though.

Also worth noting that browsers will determine (outside of the sync period specified) how often this is allowed to run based on how often the user visits the app.

Again, not ideal, but thought worth mentioning as a potential stop-gap.

1 Like
#9

I think WebRTC is kind of overkill for what we want. It is really geared more towards realtime communication. The only thing WebRTC buys us is allowing incoming connections to a mobile device.

I am not sure in the value of mobile, ephemeral wallets that are up for a few seconds or a few minutes at a time offering incoming connections. I see that as actually a bad thing.

I think just doing WebSockets on the p2p network as an alternate way to connect to a node as a client would work and already be a game-changer. Of course, all the nodes you would be connecting to would not be mobile wallets but rather would be full node peers.

#10

If full nodes could serve node data with something like libp2p-webrtc-star in ipld format over WebRTC, then web clients could store it in indexeddb with existing (or forked) libraries and communicate this ipld data with each other over WebRTC, again with existing ipfs tools.

openbazaar and filecoin establish(ed) their own ipfs networks to avoid interfering with the main network.

ipld libraries were developed for btc and eth, but never bch. The btc library has a bunch of segregated witness stuff in it.

The ipld data types are node and transaction. I believe, ultimately, a UTXO index somewhere, either in the node or some intermediary is really expedient to creating a scalable responsive app.

Perhaps this is something that @sahid.miller could offer some expertise, as it’s been a while since I’ve used the above stack.

1 Like
#11

So IPLD is the layer that parses a block of data into a DAG node.

The benefit of implementing a custom IPLD format is that you can traverse and inspect the DAG with a single path-like API. Without it, you can only fetch a block by CID, so you have to process them manually to fetch more blocks.

The commands in IPFS would be:

  1. Without IPLD: ipfs block get <cid>
  2. With IPLD: ipfs dag get <cid+path>

The downsides are, to get the benefit of IPLD on both the browser and server, the IPLD format would have to be implemented in both Go and Javascript. Also, IPFS nodes that do not have the format installed would still be able to store blocks but unable to view them in the IPFS Gateway or “Files API”.

I’ve never implemented a custom IPLD format for my projects because I didn’t think the benefit outweighed the cost compared to implementing a wrapper using raw blocks. I think the same could be said for sharing BCH blocks and transactions over IPFS as well, since the wallet itself is capable of parsing the raw ipfs block and the nodes (transactions, blocks) are rarely traversed.

That said, using IPFS at all for sharing transactions would be exciting. IPFS comes packaged with libraries like libp2p and bitswap to exchange blocks over a network - which was largely what the conversation was about so far, as far as I can tell. Libp2p comes packaged with tools for websockets, web-rtc, mdns, bluetooth-le, and much more.

Also, since IPFS has uses outside of our particular blockchain (and cryptocurrency in general) you tap into a large network of users and developers already running an IPFS node and anyone who runs one for BCH would get a lot of additional value out of it! :smiley:

@2qx Oh, for sharing BCH transactions using IPFS, there’s one additional step besides IPLD, there’s also the hashing function. Transactions are double-hashed in BCH… By default, IPFS uses a single sha256 hash.

So there’s already a layer of customization necessary if you want users to be able to fetch a block by transaction hash/id. (The go implementation of IPFS handles dbl-hash out of the box but the browser doesn’t last I checked)

1 Like
#12

Just to share, I was able to get a bcoin fork for BitcoinCash working in the browser about a year ago using a libp2p based library!

The library allowed me to connect to real BitcoinCash full nodes using a single multiplexed websocket connection (via libp2p) to a backend tunnel which connected me to the nodes… The tunnel is something someone can run themselves on desktop, server, potentially even a phone application; or just use someone else’s as a service (more on security/privacy below).

image

The experience was amazing too. Fast, secure, and pretty composable!

I was able end-to-end encrypt the connection to the destination client-side using node-forge to implement TLSv1.2/SSL client side. And for privacy, I was able to use multi-hop via libp2p so it’s possible to connect to one node and then to another (and so on) before using an “exit node” to connect to the destination.

These connections can use different transports too. So for example, WebRTC for the first hop and then WebSocket for the second, and so on. Pretty cool!

Just for comparison, WebRTC also requires some centralized (but configurable) infrastructure. STUN/TURN servers are required for two peers to establish a connection and without it, another server is needed to help them exchange information. Both would presumably need to be connected to that server at the same time since the SDP’s expire pretty quickly.

Also, IIRC, it’s best practice to use that same server as a total fallback (tunnel/media server) if the exchange fails or if the two peers can’t establish a connection after the exchange (which it does at least 20% of the time and it’s pretty difficult to debug why).

1 Like
#13

I still won’t have time to dig into this for a while, but I recently stumbled across Nimiq, another cryptocurrency that seems to have exactly the kind of WebRTC P2P network I’d like to build for BCH:

https://nimiq-network.github.io/developer-reference/chapters/nodes-and-clients.html

I haven’t reviewed anything but their docs/marketing material, but this at least describes the type of P2P network I’d love to have – mobile/web clients can easily bootstrap from known seeds/seeders, then once connected, they’re capable of receiving/broadcasting transactions purely P2P with other, similar mobile/web clients.

I’m less interested in some of the other differences from BCH (Nimiq’s NiPoPoW usage and “nano nodes”) – I think even mobile BCH wallets can realistically operate as pruned nodes (holding the full UTXO set), and clients with even stricter requirements can use progressive knockout block filters.

#14

I know the feeling. No worries.

Yeah, this is interesting. The basic concept of having a full node present a websocket to connect to would probably be my first iteration on the server side.

1 Like
#15

Another one: