Researching CPFP with on-chain history

For well over a year now we are hearing voices that it is important to increase the chain-depth of unconfirmed transactions accepted by a client. We got a token increase last May, but it doesn’t make much sense for the network to limit what kind of transactions users are allowed to create.

The main issue we hear why deeper chains are not allowed right now is because of the child-pays-for-parent feature. It would make the mining software exponentially more expensive to run. In fact, that is how some clients actually have it running.

So the simple question we need to ask at this stage of the Bitcoin Cash network is if the CPFP feature was enabled prematurely, before the software architecture was able to support it. And by asking that we need to also ask who is using CPFP now.

So, let me present a research app in my new favourite way of digging in the chain using FloweeJS.

The app here:

function getBlock(height) {
    // console.log("getBlock called with " + height);
    var findPotentials = {
        jobs: [{
            value: height,
            type: Flowee.Job.FetchBlockOfTx,
            txFilter: [Flowee.IncludeOffsetInBlock, Flowee.IncludeInputs, Flowee.IncludeTxid]
        }]
    };
    Flowee.search(findPotentials).then(function(findPotentials) {
        var hashes = {}; // txids in the block we are looking at
        for (tx of findPotentials.transactions) {
            if (tx.isCoinbase === false)
                hashes[tx.txid] = tx;
            // resolveDetails.totalTx++;
        }
        for (tx of findPotentials.transactions) {
            // a CPFP transaction is meant to increase the fee, which it effectively
            // does by lowering the 'change' we get back ourselves. It does this
            // by adding a new tx which spends the change and splits it into a fee
            // and change.
            // So, filter to transactions that have 1 input which is created in
            // the same block.
            if (tx.isCoinbase == false && tx.inputs.length == 1) {
                var prev = hashes[tx.inputs[0].previousTxid];
                if (typeof prev !== 'undefined') {
                    // console.log("  potential CPFP tx " + tx.offsetInBlock)

                    let check = {
                        jobs: [{
                            // get more info about our bad boy
                            type: Flowee.Job.FetchTx,
                            value: tx.blockHeight,
                            value2: tx.offsetInBlock,
                            txFilter: [Flowee.IncludeOffsetInBlock,
                                Flowee.IncludeOutputAmounts,
                                Flowee.IncludeInputs,
                                Flowee.IncludeFullTxData]
                        },
                        { // fetch the one it spends
                            type: Flowee.Job.FetchTx,
                            value: tx.blockHeight,
                            value2: prev.offsetInBlock,
                            txFilter: [Flowee.IncludeOffsetInBlock,
                                Flowee.IncludeOutputAmounts,
                                Flowee.IncludeInputs,
                                Flowee.IncludeFullTxData]
                        }],
                        txs: {},
                        onTxAdded: function(tx) {
                            this.txs[tx.txid] = tx;
                        }
                    };
                    // and get the transaction-output-amounts our parent spent.
                    for (input of prev.inputs) {
                        check.jobs.push({
                            type: Flowee.Job.FetchTx,
                            value: input.previousTxid,
                            txFilter: [Flowee.IncludeOutputAmounts]
                        });
                    }

                    /*
                     * The potential Tx now can have its fee calculated and the parent
                     * tx can have the same done.
                     * Then we can determine if the child adds fees to the parent or not.
                     */
                    Flowee.search(check).then(function(check) {
                        var childTx = check.transactions[0];
                        var parentTx = check.transactions[1];
                        // a CPFP only has one output.
                        if (childTx.outputs.length !== 1)
                            return;
                        // Looks like someone was experimenting or something.
                        if (childTx.outputs[0].amount == 0)
                            return;

                        let childFeeTot = parentTx.outputs[childTx.inputs[0].outputIndex].amount - childTx.outputs[0].amount;
                        let childFee = childFeeTot / childTx.fullTxData.length

                        let parentFeeTot = 0;
                        for (input of parentTx.inputs) {
                            let inTx = check.txs[input.previousTxid];
                            parentFeeTot += inTx.outputs[input.outputIndex].amount;
                        }
                        for (output of parentTx.outputs) {
                            parentFeeTot -= output.amount;
                        }
                        let parentFee = parentFeeTot / parentTx.fullTxData.length
                        let togetherFee = (childFeeTot + parentFeeTot)
                                    / (parentTx.fullTxData.length + childTx.fullTxData.length);
                        if (togetherFee - parentFee > 0.5) {
                            // console.log("CPFP tx: " + height + "]=> " + childTx.txid);
                            console.log(height + "," + childTx.txid
                                + "," + parentFee.toFixed(3) + "," +  parentFeeTot
                                + "," + parentTx.fullTxData.length
                                + "," + childFee.toFixed(3) + "," +  childFeeTot
                                + "," + childTx.fullTxData.length);
                        }
                        // console.log("  ParentFee: " + parentFee.toFixed(3) + ", child: "
                        //         + childFee.toFixed(3) + ", total: " + togetherFee);
                    })
                    .catch(function(a) {
                        console.log(a);
                    });
                }
            }
        }
        getBlock(height + 1);
    });
}

var Flowee = require('bindings')('flowee')

Flowee.connect().then(function() {
    // the header of the CSV-like output
    console.log("block,txid,fee.parent,fee.parent-total,parent.tx.size,fee.child,fee.child-total,child.tx.size");
    getBlock(630000);
});

Freetrader went and put the output in a spreadsheet which can be downloaded from here;
https://files.slack.com/files-pri/TU80WRQ0M-F015S12615G/download/20200620_freetrader_cursory_look_at_tomzander_cpfp_data.ods

3 Likes

Thanks for the data and inspiring script example Tom!
The Slack link is inaccessible for those who don’t yet have a login to BCHN Slack, so I made the spreadsheet file available here:

https://download.bitcoincashnode.org/misc/data/cpfp/20200620_freetrader_cursory_look_at_tomzander_cpfp_data.ods

I’ll repeat a few comments here that I made in Slack discussion, and upload the two graphs for those who don’t want to download the file…

The first graph shows the number of occurrences classified as CPFP by your filter script, over the entire block range. The X axis is block heights, the Y axis is number of such CPFP’s in a block. The ordinate value “0” is fairly popular because most blocks (> 85%) do not contain such parent-child transactions where the child is paying the higher fee.

The data displayed here is not clipped.

Some basic statistics:

total CPFPs:	               2181	
blocks w / CPFPs:              1431	
% blocks w / CPFPs:            14.23%	
Number of parent fees < 1 sat:  7
% parent fees < 1 sat/byte:    0.32%

Parent fee statistics (sat/byte):

Minimum            0.67
Maximum            151.111
Range              150.441
Mean               2.44663502980283
Mode               1
Median             1.06
Standard error     0.124165550009601
Variance           33.6246597878362
Standard deviation 5.79867741712162
Variance           33.6246597878362
First quartile     1.004
Third quartile     2.005
Kurtosis           243.420488965911
Skewness           12.7006055925099

The following is a graph of ratio of (child fee rate) / (parent fee rate) for those CPFP occurrences.

X axis is the occurence number (table column, essentially) in the original CSV dataset.

The Y axis is clipped at y=140 due to one outlier around a factor of 1000.

2 Likes