Mining

Introduction

Mining is the process where nodes in the Bitcoin Network assemble newly broadcast bitcoin transactions into a data structure called a block. Nodes then compete to append their block to the public blockchain by repeatedly mutating the block's header data structure, usually by incrementing the nonce field, then hashing it in an attempt to find a value that satisfies a difficult proof-of-work.

When a node finds a valid proof-of-work hash for a block, it broadcasts the block to all nodes. Other nodes accept this block only if all the transactions in it are valid and have not yet been included in a block. Other nodes in the network then express their acceptance of the block by building upon it to create the next block in the chain.

Every block is timestamped and references the hash of the block preceding it. Blocks form a backward reinforcing, timestamped chain – a Timechain, the precursor to the term ‘blockchain’.

The blockchain structures itself in a manner that is computationally and economically impractical to modify by any one entity. Due to this, transactions included in the blockchain are considered immutable. In addition to this quality, the blockchain establishes an authoritative order of these transactions throughout the network by establishing which out of any pair of conflicting transactions was seen first, thereby protecting users from attempts to re-spend coins that have already been spent elsewhere. This is the key innovation of mining and of bitcoin.

Nodes

Nodes are entities that run software that carries out the mining functions above. Nodes communicate using a P2P Network with its own protocol, and are the authors of the Bitcoin ledger and enforcers of the protocol rules.

Interestingly, the bitcoin whitepaper makes no reference to the term ‘miner’ or the concept of ‘mining’. These terms are encompassed within Satoshi Nakamoto’s description of a ‘node’. It is possible however to run the node client software without performing block assembly or proof-of-work. Such a setup can be used as an interface to the network for other systems such as block explorers, payment gateways and archives. Eventually it is expected that customised client software will be developed for each of these services so they will no longer be dependent on the node client for them to operate.

Computers that run a node client but which do not perform proof-of-work cannot append new blocks to the blockchain. This means they can neither express acceptance of valid blocks by working on extending them, nor reject invalid blocks by refusing to work on them. These computers exist as passive entities that follow the gatekeepers of the network, the miners.

Mempool

Before a bitcoin transaction can be committed to the blockchain it needs to be received and validated by a miner. If the miner deems the transaction valid, they add the transaction to one of several mempools. Mempools are temporary transaction stores and can be used to hold transactions grouped in different ways such as transactions to be mined in the next block, transactions to watch, or transactions which cannot be mined due to an nLocktime/nSequence lock.

It is important to note every miner has their own mempools and that mempools vary across miners. It is also important to note that an individual transaction can be included in different mempools at the same time.

A miner takes the transactions to be mined in the next block and hashes them into a merkle tree structure and includes the resulting merkle root within a block header candidate block header. The miner then hashes this candidate block header, attempting to find a valid proof-of-work.

Hashing

A hash is a function that converts a string of data into a fixed length value that represents the original string, called the hash value. Every hash value is unique.

Hashing is a one-way function, meaning it is infeasible to determine what the input data is by looking at the hash produced from it. Conversely, it is trivial to run the same input data through the same hash function and reproduce the same hash. Because of this, a hash value of input data can be thought of as the digital fingerprint of that data. In bitcoin mining, the input data is the 80-byte block header.

The hashing algorithm used in bitcoin mining is SHA-256. SHA-256 stands for Secure Hash Algorithm – 256 bit. Passing the same block header data through this algorithm will always output the same 256-bit number. However if the header data is modified even by a single a bit a completely different and unrelated 256-bit number will result.

Bitcoin mining passes the block header data through the SHA-256 algorithm twice (SHA-256d).

Proof-of-work

Bitcoin uses a Proof of Work function based on the earlier work of hashcash.

A miner cannot create a new block without finding a valid proof-of-work hash for the block header they are hashing. To be valid, the SHA-256d hash (which is just a number) of the block header must be less than another number, called the target. The target value is defined by the Bits field in the block header that is being hashed.

The target adjusts so that the average time it takes for the entire network to find a valid proof-of-work hash is ten minutes.

Because of the extraordinarily large SHA-256 target space, it is extremely unlikely that any given hash will be below the target. As a result of this, hashing entities invest large amounts of capital into specialized hashing hardware along with the associated electricity costs in order to produce as many of these hashes as possible in a given time period. Therefore, the only way a miner can append a block to the blockchain is through a substantial commitment of operational expenditure – proof-of-work.

Incentive

The first transaction in every block is called the Coinbase transaction. The Coinbase transaction is a special transaction which pays bitcoins to the miner of the block. The Coinbase payment is made up of the subsidy amount and the sum of all transaction fees paid by transactions within the mined block.

Competition

Propagation and validation

If a miner finds a valid proof-of-work for the block header they are hashing, they immediately announce this by sending the entire block to other miners. Meanwhile, they start working on finding the next block in the chain.

Competing miners receive this block and immediately check that all transactions within the block, as well as the proof-of-work are valid. If the block is valid they discontinue mining on top of the previous block and start working on hashing a new block header, building on the block just found by their competitor.

There are incentives at play here. First, winning 'miner A' strives to get their winning block to all other miners as quickly as possible. This reduces the possibility that competing 'miner B' (who found a valid block at approximately the same time) propagates their block to some miners before 'miner A' could. This opens up the possibility that 'miner A's' block could later be orphaned and the Coinbase reward rendered invalid and unusable by 'miner A'. The degree of bandwidth and connectivity to other miners can be seen as a competitive advantage and as such, the mining network tends toward a densely connected, small world structure with high bandwidth.

There also exists a competitive advantage in validating transactions. When validating a block, each transaction's inputs must be looked up in the miner's UTXO set to check they are unspent and the amount in satoshis is correct. Additionally, each input's locking and unlocking scripts need to be run by the miner to assess whether each transaction is valid.

Rejected blocks

At any given point in time, two or more independent miners may mine a block at the same time. In this situation, nodes can be in disagreement about which of these blocks should be the tip of the blockchain.

Miners always follow the longest chain they deem to be valid. Eventually another block will be found that builds on one of the competing chain tips. Miners then switch to this tip provided they consider it to be valid. As such, any fork scenario is eventually resolved back to one persistent chain through the actions of the majority of hash power.

In this scenario, blocks that don't end up forming part of the longest chain are rejected by the network and are called orphans. Orphaned blocks represent wasted effort on the behalf of a miner and an incentive to invest in infrastructure in order to reduce the frequency of these events. However, orphans do not reduce the overall revenues of the Bitcoin system as the wasted work is not factored into the difficulty adjustment, thus if a certain percentage of hashing work is wasted due to the orphaned blocks the difficulty will adjust downwards by a similar percentage, maintaining the same rate of valid block production overall. Additionally the fee paying transactions in the orphaned block will still be valid and included in the competing block or its descendants.

Competing valid blocks are not the only way that blocks end up being rejected. Any miner can refuse to build on any block for any reason. Such an action by a particular miner is only meaningful if the majority of miners carry out the same action. It is through this mechanism that the mining network can establish consensus on variables that are miner configurable - such as maximum block size. This is the basis of Nakamoto Consensus:

"They vote with their CPU power, expressing their acceptance of valid blocks by working on extending them and rejecting invalid blocks by refusing to work on them. Any needed rules and incentives can be enforced with this consensus mechanism."

The mining ecosystem

Asic mining

An application-specific integrated circuit, or ASIC, is a microchip designed and manufactured for a very specific purpose. ASICs designed for Bitcoin mining were first released in 2013. For the amount of power they consume, they are vastly more energy efficient than predecessor approaches to mining - using CPU, GPU or FPGA.

Pooled mining

As more and more miners competed for the limited supply of blocks, individuals found that they were working for months without finding a block and receiving any reward for their mining efforts. To address the variance in their income, miners started organizing themselves into pools so that blocks could be found more frequently and with rewards distributed among participating miners proportionally to their work contribution.

Stratum

Stratum is an open-source mining protocol used by many mining pools. Stratum facilitates coordination between the mining pool operator and individual mining machines.

See also

• Bitcoin Whitepaper