Mining
Introduction
Mining is the process where nodes in the Bitcoin Network assemble newly broadcast bitcoin transactions into a data structure called a block. Nodes then compete to append their block to the public blockchain by attempting to find a hash value that satisfies a difficult proof-of-work.
When a node finds a valid proof-of-work hash for a block, it broadcasts the block to all nodes. Other nodes accept this block only if all the transactions in it are valid and have not yet been included in a block. Other nodes in the network then express their acceptance of the block by building upon it to create the next block in the chain.
Every block is timestamped and references the hash of the block preceding it. Blocks form a backward reinforcing, timestamped chain – a Timechain, the precursor to the term ‘blockchain’.
The blockchain structures itself in a manner that is computationally impractical to modify by any one entity. Due to this, transactions included in the blockchain are considered immutable. In addition to this quality, the blockchain establishes an authoritative order of these transactions throughout the network, thereby protecting users from attempts to re-spend coins that have already been spent elsewhere. This is the key innovation of mining and of bitcoin.
Nodes
Nodes are entities that run software that adheres to the bitcoin protocol and carry out the mining functions above.
Interestingly, the bitcoin whitepaper makes no reference to the term ‘miner’ or the concept of ‘mining’. These terms are encompassed within Satoshi Nakamoto’s description of a ‘node’. It is possible however to run a node and make no attempts to find a proof-of-work. Such a setup is called a ‘listener node’ – they only participate in receiving and broadcasting transactions.
Listener nodes perform no proof-of-work and therefore cannot append new blocks to the blockchain. They can neither express acceptance of valid blocks by working on extending them, nor reject invalid blocks by refusing to work on them. Listener nodes are passive entities that follow the gatekeepers of the network, the miners. An example of a listener node is a block explorer. Block explorers are a useful service that allow users to query the blockchain for block or transaction information.
Mempool
Before a bitcoin transaction can be committed to the blockchain it needs to be received and validated by a miner. If the miner deems the transaction valid, they add the transaction to one of several mempools. Mempools are temporary transaction stores and can be used to hold transactions grouped in different ways such as transactions to be mined in the next block, transactions to watch, or transactions which cannot be mined due to an nLocktime/nSequence interlock.
It is important to note every miner has their own mempools and that mempools vary across miners. It is also important to note that an individual transaction can be included in different mempools at the same time.
A miner takes the transactions transactions to be mined in the next block and hashes them into a merkle tree structure and includes the resulting merkle root within a candidate block header. The miner then hashes this candidate block header, attempting to find a valid proof-of-work.
Hashing
A hash is a function that converts a string of data into a fixed length value that represents the original string, called the hash value. Every hash value is unique.
Hashing is a one-way function, meaning it is infeasible to determine what the input data is by looking at the hash produced from it. Conversely, it is trivial to run the same input data through the same hash function and reproduce the same hash. Because of this, a hash value of input data can be thought of as the digital fingerprint of that data. In bitcoin mining, the input data is the 80-byte block header.
The hashing algorithm used in bitcoin mining is SHA-256. SHA-256 stands for Secure Hash Algorithm – 256 bit. Passing the block header data through this algorithm will always output a 256-bit number that is unique.
Bitcoin mining passes the block header data through the SHA-256 algorithm twice (SHA-256d). The reason for this is to accommodate the presumed future fracture of mining interests into separate specialized entities. A double hash allows for a hashing entity to indicate they have a valid proof-of-work to another mining related entity without having to reveal the block header input data.
Proof-of-work
Bitcoin uses the hashcash Proof of Work function.
A miner cannot create a new block without finding a valid proof-of-work hash for the block header they are hashing. To be valid, the SHA-256d hash (which is just a number) of the block header must be less than another number, called the target. The target value is defined by the Bits field in the block header that is being hashed.
The target adjusts so that the average time it takes to find a valid proof-of-work hash is ten minutes.
Because of the extraordinarily large SHA-256 target space, it is extremely unlikely that any given hash will be below the target. As a result of this, hashing entities invest large amounts of capital into specialized hashing hardware along with the associated electricity costs in order to produce as many of these hashes as possible. Therefore, the only way a miner can append a block to the blockchain is through substantial financial commitment – proof-of-work.
Incentive
The first transaction in every block is called the Coinbase transaction. The Coinbase transaction is a special transaction which pays bitcoins to the miner of the block. The Coinbase payment is made up of the subsidy amount and the sum of all transaction fees paid by transactions within the mined block.
Competition
Propagation and validation
If a miner finds a valid proof-of-work for the block header they are hashing, they immediately announce this by sending the entire block to other miners. Meanwhile, they start working on finding the next block in the chain.
Competing miners receive this block and immediately check that all transactions within the block, as well as the proof-of-work are valid. If the block is valid they discontinue mining on the old block height and start working on hashing a new block header, building on the block just found by their competitor.
There are incentives at play here. First, winning 'miner A' strives to get their winning block to all other miners as quickly as possible. This reduces the possibility that competing 'miner B' (who found a valid block at approximately the same time) propagates their block to some miners before 'miner A' could. This opens up the possibility that 'miner A's' block could later be orphaned. The degree of bandwidth and connectivity to other miners can be seen as a competitive advantage and as such, the mining network tends toward a densely connected, small world structure with high bandwidth.
There also exists a competitive advantage in validating transactions. When validating a block, each transaction's inputs must be looked up in the miner's UTXO set to check they are unspent and the amount in satoshis is correct. Additionally, each input's locking and unlocking scripts need to be run by the miner to assess whether each transaction is valid.
Rejected blocks
At any given point in time, two or more independent miners may mine a block at the same time. In this situation, nodes can be in disagreement about which of these blocks should be at the top of the blockchain.
Miners always follow the longest chain they deem to be valid, in practice this means the chain with the most chainwork - the chain with the most proof-of-work invested in it. Eventually another block will be found that builds on one of the competing chain tips. Miners then switch to this tip provided they consider it to be valid. As such, any fork scenario is eventually resolved back to one persistent chain through the actions of the majority of hash power.
In this scenario, blocks that don't end up forming part of the longest chain are rejected by the network and are called orphans. Orphaned blocks represent wasted effort on the behalf of a miner and an incentive to invest in infrastructure in order to reduce the frequency of these events.
Competing valid blocks are not the only way that blocks end up being rejected. Any miner can refuse to build on any block for any reason. Such an action by a particular miner is only meaningful if the majority of miners carry out the same action. It is through this mechanism that the mining network can establish consensus on variables that are miner configurable - such as maximum block size. This is the basis of Nakamoto Consensus:
"They vote with their CPU power, expressing their acceptance of valid blocks by working on extending them and rejecting invalid blocks by refusing to work on them. Any needed rules and incentives can be enforced with this consensus mechanism."
The mining ecosystem
Asic mining
An application-specific integrated circuit, or ASIC, is a microchip designed and manufactured for a very specific purpose. ASICs designed for Bitcoin mining were first released in 2013. For the amount of power they consume, they are vastly superior to predecessor approaches to mining - using CPU, GPU or FPGA.
Pooled mining
As more and more miners competed for the limited supply of blocks, individuals found that they were working for months without finding a block and receiving any reward for their mining efforts. To address the variance in their income, miners started organizing themselves into pools so that rewards could be shared evenly.
Stratum
Stratum is an open-source mining protocol used by almost all mining pools. Stratum faciliates coordination between the mining pool operator and individual mining machines.