Instant Finality In Blockchains: Feature or a Bug?
Table of Contents
- What is Finality?
- Why is Finality needed?
- Finality in Proof of Stake Blockchains
- Deterministic Finality - Instant and Eventual
- Tradeoffs with Fast Finality
- Scalability Impacting Finality
- Potential Improvements & Research
- Is Instant Finality Really Needed?
- Finality in Polkadot
- Conclusion
What is Finality?
Finality in blockchains means a block (and transactions in it) cannot be reverted.
But how can blocks or transactions be reverted? Let’s first understand this. Blockchains are decentralized systems. All nodes maintain a local state and build blocks on top of it. They keep this state in sync using consensus mechanisms. Nodes are connected over a peer to peer (p2p) network. In case of a network partition, a subset of nodes could temporarily fork from the canonical chain.
Why would a network partition create forks? Because blockchains, being decentralized, choose liveness or availability over global consistency. This is important to enable censorship resistance. When the priority is liveness, in case of a network partition, all nodes would continue to accept requests and transactions and build blocks on top of their local state. This would cause the nodes on either side of the partition to follow their own fork of the chain.
When the network partition resolves, and when all nodes sync again, there are two chains. We call these forks. Based on the consensus rules, one of these chains is ignored and (in most cases) the longest chain is considered the canonical chain. Guess what happens to the transactions that are in the blocks of the shorter chain. They are ignored and hence effectively reverted. Forks caused by network partitions are just one of the reasons why blocks could be reverted.
Another reason for blocks getting reverted/rejected is forks caused by orphan or uncle blocks. When two nodes produce a block at the same time, generally in a PoW blockchain, then one of these blocks (and any blocks built on top of it) gets reverted eventually. Again following the longest chain rule. This kind of fork does not generally happen in PoS networks (you will find out a bit later in this article).
There could be several other reasons for blockchain transactions getting reverted because of forks (failed 51% attacks, governance, etc). But you get the idea that there is a slight possibility of blockchain transactions getting reverted. With that, we can go ahead and discuss finality, finally!
A block can be considered final when the possibility of it getting reverted is close to zero. In proof of work (PoW) based blockchains this is (kinda by design) probabilistic. In proof of stake chains finality is when majority (in most cases a 2/3 majority) of the nodes agree on the state including the state transition caused by the block being finalized. I’ll explain these later in the article.
It’s kind of obvious but worth mentioning - when majority of the nodes have to agree on the state changes made by the blocks, finality for a block can only be achieved when that block has reached (propagated to) and imported by majority of the nodes. This takes time; blocks propagate on peer to peer (p2p) network between the nodes. In a geographically distributed network, any data transfer would take some time. And blocks are also just small chunks of data.
Why is Finality needed?
Let’s also briefly discuss where the need for finality come from. In early days, when Bitcoin became popular, a lot of people started trading on exchanges. In the beginning the exchanges settled transactions as soon as they got included in a block. But then forks happened, and blocks got reverted. Exchanges lost that money, because they had already settled the transactions from their side. They needed a way to figure out whether a block could be considered final or not.
The community came up with a generally accepted rule that after 6 blocks (= 1 hour; Bitcoin block time is roughly 10 minutes) are built on top of a block, the probability of that block getting reversed is negligible. This is based on the amount of hash power needed to attack the blockchain and reverse the blocks by building a fork. So it was decided that a Bitcoin block could be considered final after 6 more blocks have been built on top of it. This came to be known as probabilistic finality.
A similar approach was applied to Ethereum when it used to be a PoW blockchain. Exchanges waited for 30+ confirmations (that means 30 or more blocks built on top of a block) before they called the block final on Ethereum. Although these requirements could be changed by individual exchanges depending on their trust and security assumptions.
Finality in Proof of Stake Blockchains
Things evolved and proof of stake (PoS) blockchains became a norm. The fundamental difference in PoW and PoS chains (apart from the energy consumption) is that in PoS chains the validator set (set of nodes eligible to produce blocks) is known in advance. Whoever gets enough $$ stake behind them can become eligible to be in the validator set (simply speaking). This allows us to look at finality in a different way in PoS chains. When the validators are known, we can assume that if majority of them have attested a block, then the chances of that block getting reverted are negligible. This led to the concept of deterministic finality.
For PoS systems, we could determine when a block could achieve finality depending on the total number of validators, block time, and randomness in selecting the next validator from the active validator set.
Deterministic Finality - Instant and Eventual
Instant finality is tightly coupled with block production and propagation. Generally it means a block can only be produced when the previous block has been already considered final by the majority of the nodes. In order to achieve instant finality, consistency is prioritized over liveness. If a blockchain has a large number of validators in a globally distributed network (sufficiently decentralized), it will be slow in producing new blocks.
Eventual finality is when blocks achieve finality in a deterministic time but not instantly. In this case, finality is generally decoupled from block production. The block production happens using a random or round-robin algorithm among the validators, while finality is achieved by a separate mechanism also known as finality gadget. The benefit is the blockchain continues to build blocks and serve users, while finality is guaranteed in a fixed time.
In eventual finality systems, the finality algorithm generally runs periodically and gathers attestations from validators. If validators agree on a block (majority votes in favour of a block’s validity), that block is marked final. In GRANDPA (finality mechanism used by Polkadot), this voting is done on the chain and blocks with a common ancestor are marked final in one round. This reduces the load on the p2p network. After all each of these attestations or votes are broadcasted on the same p2p network that connects the nodes.
Tradeoffs with Fast Finality
When there are a large number of validators, the number of messages passed on the network for block attestations can be a lot. It takes more time to validate so many attestations. If these messages are passed too frequently, the load on the network increases and it could impact block propagation too. Read more, here.
So now we can see the tradeoff very clearly - if the validator set is large, there are more messages on the network and more overall time needed to gather and validate attestations from majority validators. If these messages are too frequent, the blockchain network could be clogged. Hence, ultra fast or instant finality in fully decentralized systems is not that easy to achieve.
Scalability Impacting Finality
There is a whole another angle to finality - blockchain scaling techniques. Whether it is a rollup or execution sharding or data sharding, in almost all scaling techniques we end up off-boarding or offloading execution to another layer. It means the execution and block production takes place on a separate system (child chain), while validation and finality takes place on the parent chain.
When it comes to finality of the shard or rollup blocks, it is effectively dependent on the finality of the parent chain blocks. Hence the layered scaling techniques take us further away from instant finality. Here’s how - the shard or rollup will have its own reasonable block time and the parent time will have its own.
As per the definition of instant finality that the block should be final as soon as it’s produced, can we promise that for shard or rollup blocks? They will be produced and will have to wait for at least the next parent chain block for finality, even in the best case scenario. Normally, it takes a few blocks on the parent chain to finalize the shard or rollup blocks in a sufficiently decentralized network.
Potential Improvements & Research
There are a couple of things being tried by some blockchain networks to improve time to finality.
- Using advanced cryptographic signature schemes to aggregate and validate large number of attestations quickly.
- Using a random sub-set of validators to attest the blocks.
In the first case, the load on the network is still the same, even though the generation and validation of attestations becomes faster. Here’s a direct quote from this article on Ethereum docs site:
The problem with scaling aggregation by growing the number of validators per subnet is that it leads to greater load on the peer-to-peer network.
In the second case, the overall decentralization and hence security of the network is effectively reduced. There is a possibility for committees to collude.
At the end of the day, we are bound by laws of physics. It takes large amount of computing and networking resources to transfer and validate large amounts of data.
Is Instant Finality Really Needed?
There are some use-cases where instant or fast finality is required. For example payments and DeFi. When there is money to be settled, it is always a good idea to have it settled based on confidence, and finality of a blockchain transaction effectively provides ultimate confidence that the money can be settled. However, these are also the use-cases where we shouldn’t compromise with security too.
Some use case don’t really require instant finality - for example inter-bank settlements, audit trails, and identity management. These use-cases and applications require eventual data reconciliation, and hence can easily work with deterministic finality. There is no need to stress the network.
Finality in Polkadot
Now, aside from finality, when it comes to confidence in the transactions, there could be a pretty good solution - a phased approach. Let me explain.
In Polkadot, parachains are application specific chains. The user or dapp transactions are sent to parachains. All parachains are verified and secured by the relay chain. Every parachain transaction loosely goes through the following phases:
- Transaction included in a parachain block. Parachain block produced.
- Parachain block verified by initial set of relay chain validators (backing group).
- Parachain block attested by majority of relay chain validators (approval voting).
- Parachain block finalized as part of relay chain finality algorithm.
The total time taken covering all of the above steps is around 30 to 45 seconds (less than a minute). To compare, currently an Ethereum block takes 15 minutes to finalize, a Bitcoin block takes 1 hour, and an cross-border inter-bank settlement takes several days.
When we implement use-cases on top of Polkadot parachains, we have the flexibility to rely on any of the above phases of a parachain block (and transaction). For certain use-cases, we can just rely on the initial backing process, while for some others we should wait for eventual finality. There is a spectrum and not a binary choice. It helps a lot in modelling real world use-cases on Polkadot.
Conclusion
There are some blockchain platforms that claim instant finality with fast block times. Well, if they have a limited number of validators all running on extremely powerful hardware with huge network bandwidth (probably all in the same rack), then they can pass messages quickly and validate them too. But how decentralized is that network? Can we run a node to validate that network using commodity hardware? Probably not.
If instant finality comes at the cost of decentralization or timely block production (liveness), it is effectively a bug. The main characteristics of blockchains are decentralization and security, and if those are hampered because of anything, then that thing is a bug. Also, to reach mass adoption, blockchains need to scale, and we have already seen that scalability and instant finality don’t go hand in hand.
There is quite a bit of research happening to improve time to finality and we will eventually have some decent results. In any case, I would first look at the use-case and decide whether instant finality is even required or not. Let’s not make it a make or break criteria.
Thanks to Andronik Ordian for feedback and review of this post.