Aeternity: Bitcoin-NG the way it was meant to be – aeternity crypto foundation
æternity is designed and written from scratch. The current code is based on a version of the prototype from the late summer of 2017. The reference full node is implemented in Erlang/OTP. The æternity team comes from the world of distributed programming, experienced in building financial systems, massive chat applications, online markets, and a whole range of frameworks popular in the Erlang community. This technology is born to handle and serve applications created for distributed systems.
The plan was to take a leap and propose a smooth experience to both users and developers of the blockchain by including the most essential layer-2 solutions in the base protocol. The range of native transactions available from standalone rest calls and contract calls covers, among others, the following functionalities:
All of that is backed by the FATE VM and accessible through Sophia-language contracts. Both are focused on a blockchain-optimized design to provide more functionality and to lower the cost of operation. Sophia and the VM learn from all the challenges introduced by early blockchain platforms and languages, and provide more safety and security to users. FATE VM is high level VM as it operates on blockchain primitives like transaction. This is the source of low cost and high performance.
From the very beginning, it was important for us to present basic blockchain functionalities. Early full node code was a naive implementation of the Bitcoin protocol. The Erlang technology is component-oriented and lets us easily optimize subsystems one by one, so we took this path. This way of evolving the p2p network and the node gave us a chance to reason about changes and performance. Eventually, we arrived to a complete feature set protected by the protocol.
Currently, any new feature is tested on 50 nodes on the public testnet. In communicated releases, it is also deployed to well-known hosts of the mainnet network that was built since mainnet release from over 16.000 peers. Each release is tested by a multi-level test framework: unit tests, integration tests, system tests simulating network with docker containers, and a very powerful quickcheck test framework that generates random feeds for APIs of a number of components.
One of the key design goals for æternity is decentralization. We found the Proof-of-Work method of guarding the consensus as the most suitable for an open, public, and permissionless blockchain. We use the Cuckoo Cycle algorithm to implement PoW. This is a novel approach to memory intensive PoW algorithms with very cheap work evidence holding only 42 integers. Nevertheless, the last two years were very dynamic on the Proof-of-Work scene. ASICs achieved flexibility and life-cycles beyond predictions, GPUs were equipped with more complex operations, and at the same time, increased in level of parallelism. We consider going back to the topic of Proof-of-Work design to crystallize the best path to support a decentralized blockchain, which will be discussed in a separate post.
Consensus: Bitcoin-NG meets Aeternity-NG
Consensus algorithms are often misunderstood in the blockchain community. At the same time, consensus and the method used to protect it is not only a technical challenge, but also the heart of the product that we will offer to the end user. It will determine if transactions are trustless. It will determine if changes to the protocol are controlled by some groups. Finally, in the case of æternity, all the requirements revolve around users’ independence, decentralization, and freedom to transact.
The last couple of years were very fruitful in the Nakamoto-family consensus designs and adaptations of Byzantine Agreement algorithms. From a single list of blocks, through DAGs, lattices, hierarchies of trust, and PBFTs, there was a broad range of solutions to choose. Variants of choice multiply when we add Proof-of-Work, Proof-of-Stake, Space, Time, etc. By cross-checking with our goals and balancing them with the maturity of a solution, it became clear that in order to achieve our goals, we need to go with the Nakamoto way of dealing with an untrusted environment backed by Proof of Work.
So, how do we deal with 10-minute confirmation latency in a world of mobile-centric, real-time experience?
Cornell University researchers and the Bitcoin-NG paper offered some hope in this regard. [Bitcoin-NG: A Scalable Blockchain Protocol, Eyal, Gencer, Sirer, Renesse, 2016].
Bitcoin-NG is a novel way of supporting classic Nakamoto consensus. It is based on the election of a temporary leader by Proof-of-Work. The leader then publishes blocks containing transactions. We can treat those transactions as confirmed with 1 block — all of that within a matter of seconds!
Two phases of operations are reflected in the chain structure. We use key blocks to elect leaders and microblocks to hold transactions.
(Check out the Github repository).
The original paper describes the new chain structure and the crypto-economic incentives. It also covers problems like censorship, mining power, and fairness — all of which are essential for a blockchain platform. What we found the most challenging was reasoning and acting on forks and microforks. It required a couple of iterations of low level implementation of blocks, and communication with transaction handling components like mempool or block candidate generators. We also adjusted the Proof-of-Fraud mechanism. After all were rigorously tested, we waited six months for the mainnet release to bring real-world data and a bag of surprises…
Around midnight on the 27th of December 2018, the Roma release hit the live network. The genesis block held set of ERC20 tokens that were migrated to native aeternity tokens. Number of interesting events happened when real traffic and real users started to use the protocol. And what was the protocol?
We tuned Proof-of Work to mine Key blocks every 3 minutes, and we let the leader emit microblocks not faster than every 3 seconds. Each microblock can hold up to 6 000 000 of gas (which, given the initial implementation of the VM, is based on improved EVM, and is comparable to 8 000 000 in Ethereum blocks confirmed every ~15 seconds. FATE VM gives us even more leverage as it delivers more for the same amount of gas).
It means that with the network diameter in the range of 5–10 seconds, each key block should result in a 3-microblock fork. It is an expected and healthy behavior.
Our sync and gossip protocol had two modes: normal and light. The light one, gossiped only transaction hashes, making microblocks even lighter and faster. Assuming that there is a high probability that an unconfirmed transaction was already seen by peers, validation failure rate is low.
In the original paper, theoretical limits of network throughput were computed based on microblock size and capacity, and estimated at 100 tx/s. Our protocol’s theoretical limit was slightly higher as we allowed 300–400 spend transaction in one microblock, emitted every 3 seconds. Here, it is worth mentioning that we allow users to operate with stand alone transactions and with contracts calls. Underneath both, there is a common gas denominator.
The above number gives us a main constraint for network operations. The initial implementation allowed less than 50 tx/s without any backup. Further speed up was driven by optimizing the implementation of components like mempool or gossip.
Surely, the behavior of the network was also determined by crypto-economical incentives. We designed an inflation curve with very high reward over the initial 24 months (20% in the first year, 10% in the second, dropping to 5% in the third year, and 3% in the fourth year).
We initially assumed a steady network and mining power growth. What happened was quite the opposite! Within days, computation power grew so high that it made mining on personal computers and small mining rigs unfeasible (optimal cards were 1080Ti and 2080Ti). Users were very dissatisfied with the lack of opportunity to mine blocks. Also, we got reports about unconfirmed transactions and forks.
We discovered that we have an unexpectedly high number of forks. Upon inspection, we found that the network has many members, but the connection graph is very sparse. Broadcast of key blocks was affected. We had to improve (simplify) documentation and encourage users to use public IPs.
Less forks but long microforks. A microfork is a special state of the chain caused by next Key block miners following old microblocks emitted by the current leader. When it is short, it is fine. Unfortunately, we observed tens of blocks in microforks. We got mixed signals upon closer look, but decided it was still a broken network. We invested in uPnP and the network improved.
Early after the mainnet launch, the community and the core team implemented support for multi GPU rigs. It gave difficulty yet another boost. Users regained control over meaningful computational power to mine blocks. Of course, the target adjusted and lead to the formation of mining pools.
Mining pools introduced whole new dynamic. We discovered that they are based on customized stratum protocols and closed source mining clients. The problem we observed was again the growth of microforks’ lengths. It didn’t impact confirmation time, as even if a transaction is left in a fork, it is quickly reconfirmed by a new leader. But it is surely unhealthy behavior that in certain conditions can endanger the stability of peer nodes.
Dark corners of stratum
We discovered that pools are refreshing the key block candidate roughly every minute, which means that it can cause pessimistically 20 block long microforks. One minute is also a substantial part of the 3-minute key block generation. Key block generation varies a lot and it means that pools’ behavior can kick out our transaction to the next generation pretty often.
Our strategy here is to educate pools and diversify the pool landscape. This is a work in progress as currently, mining is taken by three main pools and two of them may take 95% of computational power.
We have another take on the problem. It is a work-in-progress reference implementation of stratum server that refreshes work in a Bitcoin-NG-friendly manner. Code speaks better than words, so maybe not only will it be used, but it may also inspire existing pools to adjust their configuration. Also, it may be an interesting base for working on a decentralization framework that makes pools out of full nodes instead of mining libraries. Bitcoin-NG will give the power to generate microblocks to regular users and pools will provide liquidity.
Having our bumpy road in mind, let’s look at the numbers.
Published at Tue, 29 Oct 2019 12:10:06 +0000