Solana Explains What Caused Its 5-Hour Outage This Week – Unchained

The bug, previously identified in a test network, caused Solana validators to enter into an “infinite loop” that made  them stall on a block and prevent consensus agreement.

Solana wasn’t able to reach consensus for roughly five hours on Tuesday because of a bug that triggered an “infinite loop” within the layer 1 blockchain’s validators, according to a post-mortem report prepared by Anza, a Solana-based software development firm, and posted on Solana’s website.

The report stated that the bug “created a classic infinite loop” among validators, which are tasked with the responsibility of confirming new blocks on Solana blockchain.

An infinite loop refers to a set of coding instructions for computer nodes that continues to execute without ever terminating. Since Solana validators were stuck in this infinite loop, they stalled on a specific block, making them unable to process and add new blocks to the blockchain.

“Every iteration through the mainloop triggers another compilation of the same program,” wrote Anza team member Jeff Washington in the report. “Since everyone was stalled in a recompilation loop, no one was voting and as a result, consensus halted irrecoverably.” 

Bug First Noticed on a Test Network

According to an X post following the outage written by VanEck head of digital assets research Matthew Sigel and retweeted by Solana co-founder Anatoly Yakovenko, the bug was first noticed on a test network, and while a fix was created, it was not implemented earlier “due to testing.” Blockworks reported that the Solana security team was first notified of the bug in April 2022. 

Following the outage, Solana developers deployed a patch, and after validator operators restarted the network on Tuesday, the layer 1 blockchain resumed producing blocks. SOL, the native cryptocurrency for Solana, hovered around $97 on the day of the outage and has increased nearly 9% to $105.60 since Tuesday, data according to CoinGecko shows. 

The outage this week was Solana’s first in almost a year, but its network had experienced numerous outages before that that had frustrated users.