Switcheo TradeHub experienced a chain halt on Feb. 7, which it was able to address while ensuring any funds or assets on the chain remained safe. The chain went down at 10:30 a.m. (UTC) on Sunday, and its liveliness was restored a few hours later at 3:00 p.m.
The Switcheo TradeHub chain was designed to shut down when there is a disparity between modules in its system infrastructure. Switcheo noted, “the TradeHub code was designed to prefer safety over liveliness.” In the post mortem, Switcheo Labs identified the issue:
The root cause of the incident was that an incorrect rounding direction was used when rounding decimals to integers in the new implementation that was introduced in the v1.12 upgrade.
Switcheo TradeHub is structured as multiple independent modules, and this incorrect rounding violated a safety invariant in another module — the output amount expected by the automated market maker module was slightly more than the output allowed by the liquidity pool module, triggering an internal consistency safeguard and halting the chain.
The code was not caught in pre-testing because the parameters did not include the set of values that would trigger the fault. Switcheo has updated its unit testing and fuzz test input values to catch similar issues in future updates.
A hotfix was included in an emergency upgrade to TradeHub v1.12.1. Looking forward, Switcheo Labs plans to permanently address the underlying issue in an upcoming upgrade to v1.13.1.
The full announcement can be found at the link below:
https://medium.com/switcheo/switcheo-tradehub-incident-report-7th-feb-2021-f609e3832f57
About The Author: Dylan Grabowski
Dylan is a reformed urban planner with a passion for covering the Neo ecosystem. His objective as a writer for Neo News Today is to report news in an objective, fact-based, non-sensational manner. When not behind a computer screen, he can be found in the mountains rock climbing. Find Dylan on Twitter (@GrabowskiDylan).
More posts by Dylan Grabowski