• Service Outage Report - 20231215

Service Outage Report - 20231215

Cause

In this case, the mudfish service failure was caused by the overload of the mudfish DB server. The DB table containing user information was frequently updated/selected, and the processing was delayed. As the default, it caused the failure.

Due to this issue, almost all services did not work properly. We apologize for the inconvenience.

Duration

• From 2023-12-15 8:20 AM to 2023-12-15 9:10 AM (PST)
• For approximately 1-2 hours, login and use of Mudfish did not work properly.

Actions

• To prevent the user DB table from being overloaded, we have merged it before updating the DB table when processing credit deductions, and delayed the processing when it is deemed to be overloaded.
• We no longer calculate the credit usage of each mudfish nodes.