Service Outage Report - 20231209

Cause

In the case of this mudfish service outage, it is not an internal issue of mudfish, but an issue on the side of KT ucloud, where the main servers of mudfish are located. A few hours before the failure, there was an issue of "Lost connection to MySQL server during query" between the internal authentication server and the DB server for no reason, and the connection was delayed.

Then, a few hours later, the internal network connections were starting to be abnormally disconnected, and the issue lasted for about 2 hours. We didn't do any patching on our end, but the problem disappeared naturally when KT resolved it.

Although there was no problem with the mudfish itself, almost all of our services were not working properly due to this issue. We apologize for the inconvenience.

Outage duration

• 2023-12-09 6:33 PM to 2023-12-09 7:51 AM (PST)
• For approximately 2-3 hours, using Mudfish did not work properly.

Actions and Progress

• We performed to check multiple points to analyze where this issue is from but it was determined that it was not an internal mudfish issue.

• No action was taken.