How businesses can better handle telco outages

In the past 48 hours, businesses small, medium and large have been thrown into into disarray due to the major Optus network outage which we now know was caused by a ‘technical network issue’. Without getting into the why, how and who to blame, let’s have a quick look at how this happens and how it can be prevented.

Understanding the Outage

As we come to terms with what happened, it’s becoming quite clear just how much many of our services, both personal and business are heavily reliant on the major telcos. From small startups, coffee shops, large enterprise, and even government organisations,  the impacts of the outage were felt.

What’s the impact on businesses

Businesses felt the impact in many ways.

• Cafes, restaurants and other retailers unable to take payments
• Bus and railway services impacted as communication is disrupted
• Emergency phone calls unable to be made in some areas
• Financial losses from businesses who literally had to shut their doors

How does this happen

Although root cause analysis has yet to be complete, there is many possibilities as to why something like this happens. Let’s keep this non-technical for now, but here’s a few possibilities:

• Cyber attack (hopefully not)
• Errors during maintenance changes to infrastructure coupled with poor rollback planning
• Deep routing issues and propagation

Hopefully Optus will advise customers and the rest of Australia what happened within the coming days. We must understand that these things take time to dissect, analyse and make sure the information relayed is factual, and correct. We can all agree that the communication during the outage could have been better although it is easy to jump at the throats of large organisations, but it’s even messier when the wrong details are given.

Takeaways from all businesses

Now that businesses have felt the impact of a large outage like this, we need to consider how this could have been prevented. Outages are inevitable and as we become more reliant on these telco services, it is important for organisations to take measures to mitigate these risks.

Here’s some things to consider if you are one of those who doesn’t want to deal with this problem in the future:

1. Diversify Network Providers:
   • Typically, businesses who have high impact in events like this should consider diversifying their network providers. Relying on a single entity or telco for your core services is a serious point of failure and also one that is overlooked due to costs. This might be the time to weigh up revenue loss vs having a redundant internet service from a separate provider.
2. Invest in Robust Cybersecurity Measures:
   • We have no evidence that yesterday was a cybersecurity event and nor have Optus commented regarding a cyber attack. That said, businesses can be impacted in the same manner whether the issue is telco related, or a cyber attack. Now is the time to think about your cybersecurity solutions and how you are mitigating the risk of revenue loss in the event of an attack.
3. Create a Comprehensive Disaster Recovery Plan:
   • If you don’t have a comprehensive disaster recovery and business continuity plan, now is the time to implement one. These events are likely going to become more of a common occurrence, and we suggest businesses of all size consider how they would continue operating in the future events.

The events of the Optus outage should serve as a stark reminder that all businesses rely on these services and that none of us are immune. If you have concerns of this happening in the future, your concerns are valid and now is the time to start looking into mitigations.

Elias Atie

Cloud Context