What do the recent global slump in oil demand, natural disasters, Asia’s appetite for seafood, and COVID-19 lockdowns have in common? While seemingly disparate, these developments have far-reaching consequences for the optical cables under the sea that carry 99 percent of the world’s Internet traffic.
And for the staff at Telstra’s Network Operations Centre in Singapore, that’s all in a day’s work.
Housed in a secure, state-of-the-art facility, the Network Operations Centre – or NOC, as it’s often referred to – operates with a team of more than 30 engineers working around the clock monitoring Telstra’s network infrastructure in Asia. These span six subsea cable systems that Telstra owns or has an interest in, comprising 31 cable stations and more than 65,000 kilometres of cables that carry data between key Asian economies. This vast network of fibre optic cables on the ocean floor is the largest of its kind in the Asia Pacific, and carries up to a third of the region’s Internet traffic.
Supporting other telcos and OTT firms
In a world still grappling with the COVID-19 pandemic, networks enable a large part of how we live, work, learn and play. Whether it’s video conferencing, streaming movies or playing online games, we’ve become accustomed to having these take place smoothly on our devices. While most people readily attribute these modern conveniences to their Internet service or over-the-top (OTT) providers, not many know these firms rely on a facility similar to the NOC to deliver their services, says Telstra’s Head of Network Assurance, Lee Robinson.
Lee oversees operations at the NOC, which is responsible for network transmission, or maintaining the nodes and fibre pairs that make up Telstra’s subsea cable systems. Apart from round-the-clock surveillance, operations and maintenance of cables, the NOC is also responsible for provisioning network services to other telcos and OTT companies, which are wholesale customers of Telstra.
Lee also runs two other similar facilities in India and Hong Kong, which look after managed services, and voice and Internet traffic respectively. All three facilities have complementary roles in powering the digital economy. "There’s a ripple effect when a NOC isn’t performing optimally, as you can’t have one functioning without the other," he explains.
Given the importance of Telstra’s subsea cables in connecting economies, keeping them safe is of paramount importance. However, in a region like Asia which has shallow waters, heavy maritime traffic and its fair share of natural disasters, this presents a significant challenge.
Cable cuts often take place during typhoons and earthquakes which trigger undersea landslides, while fishing trawlers can inflict similar damage with their nets. These vessels, alongside container ships and oil tankers, have anchors that can weigh tens of tonnes each – posing a significant danger to subsea cables.
Using the Automatic Identification System, which tracks the position, course and speed of each ship through its transponder, Lee’s team monitors the movement of ships into and out of the busy ports of Singapore and Hong Kong. If a ship gets too near to one of Telstra’s cables and appears to be slowing to drop anchor, the NOC team will contact the crew to alert them.
This takes place for an average of 30 large vessels a month in Singapore, a fraction of the 1,000 vessels estimated to be in its waters at any one time. With Singapore being a major oil refining and trading hub, Lee discloses that the recent slump in global oil demand has exacerbated this congestion. As the global pandemic unfolded, the number of oil-laden tankers has spiked in Singapore waters, where they effectively serve as floating storage.
Operating through lockdown restrictions across Asia
The various COVID-19 restrictions imposed by countries in the region have required the NOC team to implement and constantly refine their business continuity plan, and double down to keep operations running smoothly.
In Singapore where a partial lockdown was imposed, businesses of a non-essential nature had to shut and their employees work from home during the "circuit breaker" period. While telecommunications was classified as an essential services, Telstra had to apply for exemptions for the NOC team not once, but twice after the circuit breaker was extended by another month.
Lee adds: "This also had to be done for the field services team, which continued to physically manage the network at sites island-wide, to fulfill customer requests to increase their bandwidth for employees working from home."
While the NOC implemented safe distancing, contact tracing and split operations in line with other essential businesses, a more complex problem loomed beyond the borders of Singapore. Telstra’s subsea cables cross considerable stretches of water, and repairing damage to these cables require sending out ships with specialised equipment and crew. These ships are often out at sea for months at end, and depart from different ports depending on the location of the cable to be repaired.
With various countries entering lockdowns, it became quickly apparent that the multinational crew would face difficulties in boarding and disembarking from their ships. The NOC team, who regularly liaise with governments and regulatory authorities to lay, maintain and repair cables in territorial waters, stepped in to expedite clearance for the crew. This included booking hotel rooms to accommodate crew members who needed to be quarantined before and after their assignments.
Shifts in network traffic
As working, learning, and simply staying put at home became the new normal across the world, shifts in network traffic pattern became discernible. Demand for data on Telstra’s international network surged by up to 50 percent, though well within the additional capacity it was designed with. Peak traffic hours shifted from evening time to start from the mid-morning.
Lee describes how the NOC team was kept busy with customer orders to substantially increase their bandwidth, often at short notice: "High bandwidth connections are 10GB and above, but we received orders for 100GB which the customer asked to activate overnight. Our usual lead time is two to three weeks, but we did it in just two days as we recognise this has been a very challenging time for our customers."
While the pandemic has posed significant challenges to Lee and his team, the large-scale adoption of remote working has also opened their eyes to new possibilities in running the NOC. With technology reducing the need for physical presence, this could mean a partial shift to virtual operations when new roles are created, and help attract and retain talent when the best candidates can be considered regardless of their location. For instance, employees working remotely in a different time zone can undertake shift work during their local business hours, while having a more geographically dispersed team would support business continuity.
Amid the disruptions caused by COVID-19 in the past months, a "mystery on the high seas" continues to confound Lee and his team: the presence of numerous fishing vessels out at sea even at the height of the pandemic, when many Asian economies were under lockdowns.
While this phenomenon may well be indicative of Asia’s well-known appetite for seafood, it also meant more work for the NOC during what was expected to be a quieter fishing season. "Our automatic tracking system showed no decline in the numbers of fishing vessels at sea, and accordingly the number of cable faults we had to attend to in the same waters stayed the same," says Lee with a laugh.