Subsea Cables & Internet Infrastructure

As noted in my previous post, optical amplification allow light signals to be boosted without being first decoded into a digital representation. Optical-electrical-optical conversions go away. Hence the computer necessary for OEO conversions disappears. This in turn sharply improves the amplifier's reliability and life span. It also eliminates the conversion delay so end-to-end latency is improved. Finally, no computer means less cost. But these benefits are really secondary. More importantly, the advent of optical amplifiers led to a quantum leap in bandwidth that can be attributed to two related developments. The first is that optical amplifiers impose no transmission limits on computer technology. This means that we can lay a cable in the water and then upgrade it at regular intervals as Moore's law improves the ability of computers to process optical signals. Nothing on the wet side changes. Indeed, the introduction of digital processing allowed 10G wave subsea cables like ...

 A 1997 article celebrating how new optical amplifier technology will raise subsea cable output to an amazing five gigabits per second. It was a revolution at the time because the first optical fibre cable, TAT8, operated at 240 megabits. 😆

Layer 2 Switched Ethernet From Tokyo Data Centers To AWS Cloud 1. 10G; Layer 2 Pseudo Line Emulation Ethernet ; $775 MRC. 2. 100G; Layer 2 Pseudo Line Emulation Ethernet; $3800 MRC. Remarks: A. 802.1q frames. B. Jumbo framing available. C. See RFC 3985 to understand pseudo line emulation, which is basically a private line Ethernet service defined by a constant bit rate. D. Standard MTU size of 1522 bytes.

Picture below shows subsea amplfiers aboard a cable ship. They cost roughly $250K a piece, can support up to 40 Tbps, and are spaced every 70 kilometers or 42 US miles. These remarks apply to these specific amplifiers. As noted in Part 1 of this series, erbium doped fibre eliminated optical-electrical-optical (OEO) conversions for amplifiers. So the limiting factor of computer hardware involved in the conversions was eliminated. The result was a 'transparent' technology that placed no limits on submarine termination equipment upgrades. Subsea cables became highly scable because the amplifiers could easily handle higher throughput achieved via DWDM improvements. These amplifiers are tough buggers. They must withstand severe pressure at depths as great 8,000 meters for 25 years as well as nasty seawater corrosion. The amplifier hull is generally made of titanium or a special beryllium copper alloy, C17200, which conducts heat and electricity very well and is highly resistant to ...

Canada and US account for slightly over 45% of the world's IVP4 IP addresses. Europe is far less. Yet the 2025 budget for ARIN, the regional registry that serves the US, Canada, and Caribbean, is $28.8 million versus 41 million Euros for RIPE. If I use purchasing price parity to convert Euros to USD, and this is the correct approach, then the RIPE budget is 1.3*41 million Euros or $53 million USD. So RIPE's budget is almost twice the ARIN budget, yet their core functions are the same. The budgets are recouped via fees on IP addresses. It is plausible in my opinion that RIPE is feather bedding, which means it has a lot of unnecessary positions. There have been rumors for years that RIPE maintain a huge, but generally useless staff of community representatives. Let me add, that I have used ARIN to get /24s for an ISP client. The organization is highly efficient and professional. So is Arin underfunded and RIPE just the right size? Highly unlikely. Given the price pressures that I...

***It is worth emphasizing that the DRC 2Africa CLS is a carrier neutral facility where all clients are treated equally with non-discriminatory and cost-based pricing and consortium-imposed performance standards. Moreover, there are several diverse fibre routes from the CLS to Kinshasa, which has several new carrier neutral data centers. The DRC market could be particularly interesting as it traditionally been a hell hole for telecom carriers. This means wholesale ISPs can sell at high transit prices. ***The other Congo (Brazzaville) is also on-net for 2Africa. And I can get you fibre transport across the river at wholesale prices to the DRC. 🙂 ***Special Note: ACR2, a Digital Realty data center in Accra, Ghana, is where most 2Africa capacity carriers keep their SLTEs.

TAT-8 was the first Atlantic optical cable. It was RFS in 1988 and signified a bandwidth revolution with its capacity 10x that of its coaxial predecessors. By modern day standards it was a pygmy with 280 megabits total throughput. But it heralded the beginning of a long period of rapid throughput growth. Optical repeaters were spaced every 50 kilometers or 30 US miles. Photo diodes received the weak incoming signals, converted them into a digital representation of zeros and ones, and then laser diodes generated fresh light signals. A copper conductor provided power. Lots of redundancy in terms of components were built into this amplifiers to avoid repairs. However, this approach had a severe drawback, namely the throughput could not be increased in a time where Moore's law was rapidly increasing transmission rates. The pace of the optical-electrical-optical conversion was set in stone because an upgrade would require replacing all the hardware on all the amplifiers. TAT-8's ca...

***Senegal is the one of the most challenging on-net 2Africa countries. 1. Sonatel appears to have a monopoly on Dakar metro fibre. This spells trouble. 2. Sonatel's landline infrastructure monopoly may be one reason many 2Africa consortium members declined capacity into Senegal, despite its strategic location as the natural Northwest Africa telecom hub. Another reason might be their lack of mobile operations in that country as Senegal is a small market. 3. ONIX data center is the place to be in Senegal. Carrier neutral and on-net for 2Africa. Most African ISPs coming to me for advice plan to make ONIX their home. 4. I expect 100G pricing to be high due to the dearth of capacity. Figure low 30s for 100Gs to Europe and upper 20s for 100Gs to West African countries. ***Ghana is much friendlier. Many 2Africa carriers are on-net at the CLS and also the very popular PAIX data center. In fact, I know one that has built a ring from the CLS to PAIX reflecting strong demand for the latter. ...

Facebook engineers during an interview on the Waterworth project provided details about their next big project. 1. Facebook is planning to build the first one petabit per second Atlantic cable. No details yet available on end points. Given traffic flows it is most likely to directly connect Continental Europe and the US. 2. META engineering is looking at three options to achieve one petabit per second throughput. A. Using both the C and L spectrum. This would effectively double the bandwidth. B. Cable will definitely be SDM (spatial division multiplexing). Strong likelihood that will be 48 pairs. C. Another possibility is two core optical fibre in order to double the bandwidth per pair. 3. I believe the most likely option is using C and L bandwidth. Arelion has incorporated L band spectrum into its DWDM Layer 1 service between Atlanta data centers and Ashburn Equinix using Infinera gear. Colt and Sparkle have used the L band on terrestrial routes. Most DWDM equipment today offers both ...

1. The Waterworth cable will link the US to South Africa with a branching unit to Brazil. It extends from South Africa to India & onto to the Pacific and the US. The total length is greater than the Earth's circumference at approximately 50,000 kilometers. 2. Subsea cable projects are taking 3 to 5 years from initial idea to commercial service. 3. US traffic goes to Europe. It's aggregated with European originated traffic and then traverses the Mediterranean Sea to reach Egypt and takes terrestrial routes (Telecom Egypt) to the Red Sea. Then the traffic flows down the Red Sea. From there it either heads to India or bypasses it with Southeast Asia being the destination. 4. In the eyes of Facebook's subsea engineering team, the standard cable routing described above creates a host of problems. First of all , the Red Sea is a single point of failure. Same holds for Egypt. Secondly, the Mediterranean Sea requires many government permits as cables inevitably goes through ter...

First Point: The Portugal and Spanish grids are tightly integrated with limited power connector to the rest of Europe. Solar and wind play a big role and both power sources suffer from what is known as frequency instability. Solar and wind generated power is much more volatile than traditional power sources. Traditional power generators have angular momentum inertia. It takes a while to up or lower the power due to the inertia in the spinning components. A natural gas turbine takes a few minutes to spin up. A nuclear reactor an hour to lower or increase output by 10% (French reactors do load following). Solar and wind create very volatile power fluctuations that can easily trigger a circuit breaker. In an isolated grid if a circuit breaker is triggered, the power in the remaining active part of the grid increases. This triggers more circuit breakers and usually brings down the entire grid. 2. The consequence of the first point is that avoiding a repeat of the Iberian Penisula outage re...

Top 10 ASN Ranking! Not Cogent. Not Hurricane.

`This article reflects discussions I have had with war planners, government officials, policy analysts, and subsea cable colleagues.  1. A striking fact is that there is no compelling evidence of subsea cable outages due to sabotage since the end of WWII. Subsea cables are poor terrorism targets. Terrorists create terror by killing and maiming people and damaging highly visible and important infrastructure like bridges, skyscrapers, prominent buildings or sites having symbolic importance. Intentional damage of a thin cable buried two meters deep in the English Channel does not have the shock value or cause sufficient disruption by itself to justify the great effort of clandestinely locating and severing it. Secondly, there are so many cables now that sabotage of one or two has little impact on voice or data traffic. RIPE analysis indicated that a country like Estonia experienced little layer 3 degradation despite losing subsea cables landing in the country or adjacent Finland, a r...

 4. The star-shaped organization model defines each department by function such as sales, provisioning, procurement, network operations and the like and has it directly report to the COO or CEO. Each department has a large degree of autonomy and independence which encourages kingdom building and departmental bickering.  We improve cooperation by recognizing that accountability requires some departments be subordinate to others. For example, procurement falls into two categories: backbone and the customer driven requests where a third party component like a long haul circuit, metro dark fibre pair or a local loop is required. Third party sourcing for customer driven requests should in the sales department. I know salesmen at one European carrier that source their own off-net requirements because procurement takes to 2 to 6 weeks to do it. Once a verbal yes is received, the salesman then points procurement to the third party offer. If sales determined the compensation and perfor...

Retelit is a competitive Italian carrier with a pan-European network. Retelit together with the Italian Ministry of Economy and Finance have purchased Sparkle with a 30/70 ownership split. Why the current Italian government keeps insisting on public ownership of telecommunications infrastructure is beyond me. There are no real benefits. Italian government interference in important sectors like banking and now telecom have brought only problems, no solutions.  Government ownership often leads to management sacrificing long term financial health to goals such as preserving head count. This purchase together with the recent EXA acquisition of Aquacomm for only $45 million despite a couple hundred million dollars in network investments raises alarm bells about the financial health of the subsea cable industry. Sparkle generated a billion Euros in fiscal 2023 or 1.14 billion dollars at the current exchange rate. Yet it was purchased for far less than current revenues, a sig...

I've been in telecom since 1992. This qualifies me as an 'old fart' or 'dinosaur fossil' as American teenagers would say. This means I've seen every mistake made by subsea cable capacity owners. 1. Buying lots of capacity between cable landing stations, but owning no fibre from the CLS to the customers' destinations, namely the popular carrier neutral data centers. You can't be competitive if you must buy 100G or 400G metro waves from a UK landing station to Slough Equinix. Lease a dark fibre pair ring and light it with DWDM. Don't be penny wise and pound foolish. Those network investments will dramatically improve operating margins. The amazing thing is that PPT members of cable consortiums make this mistake all the time. If it is worth spending $55 million for an undersea fibre pair, then it is worth adding a couple million to the pot for back haul IRUs. 2. Refusing to extend the network to new locations unless the order achieves an investment thre...

When you come to Europe, you go to Amsterdam. It's inevitable. Amsterdam is one of Europe's two main Internet hub, the other being Frankfurt. Also the city is a major logistics hub that is a good hunting ground for commercial telecom deals. 1. AM5 is one of the best data centers for Internet peering, but has no power for new clients or upgrades for existing ones. Novation is your best bet.  2. AM3, Nikhef, and AMS17 do have power.  3. Nikhef is the best deal for smaller players because there are no recurring cross connects fees, power is available, and fractional racks have no install charges. Peering opportunities are excellent with both AMS-IX & NL-IX present. Excellent site for both private ISPs and financial trading firms.  4. NorthC's two Amsterdam facilities have plentiful space to lease. Not a lot of connectivity providers yet, but could prove highly attractive to the server intensive crowd which needs a lot of racks and power.  5. Relined is highly reco...

The conflict between China and the West is exacerbating bandwidth shortages on key routes like Marseille to Singapore. AAE1 is maxed out just like SWM5. Both will be upgraded this year, but I believe all the incremental capacity will be snatched up even before upgrades are finished and the capacity delivered to customers. Furthermore, China Unicom is the lead AAE1 consortium member with China Mobile also selling capacity on the system. Avoiding carriers incorporated in China makes intercontinental capacity shopping is an excruciating exercise. I've been seeking Express AAE1 100G for almost a year for clients for whom China is a red line. Bandwidth sourcing has become a marathon. 😄 In light of this, I recommend buyers consider Peace despite the fact that it is a Chinese financed project. Encryption does work. It will not protect the IP overhead, which include the IP addresses, but the data payload itself will remain safe. Moreover, there are Peace providers such as PCCW or TELIN t...

6. Inability to fully exploit and integrate acquisitions. Lumen bought denial-of-service cybersecurity firm Black Lotus in 2015, but the company's full potential has not been realized. Lumen uses Black Lotus for its highly rated IP network, but it has not really fully developed commercial services for this acquisition. 7. Collapse of transit pricing. Cogent and Hurricane Electric has caused pricing to collapse. I just heard of a deal where a client got a 10G commit on a 100G port for $500 a month plus bursting charges at an Amsterdam data center. 8. Inability to control network expenses. The graph below shows gross margins declining sharply over the last 15 years. A lot of that reflects transit and wavelength price declines. But it also reflects the company's arthritic bureaucratic culture: revenue per employee is $509,000 versus over a million dollars for Verizon. Market pricing is often out of a company's control. In the competitive telecom industry there is only one way ...

Lumen continues its restructuring. It aims to sell its consumer fiber division to AT&T for a rumored $5.5 billion dollars. It sold its European and South American assets as well as its Atlantic cables over the last 24 months. In 2021 Lumen solds it local exchange assets, the old Centurylink copper line business, for $7.5 billion. Lumen is paying a high interest rate on its recently renegotiated debt of $17.5 billion. It is under severe pressure to continue paying down debt while also investing in network infrastructure. Lumen once had aspirations of being a global telecom player. It built South American and European networks from scratch during the New Millenium telecom frenzy. The carrier acquired a host of distressed competitors including Global Crossing, Progress Telecom, and US CLECs. In turn, Centurylink bought Lumen in a friendly merger in 2017. The idea was to infuse capital into the so-called sexy fibre optic business. The underlying idea was that Centurylink had a lot of c...