Beneath the surface of the internet are services that keep the whole thing alive (Image: Archi Banal)
Beneath the surface of the internet are services that keep the whole thing alive (Image: Archi Banal)

IRLJuly 14, 2022

The invisible infrastructure of the internet

Beneath the surface of the internet are services that keep the whole thing alive (Image: Archi Banal)
Beneath the surface of the internet are services that keep the whole thing alive (Image: Archi Banal)

Everybody online uses these services, even if most can’t name them or explain what they do exactly. For IRL, Shanti Mathias explores the internet’s second layer.

It’s another bright sunny day, and you’re logging on to the internet. A distant acquaintance has started a shop to sell candle-making kits; you tap the link in their bio, and purchase one. You’re trying to figure out why your washing machine is making a weird sound, so you read some help forums. You need to find a PDF of a form from years ago; your old laptop is long gone, but you can easily dig out a copy on Dropbox. You search for a recipe to use the halloumi and broccoli at the back of your fridge and refer to it as you cook. You watch some Netflix while messaging people before going to bed. 

In the course of this ordinary day’s internet use, you haven’t just used Facebook, Google, Netflix, goodrecipes.com, Dropbox, and iheartcandles.co.nz – you’ve also used Amazon Web Services (AWS), Stripe, Shopify, and Cloudflare. “You have layers of services that sit behind each other; Facebook or Twitter is the first layer, and Shopify and Cloudflare and the rest of them are the second layer,” says Andrew Chen, research fellow at Auckland University’s Centre for Informed Futures, who studies digital ethics. Picture the internet like other kinds of infrastructure: for most people, it’s enough to trust that water flows through pipes and electricity sizzles through wires, without needing to know the mechanics of water pressure and ions. Services to make websites load fast, hold data remotely, and manage the financial transactions of an online business can be useful without being visible from the outside. Ordinary internet users can effectively use the outer, visible layer of the internet – website interfaces – without comprehending the technical and financial complexities of their inner workings. 

Widespread understanding of how a service works isn’t essential for it to be big business. Take AWS, for example: the cloud provider, which offers digital storage for companies from Netflix to Kiwibank to Pfizer, generates twice as much profit as Amazon’s retail arm, amazon.com. AWS is so widespread that it’s nearly impossible to use the internet without it; your device probably accesses information stored in AWS servers dozens of times a day. 

Behind the curtain of the internet, there’s a lot happening that most users don’t see (Photo: Simon Day)

Meanwhile, online store builder Shopify, probably used by any small business with products available online, is worth nearly $40b; Stripe, a secure payments service that allows websites to process customer’s money, was valued at $95b in April; and Cloudflare, a content delivery network that distributes websites to servers across the world to improve load times and protect from DDOS attacks, has a market cap of $16b. 

“It’s probably a good sign that there isn’t much public awareness of these services,” says Chen. “If something bad happened, people would probably know about it.” When these services don’t work – Cloudflare going down damages Discord, for instance, and an AWS outage will hurt Netflix – users will hear about it, even if they don’t understand the technical reasons for the disruption. 

Despite their low profile, Chen says that these digital services are mostly good for consumers. Using digital infrastructure services means that developers don’t have to build their products from scratch “which means that services and products get developed faster, [and] businesses and consumers have more choice”. If a website you use every day uses Cloudflare so that its website takes two seconds less to load, that can reduce a flare of impatience each time you open it. Small businesses using Shopify to easily sell things online means more consumer alternatives from mainstream retail stores. 

These secondary services also hold data about millions of users, but Chen says it’s important not to let “tinfoil hat thinking” go too far. Most internet users are at least vaguely aware that the utility of the internet is funded by user data that can be sold to advertisers. If the terms and conditions allow it, both primary services – the website you’re accessing – and secondary services – the features built by other companies which that website uses – will hold some data that will help advertisers target you. 

Services like Shopify and Stripe help to monetise the internet, making it profitable for businesses to work online. Photo: The Spinoff/Getty Images

For the most part, infrastructure providers want to stay out of the headlines, so they won’t do anything egregious with user information. “All of these companies are still subject to laws and regulations, as well as the court of public opinion,” Chen says. There are competitors that would be more widely used if any of these services committed egregious data breaches; for the most part, the infrastructure providers are used because they’re effective at providing a service at an affordable price, and they want to keep it that way.

It’s also difficult to know how applicable concerns about data are in Aotearoa. After all, the internet is a global thing; many technology companies based here will be using overseas-based services like AWS and Shopify. While any technology company with New Zealand operations is subject to the Privacy Act 2020, a lot of the reporting on these services is not based in the local context. “A lot of concerns about data come from the US,” Chen says. “We’re a small market – it doesn’t necessarily translate to what’s happening here.”

While awareness of the commercial and technical complexities of the internet is a positive development, it’s good that technical knowledge is largely not a barrier to internet access, Chen says. “If we required that everyone understood how it works, there would be fewer people using the internet.”

Public Interest Journalism funded through NZ On Air.

Mad Chapman, Editor
Aotearoa continues to adapt to a new reality and The Spinoff is right there, sorting fact from fiction to bring you the latest updates and biggest stories. Help us continue this coverage, and so much more, by supporting The Spinoff Members.Madeleine Chapman, EditorJoin Members

Get The Spinoff
in your inbox