Danyl Mclauchlan stares into the abyss that is Google and wonders if we are about to experience the birth of a new, even more terrifying capitalism.

I feel it most when I’m at the supermarket. I’m standing there looking at jellymeat but at the same time, I’m aware of being embedded in a web of data and analytics. I’m watched by the store’s security system but doubt any human will ever see the footage. I wonder if my presence – my choices, expressions, conversations – is quantised, aggregated into behavioural datasets, auctioned on prediction markets to vendors and consumer research companies, used to optimise product design, packaging and store layout. I sense that my thoughts and feelings are not my own, that they’re mediated by a vast accumulation of psychology and machine learning. I never knew what to call this feeling: it was just a vague sense of powerlessness; a background paranoia. Shoshana Zuboff calls it surveillance capitalism.

Zuboff is a professor emerita of social psychology at Harvard Business School, one of the first tenured women on the faculty. Her book The Age of Surveillance Capitalism: The Fight for the Future at the New Frontier of Power is huge, both in the sense that it’s important; vital; essential; all the superlatives, and also just literally very long, a little over 700 pages. It’s a work of history, telling the story of the tech industry over the last two decades, from the collapse of the dotcom bubble to Silicon Valley’s resurgence and global ascendancy. It’s a work of undercover journalism: she interviewed over 50 tech industry sector data scientists, almost all of them under conditions of confidentiality and anonymity. It’s a work of economic and political and social analysis. Somewhat inexplicably there is also a lot of poetry by WH Auden.

Zuboff begins her story at Google in 2002. The tech bubble had burst. Google had investors but no business model; great search technology but poor revenue streams. There was Google AdWords which displayed advertising adjacent to search results based on keywords. If you typed in, say, ‘pizza Auckland’ and an Auckland based pizza company had paid to link to that result, then you saw their ad. But not many people clicked on the links. Many users of Google and Facebook, and the countless free apps and other ubiquitous forms of surveillance capitalism mistakenly believe that this is still the deal: that you get a free service or product for in exchange for exposure to advertising. This is definitely what we’re supposed to think.

Google’s founders, Sergey Brin and Larry Page, were sceptical of online advertising: it corrupted the purity of the algorithm, and besides, there just wasn’t much money in it. AdWords wasn’t a cool place to work. All of that changed when Google’s investors got nervous because the tech industry was collapsing around them, and at the same time, the ad department figured out how to monetise search histories.

Prior to 2002, Google didn’t do anything with your history: it was just surplus data – “digital exhaust” – cached on a server someplace. But while each individual search was near worthless, if they looked at your history in its totality, built up behavioural profiles, binned them with other users, cross-referenced them with metadata, and performed other forms of analysis designed and honed by some of the smartest statisticians and computer scientists on the planet, you could build astonishingly complete pictures of almost everyone who used Google search. And – here’s what’s really crucial – before posting an ad or a link you could make a prediction about whether people would make a purchase subsequent to it being displayed.

If you’re a pizza company and you sell advertising to a newspaper, TV station or billboard company, you’re making a bet that some subset of the audience – probably a microscopic one – will see it and then go out and buy your product, hoping that this will lead to higher profits than the cost of the ad. What Google sells is so much better. They sell the probability that your ad will have value targeted at each individual customer. They can post a link to one person in Auckland located at a precise geographic location searching for pizza on their phone, and they can estimate a 95% probability that it will lead to a purchase, based on everything Google knows about them, which is effectively everything. Now, Google asks, who wants to buy that ad and how much will you bid for it?

Google stopped being a search company in late 2002. After all, it’s just not a very profitable thing to be. Now the search engine is merely one of many tools in a vast supply chain, a global extraction architecture designed to harvest as much information as possible about as many people as possible and to make their predictions about our future behaviour as accurate as possible. They give us Gmail so they can read our mail; Google Maps so they can see where we go, and how we get there; they give away Chrome so they can see everything we do on the internet, especially everything we buy; YouTube records what we watch; licenses for Android, the smartphone operating system are free so they can read all our other messages, track all of the media we consume, listen to our conversations.

The behavioural predictions they build from all this data and analytics are the holy grail of marketing, and because they’re such valuable products Google can sell them to the highest bidder in one of the trillions of simultaneous auctions they run on their futures market. Instead of trading shares, or currency or financial products, Google and their imitators trade predictions about individual behaviour and options to modify that behaviour. This is their real business: their world-changing elaboration of free market capitalism.

It’s a cliche to say of media and tech platforms that the users aren’t the customers – they’re the product. But Zuboff argues that this cliche is no longer true, and it’s the reason tech platforms are so different from the free service or content models adopted by previous iterations of media. Now we – the users – are the raw material. We’re like seabeds being dredged for minerals or forests chopped down for logs, without any awareness that this is happening and no legal or political framework to protect us.

Google’s most aggressive and successful competitor is Facebook. Facebook’s COO, Sheryl Sandberg, a former Google sales executive is credited with duplicating the futures market model. Because Google and Facebook are now two of the most successful companies in the history of capitalism, their model is being widely adopted, metastasising to every other industry: finance, health, insurance, recruitment, retail. Supermarkets. And the behaviour modification is not always commercial. It can be a link or a video or a news story encouraging support for a conspiracy theory, or a policy, political candidate or content designed to alter your mood, typically to make you more angry, anxious or depressed and therefore more vulnerable to future manipulation. Advertising was the beginning of the surveillance project, Zuboff warns, not the end.

Back in the 1850s, Karl Marx was still working out his critique of industrial capitalism and he developed a famous model of how to understand complex societies and historical change: the theory of the base and the superstructure. Zuboff is not a Marxist in a political sense – she’s a professor at Harvard Business, after all. She describes herself as a ‘rational capitalist’. but she uses some of the framework and concepts of orthodox Marxism, including the base-superstructure model.

The superstructure is everything we notice when we look at society: all our political systems, our beliefs, our values, our culture, our art, our language, our institutions. This is what is easily seen and what we think is important. But beneath all of this, Marx claimed, is the base: the deep economic and social relations defining the true nature of power in society: it fixes the possibilities and limits of everything that happens in the superstructure. The base is unseen, either because it is deliberately opaque, or because we just take it for granted, or we’re too distracted by events in the superstructure. And when things in the base change, everything up in the superstructure changes  – but it’s hard to see why.

Zuboff argues that behavioural futures are a new form of capital, comparable to shares, property, currency, or financial derivatives. But also profoundly different. Just as the development of industrial capital – mines, factories, machines – drove the industrial revolution and caused the rapid and radical transformation of the world, the emergence of surveillance capital is transforming our society and our lives, but it is complex and fast and hard to see. It’s happening in the base, in other words: it constitutes a change in our social and economic relations.

The nature of that change is the appropriation of our lives. There’s this assumption – in liberal democracies, at least – that our experiences and thoughts and choices are our own. We can trade our labour for money or agree to watch TV in exchange for viewing ads. But the choices and experiences and thoughts belong to us. Surveillance capitalism nullifies that assumption. It appropriates and monetises as much of our lived experience as it can, asserting its ownership then auctioning it off, and it does so in a way that is both aggressive and covert.

This is changing our politics, especially the politics around privacy, because privacy is now inimical to corporate profit. And of course, it’s changing the internet which was supposed to be this democratising, distributed network where we all talk to each other and find out about the world. But the way the information flows is increasingly asymmetric: instead of talking to each other with data routing back and forth, the data flows from us, captured by the sensors in our home, or car, or office, or on our person, to them. And it has to be this way because surveillance capitalism isn’t a company or a group of companies. It’s not a technology and it’s not a platform, although it touches all of these things. At heart, it is an economic logic that asserts that our lives are someone else’s commodity.

It’s impossible not to compare The Age of Surveillance Capitalism with Piketty’s Capital in the Twenty-First Century. They’re both monumental, deep studies of contemporary capitalism. Zuboff’s book is a lot more accessible than Piketty’s, and it’s a lot closer to the sociology literature, Marx especially. Meanwhile, Piketty is scathingly anti-Marxist (I think this has something to do with being an intellectual in France.)

This gives Zuboff a wider perspective. It feels like she’s seeing a lot more and explaining a lot more. But it also means the book is steeped in the apocalyptic rhetoric that’s so ubiquitous in Marxist discourse. There’s no such thing as a moderately serious problem in Marxist analysis: everything is existential. Google and Facebook can’t just be bad actors doing bad things: they’re erasing our humanity and destroying the possibility of the future.

So it’s very disconcerting when Zuboff switches from problems to proposed solutions: she doesn’t want a revolution or to drown the streets of Palo Alto in blood. She wants a more attenuated version of capitalism; sanctuary spaces; more robust privacy. But even the chances of these very modest policy changes feels low. Google, she notes, is one of the most lavish, aggressive and litigious political lobbyists on the planet. She doesn’t spend a lot of time on solutions.

More and more people seem to use the phrase ‘late capitalism’ to describe the savage absurdity of our present moment in history. It’s an ultimately optimistic term. It looks at the world and assumes, very reasonably, that things cannot go on like this: that capitalism itself is in a decadent phase, that it will soon collapse under the weight of its internal contradictions to be replaced with something new and better, although no one knows quite what that is.

Zuboff’s book feels like a harsh and bitter laugh directly in the face of that notion. It tells us to look deeper, past the fashionable scorn for the tech bros of Silicon Valley, and pay attention to what they’re really doing. It shows us that present-day capitalism is horrifyingly robust; that it is being reinvented and renewed by some of the smartest and most ruthless people in the world at a scale, velocity and complexity the rest of us struggle to comprehend.

Maybe it is all just an illusion. Maybe the hour really is late, and capitalism itself will just melt into air. But maybe all these modern manifestations of capitalism – information capitalism, financial capitalism, surveillance capitalism – are different aspects of the same creature, a logic we cannot yet see in its totality. Maybe something vast and unseen is still unfolding itself, gradually awakening; opening its eyes in the automated trading floors and server farms and data warehouses of the world. Maybe the last few centuries of history were merely a transitional phase, a prelude – Precapitalism. Maybe the age of Actual Capitalism has only just begun?

The Age of Surveillance Capitalism: The fight for the future at the new frontier of power by Shoshana Zuboff (Allen & Unwin, $55) is available at Unity Books.