Introduction
Though we frequently forget about it nowadays, the idea that the internet would give rise to some of the largest and most profitable businesses in the world was not at all obvious at the outset. In 1996, Viacom’s CEO, Ed Horowitz, was able to remark dismissively that “the Internet has yet to fulfill its promise of commercial success. Why? Because there is no business model.”1
The answer to the question of how to make boatloads of money on the internet has been, resoundingly, advertising. From the biggest technology giants to the smallest startups, advertising remains the critical economic engine underwriting many of the core services that we depend on every day. In 2017, advertising constituted 87 percent of Google’s total revenue and 98 percent of Facebook’s total revenue. Advertising funds the production of online content. From long-standing publications like The New York Times to more recent outlets like BuzzFeed, advertising remains the core business model for online media despite massive technological changes over the decades.
Digital advertising is highly consolidated. It is dominated by a few major types of advertising and a few major companies. Search advertising, in which ads are placed alongside search engine results, accounts for about 46 percent of overall digital ad revenue.2 Google, not surprisingly, dominates this segment, accounting for 78 percent of the overall revenue from online searches.3 Display advertising—where the ads are delivered through image “banners” or similar media on a website—accounts for another 32 percent of overall digital ad revenue.4 Facebook is the biggest player in this segment, capturing about 39 percent of ad dollars spent in this format.5 Advertising delivered through formats other than search and display (such as video, audio, or other media) makes up a far smaller part of the revenue pie.6 Google controls around 37.2 percent of the overall U.S. digital ad spend, accounting for around $40 billion.7 Facebook controls another $21 billion of this market, accounting for another 19.6 percent of the U.S. market.8
This path for funding the web has had major implications on the development of the technology itself. Core services like online search and social media are available free of charge in large part because advertisers underwrite the costs of developing them. The basic building blocks of our present-day experience of the web—from the “user profile” to the “like”—allow advertisers to more effectively target messages to users.
While advertising has made services online more accessible, numerous voices have long pointed out that advertising has generated its own fair share of negative impacts as well. In The Attention Merchants, the law professor Tim Wu argues that the ever-expanding reach of advertising is responsible for the “widespread sense of attentional crisis” produced by the modern technological environment.9 The researcher Zeynep Tufekci has noted that the “deep surveillance-based profiling” and “bias toward inflammatory content” that are endemic online are natural outcomes of an advertising-based business model.10 Investigations into Russian meddling in the 2016 U.S. presidential election have underscored the degree to which advertising channels can be leveraged to enable state-sponsored campaigns of propaganda and disinformation.11
These concerns are even shared by the creators of the services that have come to dominate the web. Infamously, Google’s cofounders, Larry Page and Sergey Brin, worried about the perverse incentives of advertising in their seminal 1998 paper laying out the rudiments of the core algorithm behind web search. “The goals of the advertising business model do not always correspond to providing quality search to users,” they wrote. “We expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers.”12
In the midst of all this criticism, it’s worth taking a moment to think about what precisely advertising is. Media buyers—whether they are a global company, a mom-and-pop shop, or a nefarious band of trolls seeking to influence an election—are all looking to get their message in front of people. Online platforms—whether they are a billion-user social media platform or a small neighborhood newsletter—offer ways to get that message to those people. The platforms sell access to their users, and the buyers pay to acquire that access to distribute their message.
Buyers and sellers. At its core, advertising is a marketplace for attention. When your eyes breeze over an advertisement as you scroll through your news feeds or read an article, a transaction has occurred. Your attention has been sold by the platform and bought by the advertiser.
Even though all markets involve some kind of buying and selling, not all markets are made equal. The kinds of buying and selling that take place in a rural farmers market, for instance, are vastly different from what takes place every day on the New York Stock Exchange. We can distinguish between different types of markets in lots of different ways. Are people trading for their own livelihoods, or are they speculators looking for opportunities for profit? Are there a lot of buyers and a small number of sellers in the market? The opposite? What is being sold? How much of it is there? How easy is it to close a transaction?
Markets come in many shapes and sizes. They also are entwined with society in different ways. Changes in the marketplace for oil have wide-ranging impacts that can shape the entire course of the global economy. The failure of a local coffee shop may cause an appreciable impact only on the surrounding neighborhood. In other words, it is not enough just to think of the internet as an attention marketplace. To fully understand the implications for society at large, we must ask precisely what kind of marketplace online advertising is.
This book explores exactly this question. Advertising as a marketplace for attention has undergone major structural changes in the past decades, largely driven by the rise of the internet. As a result, the practice of buying and selling billboards in the Mad Men era of the 1960s bears little resemblance to the modern-day marketplace of online advertising.
Today, online advertising is more Wall Street than local bookfair. The exchange of attention is facilitated through a massive global system of digital marketplaces. Attention is not just for sale online: it has been automated and streamlined in ways that we often fail to appreciate.
As they do in modern-day capital markets, machines dominate the modern-day ecosystem of advertising on the web. This method, known in the industry as programmatic advertising, leverages software to automate the buying and selling of advertising inventory.
This is how Facebook and Google—the dominant duopoly—sell the attention they capture on their platforms. But, even beyond these leading companies, advertising both online and offline is increasingly bought and sold by machines. Search advertising, which was the first form to fully adopt this approach, is almost entirely transacted in a programmatic way. Programmatic advertising is dominating the other segments of the digital advertising market as well. As of 2017, programmatic advertising claimed fully 78 percent of the total digital display ad spending in the United States, representing more than $32 billion of activity.13 This trend is set to grow, with projections showing this amount to increase to 86 percent in 2021 as automation continues to replace human buyers and sellers in this marketplace.14
New channels of advertising have also become integrated into the programmatic ecosystem. Advertising distributed alongside online video is now mostly facilitated through this method, accounting for 76.5 percent of the total U.S. digital video ad spend.15 It accounted for some 74.1 percent of U.S. ad spending on mobile advertising in 2017.16 The share of dollars going into programmatic advertising in each of these domains is also projected to rise in the coming years.17 The global infrastructure of programmatic advertising makes it possible to access a seemingly limitless well of opportunities to place ads.
These markets have also radically expanded the scope of who can participate. Programmatic advertising tools are increasingly streamlined for easy use by nonexperts, enabling everyone from the marketing department of Coca-Cola to a local hobbyist blogger to buy and sell attention online. Sophisticated tools for targeting ads and analyzing audiences are more and more available, giving even private individuals the ability to deliver advertisements with a precision that would have been considered unthinkable to the biggest multinationals just a few decades ago.
This was not a foregone conclusion. In an alternative universe, advertising might still have ended up the dominant business model for the web, but the marketplace might have been structured in a very different way depending on how certain choices were made in the history of its development. Markets are designed and implemented by people. Those people make decisions around these economies that subsequently have wide-ranging impact.
In this respect, the resemblance of the online advertising marketplace to the financial markets is no accident. This book investigates the fascinating connections between the finance industry and the evolution of the modern programmatic advertising ecosystem. Financial markets have served as a major source of inspiration in the design of the online ad economy throughout its history, and many of the founding leaders who built these marketplaces hailed from previous careers as brokers and traders in finance.
Why does this matter? Telling this history and exploring these nuances are more than just academic exercises. Because advertising is responsible for such a colossal portion of the money that drives the internet, it is impossible to think about the future of the web without thinking about the future of advertising. Shifts in how attention is bought and sold will have major consequences not only for our everyday experience of the web but also for how the internet affects broad questions of expression, identity, and democracy.
On this count, the parallels between the financial markets and the attention markets reveal crucial hints about what the internet might be evolving toward. Indeed, the rush to architect the buying and selling of attention in the model of the financial markets raises the question of whether some of the problems of the financial markets will follow the attention economies of the web.
There is good reason to believe that the financial foundations of the web are perhaps shakier than we think, maybe even producing the conditions for a “subprime” crisis in attention, similar to the dynamics that brought down the global economy in 2008. Examining these parallels is a key step toward thinking about how society, if it’s not too late, might want to re-architect the web for the better, and about whether the web as we understand it will endure.
1
The Plumbing
The jargon of programmatic advertising evokes a vision of the Wall Street of the 1980s. “Trade desks” buy and sell “inventory” on “exchanges.” You might imagine something out of movies like The Wolf of Wall Street and Trading Places: big, cavernous rooms packed with shouting traders in colorful vests. But this vision of how business is done is long out-of-date. The reality resembles the ultra-quantitative, high-frequency trading of Michael Lewis’s Flash Boys more than it does the predigital, smoke-filled rooms of Mad Men.
The manual processes for negotiating the sale of advertising inventory have been replaced over time with something far more mechanized. The trading floors of the global exchanges for attention are a mostly silent affair, a global network of humming servers and software that buy and sell slices of attention millions of times per second. In this marketplace, the advertiser operates as a mere supervisor to a large-scale, data-driven, automated process. This transition has been made necessary as the speed of transactions in this marketplace has outstripped the ability of any human to keep up.
A Tour of Programmatic Advertising
Being served an ad online is the textbook case of an unremarkable experience. You load up a website or an app on your phone, and an ad appears alongside, in front of, or under the thing that you wanted to see. It’s a small distraction as you browse around online, and for the most part it goes unnoticed. If you—like millions of others—have installed an ad blocker on your phone or web browser, you may never see these advertisements at all. It’s difficult to recall clearly the last five or six ads that you might have run into online. It’s probably even more difficult to recall the last time that you actually clicked on one.
Yet this boring, mundane occurrence online belies the intricate machinery running under the hood of each and every one of these ads. The modern online advertising ecosystem is a mind-bending, globe-spanning infrastructure designed to deliver billions of advertisements at split-second speeds every minute of every day. What you see at the end point—a banner advertisement that you may instantly forget, or a preroll video that you click to skip as quickly as you can—is only the tiniest piece of an incredibly complex system that is otherwise invisible as you move through the internet.
The players in the marketplace for attention are the same as they always have been: those who have attention they are willing to sell, and those who are willing to buy. In online advertising parlance, those who sell attention are the “publishers.” Attention can be captured in many different ways. A YouTube star who has millions of viewers tuning in to see her latest video and a social media platform where users are logging on to chat and comment on each other’s photos are examples of publishers with a surplus of attention—other people’s attention—to sell. Publishers sell “inventory”: the opportunity for the buyer to display its message to someone who is paying attention to the publisher. The buyers vary: they include large companies buying advertising to promote their products and marketing agencies working on their behalf, as well as what are known as agency trading desks—specialized companies focused on navigating the programmatic advertising ecosystem.
The infrastructure of programmatic advertising is architected so that, theoretically, publishers are able to sell their inventory of attention to the highest bidder among a pool of buyers. This is generally done through an arrangement known as real-time bidding (RTB). RTB is initiated by the tiniest of actions—clicking on a link or loading a piece of content—which sets off a rapid, orderly cascade of events, Rube Goldberg–style. As soon as an opportunity for delivering an advertisement appears, an ad server leaps into action, announcing to the marketplace the opportunity to bid for inventory.
One of the most incredible aspects of the RTB system is that the entire process takes place in real time. The advertisements you see online are not predetermined. At the moment you click the link and load up the page, a signal from the ad server triggers an instantaneous auction to determine which ad will be delivered. The highest bidder gets to load its ad on the website and into your eyeballs.
This process happens at the speed of light. The inventory must be bid upon and the actual advertisement delivered in the split-second moment between when you click to load something online and the time that the website or content on an app is finally loaded. The entire process of putting out a request for bids, making the bids, evaluating the bids, and delivering the advertisement takes place in under a hundred milliseconds—about a quarter of the time it takes you to blink.1 This happens millions and millions of times across the internet every second, without ceasing and largely without hiccups.
This combination of rapid bidding and massive scale means that humans cannot be directly responsible for initiating the auction, bidding, and evaluating the bids. There simply is not enough time. Programmatic advertising is facilitated entirely through software platforms that automate this process. These are known as demand-side platforms (DSPs) or supply-side platforms (SSPs). DSPs service the buyers, providing tools to specify certain parameters around their bidding and specify the audiences they would like to target. SSPs provide a suite of tools that are a mirror image of this for publishers, enabling sellers to specify auction rules, set price floors, and control the kinds of advertisers able to gain access to their inventory.
Once configured manually, programmatic advertising relies on the interactions between algorithms to make the discrete choices to bid on available blobs of advertising inventory. These competing algorithms working on behalf of ad buyers then interact with algorithms working on behalf of the publisher, which choose the winner of the auction and deliver the selected advertisement.
As with the Nasdaq and the New York Stock Exchange, the buying and selling of advertising inventory does not take place all in one location. There exists a dense, networked set of ad exchanges that facilitate the meeting of supply and demand in this marketplace. At the time of this writing, companies like AppNexus, OpenX, and Smaato are leading players in this space, alongside exchanges operated by larger long-standing players like Verizon Media (formerly Oath) and Microsoft. The largest platforms—Google and Facebook—offer their advertising inventory on their own specialized tools, which operate much like a DSP for buyers.
Perhaps most remarkably, all of this takes place seamlessly. We are bombarded by ads online but give little thought to the complex, behind-the-scenes machinery of bidding and delivery that makes them possible. This invisibility belies the huge importance of this infrastructure and the implications of the failure of this system for the web and for society as a whole.
What Happens If This Breaks?
The economic significance of programmatic advertising is huge. Most obviously, the iconic services of the present-day internet—search engines, social media platforms, and more—are built on a bedrock of advertising both in the United States and beyond. Whether it is Google or Baidu, Facebook or Weibo, all these services rely on a robust, automated market for buying and selling attention to generate revenue. But programmatic advertising is critical not just because it is the business model for a handful of dominant technology players. It is important because it is increasingly intertwined throughout the broader economy.
Certain goods and services are more interlinked throughout the economy than others. Oil is a prototypical example. The price of oil is important not only because it influences the balance sheets of the companies that extract and refine it. Its influence echoes through the economy, affecting things like the costs of transportation and of the production of certain goods. Because it forms such a fundamental part of the economies of certain nations, the price of oil shapes the balance of power among countries as well.2
This relative interconnectedness of particular industries has important implications for what happens to the broader economy when those industries run into trouble. One reason that the 2007–2008 subprime mortgage crisis was so damaging was that banks and other major financial institutions at the core of the economy relied on the continued health of what turned out to be shoddy mortgages. When the mortgages failed, so did the banks, which in turn were connected to a vast range of economic activities throughout the world. This served to make a downturn in the mortgage market one that placed intense pressure on other parts of the economy at the same time. Economists contrast the 2008 downturn with the 2001 dot-com bust in technology stocks, which was more isolated and had a less enduring impact in part because the tech companies were less interlinked with the rest of the economy.3
Programmatic advertising more closely resembles oil than an overhyped mid-1990s technology stock in this respect. For one, advertising is still the primary financial engine funding the creation of most media. The ad exchange infrastructure is deeply integrated into the online world because everyone, from independent stars on YouTube to massive media companies, leverages the programmatic advertising infrastructure. Increasingly, advertising on “nondigital” media like television, radio, and even billboards is facilitated through these global exchanges.4
The influence of programmatic advertising, however, extends beyond its role in bankrolling the media. The online advertising ecosystem has proven to be a highly flexible infrastructure, one that can transform products and services that weren’t previously monetized through advertising into advertising businesses. It’s easy to forget that products like email were originally operated as fee-based businesses in which various features like storage and spam filtering were monetized as a subscription.5 Services like Gmail transformed this, using the advertising model to make email free to use and monetizing it through the sale of user attention.6
One way of looking at the interconnectedness of programmatic advertising is to follow the money, asking what kinds of businesses rely on the flow of money from these systems. It’s equally revealing to follow where the wealth generated by advertising is subsequently invested.
Google is a perfect example of advertising’s remarkable ability to fund other ambitious ventures. Whether underwriting a massive effort to scan the world’s books or enabling the purchase of leading robotics companies, Google’s revenue from programmatic advertising has, in effect, reshaped other industries.7 Major scientific breakthroughs, like recent advances in artificial intelligence and machine learning, have largely been made possible by a handful of corporations, many of which derive the vast majority of their wealth from online programmatic advertising.8
Online advertising is also important for the wealth it has generated on the individual level. The advertising economy turned Mark Zuckerberg into one of the richest people in the world, and that wealth now supports a range of other endeavors. Zuckerberg has pledged Facebook shares worth more than $45 billion to his Chan Zuckerberg Initiative, which focuses on “health, education, scientific research, and energy,” making it instantly one of the most well-resourced entities in these domains.9 And for every Zuckerberg, there are thousands of other technology entrepreneurs who have made smaller, though still significant, fortunes through businesses funded by programmatic advertising. How these lucky few spend their wealth connects the fate of the advertising economy with a range of other, unrelated causes.
The fact that these invisible, silent programmatic marketplaces are critical to the continued functioning of the internet—and the solvency of so much more—begs a somewhat morbid thought experiment: What would a crisis in this elaborately designed system look like? What if the advertising revenue generated by the online attention marketplace were to decline rapidly and remain depressed for a long time?
The immediate and most obvious impact would be on the online platforms and ad exchanges themselves. There would be not just a sharp decline in the revenue generated by their core businesses but also a corresponding pinch in stock prices driven by investor panic in the financial markets. Recall that—for all their market dominance—platforms like Google and Facebook are heavily dependent on their continued revenue from advertising.
Interestingly, the impact might not immediately be felt by the everyday user of the web. Companies like Facebook and Google hold substantial reserves in cash, and they might be able to weather the short-term impact by drawing on these resources to maintain the status quo. The platforms could continue to offer their services for free to consumers in the hopes that confidence would be restored and ad buyers would return to the market. They could continue to fund product development, hire top engineers, and ensure the flow of quality snacks to their corporate campuses.
This crisis would generate an intense phase of consolidation beyond the rarefied confines of the Google-Facebook duopoly. Companies without similarly massive cash reserves would face pressure to close or merge to remain in operation. But the net effect in the short term would be that while businesses would be hurting, end users would still interact with their products and services in largely the same way.
This would be a temporary state of affairs: longer-term, the impact of this crisis would become more apparent to the average user. Intense dysfunction in the online advertising markets would threaten to create a structural breakdown of the classic bargain at the core of the information economy: services can be provided for free online to consumers, insofar as they are subsidized by the revenue generated from advertising. Companies would be forced to shift their business models in the face of a large and growing revenue gap, necessitating the rollout of models that require the consumer to pay directly for services. Paywalls, paid tiers of content, and subscription models would become more commonplace. Within the various properties owned by the dominant online platforms, services subsidized by advertising that are otherwise unprofitable might be shut down.
Imagine waking up to the announcement that searching the web would now require a monthly subscription fee, or that your favorite social network would have limited features until you added a credit card to sign up for a premium version. Imagine being charged on a per-trip basis for navigating with Google Maps. Think of WhatsApp—which was acquired by Facebook in 2014 but has not made significant money for the platform10—being shut down in order to preserve Facebook’s bottom line. How much would you be willing to pay for these services? What would you shell out for, and what would you leave behind? The ripple effects of a crisis in online advertising would fundamentally change how we consume and navigate the web.
Silicon Valley would not be the only region hurt by such a crisis. The failure of the programmatic advertising economy would have follow-up impacts throughout the media, considering how many media businesses rely on the programmatic ecosystem. Already in 2013, a survey of publishers revealed that 72 percent of them were offering space to advertisers through automated real-time auctions.11 Condé Nast—the leading mass media company, which owns publications like Wired, The New Yorker, Vanity Fair, and GQ—makes advertising on all of its digital properties buyable through programmatic marketplaces.12 Because programmatic advertising is increasingly a means of selling advertising through channels other than just banner ads on websites, a downturn in this economy might negatively affect some services that you might not immediately suspect. The audio streaming service Spotify, for instance, receives 20 percent of its advertising buys through programmatic means.13
All of these businesses rely on the continued strength of automated advertising marketplaces to generate revenue. The obvious outcome of a downturn would be for these businesses to cut costs. We might expect a massive round of layoffs in digital media, similar to but much larger than the 2019 firings at prominent content channels like Vice, BuzzFeed, Vox, and Mashable.14 Even a short-term decline in advertising could create massive disruptions. As the 2020 newsroom layoffs triggered by the COVID-19 outbreak show, most media businesses are not well positioned to weather even brief periods of weakness in the market for ads. Media companies dependent on online advertising might also become attractive acquisition targets for the well-resourced online platforms and other buyers in such a financially distressed environment.
Consider for a moment all of the advertising-supported media that you consume for free on a daily basis: podcasts, videos, articles, email newsletters, and more. How would it feel to have a large portion of that media suddenly disappear behind a paywall—or disappear entirely? How would your experience of the web change? What would it feel like for Facebook to announce that it was purchasing The New York Times or for YouTube to announce that it was purchasing Vice? What we consume and who controls what we consume would radically change in such an environment.
We might be able to stomach these changes as users of the web, but it’s important not to lose sight of the significant human costs of such changes. Paywalls rising throughout the web would exclude large populations of consumers unable to afford services that until recently were free. A failure of the online advertising markets would have a serious impact on a wide range of journalists, videographers, and other media creators great and small. Social media and platforms like YouTube serve as free distribution channels, allowing creators to reach much broader audiences than they otherwise would. A changing business model that prioritized subscription and paid access would narrow these prospects and make content creation less sustainable. Not to mention the knock-on effects that might emerge from a radical slowing of the spigot of philanthropic funding—from supporting medical research to fighting climate change—driven by a contraction of wealth in the technology sector.
In the most dramatic case, a sustained depression in the global programmatic advertising marketplace would pose some thorny questions not entirely unlike those faced by the government during the darkest days of the 2008 financial crisis. Are advertising-reliant services like social media platforms, search engines, and video streaming so important to the regular functioning of society and the economy that they need to be supported lest they take down other parts of the economy with them? Are they, in some sense, “too big to fail”? In an era when the relationship between the government and these platforms is becoming increasingly adversarial, state action to bail out a fragile tech industry might massively reshape the relationship between the state and the internet.15 What if the White House were to offer to keep key internet platforms solvent on the condition that they accept greater regulation and deeper involvement by government appointees in their operations?
This is a long way of saying that the web that we experience when we roll out of our beds and unlock our phones is sustainable only because of the continued health of programmatic advertising. Change that, and the whole edifice of the economy built on top of it begins to change with it. Persistent lack of confidence in the online advertising markets would produce a web that would be more fragmented, less accessible, and stuck at a slower rate of growth quite distinct from the last few decades of rapid growth in the technology sector. Reasonable minds can disagree about whether these changes are long overdue or desirable, but it is certain that a sudden transition would be painful.
The skeptic might point out here that it is easy to imagine a doomsday scenario but quite a different thing for that speculation to become reality. Would a sustained, global collapse in programmatic advertising actually be possible? How might it come about?
Even as the world grapples with a global recession in the wake of the COVID-19 pandemic, programmatic advertising appears poised to grow, grow, grow over the long term. The dominant, ad-driven platforms of the web continue to be some of the largest and most profitable businesses in the world. Traditional forms of advertising—the dollars that put ads in newspapers and magazines, for instance—are not yet fully integrated into the ecosystem of online programmatic advertising. This means that there is room for growth: these dollars may yet become part of the ad exchange system.
But such a rosy situation may mask deeper systemic risks lurking within the system. Like the financial markets prior to the crisis of 2008, the modern infrastructure of the ad economy might have produced explosive growth while simultaneously introducing a set of vulnerabilities that could make this money machine less stable over time. Indeed, the parallels between the financial industry and the evolving economy of online advertising give us clues as to where things might be going and the less evident cracks that might lie beneath the surface.
Copyright © 2020 by Tim Hwang
