A forward-thinking manifesto from three Stanford professors—experts who have worked at ground zero of the tech revolution for decades—which reveals how big tech’s obsession with optimization and efficiency has sacrificed fundamental human values and outlines steps we can take to change course, renew our democracy, and save ourselves.
In no more than the blink of an eye, a naïve optimism about technology’s liberating potential has given way to a dystopian obsession with biased algorithms, surveillance capitalism, and job-displacing robots. Yet too few of us see any alternative to accepting the onward march of technology. We have simply accepted a technological future designed for us by technologists, the venture capitalists who fund them, and the politicians who give them free rein.
It doesn’t need to be this way.
System Error exposes the root of our current predicament: how big tech’s relentless focus on optimization is driving a future that reinforces discrimination, erodes privacy, displaces workers, and pollutes the information we get. This optimization mindset substitutes what companies care about for the values that we as a democratic society might choose to prioritize. Well-intentioned optimizers fail to measure all that is meaningful and, when their creative disruptions achieve great scale, they impose their values upon the rest of us.
Armed with an understanding of how technologists think and exercise their power, three Stanford professors—a philosopher working at the intersection of tech and ethics, a political scientist who served under Obama, and the director of the undergraduate Computer Science program at Stanford (also an early Google engineer)—reveal how we can hold that power to account.
Troubled by the values that permeate the university’s student body and its culture, they worked together to chart a new path forward, creating a popular course to transform how tomorrow’s technologists approach their profession. Now, as the dominance of big tech becomes an explosive societal conundrum, they share their provocative insights and concrete solutions to help everyone understand what is happening, what is at stake, and what we can do to control technology instead of letting it control us.
About the Author
ROB REICH is a philosopher who directs Stanford University’s Center for Ethics in Society and is associate director of its new Institute for Human-Centered Artificial Intelligence. He is a leading thinker at the intersection of ethics and technology, a prizewinning author, and has won multiple teaching awards. He helped create the global movement #GivingTuesday and serves as chair of its board.
MEHRAN SAHAMI was recruited to Google in its start-up days by Sergey Brin and was one of the inventors of email spam-filtering technology. With a background in machine learning and artificial intelligence, he returned to Stanford as a computer science professor in 2007 and helped redesign the undergraduate computer science curriculum. He is one of the instructors of Stanford’s massive introductory computer programming course taken by nearly 1,500 students per year. Mehran is also a limited partner in several VC funds and serves as an adviser to high-tech start-ups.
JEREMY M. WEINSTEIN went to Washington with President Obama in 2009. A key staffer in the White House, he foresaw how new technologies might remake the relationship between governments and citizens, and launched Obama’s Open Government Partnership. When Samantha Power was appointed US Ambassador to the United Nations, she brought Jeremy to New York, first as her chief of staff and then as her deputy. He returned to Stanford in 2015 as a professor of political science, where he now leads Stanford Impact Labs.
"Enough with the breathless dreams of digital utopias and poisonous polemics about technological dystopias! In System Error, we finally have a book about the digital revolution that is serious rather than sensationalistic. Read this if you want to understand how to shape our technological future and reinvigorate democracy along the way."
— Reed Hastings, co-founder and CEO of Netflix
“System Error offers a powerful account of how our lives, our politics, and our values have been reshaped by technology in ways that we are just starting to comprehend. Full of stories and insights, this remarkable book charts a path forward for creating a healthy digital future.”
— Darren Walker, President of the Ford Foundation
"Albert Einstein once lamented that 'our technology has exceeded our humanity.' That danger is ever more pressing as powerful artificial intelligence technologies are transforming society at a pace never seen before. From the heart of Silicon Valley comes a profoundly important book that examines the ethical and social impact of the digital technologies and offers a more human-centered framework. This is a must-read for every student, engineer, businessperson, policymaker, or anyone who cares about our society’s collective future."
— Dr. Fei-Fei Li, Professor Computer Science, Co-Director of Stanford's Institute for Human-Centered AI, and member of the National Academies of Engineering and Medicine
"System Error is a triumph: an analysis of the critical challenges facing our digital society that is as accessible as it is sophisticated. Best of all, the authors offer actual solutions for a reboot that are both timely and feasible."
— Anne-Marie Slaughter, CEO of New America
"It's not about the obvious villains. This wise, nuanced, quietly brilliant book reveals how technology is reshaping our society and our values in ways that are insidious, hidden—sometimes even from their inventors—and far more fascinating. Anyone who believes this reshaping shouldn't be entrusted to private companies needs to read it. Now."
— Larissa MacFarquhar, author of Strangers Drowning: Impossible Idealism, Drastic Choices, and the Urge to Help
"This is the best overview of the most severe problem facing the world today: that technology has become a weapon aimed at the heart of democracy. Balanced, thoughtful and constructive, this is exactly the kind of thinking we need more of."
— Glen Weyl, Microsoft’s Office of the Chief Technology Officer and Founder of the RadicalxChange Foundation
"The authors explore major issues that they posit society needs to grapple with: the rise in the outsourcing of decision-making to algorithms, the immense amount of user data collected by tech companies, increasing automation, and the proliferation of hate speech and disinformation online. Their suggestions for how the country might better balance democracy and technology are evenhanded and nuanced . . . Never falling into the trap of offering easy answers over deep analysis, this study is worth a look for readers worried about the outsize influence of technology on their lives and society."
— Publishers Weekly (starred review)
"Such important work."
— Joe Scarborough, Co-host of Morning Joe on MSNBC
"System Error is a powerful and important, timely book. It feels like a public service you have done for us, and I encourage everyone to pick up a copy."
— Julian Castro, Former Secretary of Housing and Urban Development
"This book’s contribution, and it’s an important one, is to spell out what needs to be fixed, and to provide two important reminders. First, the transition to digital in a democratic society was going to be messy no matter what—look at the antitrust fights that accompanied the telephone in the analog era. And second, solutions are going to be less about achieving utopia than about setting parameters to make sure certain things don’t happen. We live at best in a good-enough world.”
— Wall Street Journal