Imagine that Jill from 2006 and Jack from, say, 1990 are able to communicate with each other using an instant messenger program. (Think of it as The Lake House meets the Turing Test.) They discuss a wide range of topics, and Jack is stunned by Jill's intelligence – or, at least, her breadth of knowledge. She is familiar with every cultural reference Jack throws at her, no matter how obscure; she knows the plot of every novel, movie, and Cheers episode he mentions; she is aware of the critical and commercial reception to every record album in existence; she seems to have intimate knowledge of events that took place ten years before she was born. True, her responses are occasionally a bit sluggish, but no one could find these facts in an encyclopedia volume quickly enough to simultaneously hold a conversation in real time. Most of these facts aren't even in any encyclopedia.
So Jack asks Jill how she got so smart. He doesn't fully grasp her answer – what's a google? – but from what he understands, she's interfacing at extremely high speeds with a global data network that contains most of the important information in the world. To Jill, and to us, this is everyday technology and it’s nothing terribly impressive. But to a person from sixteen years ago, Jill and her Internet connection form a cybernetic organism with the entirety of human knowledge at her mind’s disposal, and the future she inhabits is a strange place indeed.
Now pivot forward another sixteen years and imagine a similar conversation between Jill in 2006 and Jeff in 2022. This time, let’s make it a phone call. Jeff comes across as highly intelligent and thoughtful. Not only is he knowledgeable, but even his opinions seem to be incredibly well-grounded, consistent, and supported by giant webs of facts and ideas. All of his responses come instantly; if he’s culling his information from the Internet or elsewhere, there’s no indication of it. It’s as if he’s already spent huge amounts of time researching and contemplating every conceivable topic of discussion. Jill knows no amount of frantic googling would allow her to keep up with him, especially in a voice conversation.
So she asks him how he got so smart. She doesn’t really understand his answer – something about “trained surrogates” and “situational analysis” and “rapid belief integration” – but basically, he’s some kind of cyborg hooked into a swarm of intelligent agents that are tuned to his environment, personality, actions, and goals. The future, Jill decides, must be a strange place indeed.
Humankind is in the process of inventing one of the most transformative tools in its history. It lies at the intersection of the human mind, global networks, and artificial intelligence. As our networks grow more complex and our software grows smarter, the ways in which our mind handles information will deeply change. The cognitive frontiers of cybernetics will move from information retrieval to information interpretation to information construction, and the nature of our cognition will be altered forever.
This article will examine the nature of the modern interface between digital information and the human mind, and how this interface relates to two distinct ideas: artificial intelligence and intelligence amplification. I’ll draw on concepts from computational linguistics, data mining, and cognitive psychology in an attempt to chart the cybernetic marriage of our minds to the algorithms and networks we use in our day-to-day lives. Hopefully, we’ll wind up with a clearer map of the road from 1990 to 2006 to 2022 and beyond.
The emphasis in this discussion will rest on computer software rather than hardware. In a networked era, a device’s ability to augment its user’s intelligence lies more in its software- and network-related assets, like the network’s bandwidth and the quality of the data and software on the network, than in its hardware resources, like memory and processor speed. A state-of-the-art PC is no better at browsing the Web than a six-year-old model. In other words, the Internet is useful to us because of the information and software that exists on it, and not because of the hardware we use to access it. (The hardware infrastructure of the Internet is still relevant because it determines bandwidth.) As applications increasingly migrate to Web-based implementations, the resources on the user’s end become deemphasized further, and the slack is picked up by centralized servers and network bandwidth.
This helps motivate recalibrating the connotation of phrases like “cybernetic organism” and “machine-augmented intelligence.” These terms evoke images of silicon chip implants and bionic women and the Borg Collective – in essence, a very hardware-oriented (and somewhat frightening) form of human-computer integration. What we see in reality are the beginnings of intelligence amplification with no direct integration of human physiology and computer hardware, but a high degree of interaction between human minds and computer networks through software. Instead of a direct brain-computer link, the computer-to-human interface is provided by the traditional senses of sight and hearing, and the forms of human-to-computer input are just as mundane.
Is today’s network-connected individual really a cybernetic organism? The term is meant to designate an organism that’s a mixture of organic and synthetic components. We can stretch this idea to describe a mind that’s a mixture of human cognition and software. I’ll call this type of cybernetic organism a soft cyborg. A human being armed with a good search engine is very nearly a soft cyborg who “knows” all the information that can be easily obtained on the Internet, since the speed of finding information online is approaching the speed of searching one’s own mind for information. In situations where this delay can be smoothed over as a delay in response time, like in an IM conversation or an email exchange, the soft cyborg of 2006 behaves like a human mind with limitless knowledge.
“Artificial intelligence” is a term that warrants a similar reevaluation. It suggests systems focused on reasoning and planning, and, in the extreme case, synthetic human-level consciousness, or “strong AI.” Judged on these dimensions, half a century of AI research has yielded disappointing results. Compared to humans, planning agents remain primitive and overly specialized, and despite progress in cognitive science and neurological modeling, strong AI is still struggling for a foothold.
But there is one human-level task at which today’s computers excel: the interpretation of data. Pattern recognition is among the skills considered most fundamental to human intelligence, and we routinely use machines to scale pattern recognition to particularly large or complex sets of data. This is usually referred to as data mining or knowledge discovery. Google’s PageRank algorithm, which analyzes the World Wide Web’s vast hyperlink structure to determine which Web sites are most “important,” is one data mining technique; the algorithms used by intelligence agencies to root out terrorists are another. The recent proliferation of social networking sites has led to new forms of data mining that leverage the intelligence and behavior of millions of users in the same way that PageRank leverages the “wisdom” of hyperlinks.
I contend that data mining systems represent artificial intelligence every bit as much as systems devoted to reasoning or decision-making. It’s true that data mining sometimes makes use of the same machine learning techniques as traditional AI, but it deserves to be categorized as bona fide artificial intelligence because of its deep connection to human thought: just as traditional AI seeks to emulate the distinctly human skills of reasoning, planning, and high-level perception, data mining strives to produce humanlike pattern recognition in a computational domain.
A significant difference between traditional AI systems and data mining systems lies in their usefulness to contemporary human cognition. In an information age, reasoning isn’t any harder than it used to be, but managing information is, due to the explosive rate at which information grows, spreads, and mutates, and the increasing speed with which we need to obtain specific pieces of it.
The traditional perspective sees AI systems as appliances: by focusing on planning and reasoning, we implicitly expect intelligent machines to work independently of us and to make our lives more convenient by freeing us from low-level tasks. A fresher perspective would allow for AI that works with us to expand and accelerate human cognition. Instead of an appliance or a human replacement, AI can be seen as a cognitive tool.
Data mining falls under the latter view as a specific type of artificial intelligence that, like all tools, serves to augment natural human ability. A hammer is useless in the absence of a person to intelligently manipulate it. By the same token, a search engine doesn’t really do anything independently of its human user, and it’s not weak AI nor a primitive type of strong AI nor a component of a strong AI, but when combined with a human that can choose keywords and select the correct results, the search engine becomes extremely useful.
I propose the term quiet AI for this kind of artificial intelligence – “quiet” because it is augmentative and unassuming, because it has nothing to say about machine sentience and is thus independent of the weak-strong axis, and because it has the potential to be folded so completely into our behavior that it can become nearly imperceptible. Quiet AI is vital to the notion of the soft cyborg; Web search, a form of data mining, is the basic artificial component of today’s soft cyborg, and the soft cyborg of the future will rely on more advanced varieties of quiet AI.
The difference between quiet AI and strong AI is an important one to note. While the end goal of strong AI is a total simulation of human intelligence in an artificial (and, presumably, accelerated) form, quiet AI aims toward the amplification of human intelligence using a combination of the artificial and the biological. At the precipice of a technological singularity, strong AI pushes us aside and jumps off alone, while quiet AI rather politely holds our hand on the way down.
THE CYBERNETIC MEMORY
Modern cognitive psychology divides the human long-term memory into two fundamental information types: declarative information and procedural information. Declarative memory, based on conscious recall, holds information that can be explicitly stored and retrieved. Procedural memory holds information derived from practice and implicit learning, and is closely linked to motor skill. Explicit knowledge about the state capitals is stored in your declarative memory; implicit knowledge about how to ride a unicycle is stored in your procedural memory.
Declarative information can be further divided into two subtypes: semantic information and episodic information. Semantic memory encodes abstract concepts and information about the world, while episodic memory stores personal sensations and emotions tied to specific experiences and contexts.
From this, it’s clear that the soft cyborg mind extends only the semantic memory store. It’s not possible for current technology to interface closely enough with either your motor system or your sensory organs to be able to handle procedural or episodic memory, and this is unlikely to change in the near future.
To put it another way, semantic memory is the only type of memory that can be easily communicated: a concept can be encoded as language by one person and transmitted to another, and once the language is interpreted, roughly the same concept should exist in the minds of both people. (Communicating episodic information is a lot harder, and it’s usually the purview of artists.) This underscores the importance of language to cognition, and, similarly, the importance of computational linguistics to intelligence amplification.
THE LOCATION OF MEANING
As noted by Graeme Hirst of the University of Toronto, the past three decades of computational linguistics have vacillated between three philosophies on where the meaning of a text is located:
- Meaning is in the reader.
- Meaning is in the writer.
- Meaning is in the text.
(“Text” is used here to denote any kind of utterance, short or long, speech or writing, and “meaning” denotes the complete semantic information encoded in a text.)
These three views were motivated by three corresponding paradigms in artificial intelligence research. The “in-reader” view of meaning prevailed from the mid-seventies to the mid-eighties, when research focused on creating intelligent agents that could assimilate knowledge about the world and use that knowledge to reason and make decisions. Computational linguistics saw texts as ambiguous conveyers of knowledge, and the goal of an intelligent agent was to find a semantic interpretation of a text that was most consistent with its existing knowledge. The more abstract knowledge an agent had, the better it was at interpreting a text.
Research then shifted to developing interactive systems that could converse with human users and determine the user’s intent, triggering an “in-writer” view of text meaning. Rather than learning about the world to understand text, it became necessary for intelligent agents to learn as much as possible about the user’s plans and goals so that they could interpret text from the user’s perspective.
From the mid-nineties onward, computational linguistics veered closer to quiet AI, and its principal application became information search and retrieval. The philosophy in this case is that objective meaning exists in the text itself, and it arises from the combined effect of the words in the text. As Hirst puts it, “meaning is ‘extracted’ from the text by ‘processing’ it.”
The language processing algorithms of this paradigm rely on statistics and data mining. One of the more well known of these methods is latent semantic analysis (LSA), which performs a type of mathematical reduction on text and represents words as points in a concept space of about three hundred semantic dimensions. By adding up the point vectors of words in a sentence, we obtain the position of that sentence in the concept space, and hence its meaning. We can find the meaning of a paragraph or a document the same way.
Of course, a set of coordinates in concept space seems like a crude representation of the semantics in a document, but it’s useful if you’re interested in sorting documents by their meaning or retrieving documents that correlate to some query concept. More creative applications of LSA include text summarization, automated essay scoring, and psychiatric diagnosis from patient writing samples.
As it turns out, hyperlink structure is rich enough with information that we don’t really need LSA-type algorithms for finding the Web pages we want, but computational representations of semantics will come to the fore as the objective of quiet AI expands from information search to information interpretation.
Hirst suggests that, over the next decade, both the in-reader and in-writer views of text meaning will reemerge as we use computers for two types of interpretation: interpretation on behalf of the reader and interpretation on behalf of the writer. In the first case, the computer acts as the user’s surrogate, tailoring its interpretation of information to the user’s goals, agenda, and beliefs. In the second case, the computer attempts to consider a text from the author’s point of view in order to understand what the author wishes to say, so that it can communicate meaning to the user as faithfully as possible.
SEMANTIC PUTTY AND PSYCHOLOGICAL FOOTPRINTS
Future progress in human-AI integration will be driven by progress along two dimensions: the plasticity of text and the amount of knowledge about the user.
Presently, quiet AI is very good at analyzing the implicit semantics of hyperlinks and the coarse meanings of objective texts for the purposes of information retrieval. But contemporary applications like text summarization give us a foretaste of plastic text that maintains semantic integrity. The goal is to view text as semantic putty: it can be stretched or shrunk as needed while its semantic substance – that is, its overall meaning and intent – doesn’t change. You’re given a paragraph-long summary of a breaking news event. You ask for more, and the paragraph stretches into a page of information constructed from bits of news floating around online. You ask for less, and the paragraph shrinks into a headline.
With a more precise understanding of a text’s intent, and hence a more faithful interpretation on behalf of the writer, we can extend this technique to opinion texts. A blogger writes a short post expressing a nuanced antiwar sentiment that you find appealing. You ask for more. Your AI seeks out documents that express a similar sentiment and algorithmically builds a long opinion piece on demand. You ask for less, and the sentiment is compressed into a sentence-long soundbite.
There are a number of ways for software to understand its user’s needs and goals, but they all rest on a constant monitoring of the user’s environment and actions. Real-time detection of new utterances and objects in your immediate surroundings will allow an AI to make predictions about the information you need in the here and now. Long-term analysis of your observable actions and communications – your psychological footprint – will give your AI a sense of your goals and beliefs. By examining its history with the user, an AI will be able to make educated guesses about what knowledge the user already holds in his or her head, and it will prune new information accordingly.
This can sound a little Orwellian, but it’s important to note that the monitoring is private and personal, and the goal of the AI is effectively to act as much like its user as possible. It’s not Big Brother; it’s Kid Brother. Through its interaction with you and with your environment, the quiet AI learns to mimic you and becomes capable of acting as your digital surrogate. What’s more, the merging of mind and software will see a blurring of the distinction between the monitoring of experience and the experience itself. At that point, to say that your software is spying on you is a little like saying your cerebral cortex is spying on your reptilian brain.
So, what do you get when you combine plastic semantics with a digital user surrogate? The implications are kind of staggering. This sort of AI would be able to independently seek out novel information that is compatible with your goals. It could read a long text and automatically form an opinion about it that would approximate the opinion you’d arrive at if you read the text yourself. It could locate other soft cyborgs in your “belief zone” and, through computation alone, interact with them to construct an entire self-consistent ideology. It could, with permission, load up the surrogate personality of another individual and tune its own information delivery accordingly, lending you a very real window into that person’s worldview. Less open-minded (or maybe just less careful) people could turn their AIs into confirmation drones that disfigure external information so that it never conflicts with their existing beliefs.
The potential uses of this technology are at turns exhilarating and frightening. The following section looks in on people living at various points in the future and describes how their mental life is transformed by quiet AI.
Sometime in the second decade of the twenty-first century, Jerome is interviewing for a Rhodes Scholarship. Wearable computing is big these days; Jerome wears a pair of networked glasses that act as a head-up display, overlaying information on top of his vision when needed.
The interviewer asks, “What are your thoughts on current trends in lumber imports from Canada?”
One of Jerome’s wearable devices has a microphone for voice communication. In real time, the device recognizes the text of the interviewer’s question and relays it to Jerome’s glasses. The text is interpreted, the salient phrases are extracted – “trends,” “lumber imports,” “Canada” – and, virtually instantly, a series of graphs and point-form facts are displayed for Jerome. He vamps for a second or two to take in the information, and then starts to answer the question confidently and knowledgeably. His opinion about the issue forms as he talks through the data. Five seconds ago, he didn’t know anything about Canadian lumber imports.
This is an example of anticipatory search – information before you ask for it. By monitoring his surroundings, Jerome’s wearable network located information on a relevant topic before he would’ve even begun to search for it. Jerome is a soft cyborg, accelerated and gone mobile.
The Passionate Wonk
A few years later, Julia is running for local political office. She’s in the midst of a debate against her opponent. By now, wearable devices and augmentation software have become so natural and ubiquitous that no one objects to politicians using them to answer questions. (Increasingly, it reflects poorly on leaders not to augment their intelligence.)
By monitoring the moderator’s questions and her opponent’s responses and rebuttals, Julia’s AI determines the debate issues being discussed. Based on what it knows about Julia’s political views, her AI builds specific opinions about these issues. Semantic analysis algorithms condense that information into small chunks to transmit to her contact lenses. The information is scaled based on how much time she has to speak, and it comes packaged with verified facts and figures to support her opinion. In effect, it’s a personal, automated, dynamic teleprompter.
After the debate, most observers declare Julia the winner. She is seen as passionate and knowledgeable and her arguments are convincing. But is this impression based on her human qualities, or the effectiveness of her algorithms? Is the electorate voting for a person or for software? Perhaps more importantly, should it matter?
Several years later, Janet is a college student majoring in world literature. She only had one class today, and she didn’t feel much like going; instead, she spun off a surrogate to monitor the perceptual feed of a friend who attended the lecture. Based on the notes it came back with, she didn’t miss much.
Now she’s starting a 2000-word paper that’s due in about fifteen minutes. The topic is the use of free indirect discourse in Thomas Mann’s Death in Venice. She hasn’t gotten around to reading the book yet, but that’s no big deal. She retrieves some representative examples of free indirect speech from the text and computes approximate opinions about the style and content of the book. She looks over this information and develops a rough thesis. Little is found to support this thesis, so she adjusts it slightly. After a few more iterations, she converges to something she’s satisfied with. She then constructs 2000 words of supporting arguments, fully cited and written in her own style.
The paper mentions quite a few philosophical concepts and literary terms that her biological half has never heard of, but her brain trusts her software. Her homework is submitted with three minutes to spare.
Three or four decades from now, Jacob is an academic who specializes in foreign policy. He’s just come back from a day-long “culture sabbatical”: every few minutes, he loaded up a different personality file in order to experience information from a new cultural perspective. It’s something he does every so often to keep his belief system from growing stale. Meanwhile, the twelve surrogates he’s currently running didn’t get to enjoy the vacation – as always, they were busy churning out essays and giving lectures. It’s publish or perish, after all, and like most thinkers of his time, Jacob produces several dozen publications per day.
He pulls all of his surrogates back for reintegration. After taking a moment to relish the undivided bandwidth, he starts talking to himself to verbally explain the ways in which his sabbatical has shifted his beliefs. He does this in order to train the software portion of his mind to see the world the same way that his biological mind now does. He then spins his surrogates off again and gets back to work, confident that the arguments put forth in his essays will now reflect his new worldview.
INCIDENTAL UPLOAD AND BELIEF PILOTING
What happens when a quiet AI becomes such a faithful user surrogate that the way it perceives and produces information is indistinguishable from the way its user behaves? Maybe the user has accomplished something akin to uploading his biological mind to an artificial medium. It’s an incidental upload: in the pursuit of intelligence amplification, we wind up offloading so much cognition to software that the software alone becomes capable of what looks like cognition.
Consider Jacob, the foreign policy expert. His quiet AI is so advanced and tuned to his psyche that it can produce “Jacobesque” cognitive behavior without any direct human input. This allows him to spin off several copies of his AI to work on cognitive tasks in parallel while he devotes himself to honing what he believes. He’s the belief pilot of his flock of artificial minds, turning his life experiences into a unique belief system that influences the behavior and information handling of his software, and providing minor course corrections when necessary. This may wind up being the most important vehicle for individuality in the information culture of the future.
I’ve concentrated so far on the interaction between mind and software, but there’s another important player on the cybernetic scene: the network. We use quiet AI to mine and manipulate information, but where does that information come from? The information retrieved by a search engine doesn’t live in an encyclopedia. It’s the information that we spew out in our everyday lives, through news articles, blogs, and commercial enterprises. Quiet AI is getting smarter largely because a growing subset of human behavior is reflected in digital information. That information is useful to us even if it wasn’t originally created to be searched by others.
Our developing symbiosis with technology will only accelerate the reflection of our behavior as publicly accessible information. (This, unlike AI monitoring, is a privacy concern, but it’s privacy that we’re already giving up voluntarily, and there’s no sign that we’re going to stop.) As we increasingly use our software to help us think, an increasing amount of our cognition is likely to appear online. This information will, in turn, feed back into the intelligent software of every other individual in our society. The amplification of our intelligence will result from the manipulation of valuable information that exists in the world, and the value of this information will hinge on our intelligence.
There’s nothing fundamentally new about this trend; our civilization is built on a cultural feedback loop that operates through the cyclical dissemination, learning, and production of knowledge, and this cycle has existed since we became able to communicate with each other. But for the first time, technology will allow the speed of the information cycle to match the speed of cognition.
You can see where this leads. As a feedback loop, it sets in motion an extremely rapid intelligence takeoff. In a broader sense, it effectively gives birth to a new structure in the human mind: an internalized copy of the information society itself.
The modern ease of information access isn’t just another nifty convenience, or the result of some single gadget. It represents the beginning of the most dramatic shift in human consciousness since the invention of writing: a smooth, quiet ramp-up to posthuman cognition through an utter detonation of the divisions between mind, software, networks, and society. The particulars of our cybernetic revolution are hard to predict with certainty, but we can say this for sure: the future will be a strange place indeed.