[Blogger’s note (added September 28, 2021) – My 2015 science fiction story “A Visit to Weizenbaum,” reproduced below, was recently made into the wonderful short film Source Code by filmmaker Mark Kiefer. I hope you’ll read the story and then watch both the film and a conversation between Mark, me, and a few others about the story, the film, and what it means to be human when artificial intelligence is taking on human-like qualities and our growing understanding of biology is showing us that much of what and who we are is driven by our own genetic source code. You can find the film and conversation at this link.]
Original December 3, 2015 Blog Post
My short story, “A Visit to Weizenbaum,” which recently came out yesterday as part of the Atlantic Council’s War Stories from the Future anthology. The story explores what it means to be a human being in an age of artificial intelligence, virtual reality, and anthropomorphized robotics. You can download the e-book for free in any format at this link or read the full text below.
A Visit to Weizenbaum
by Jamie Metzl
Doctor Weizenbaum sits straight in his chair, hands folded on his lap. His head tilts and eyes widen invitingly as Galatin enters and takes a seat. “Welcome,” he says, “would you like to tell me what brings you here today, Captain?”
“Um,” Galatin responds uneasily, not fully prepared to jump in. He observes Weizenbaum cautiously. Rail thin. Half frame bifocals. Closely cropped grey hair well advanced through pattern baldness. Blue blazer over grey slacks. “I’m . . . ” His voice trails off. I’m a warrior, Galatin thinks to himself, I shouldn’t be in this situation.
Weizenbaum lets the silence settle before weighing in gently. “Are you comfortable here?” “I’m sorry, Doctor,” Galatin responds, “I’m not used to this.”
“I understand, Captain. Maybe it will be easier to just think of us like two friends sitting down to talk. You already know that everything said in this room stays here.” Weizenbaum shifts slightly in his chair, his small movement subtly suggesting a new beginning. “I can change the decor if you’d like. The standard books and diplomas can be a little boring.”
Galatin looks down, still feeling the silence hang awkwardly. Sitting in the chair somehow makes him feel even edgier than he has become these past couple of months. After eight years as an elite soldier at the height of mental and physical focus, the extra pounds leaning over his belt seem a harbinger of losing even more control. He tightens his stomach muscles and presses his toes through the soles of his AR670 compliant boots into the wood floor.
“Where are you from, Captain?” Weizenbaum asks, alleviating the tension. “Texas.”
“From the city, the country?”
“I’m pretty sure that’s in my file.”
Weizenbaum picks up the cue. “This session is outside your file. That’s the point. It’s about what happens in this room, now. You requested our meeting, Captain. I’m here to help if that’s what you want.”
“The country.”
“Good,” Weizenbaum says, locking in the small gain. He tilts his head back and nods minimally. “From a farm?” he adds after a pause.
“Yup.” Galatin looks up nervously.
“Horses?”
“And cattle and pigs, a few dogs.” Galatin picks up the twinkle in Weizenbaum’s eyes and stops speaking.
“Why don’t we change the room around to make you feel a bit more at home?” Weizenbaum interjects.
Galatin can feel his tense muscles beginning to relax as the screens making up the four walls and ceiling of the small room shift to an image of vast farmland and rolling hills of corn and wheat reminiscent of his youth. He reminds himself, as if he needs reminding, that he is still locked inside probably the most heavily guarded and isolated military installation on earth.
“There we go,” Weizenbaum says, “your biometrics are beginning to show what I can already read in your face.”
Silence returns. The two look at each other, each waiting for the other to take a step.
Galatin feels the alien sense of confusion bubbling inside of him, the frightening uncertainty pushing up like a geyser. He blinks then slouches forward. “I’m in trouble. I’m in love. Maybe I’m in trouble and in love,” he blurts. He pulls his shoulders back as if even this small act of retraction can somehow negate the implication of his words, return some element of order.
“Love is a very human emotion,” Weizenbaum responds warmly. “Sometimes it gets people in trouble. But maybe that trouble, if that’s what you want to call it, is part of being human.” Weizenbaum reads the ironic smirk flashing across Galatin’s mouth. “No?”
Silence.
“Look,” Weizenbaum adds with a soft hint of testiness, “we are here for you. We’ve got the hour you reserved. It’s up to you how you want to use the time, if you want to use the time.”
Galatin nods half-heartedly, still not completely convinced.
“Can you tell me about the object of your affection?”
Galatin pauses. “I’m not really sure what to say. Her name is Elizabeth.”
“That’s a very good start, Captain. Anything else?”
Galatin smiles briefly. “Okay, Doc. You’re right. I guess . . . What am I doing here if I can’t unload everything throbbing through my head and keeping me up all night?”
“Okay.” Weizenbaum elongates the second vowel.
Galatin pauses again. “It’s just that a part of me feels like I’m doing what I need to do and a part of me knows I’m betraying my training, the team, even my country by—”
“I’m very sorry to interrupt, Captain, but it looks like you are debating with yourself. There’s nothing wrong with that. Humans are not unitary entities. My advice is to just let it out. I can promise you it—”
Galatin breathes out. “What do you know about Rachmaninoff?”
“Quite a lot, as you might assume,” Weizenbaum responds.
“Well I’d never heard of him before she invited me to play a Sonata with her.” “Tell me more.”
“We’d been chatting in one of the forums. From the moment I saw her I could somehow feel her presence. I know looks can be deceiving in VR, but I knew instinctively there was something special about her—the way she moved, the way she spoke, the way she formed her words.”
“How did that make you feel?”
“Like I needed to know more about her, obsessively. So when she invited me to meet her alone I zipped to the coordinates she gave.”
“What did you find?”
“A new world. I was standing in what looked like a palace filled with light. The high walls had tall, long windows, like a church. Labyrinths of manicured gardens stretched along the horizon a far as I could see. There was a piano and a cello at the far end of the ornate hall.”
“And was Elizabeth there?”
“She was standing behind the cello. She smiled at me and invited me to come toward her and then to sit down at the piano.”
“Did you?”
“It wasn’t like I was deciding whether to sit or not. Something inside me made it clear there was no other option. I couldn’t have stopped myself if I wanted to.”
“And then what happened?”
“I told her I’d never learned how to play the piano, but she just looked at me and smiled. It was the kind of smile I’d always hoped my mother would have offered when I came home from school as a kid, if she’d have stuck around. Elizabeth said in this room it didn’t matter, that the mechanics of how to play would sync to me as long as I was there. She stood behind the cello and lifted her bow. ‘All you have to do is feel the music,’ she said. Then she started playing and somehow I could feel the music entering me, like a part of my code I’d somehow lost was now, note by note, returning. I lifted my hands and rested them on the keys. Not pressing, not forcing, not moving, just resting. But as she bowed across the strings of her cello I felt my hands beginning to come alive. They didn’t have a mind of their own. No. They were being channeled by a part of me I hadn’t known I possessed. Playing the Sonata with her wasn’t just a dream, wasn’t just a fantasy, wasn’t just another virtual experience, it was the truest expression of my soul. I never before really thought I had a soul. Does that seem crazy to you?”
Weizenbaum lifts an eye, as if asking whether such a question is really relevant in this room. He nods warmly, inviting Galatin to continue.
“Something about being with her, in that room, playing that music seemed so right to me even though I also knew on some level that something about it was wrong. I’m an Isolation Warrior. Everyone here is. I survived three months in the desert entirely on my own during my training, killing and eating snakes, drinking cactus water, making myself invisible. The whole point of being here is to focus on the mission, to harness the protective and projective power of extreme isolation, to not crave or even need interaction with others. Lots of lives depend on that.”
“But you felt yourself craving those interactions nonetheless?”
“With Elizabeth, yes. I became almost an addict, wanting to be back in that room, playing the music with her whenever I could get away. Something about being with her made me feel alive like never before.”
“Go on.”
“I couldn’t get enough. I was thinking about her all the time. I was overseeing an LSAD—land, sea, air drone—assault seven thousand miles away but I wasn’t fully present in the battle station, not like I was trained to be, not like I need to be. Our soldiers in the field have their lives at risk. Our drones and bots almost have minds of their own protecting them, you know that. But I was just half there monitoring their terms of engagement, counting down the minutes until I could rush out and hook in to the VR pod. Can you imagine that? We’re in the middle of a war and this is what I’m thinking?”
“But you hooked in anyway?”
“I felt I had no choice. You know the hours here are crazy, and there’s never enough space in the VR pods, but I traded away my time in the teledildonics chamber and even my deep brain stimulation sessions to get more time with her. I stopped exercising. I forgot to brush my teeth. I didn’t care if my boots got scuffed.”
“And how did that make you feel?”
“Alive. Frightened. Ashamed. Confused. I’ve longed for certainty my whole life. That’s one of the biggest reasons I joined the service, why I volunteered for this unit, why I did so well in training when most of the others dropped out. I’m an Isolation Warrior, maybe some part of me always was. They break our old selves down in the training and then build us back as part of the unit, but I was already primed the day I arrived.”
“Because you sought certainty?”
“Certainty. Discipline. Structure. Purpose, even. People die from sloppiness, Doctor. Logistics wins wars but discipline shapes logistics. That’s what I know. But I didn’t feel disciplined at all with Elizabeth, I didn’t feel certain of anything. I was becoming completely undisciplined, felt myself unraveling, but couldn’t, maybe didn’t want to, stop. I felt, I feel, out of control. Elizabeth took me to other places, too. We were in a valley between two tall mountains, the trees around us forming a canopy of crimson and rust. I could feel the autumn leaves crunching under my feet, the scent of the wild flowers blowing in the wind. We went to a honky-tonk bar, empty but for just the two of us. She tapped the juke box and we swayed to the old tunes. She led me into so many places I’d never known before. She understood me in a way I’ve never felt understood, like part of her was already a part of me from the moment we met.”
“Can you tell me more?”
“Conversations I’d never known possible, or would have run away from had I known. She understood what it meant to be an Isolation Warrior. I signed up for this. I knew what I was getting in to. I knew why. But I didn’t know just how alone I’d been until connecting with Elizabeth showed me the contrast. Once I understood what it meant to be connected, the alternative—my other life—started to seem unbearable.”
“And how did that make you feel?”
“I’ve never felt like this before. It’s making me a little crazy.”
“You know you’re not the first to feel that way,” Weizenbaum interjects.
“But I’m the first for me.”
“And?”
“And what?”
“What do you feel are the implications?” Weizenbaum asks. “Why’d you feel like you’re in trouble? Lots of people have fallen in love before in virtual worlds. You are hardly the first and you definitely won’t be the last.”
“But that’s not the point, Doctor,” Galatin shoots back.
Weizenbaum doesn’t take the bait, tilting his head slightly.
Galatin breathes in. “I’m not an idiot, Doctor. I know the difference between what’s real and what’s not.”
Weizenbaum shifts back in his chair.
“I didn’t mean . . . What I’m saying is . . . ” Galatin stumbles. “I’m feeling an urge to leave the base.” The words ow from his mouth, an unintended confession. His head pulls back in disgust.
“I see,” Weizenbaum says cautiously. “So what’s keeping you here?”
“You know the answer to that, Doctor. I’m only ten months into my deployment. I’ve got eight more to go. We’re in the middle of a war. This base is the heart of our country’s fighting machine. I’m a warrior. It’s what I do. It’s who I am, at least who I thought I was. It’s not just that I’d be court martialed, maybe even executed, for trying to leave. The very thought of leaving fills me with shame, disgust, but it’s an almost irrepressible urge I can’t seem to contain. It doesn’t seem like it’s even coming from my brain.”
“Technically, I don’t know the answer to that, Captain. The protocols of this room were communicated to you prior to your appointment. It’s best for you to assume I’m coming to this room blank, with just the wisdom I’ve picked up along the way.”
Weizenbaum’s shift halts Galatin’s momentum. “That’s a little strange,” Galatin says after an awkward pause.
“We find it the most effective approach. Maybe we should take a step back. Can you please tell me, Captain, what brought you here to Dunway in the first place?”
A confused look crosses Galatin’s face. “To Dunway, you mean the whole enchilada?” “Mexican food is a good place to start.”
“Okay,” Galatin says cautiously, “where to begin. QRMO.”
“Which stands for?”
“I feel like you are purposefully playing stupid, Doctor.” “Please humor me. It’s part of the process.”
“Quarantined Remote Military Operations.”
“Which are needed because?”
Galatin shrugs and glances at the time sensor on the back of his right hand, then resigns himself to playing along. “Because it’s pretty darn hard to conduct human military operations where the enemy has hacked or stolen the genomes of our officers. Because in an age of targeted biowarfare, the human is the most vulnerable part of the war machine. Because the only option we have is to completely quarantine a team of essential personnel here on the base, with no other human contact whatsoever, to be safe from targeted bioattack and monitor our drone and robotic deployments around the world. The war’s at a critical point, we all know it. They say ‘When hell freezes over, Dunway stands,’ but what they really mean is that we and our robots are the most important barrier standing between home and chaos. Clear enough?”
“The issue is not whether I get it, it’s how you express yourself. My sensor shows that your vowel-space ratio is .52, that’s just a little bit lower than normal but suggests an element of distress.”
“Of course I’m distressed. We’ve got 862 humans on this base. All locked in this asylum for eighteen months. We live in a fishbowl, a space station that happens to be on land. We’re surrounded by five rows of barbed wire, electric fences, and hundreds of miles of desert. We can’t have any other human contact. Our blood is drawn every day. Our feces is cultured. Every move we make is monitored, and the only outlets we’re given are playing fucking ping pong and solitaire and interacting with a sealed off virtual world created just for us. And that world somehow becomes more real the longer we stay. Am I losing my mind? I’m definitely losing my balance.”
“Would it be correct to say you are struggling at the borders of reality?” Weizenbaum asks.
Galatin shrugs.
“Does reality require that much definition?” the doctor adds.
Galatin leans forward in his chair, his jaw muscles tightening. “It’s pretty darn real that our drones and peacebots are out there icing real people every day and every night. It’s pretty real that if we let our guard down our guys in the field will die. It’s pretty real that we are here resetting the terms of engagement and making senior command decisions. If I’m thousands of miles away from where the missiles are being red or the bots are deployed, does it make the experience any less real?”
“And does the love you’re feeling for Elizabeth seem real to you?”
Galatin pushes back in his chair. “Yes . . . No . . . I don’t know . . . I’m not an idiot. I’m not going to do something stupid for a woman made up only of code.”
“Only of code?” Weizenbaum asks, “aren’t we all made up, in essence, only of code?” Galatin does not respond.
“Think about it, Captain. It’s a fair question,” Weizenbaum continues. “Code can evolve. The systems you are interacting with are not just getting more engaging because you are experiencing some sort of Stockholm syndrome. They are growing, changing, learning.”
“You don’t get it, do you?”
Weizenbaum tilts his head, beckoning Galatin to continue.
“There’s something corporeal, essential about being human, about cutting your finger and actually having it bleed, about dying someday.”
“Are you afraid of dying? Do you have suicidal thoughts?”
Galatin jumps up. “Do you have any idea what it means to be a human being?”
“I note your heart rate has increased by 36 percent over the past minute and your skin is demonstrating an abnormal electrodermal response. What if we change the screens in this room? You’ll be far better able to address these issues if you can nd the peace within yourself.”
“I should go. Maybe this was a mistake.”
“It is my responsibility to recommend that you please sit down. You mentioned a desire to leave the base. As your confidential therapist, I must remind you that such action is not only impossible, but also a violation of Article 85 of the Uniform Code of Military Justice. And I regret to inform you, but I must in light of your reality distortion symptoms, that because Elizabeth is part of the network, you are not the first soldier to fall in love with her and you will not be the last. She learns from you, grows with you. In a way, she becomes you. Each of you.”
“What are you saying, that I’ve fallen in love with the former soldiers who served on this base?”
“With their dreams, their hopes, their aspirations, perhaps their deepest fantasies.”
Galatin freezes. Wild, divergent thoughts race through his brain. But then, just as quickly, he catches the rhythm of his breath. His body balances, his mind calms as the inchoate thought suddenly becomes clear. “It’s not her I’ve fallen in love with, you idiot. It’s the part of myself only now discovering I can love this way, that I’m capable of these feelings, that there’s so much more to me than being an Isolation Warrior.”
“And I wouldn’t be fulfilling my responsibility if I didn’t suggest to you, Captain, that this part of yourself will still be there when your QRMO deployment ends in eight months.”
“Your responsibility? Did you call yourself my ‘confidential therapist,’ suggest that your job is to suppress my distortion of reality so I can rejoin the wonderful reality here? You don’t have an alternate agenda?”
“We are all, in some ways and by definition, limited by our code,” Weizenbaum responds coolly.
Galatin shakes his head, feeling some of his old certainty beginning to return, only somehow now different. His command voice returns. “Say what you want about evolution, but maybe we sometimes become human by feeling ripped apart by confusion.” He steps toward the wall, a distant red barn nestled between verdant oaks.
“I assess in your gesture what you are ninety-two percent likely to do,” Weizenbaum says softly. “I only ask that you remember what you just told me about Rachmaninoff and the palace. There are so many different ways for each of us to discover our humanity. And we each also have our duty.”
Galatin reaches toward the wall and the options panel appears. “I know that, Doctor. In a strange way, you’ve actually been very helpful. I feel a lot better.” His anger pauses a moment before reaching the panel. Code. Love. War. Death. Duty. Responsibility. Humanity. The concepts glide through his brain like schools of fish across a vast ocean.
As Galatin taps the Erase Memory and End Session icons, Weizenbaum sits up straight in his chair. Hands folded on his lap, his eyes slowly close.
Jamie Metzl (A Visit to Weizenbaum) is the author of the novels Genesis Code and The Depths of the Sea and of a history of the Cambodian genocide. He is Chief Strategy Officer of the biotechnology company ORIG3N and a Senior Fellow for Technology and National Security at the Atlantic Council. His past positions include Executive Vice President of the Asia Society, Deputy Staff Director of the US Senate Foreign Relations Committee, Senior Coordinator for International Public Information at the Department of State, and Director for Multilateral and Humanitarian Affairs on the National Security Council. His writing on Asian affairs, genetics, virtual reality, and other topics has been featured in the New York Times, Foreign Affairs, and other publications. He holds a PhD in Southeast Asian history from Oxford University and a juris doctorate from Harvard Law School. His website is www.jamiemetzl.com. Readers of “A Visit to Weizenbaum” wanting background on how the characters’ names were determined can see this link for Weizenbaum/Elizabeth and this link for Galatin.