In case you’re new to Medium Cool, BGinCHI is here once a week to offer a thread on culture, mainly film & books, with some TV thrown in.
In this week’s Medium Cool, let’s talk about what it means to be human.
MattF (thank you!) recommended Martha Wells’ “Murderbot” series in a thread, and I’ve been blazing through them for the past few weeks. I really needed something fun, engaging, and with a captivating voice, and these novellas have it.
At their heart is the question of what it means to be human. As we increasingly move toward an AI and robotics-dominated world, this question is going to become more than just a fictional curiosity.
Where have you read or seen the most interesting accounts of the intersection of AI/robotics/human being/consciousness? It could be fiction, or film, but also science and engineering, or anywhere the subject comes up.
JanieM
The first thing that comes to mind is Dan Simmons’s Hyperion series.
ETA: Resisting any elaboration lest I generate spoilers….but there are several kinds of human+X hybrid creatures in the 4-volume story, and one of my favorite moments in all the sci-fi I’ve read, which admittedly isn’t tons
ETA2: And human+X hybrid doesn’t really capture the flavor, either. I’d better stop now… ;-)
Craig
John Scalzi is really good with this. Old Man’s War, and Androids Dream deal with what does it means to be human in different ways.
Ken
Some of these might not quite be what you asked for, but all have made me think about what it meant to be human.
The golems in Terry Pratchett’s Feet of Clay.
The angel in Donald Westlake’s Humans.
The posthuman androids in Charles Stross’s Saturn’s Children and Neptune’s Brood.
The software entities in Greg Egan’s Permutation City and Diaspora.
Frankensteinbeck
This is a personal obsession and a major thread in almost my entire bibliography. Wild Children, a book about being cursed to lose your humanity. Sweet Dreams Are Made of Teeth and A Rag Doll’s Guide To Here And There, both about how human an extremely not-human protagonist can be. No spoilers, but anyone who’s read the Supervillain books knows how important questions of AI humanity become. You Can Be A Cyborg When You’re Older, filled with insane AIs who are much more than just murderbots when you see them up close. Quite Contrary and I Did Not Give That Spider Superhuman Intelligence are not about that ‘what is human?’ question, but they sure dip into it in several places. This shit is my JAM, yo. Heck, one of my first published stories was a short story about steampunk airships falling in love.
BGinCHI
@JanieM:
Simmons has the honor of being the biggest asshole who’s written the most books I’ve read.
Baud
While it’s early, I’m a little surprised Blade Runner hasn’t been mentioned yet.
BGinCHI
@Craig:
Yes, agreed. And Jeff Vandermeer in a different way/vibe.
BGinCHI
@Frankensteinbeck:
Very cool. Glad this hit your sweet spot.
I’m glad it’s not just Data from STNG.
Mousebumples
I love Gattaca. Not really AI related, but it grapples with genetic engineering and “naturally” conceived people versus those that were engineered to have fewer flaws.
BGinCHI
@Baud:
That and Data (STNG) were, I thought, the most obvious and prominent, though I suppose even “Wizard of Oz” is in on the game.
schrodingers_cat
Star Trek, BSG the aughts version and last but not least, the Mahabharata.
JanieM
@BGinCHI: Personally, or in terms of his books?
I don’t know anything about him as a person, but I tried one of his other books and hated it, so didn’t read anything more. I did like the Hyperion series, though. Then again, it’s been a while since I read it, so I might look upon it with a different eye now.
A Ghost to Most
You wanna talk culture? How about the legendary performance of GG Allin at the Antenna Club in Memphis 30 years ago! Back before so many of my generation stopped living.
trollhattan
Am halfway through the final season of “Westworld” where they take the basic premise of “Blade Runner” and set the dial to 11. One is left pondering whether the human guests being freed to screw and drink and kill as much as they want will all gravitate to their basest instincts and desires. Jan 6 might have been an unfilmed episode from season 4. Very fine people.
debbie
Ishiguro’s Klara and the Sun is AI from the AI’s point of view. I’d be interested in hearing from someone who’s a fan of the genre and has read it.
dmsilev
A somewhat older story: Isaac Asimov, ‘The Bicentennial Man’
hitless
Ancillary Justice by Ann Leckie is spiritually similar to the murderbot series, although the voice and tone of the book is very different. Both excellent.
Redshift
Murderbot is soooo good. I ripped through the novellas and waited for the first novel. I’m holding off on buying the new one until I have time because I know I won’t be able to put it down
I agree its best aspect is using a not-entirely-human character to explore the meaning of humanity.
SFBayAreaGal
I, Robot
jeffreyw
Iain M Banks is all over this with his Culture novels.
BGinCHI
@JanieM:
I liked the first Hyperion book, then I thought they went downhill. His hard-boiled crime series set in Buffalo is terrific, as is his book The Terror.
But he’s become a libertarian crank. As long as I stay away from the actual human being, I can enjoy his writing. Not easy.
SFBayAreaGal
The robot from Lost in Space. That was one conflicted robot.
BGinCHI
@A Ghost to Most:
Sorry, coprophagia is down the hall.
BGinCHI
@debbie:
Haven’t read it yet! So many good books on my list right now, including Djinn Patrol on the Purple Line, by Deepa Anappara, which just won the Edgar.
MattF
FYI, there’s a Murderbot Twitter bot @Murderbotbot that, if you follow it, regularly emits quotes from the Murderbot books into your Twitter timeline. It’s a reminder, if you needed one, that Murderbot is spectacularly funny and smart. I think it’s great, but going off-topic all the time may not be for everyone.
In any event, Murderbot’s future is assured: Wells has written another Murderbot novella and has signed a contract for three more.
cope
How about Pinocchio?
trollhattan
@SFBayAreaGal:
I love that the same robot–Robby–kept being recycled from film to show to show. I wonder if he had arguments with his agent about being typecast?
BGinCHI
@MattF: She’s got such a great formula there, and the writing is super sharp. Similar to the Mandalorian, but even better.
BGinCHI
@cope:
That bugiardo?
MattF
@cope: ‘Pinocchio’ raises a good question— what about an AI who doesn’t aspire to be a ‘real boy’? Unlike, e.g., Data in Star Trek.
Starfish
@BGinCHI: My husband used to really like him too. He said that there is definitely an anti-Muslim sentiment in some of the Hyperion books. I think I read one because he liked them so much.
Gravenstone
Seems like an opportune subject to toss in an anime currently airing Vivy -Fluorite Eye’s Song
The premise is an AI is sent 100 years into the past to contact the first autonomous AI for her assistance in preventing a pending AI apocalypse in his era of origin. It’s based on a (not yet released) novel by the writers behind Re:Zero and Chaos;Child and it’s a trip which is getting darker by the episode (as might be expected from the author’s pedigrees). Unfortunately, it’s only available on Funimation in the US at present. Might wander over to Hulu after it finishes airing this season.
BGinCHI
@Starfish:
Fun Dan Simmons fact: he went to undergrad at Wabash College.
If you’re a Hoosier (current or recovering), you’ll get the ramifications.
Starfish
@Redshift: How many novellas were there? I read four. Am I caught up?
TomatoQueen
Asimov was a first class asshole, obvs, but the Three Laws as the basic framework stand up well. What it means to be human? Well now, Herbert spent quite a bit of substance on that question, yet another loonbucket project with unforgettable moments. And all the others which mostly look at the Three Laws and push against ’em. Nietzsche (or was it his sister…) went round and round and by the time he got anywhere, his own humanity was claimed by tertiary syphilis; he did know from Wagner, however. Strange intelligences: first one comes to mind is Moya. Next one is of course Mike, in Moon is a Harsh Mistress. Another would be any whale, most especially the less-studied ones, Right Whale, Minke, Finback, all older that we ever expected. So what do they know, and when did they know it?
Amir Khalid
@dmsilev:
Despite having Robin Williams in the title role, the movie left me vaguely disappointed.
JanieM
@Starfish: That’s interesting, because all I remember is a strong anti-Catholic strand (not that I objected to it). But as I said, it’s been a while, and I might not have been as sensitive to anti-Muslim content as I would be now.
Delk
Pretty much any William Gibson.
JanieM
Marge Piercy, He, She and It
BGinCHI
@Delk:
He’s kind of always writing about that subject, isn’t he?
Craig
Neuromancer. Where is the line that Molly would stop being human and become something else with all her implants and enhancements? The AIs seem to be becoming more and more human while the TAs are becoming more and more detached from humanity.
MattF
@Starfish: The first four are novellas, the fifth, ‘Network Effect’ is a novel, and the latest, ‘Fugitive Telemetry’ is novella-length. There’s also a short story from Mensah’s POV that you can find on the Tor website.
ETA: Also, I have no idea what the titles signify.
dnfree
@debbie: I’m a fan of SF, but more so I’m a fan of Kazuo Ishiguro and have read most of his work. Klara and the Sun is what I was going to recommend here. For someone who has read him, it’s in the same general ballpark with Remains of the Day (which by the way, the movie ruined, especially by changing the ending), with a little tinch of Never Let Me Go, but distinctly its own. He raises a lot of issues very subtly, as is his habit. There were two moments that made me gasp out loud, which seldom happens.
Coincidentally, the book I started after Klara is nonfiction, 21 Lessons for the 21st Century by Yuval Noah Harari. Right away it touches on one of the background issues in Klara, the loss of jobs caused by automation. I think I got this book because it was recommended by Bill Gates a year or so ago. He’s a busy man, besides the microchipping.
cope
@BGinCHI: You mean he didn’t want to be a real boy?
prostratedragon
@MattF: It is an interesting question. HAL?
Poe Larity
I think the intersection is closer not because AI is getting better, but because human intelligence is plummeting to a point where your washing machine is smarter than your neighbor.
Craig
@Poe Larity: relevant works being Mike Judge’s Idiocracy, and Beavis and Butthead.
Starfish
Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins by Gary Kasparov about Deep Blue and the various human v machine competitions was more interesting than it had any business being.
The AI Does Not Hate You: Superintelligence, Rationality and the Race to Save the World by Tom Chivers was deeply disappointing because it was more about The Rationalists and whether they are sex cult than about AI.
I am interested in suggestions on AI ethics and how racist training data leaves us with bad models.
Laura
I love that series I’ve read them all and have reread them several times too
Starfish
@MattF: Thanks. I put a library hold on The Network Effect.
Craig
Does Frankenstein fit here? Asking, because I’m realizing that I need to go and reread that.
MattF
@Starfish: It’s an established legend that neural nets trained on random internet data become nazi trolls– one may wonder if this is actually true, and if it is, why it’s true. Neural net training is, of course, opaque– you change some weights somewhere in a non-linear feedback loop and then watch what happens– but I don’t think that just saying ‘Well, we won’t do that again’ is a good enough response.
badgirl
I love Murderbot! And Simmons earlier work but he is as you say an asshole. Ann Leckie’s books are wonderful as well and there is so much tea!
Mike in NC
Last night we watched something called “Tenet” on HBO. Still have no idea what it was all about.
Ken
The first novel I remember reading that touched on that idea was Martin Caidin’s Cyborg (a.k.a. the source material for The Six Million Dollar Man). In the first part of the novel, Steve Austin hates what has been done to him. Near the end, he’s stranded in a desert with someone else, and walks out carrying that other person. On the way he thinks, as a human I could never have done this, we would both have died, but the machine just keeps going.
Starfish
@MattF: The training data for a lot of models is very specific.
When you are building stuff up in college, the cheapest resource that you have access to is other college students, so sometimes you are overfitting to a group of people that are not a good measure of people in general.
A lot of language training stuff was done on a Wall Street Journal corpus. Apparently, there is a corpus based off of TED Talks out there now too.
debbie
@dnfree:
Thanks. It’s not a genre I generally read, but I have been a rabid Ishiguro fan since forever (for me, it’s Artist of the Floating World and Buried Giant). I think he did a better job than other literary-type authors’ attempts I have read (McEwan, Chang Rae Lee), but not even Ishiguro could get me to want to read more dystopian books.
Brachiator
As a pre-teen, I dug into SF and soon focused on robot stories as some of my favorites in the genre (along with time travel stories). I find that some early excellent examples from TV and movies still resonate with me. Ultimately, all good SF, no matter how strange or exotic, is about the human condition.
One of the best is the Twilight Zone episode, “The Lonely,” written by Rod Serling and starring Jack Warden as a convicted murderer from 2046 (not too far in the future now!) who has been sentenced to solitary confinement on an asteroid. An Earth ship stops for short visits once every four years. On one trip they leave him an old car that he can restore. After all he’s got lots of time. One another trip they leave him with a mechanical companion played by Jean Marsh, a robot which can adapt to his personality and emotions. The convict initially rejects the device, but then…
Ultimately one of the themes of this gentle story is that we yearn to make connections with others, and without this we lose part of what makes us human.
And then there is the magnificent Outer Limits episode written by Harlan Ellison, “Demon With a Glass Hand.” It is about a mysterious stranger Trent, played by Robert Culp, who may be the key to how all of future humanity was able to hide from alien conquerors. But there is a touching parallel story about whether Trent and a young woman (played by Arlene Martel), who has become trapped with him can save each other as well as save humankind.
Parts of “Demon With a Glass Hand” was filmed at the glorious Bradbury Building in downtown Los Angeles, which is also the setting for parts of another great robot film, “Bladerunner.”
A more recent robot story which I think is great is the 2013 film “Her,” which is again about the human need for connection. Where this film excels is in the sly, but detailed way that it depicts the variety of ways that humans and AI can connect, but also in suggesting that truly sentient and intelligent AI might develop a sense of destiny entirely independent of what human beings might desire.
I don’t think that AI or robots will dominate humans anytime soon. But I think it is very interesting to see how we continue to incorporate AI into our lives. We want “human like” interactions with AI. Children are growing up with AI interfaces, and see it as a normal part of their world, so much so that some parents are having to teach their kids not to be “rude” to AI interfaces like Alexa or Siri.
I wonder if, instead of teddy bears and imaginary friends, kids will soon grow up with something that is not the AI “brother” of Spielberg’s AI. but an AI interface which can serve as a companion and friend, and adapt to the child as they grow and mature.
Aspects of this bring to mind another Twilight Zone episode, “I Sing the Body Electric,” written by Ray Bradbury,
This also brings to mine the line from Shakespeare’s “Tempest,”
We now have to include all the variations of AI and robots.
BGinCHI
@Poe Larity:
This is a good take.
Ken
@MattF: This reminds me of James P. Hogan’s The Two Faces of Tomorrow, which revolves around the training of AIs.
IIRC, the novel opens with a scene on the Moon, where some surveyors log a request that a ridge which blocks a proposed route be removed. The construction AI tells them it will take about ten minutes, and just as they realize that’s much too soon for the automated bulldozers to reach them, a few hundred tons of rock launched by the lunar mass drivers slams into the ridge. The most efficient solution, you see.
This isn’t the only reason I’m skeptical about driverless cars, but it’s one.
RSA
I’m thinking of work in computer science, including AI and machine learning, that aims toward a goal Douglas Engelbart (famous computing pioneer) called Augmenting Human Intellect. Along those lines is Andy Clark & David Chalmers’s 1998 article, “The extended mind”, in the philosophy of mind literature. An excerpt:
As Clark and Chalmers put it, “Where does the mind stop and the rest of the world begin?” The computational world has been changing quickly and drastically; it’s interesting to think about the implications for what we consider our minds.
Brachiator
@TomatoQueen:
I am not sure that these stand up at all, even though I know a lot of SF fans who strenuously believe that these fictional rules can be applied in the real world.
Quick example:
In the real world, we will have to program self-driving cars to decide whether to kill pedestrians or let the occupants of the car die in the case of some collisions.
There are subsidiary issues as to whether the owner of the vehicle or the auto manufacturer could ever be considered at fault in situations like this.
I can also see robots employed as personal security devices with specific instructions to kill an intruder in order to protect its owners.
WereBear
That you for bringing up a subject I wanted to broach: we have species on this planet who can teach us what it means to be human, and they are not of our creation like AI. They are very distant relatives who nonetheless can “speak” to us.
My most recent post on my cat blog touches on the decreasing inability of science to understand cats (the past few years has been a cascade of vindication) and yet there are prejudices visible.
One of them is thinking of language solely as a product of created sounds, and ignoring the cat’s ability to convey rather complicated information through expression and posture. Yet, they teach apes sign language.
I do communicate with my cats. And any of us with beloved pets know it’s possible.
I’d like to see us pay more attention to the alien species some of us live with, and who we should all be responsible for.
Yet Another Haldane
I love Murderbot, it’s thought-provoking without feeling like a punishment to read. Eagerly awaiting Fugitive Telemetry. Also, the title character has zero interest in becoming human. Rights as a person, heck yeah! Bodily functions and other indignities, heck no!
Also, yes, Leckie’s Ancillary trilogy, I’ve read them, um, twice? Three times? Plus the sorta-sequel Provenance, which is lighter and from a very different POV.
Iain M. Banks’s novel The Algebraist deals most directly with the personhood of AIs. It’s not a Culture novel, set in a society that prohibits AI very energetically.
The magnificent Ted Chiang has a couple of relevant stories in his collection Exhalation, especially “The Lifecycle of Software Objects” and the steampunk(!) “Dacey’s Patented Automatic Nanny.”
So little time, so many good stories!
Ken
Why not build the AI interface into a teddy bear? It could not only be a companion, but could teach the child, and even mold them morally as they grow. What could go wrong?
“I Always Do What Teddy Says” by Harry Harrison, of course. Kage Baker also touched on this in her Company novels, when young Alec Checkerfield subverts the AI that was supposed to provide moral instruction.
Brachiator
@MattF:
The show The Orville features an AI character which is happily non-human. Unfortunately, his fellow robots think that humanity and other non-mechanical life are an existential threat that must be wiped out.
Yet Another Haldane
Oooh, and Naomi Kritzer’s story “Cat Pictures, Please” and novel Catfishing on Catnet are loads of YA fun. The sequel novel came out recently (last week?).
And Marvin in the Hitchhiker’s Guide stories would really rather not be human, I think. A bit like Murderbot that way (but not as heavily armed).
RSA
@Brachiator:
I think your view is the consensus in the robotics literature as well, though some notable efforts have been made. Here’s Ron Arkin, 2008, in “Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture”:
WereBear
More than Human, Theodore Sturgeon, explores PSI expressing in different people, and how they can act as a being and a group.
Cordwainer Smith built an alternate world of stories involving human/animal hybrids. Probably the most famous was The Ballad of Lost C’Mell. But my all time favorite of his work (and I adore it) is how astronauts cope with the loneliness of space by allowing themselves limited access to their own sensory system, Scanners Live in Vain.
BGinCHI
@Brachiator:
The Tempest raises the giant question, which is also a very old one, of “who gets to be considered human?” This has taken many forms historically, and Shakespeare’s interest there in how New World exploration was dealing (badly) with natives is a provocative moment.
BGinCHI
@Ken:
Harry Harrison one my earliest favorites.
Brachiator
@Starfish:
There are examples of this all over the place and it is a serious area of study. One of the problems seems to be that a degree of racism and sexism is just built into the culture, and it’s not just a matter of bad training data. AI “learn” to be racist and sexist even when they extrapolate from the material they have been fed, because of the biases of their designers, and because of the biased nature often built into the material itself.
billcinsd
I think I’ll add John Sladek’s Tik Tok
Starfish
@Brachiator: Has anyone done a book on this topic?
Seeing the comments section made me reflect on what I have read in the past year or two. I really like science fiction; but as I have intentionally tried to read more diverse authors, I have been reading less science fiction.
RSA
@Starfish:
Two good overviews are Moral Machines: Teaching Right from Wrong, by Wendell Wallach and Colin Allen, and the collection Robot Ethics: The Ethical and Social Implications of Robotics, edited by Patrick Lin, Keith Abney, and George Bekey.
A woman from anywhere (formerly Mohagan)
Haven’t read all the comments so this may be a repeat, but Battlestar Galactica (the one with Edward James Olmos etc.) really went into the subject in-depth, as it did with many issues.
Ken
@billcinsd: Ian Tregillis has a trilogy (The Mechanical is the first) with somewhat similar clockwork robots, operating under geases. One of their constraints is that if they perceive another mechanical acting in a way that indicates a geas has failed, they must pursue, capture, and destroy the defective one.
SFBayAreaGal
@Brachiator: I loved that episode from the Twilight Zone that Jack Webb was in.
The Outer Limits was also a favorite series. I need to track that episode down with Robert Culp.
Ken
By the way, Randall Munroe has several comics on robots and ethics. I like his analysis of the Three Laws, and not just because it introduced the phrase “killbot hellscape” to my vocabulary.
Starfish
@RSA: Thanks.
Anotherlurker
Does anyone have a good take on what that RWNJ, Larry Nivin, tried to do with the concept of humaity in his “Ringworld” series?
Brachiator
@BGinCHI:
In Shakespeare’s time, New World exploration was still new. There was writing about natives as exotics, as cannibals, or as innocents in a New Eden, but I don’t think that knowledge of, or concerns about, exploitation were as yet widely considered.
But I note that there is a good deal of Shakespearean criticism that gets into neo-colonial aspects of “The Tempest.”
There are tech people and SF fans I know who object to the idea of sentient AI, even fictional Data. They would see robots as the equivalent of a can opener, a device that humans created to do something for a human.
Every now and then, I would cause great upset among some of these people when I suggested that they were ignoring the elephant in the room, that what some of them clearly wanted was a perfect slave that could be exploited without consideration because it was programmed to serve and presumably had no feelings or rights.
In some ways I think that the creation of perfectly humanoid looking robots with human-like reactions could be a problem because they could encourage some people to abuse them.
Some of this is also one of the themes of the Star Trek: Next Generation episode, “The Measure of a Man,” where Data is put on trial to determine whether he is a being whose self-determination must be respected, or simply the property of Star Fleet, and which could be disassembled by Federation scientists on a whim.
Ken
@Anotherlurker: Not terribly much. Niven’s main theme was that humanity is not native to Earth; we are a (failed) colony of Pak. The Pak also built the Ringworld. Humans, and all the humanoids on the Ringworld, are evolved from Pak breeders (Australopithecus, IIRC).
Brachiator
@Starfish:
This is becoming a big deal, but I don’t know what the best books are. This google search comes up with a lot of interesting citations: “bias and racism in Artificial intelligence”
There was an interesting documentary that dealt with this called “Coded Bias.” Some background here.
A 2017 story from The Guardian had a lot of interesting references.
piratedan
tried to wait before I weighed in…
David Weber – Safehold series, his main character is a human consciousness embedded into a human/robotic hybrid as Mankind has had to flee an alien invasion (complete with corrupt psychologists who embed in their religion a dogma against scientific advance)
Becky Chambers – Wafarer series, but the second book in the cycle (A Closed and Common Orbit) that deals with the identity of a sentient AI and the development of their personalities as it treats their exploration of knowledge with programmed guidelines and the idea of a discussion of “like” and decisions.
Lois McMaster Bujold – If the question is about what can you do to the human genome and remain human, Lois has some thoughts.
Tim Pratt – The Stars series, premise is that space travel of an interstellar kind can only be managed by an AI based on human consciousness in regards to the AI is the ship itself (also explored by Anne McCaffrey I believe)
Sharon Lee/Steve Martin – Liaden series where we have sentient AI’s seeking recognition as entities in their own right and in other instances, also brought into fully expressions as being the house computer via service bots and home protection systems, personal secretaries and the like.
Uncle Omar
@billcinsd: So, it’s about Dick Cheney’s life?
This is going back into the memory banks, but in 1963 or 64 or so, there was a short story in Playboy (no, really I read it for the stories and interviews) in which the writer imagined the internet and eventually the computers, after total link-up, decided to dispose of people as being useless. That was when a computer with a tenth the capacity of your smartphone needed a 1500 square foot room with total climate control in order to operate correctly. The Lesson? Keep your eyes on the sneaky bastards and turn them off every so often.
Bill Arnold
Blindsight, Peter Watts, 2006 is excellent. The characters are early posthuman specialists, mostly (including a super-smart vampire[1]) The lead character specializes in translation between specialists in different specialties (including leadership), and is probably the most understandable – “I’m as much of a specialist as you. I specialize in processing information topologies”
Echopraxia by the same author is interesting so far. Haven’t finished it yet.
Karl Schroeder’s “Lady of Mazes” (2006) remains interesting. Various subpopulations live in tech-enforced consensus realities, and the most hard core posthumans live in the the Archipelago. We complain about filter bubbles; in the Archipelago, the entities are almost all living in (possibly/partly) opaque individualized consistent (somehow:) narratives technologically optimized for their emotional well-being. (Or optimized for something that differs for different individuals.)
I’ll assume people have read all the Iain M. Banks culture novels. :-)
[1] Blindsight: Notes and References (Peter Watts) includes “A Brief Primer on Vampire Biology”
Yutsano
Sigh.
No love for Marvin the Paranoid Android?
Plus the effects of the Butlerian Jihad run all through the Dune novels. The tension between humans and their studious avoidance of any advanced AI actually winds up stunting human growth.
AWOL
Well, the obviously answer is “The Forever Watch,” which was written by David Ramirez and produced by moi.
hueyplong
“My instructor was Mr. Langley, and he taught me to sing a song. If you’d like to hear it I can sing it for you.”
Mary G
Fifth or sixth concurrence on Ann Leckie’s Ancillary series. Extra credit for the fact that the first novel, starring a ship AI, can’t distinguish human sexes and calls every human she/her.
This so outraged some SciFi fans that they started the Sad Puppies, devoted to Making SciFi White Male Again. They succeeded in the short term, sabotaging a couple of years of Hugo Awards with ballot stuffing, but in the long run probably got a lot of good writers more attention and praise than they might have achieved buried in the masses published.
The Hugo Award was awarded to N.K. Jemison, a Black woman, three years straight for her Broken Earth series. I recently finished her “The City We Became,” in which each borough of New York is represented by a spirit reflecting its essence. Only Staten Island’s is White.
Brachiator
@Yutsano:
Only read the first Dune novel. I did not know how popular or influential the novel was. I picked it up in a convenience store market that sold a scattering of used books. It was a lucky find. I really loved it.
I am cautiously waiting for the new movie based on the novel.
Benw
I’d like to thank folks for all the awesome recs!
I recommend Her, no spoilers but the movie does have an AI that is not interested in being “more human”.
Also, if you don’t mind a lot of testosterone, Richard Morgan’s Altered Carbon trilogy is pulpy fun that blurs the lines between human and machine. The Big Sleep of the future.
UttBugly
“Demon with a Glass Hand” from The Outer Limits. I believe the lady trapped with him was Spock’s wife.
Brachiator
@Starfish:
Another good article on bias in data from MIT Technology Review.
Anotherlurker
@Ken: It has been so long since I read “Ringworld”, I forgot Nivin’s cynical take on humanity. I found the series fun to read and I guess I didn’t take away anything deep.
I’m able to have fun with a story or a novel or a film because I can fully suspend my disbelief. The only author that gaged me from word one was Ayn Rand. I think I was able to read 10 pages of “Atlas Shrugged” before I decided that her drivel was not worth my time.
Thanks for your take . I will read “Ringworld” again.
HinTN
@BGinCHI:
Same with Heinlein. Funny how that works.
BGinCHI
@HinTN:
Narcissism?
Bill Arnold
@Brachiator:
Magic of the marketplace! There are potentially 10s of milliseconds available to make the decision. The AI/automation for the potentially-involved insurance companies can negotiate an outcome. (The negotiation exchange would need to be sited within a few milliseconds of the dispute.) The very rich would be self-insured and have high speed AI lawyers negotiating for them.
BGinCHI
@Benw:
The Altered Carbon TV series was not bad either. I enjoyed the books.
BGinCHI
@AWOL:
This looks fabulous. Will give it a read.
Brachiator
@Bill Arnold:
we will have to program self-driving cars to decide whether to kill pedestrians or let the occupants of the car die in the case of some collisions
Being very rich doesn’t really solve anything.
Let’s presume that most people buying a self-driving car that includes the option “in a collision, try to save the vehicle occupants no matter what.”
Lawsuits would still be flying. The very rich might have some protection, but the potential liability might still be out of this world.
And the larger question still might be, if you can build some accident avoidance measures into self-driving cars, what are the ethical considerations to be followed? Can ethical considerations override what the vehicle owner might want.
ETA: The Altruism Option Pack might be, “if there is a potential collision and there is only one vehicle occupant and a crowd of people, the vehicle should take reasonable steps to avoid the crowd even if it results in injury or death to the vehicle occupant.”
Ken
@Brachiator: I think we should come up with a solution to the Trolley Problem that all humans can agree on, before trying to program a car to implement it.
Bill Arnold
@Brachiator:
I was joking about a libertarian dystopia that we should try to head off, but yeah.
The lawsuits will be a mess, particularly while there is a mix of human-driven and self-driving cars, including with hybrids (assists of various sorts) . The exchange idea/joke was to offload some of the potential liability onto the vehicle owner (or their agent(s)) rather than the manufacturer. It depends on how much responsibility is placed on the manufacturer in the legal code, in various jurisdictions and various times. The lobbying will be intense. (Probably already is; haven’t looked.)
Professor Bigfoot
Reading Murderbot led me to this:
”Every sophont has the inherent right to self definition and self determination.”
Bill Arnold
This is worth a look. It includes some discussion of AI hacking of the legal code. (He’s not talking about superhuman AIs, but about next 10-30 year extensions of current research/implementations.)
When AIs Start Hacking (Bruce Schneier, April 26, 2021)
The Coming AI Hackers (Bruce Schneier, April 2021) (pdf)
Brachiator
@Bill Arnold:
Noted. And I am not a lawyer or automotive vehicle expert and am just idly speculating.
But in looking at links related to data and bias for another commenter here, I ran across this little story.
This was from a 2019 article. I hope things have improved since then.
But again, the liability issues related to self-driving vehicles has got to be wild.
dnfree
@debbie: Im not sure Klara and the Sun is any more dystopian than Buried Giant! I just think he has a way of presenting things that is subtle and understated and leaves you to fill in the blanks. You won’t know unless you try it! Don’t read too many reviews. I saw a few that gave away more of the plot than they should have.
RSA
@Brachiator:
This is already baked into some systems in self-driving cars. From 2016:
I’ve said elsewhere that business people and software engineers are some of the last people we want to be making ethical decisions on our behalf. Yet it’s happening and largely flying under the radar of the general public’s attention.
James E Powell
@BGinCHI:
Really? Any film/tv adaptations in the works?
Bill Arnold
The Quantum Thief (Hannu Rajaniemi, 2010) and its two sequels are rather dense conceptually.
His short story collection “Hannu Rajaniemi: Collected Fiction” is mind-bending as well. Much of it is edge-of-humanity material.
In our mundane current world, non-neurotypical humans are often quite alien to most biological humans. There is a lot of potential for variation in mental schemas(/topologies) that has not been explored. (Some in the psychedelics research literature; entheogens is a good search keyword.)
Brachiator
@RSA:
Very interesting snippet. Thanks.
This might be the most ethically conservative option.
Wouldn’t most people expect the same thing if they were riding in a cab with a human driver? Even though they might make a different decision if driving their own car and involved in a potential collision.
Yet Another Haldane
Elizabeth Bear’s trilogy of Dust, Chill, and Grail has some truly wild speculation on AIs and their interaction with (post-)humans. Most of the action takes place on a generation ship that suffered Complications between launch and the start of Dust.
debbie
@dnfree:
Actually, I did read it. So bleak!
RSA
@Brachiator:
Yes. I think you raised a good point earlier on, that if a self-driving car manufacturer didn’t prioritize vehicle occupants over others, no one would buy their cars.
Benw
@BGinCHI: I only watched the first season of the TV show, but yeah, it was pretty good.
Larch
Not tech/AI-related, but Octavia Butler explored how far the concept of “human” could stretch. Clay’s Ark was perhaps the most direct, as something inadvertently brought back from space on one of the (I think) Apollo missions changes those who come into contact with it, and the changes spread until it’s clear that there will soon be almost no one left on earth who’s still “human”. Some of her books explored aliens interacting with & being raised by earth humans, or people who could be humans with extra abilities or could be something else.
I devoured her books in the 80s, and the details are fuzzy, but I still remember the realization of where the Clay’s Ark scenario was headed.
Geminid
This thread references some good science fiction I remember from long ago. I am reminded of a short story whose author I have forgotten. It takes place in a dystopian world dominated by robots. Humans coexist, but have to use wooden and stone tools to raise crops, weave cloth etc., because every robot’s sole purpose is to build more robots, and to that end they take any and all metal they can find. At the end of the story, the humans observe a rickety robot rolling mindlessly along. It hits a bad bump, falls and breaks. Other robots immediately rush in and scavenge all the pieces, except for a gear that goes bouncing down the hill. Then a little robot scurries out, grabs the gear, and scurries back to it’s hiding place.
Steeplejack (phone)
Ex Machina (2014). Absolutely chilling, with a killer ending.
dm
If one takes the entirety of the Ghost in the Shell franchise, the question of what is human comes up a good deal. It’s something the tachikoma think about a lot in the TV series (and often they are the most human characters). And it’s at the heart of Kusanagi’s personality: how much (if any) of her humanity is left.
Poe Larity
@Ken: Hogan had a lot of interesting takes, Code of the Lifemaker was another favorite connecting robots and evolution.
Of course, he was JAFRWNJSF Writer.
HeartlandLiberal
Cannot recommend the Jon and Lobo series by Mark L. van Name highly enough. From van Names’s web site:
One of my all time favorite good read sci-fi series.
Truly a warning on the Internet of things. The main characters is able to talk to the AI’s in coffee makers and toasters, and garner intelligence to help him in his quests. Turns out their limited ability AI makes them eager to gossip!
Michael Cain
The first test case is much more likely to be like Heinlein’s Moon Is a Harsh Mistress (ignore the politics, please) than a robot: a warehouse full of hardware drawing down megawatts. Can tons of hardware running software whose state is not predictable because random numbers get tossed in from time to time (it’s not called stochastic gradient descent for nothing) be human?
“You listening, Bog? Is a computer one of Your creatures?”
AWOL
@BGinCHI: I had to read it three years after I produced it so I could forget the parts of the novel I did detail work on. It’s an exceptional work by a Filipino visionary.
Nelson
@debbie: Read it recently and enjoyed it very much. One of the interesting things was explicitly dealing with the concept of an AI’s mortality.