||It has been suggested that Clanking replicator be merged into this article. (Discuss) Proposed since March 2015.|
A self-replicating machine is a type of autonomous robot that is capable of reproducing itself autonomously using raw materials found in the environment, thus exhibiting self-replication in a way analogous to that found in nature. The concept of self-replicating machines has been advanced and examined by Homer Jacobsen, Edward F. Moore, Freeman Dyson, John von Neumann and in more recent times by K. Eric Drexler in his book on nanotechnology, Engines of Creation and by Robert Freitas and Ralph Merkle in their review Kinematic Self-Replicating Machines which provided the first comprehensive analysis of the entire replicator design space. The future development of such technology is an integral part of several plans involving the mining of moons and asteroid belts for ore and other materials, the creation of lunar factories, and even the construction of solar power satellites in space. The possibly misnamed von Neumann probe is one theoretical example of such a machine. Von Neumann also worked on what he called the universal constructor, a self-replicating machine that would operate in a cellular automata environment.
A self-replicating machine is an artificial self-replicating system that relies on conventional large-scale technology and automation. Certain idiosyncratic terms are occasionally found in the literature. For example, the term "clanking replicator" was once used by Drexler to distinguish macroscale replicating systems from the microscopic nanorobots or "assemblers" that nanotechnology may make possible, but the term is informal and is rarely used by others in popular or technical discussions. Replicators have also been called "von Neumann machines" after John von Neumann, who first rigorously studied the idea. However, the term "von Neumann machine" is less specific and also refers to a completely unrelated computer architecture that von Neumann proposed and so its use is discouraged where accuracy is important. Von Neumann himself used the term universal constructor to describe such self-replicating machines.
Historians of machine tools, even before the numerical control era, sometimes figuratively said that machine tools were a unique class of machines because they have the ability to "reproduce themselves" by copying all of their parts. Implicit in these discussions is that a human would direct the cutting processes (later planning and programming the machines), and would then be assembling the parts. The same is true for RepRaps, which are another class of machines sometimes mentioned in reference to such non-autonomous "self-replication". In contrast, machines that are truly autonomously self-replicating (like biological machines) are the main subject discussed here.
- 1 History
- 2 Recent work
- 3 Partial construction
- 4 Self-replicating spacecraft
- 5 Other references
- 6 In fiction
- 7 Prospects for implementation
- 8 See also
- 9 References
- 10 Bibliography
- 11 External links
The general concept of artificial machines capable of producing copies of themselves dates back at least several hundred years. An early reference is an anecdote regarding the philosopher René Descartes, who suggested to Queen Christina of Sweden that the human body could be regarded as a machine; she responded by pointing to a clock and ordering "see to it that it reproduces offspring." Several other variations on this anecdotal response also exist. Samuel Butler proposed in his 1872 novel Erewhon that machines were already capable of reproducing themselves but it was man who made them do so, and added that "machines which reproduce machinery do not reproduce machines after their own kind".
In 1802 William Paley formulated the first known teleological argument depicting machines producing other machines, suggesting that the question of who originally made a watch was rendered moot if it were demonstrated that the watch was able to manufacture a copy of itself. Scientific study of self-reproducing machines was anticipated by John Bernal as early as 1929 and by mathematicians such as Stephen Kleene who began developing recursion theory in the 1930s. Much of this latter work was motivated by interest in information processing and algorithms rather than physical implementation of such a system, however.
von Neumann's kinematic model
A detailed conceptual proposal for a physical non-biological self-replicating system was first put forward by mathematician John von Neumann in lectures delivered in 1948 and 1949, when he proposed a kinematic self-reproducing automaton model as a thought experiment. Von Neumann's concept of a physical self-replicating machine was dealt with only abstractly, with the hypothetical machine using a "sea" or stockroom of spare parts as its source of raw materials. The machine had a program stored on a memory tape that directed it to retrieve parts from this "sea" using a manipulator, assemble them into a duplicate of itself, and then copy the contents of its memory tape into the empty duplicate's. The machine was envisioned as consisting of as few as eight different types of components; four logic elements that send and receive stimuli and four mechanical elements used to provide a structural skeleton and mobility. While qualitatively sound, von Neumann was evidently dissatisfied with this model of a self-replicating machine due to the difficulty of analyzing it with mathematical rigor. He went on to instead develop an even more abstract model self-replicator based on cellular automata. His original kinematic concept remained obscure until it was popularized in a 1955 issue of Scientific American.
Moore's artificial living plants
In 1956 mathematician Edward F. Moore proposed the first known suggestion for a practical real-world self-replicating machine, also published in Scientific American. Moore's "artificial living plants" were proposed as machines able to use air, water and soil as sources of raw materials and to draw its energy from sunlight via a solar battery or a steam engine. He chose the seashore as an initial habitat for such machines, giving them easy access to the chemicals in seawater, and suggested that later generations of the machine could be designed to float freely on the ocean's surface as self-replicating factory barges or to be placed in barren desert terrain that was otherwise useless for industrial purposes. The self-replicators would be "harvested" for their component parts, to be used by humanity in other non-replicating machines.
Dyson's replicating systems
The next major development of the concept of self-replicating machines was a series of thought experiments proposed by physicist Freeman Dyson in his 1970 Vanuxem Lecture. He proposed three large-scale applications of machine replicators. First was to send a self-replicating system to Saturn's moon Enceladus, which in addition to producing copies of itself would also be programmed to manufacture and launch solar sail-propelled cargo spacecraft. These spacecraft would carry blocks of Enceladean ice to Mars, where they would be used to terraform the planet. His second proposal was a solar-powered factory system designed for a terrestrial desert environment, and his third was an "industrial development kit" based on this replicator that could be sold to developing countries to provide them with as much industrial capacity as desired. When Dyson revised and reprinted his lecture in 1979 he added proposals for a modified version of Moore's seagoing artificial living plants that was designed to distill and store fresh water for human use and the "Astrochicken."
Advanced Automation for Space Missions
In 1980, inspired by a 1979 "New Directions Workshop" held at Wood's Hole, NASA conducted a joint summer study with ASEE entitled Advanced Automation for Space Missions to produce a detailed proposal for self-replicating factories to develop lunar resources without requiring additional launches or human workers on-site. The study was conducted at Santa Clara University and ran from June 23 to August 29, with the final report published in 1982. The proposed system would have been capable of exponentially increasing productive capacity and the design could be modified to build self-replicating probes to explore the galaxy.
The reference design included small computer-controlled electric carts running on rails inside the factory, mobile "paving machines" that used large parabolic mirrors to focus sunlight on lunar regolith to melt and sinter it into a hard surface suitable for building on, and robotic front-end loaders for strip mining. Raw lunar regolith would be refined by a variety of techniques, primarily hydrofluoric acid leaching. Large transports with a variety of manipulator arms and tools were proposed as the constructors that would put together new factories from parts and assemblies produced by its parent.
Power would be provided by a "canopy" of solar cells supported on pillars. The other machinery would be placed under the canopy.
A "casting robot" would use sculpting tools and templates to make plaster molds. Plaster was selected because the molds are easy to make, can make precise parts with good surface finishes, and the plaster can be easily recycled afterward using an oven to bake the water back out. The robot would then cast most of the parts either from nonconductive molten rock (basalt) or purified metals. A carbon dioxide laser cutting and welding system was also included.
A more speculative, more complex microchip fabricator was specified to produce the computer and electronic systems, but the designers also said that it might prove practical to ship the chips from Earth as if they were "vitamins."
Much of the design study was concerned with a simple, flexible chemical system for processing the ores, and the differences between the ratio of elements needed by the replicator, and the ratios available in lunar regolith. The element that most limited the growth rate was chlorine, needed to process regolith for aluminium. Chlorine is very rare in lunar regolith.
Lackner-Wendt Auxon replicators
In 1995, inspired by Dyson's 1970 suggestion of seeding uninhabited deserts on Earth with self-replicating machines for industrial development, Klaus Lackner and Christopher Wendt developed a more detailed outline for such a system. They proposed a colony of cooperating mobile robots 10–30 cm in size running on a grid of electrified ceramic tracks around stationary manufacturing equipment and fields of solar cells. Their proposal didn't include a complete analysis of the system's material requirements, but described a novel method for extracting the ten most common chemical elements found in raw desert topsoil (Na, Fe, Mg, Si, Ca, Ti, Al, C, O2 and H2) using a high-temperature carbothermic process. This proposal was popularized in Discover Magazine, featuring solar-powered desalination equipment used to irrigate the desert in which the system was based. They named their machines "Auxons", from the Greek word auxein which means "to grow."
Self-replicating rapid prototypers
Early experimentation with rapid prototyping in 1997-2000 was not expressly oriented toward reproducing rapid prototyping systems themselves, but rather extended simulated "evolutionary robotics" techniques into the physical world. Later developments in rapid prototyping have given the process the ability to produce a wide variety of electronic and mechanical components, making this a rapidly developing frontier in self-replicating system research.
In 1998 Chris Phoenix informally outlined a design for a hydraulically powered replicator a few cubic feet in volume that used ultraviolet light to cure soft plastic feedstock and a fluidic logic control system, but didn't address most of the details of assembly procedures, error rates, or machining tolerances.
In 2005, Adrian Bowyer of the University of Bath started the RepRap Project to develop a rapid prototyping machine which would be able to manufacture some or most of its own components, making such machines cheap enough for people to buy and use in their homes. The project is releasing its designs and control programs under the GNU GPL. The RepRap approach uses fused deposition modeling to manufacture plastic components, possibly incorporating conductive pathways for circuitry. Other components, such as steel rods, nuts and bolts, motors and separate electronic components, would be supplied externally. In 2006 the project produced a basic functional prototype and in May 2008 the machine succeeded in producing all of the plastic parts required to make a 'child' machine.
NIAC studies on self-replicating systems
In the spirit of the 1980 "Advanced Automation for Space Missions" study, the NASA Institute for Advanced Concepts began several studies of self-replicating system design in 2002 and 2003. Four phase I grants were awarded:
- Hod Lipson (Cornell University), "Autonomous Self-Extending Machines for Accelerating Space Exploration"
- Gregory Chirikjian (Johns Hopkins University), "Architecture for Unmanned Self-Replicating Lunar Factories"
- Paul Todd (Space Hardware Optimization Technology Inc.), "Robotic Lunar Ecopoiesis"
- Tihamer Toth-Fejel (General Dynamics), "Modeling Kinematic Cellular Automata: An Approach to Self-Replication" The study concluded that complexity of the development was equal to that of a Pentium 4, and promoted a design based on cellular automata.
Cornell University's self-assembler
In 2005, a team of researchers at Cornell University, including Hod Lipson, implemented a self-assembling machine. The machine is composed of a tower of four articulated cubes, known as molecubes, which can revolve about a triagonal. This enables the tower to function as a robotic arm, collecting nearby molecubes and assembling them into a copy of itself. The arm is directed by a computer program, which is contained within each molecube, analogous to how each animal cell contains an entire copy of its DNA. However, the machine cannot manufacture individual molecubes, nor do they occur naturally, so its status as a self-replicator is debatable.
New York University artificial DNA tile motifs
In 2011 a team of scientists at New York University created a structure called 'BTX' (bent triple helix) based around three double helix molecules, each made from a short strand of DNA. Treating each group of three double-helices as a code letter, they can (in principle) build up self-replicating structures that encode large quantities of information.
Self-replication of magnetic polymers
Self-replication of neural circuits
In 1968 Zellig Harris wrote that "the metalanguage is in the language," suggesting that self-replication is part of language. In 1977 Niklaus Wirth formalized this proposition by publishing a self-replicating deterministic context-free grammar. Adding to it probabilities, Bertrand du Castel published in 2015 a self-replicating stochastic grammar and presented a mapping of that grammar to neural networks, thereby presenting a model for a self-replicating neural circuit.
The von Neumann model of self-replication envisages that the mother automaton should construct all portions of daughter automatons, without exception and prior to the initiation of such daughters. Partial construction alters the construction relationship between mother and daughter automatons, such that the mother constructs but a portion of the daughter, and upon initiating this portion of the daughter, thereafter retracts from imparting further influence upon the daughter. Instead, the daughter automaton is left to complete its own development. This is to say, means exist by which automatons may develop via the mechanism of a zygote.
The idea of an automated spacecraft capable of constructing copies of itself was first proposed in scientific literature in 1974 by Michael A. Arbib, but the concept had appeared earlier in science fiction such as the 1967 novel Berserker by Fred Saberhagen or the 1950 novellette trilogy The Voyage of the Space Beagle by A. E. van Vogt (see In fiction, below). The first quantitative engineering analysis of a self-replicating spacecraft was published in 1980 by Robert Freitas, in which the non-replicating Project Daedalus design was modified to include all subsystems necessary for self-replication. The design's strategy was to use the probe to deliver a "seed" factory with a mass of about 443 tons to a distant site, have the seed factory replicate many copies of itself there to increase its total manufacturing capacity, and then use the resulting automated industrial complex to construct more probes with a single seed factory on board each.
- A number of patents have been granted for self-replicating machine concepts. The most directly relevant include U.S. Patent 4,734,856 "Autogeneric system" Inventor: Davis; Dannie E. (Elmore, AL) (March 1988), U.S. Patent 5,659,477 "Self reproducing fundamental fabricating machines (F-Units)" Inventor: Collins; Charles M. (Burke, VA) (August 1997), U.S. Patent 5,764,518 " Self reproducing fundamental fabricating machine system" Inventor: Collins; Charles M. (Burke, VA)(June 1998); Collins' PCT: and U.S. Patent 6,510,359 "Method and system for self-replicating manufacturing stations" Inventors: Merkle; Ralph C. (Sunnyvale, CA), Parker; Eric G. (Wylie, TX), Skidmore; George D. (Plano, TX) (January 2003).
- Macroscopic replicators are mentioned briefly in the fourth chapter of K. Eric Drexler's 1986 book Engines of Creation.
- In 1995, Nick Szabo proposed a challenge to build a macroscale replicator from Lego robot kits and similar basic parts. Szabo wrote that this approach was easier than previous proposals for macroscale replicators, but successfully predicted that even this method would not lead to a macroscale replicator within ten years.
- In 2004, Robert Freitas and Ralph Merkle published the first comprehensive review of the field of self-replication (from which much of the material in this article is derived, with permission of the authors), in their book Kinematic Self-Replicating Machines, which includes 3000+ literature references. This book included a new molecular assembler design, a primer on the mathematics of replication, and the first comprehensive analysis of the entire replicator design space.
Many types of fictional self-replicating machines have been featured in literature, and particularly in science fiction.
Fictional self-replicating machines in literature Year Work Author Notes 1943 "M33 in Andromeda" A. E. van Vogt A. E. van Vogt used the idea as a plot device in his story "M33 in Andromeda" (1943) which was later combined with the three other Space Beagle short stories to become the novel, The Voyage of the Space Beagle. The story describes the creation of self-replicating weapons factories designed to destroy the Anabis, a galaxy-spanning malevolent life form bent on destruction of the human race. 1953 "Second Variety" Philip K. Dick In the short story a nuclear war between the Soviet Union and the West has reduced much of the world to a barren wasteland. The war continues, however, among the scattered remains of humanity. The Western forces have developed "claws", which are autonomous self-replicating robots to fight on their side. It is one of Dick's many stories in which nuclear war has rendered the Earth's surface uninhabitable. The story was adapted into the movie Screamers in 1995. 1955 "Autofac" Philip K. Dick An early treatment was the short story Autofac by Philip K. Dick, published in 1955. Dick also touched on this theme in his earlier 1953 short story "Second Variety". Another example can be found in the 1962 short story "Epilogue" by Poul Anderson, in which self-replicating factory barges were proposed that used minerals extracted from ocean water as raw materials. 1955 "The Necessary Thing" Robert Sheckley In the short story the Universal Replicator is unwittingly tricked into replicating itself. 1958 "Crabs on the Island" Anatoly Dneprov In his short story "Crabs on the Island" (1958) Anatoly Dneprov speculated on the idea that since the replication process is never 100% accurate, leading to slight differences in the descendants, over several generations of replication the machines would be subjected to evolution similar to that of living organisms. In the story, a machine is designed, the sole purpose of which is to find metal to produce copies of itself, intended to be used as a weapon against an enemy's war machines. The machines are released on a deserted island, the idea being that once the available metal is all used and they start fighting each other, natural selection will enhance their design. However, the evolution has stopped by itself when the last descendant, an enormously large crab, was created, being unable to reproduce itself due to lack of energy and materials. 1963-2005 Berserker series Fred Saberhagen The Berserker series is a series of space opera science fiction short stories and novels, in which robotic self-replicating machines (The berserkers) strive to destroy all life. 1964 The Invincible Stanisław Lem Stanisław Lem has also studied the same idea in his novel, in which the crew of a spacecraft landing on a distant planet finds a non-biological life-form, which is the product of long, possibly of millions of years of, mechanical evolution (necroevolution). This phenomenon is also key to the aforementioned Anderson story. 1968 The Reproductive System John Sladek John Sladek used the concept to humorous ends in his first novel The Reproductive System (1968, also titled Mechasm in some markets), where a U.S. military research project goes out of control. 1970 "The Scarred Man" Gregory Benford Long before the existence of the Internet, author Greg Benford was inspired by his work on ARPANet in the late 1960s to write this first account of a self-replicating program - a computer virus. His con men program a computer to randomly dial phone numbers until it hits a telephone modem that is answered by another computer. It then programs the answering computer to begin dialing random numbers in search of yet another computer, while also programming a small delay on each computer's processing time. The virus spreads exponentially through susceptible computers, like a biological infection, and the creators profit by "fixing" the slowed computers. (Story text on author's website.) 1975 The Shockwave Rider John Brunner An early example of a fictional account of a computer virus or worm. 1977 The Adolescence of P-1 Thomas J. Ryan Another early fictional account of a computer virus or worm. 1977-1999 Galactic Center Saga series Gregory Benford The series details a galactic war between mechanical and biological life. In it an antagonist berserker machine race is encountered by Earth, first as a probe in In the Ocean of Night, and then in an attack in Across the Sea of Suns. The berserker machines do not seek to completely eradicate a race if merely throwing it into a primitive low technological state will do as they did to the EMs encountered in Across the Sea of Suns. 1982 2010: Odyssey Two Arthur C. Clarke The novel is the sequel to the 1968 novel 2001: A Space Odyssey, but continues the story of Stanley Kubrick's film adaptation with the same title rather than Clarke's original novel. Set in the year 2010, the plot centers on a joint Soviet-American mission aboard the Soviet spacecraft Leonov. Its crew flees Jupiter as a mysterious dark spot appears on Jupiter and begins to grow. HAL's telescope observations reveal that the "Great Black Spot" is, in fact, a vast population of monoliths, increasing at an exponential rate, which appear to be eating the planet. By acting as self-replicating 'von Neumann' machines, these monoliths increase Jupiter's density until the planet achieves nuclear fusion, becoming a small star. 1983 Code of the Lifemaker James P. Hogan NASA's Advanced Automation for Space Missions study directly inspired the science fiction novel. 1985 The Third Millennium: A History of the World AD 2000-3000 Brian Stableford
In the book—a fictional historical account, from the perspective of the year 3000, giving a future history of humanity and its technological and sociological developments—humanity sends cycle-limited Von Neumann probes out to the nearest stars to do open-ended exploration and to announce humanity's existence to whoever might encounter them. 1986 The Songs of Distant Earth Arthur C. Clarke In the novel humanity on a future Earth facing imminent destruction creates automated seedships that act as fire and forget lifeboats aimed at distant, habitable worlds. Upon landing, the ship begins to create new humans from stored genetic information, and an onboard computer system raises and trains the first few generations of new inhabitants. The massive ships are then broken down and used as building materials by their "children". 1986 "Lungfish" David Brin In the short story collection, The River of Time, the short story "Lungfish" prominently features von Neumann probes. Not only does he explore the concept of the probes themselves, but indirectly explores the ideas of competition between different designs of probes, evolution of von Neumann probes in the face of such competition, and the development of a type of ecology between von Neumann probes. One of the vessels mentioned is clearly a Seeder type. 1987 The Forge of God Greg Bear The Killers, a civilization of self-replicating machines designed to destroy any potential threat to their (possibly long-dead) creators. 1990 The World at the End of Time Frederik Pohl  1992 Cold as Ice Charles Sheffield In the novel there is a segment where the author (a physicist) describes von Neumann machines harvesting sulfur, nitrogen, phosphorus, helium-4, and various metals from the atmosphere of Jupiter. 1993 Assemblers of Infinity Kevin J. Anderson and Doug Beason This novel describes self-replicating robots that are programmed not to harm biospheres but instead use materials on the moon for an alien civilization to reproduce and colonize the moon. While this is happening a human scientist on Earth reverse engineers the dormant nanomachines found on Earth (since Earth is a biosphere they don't harm the environment) to make medical nano-machines and is successful at first when he revives a medically dead scientist, but accidentally removes the safety measure, creating a grey goo scenario that he stops at the cost of his life when he activates a high powered x-ray machine built as a safety guard. 1993 Anvil of Stars Greg Bear The novel is the sequel to The Forge of God and explores the reaction other civilizations have to the creation and release of berserkers. 1995 The Ganymede Club Charles Sheffield A mystery and a thriller, the story unravels in the same universe that Sheffield imagined in Cold as Ice. In it humans have colonized the solar system with the help of self-replicating machines called Von Neumanns. 1995 The Diamond Age Neal Stephenson The novel depicts a near-future Earth society wherein nanotechnology, including self-replicators, both exist and influence daily life greatly. 1996 Excession Iain Banks In the novel hegemonising swarms are described as a form of Outside Context Problem. An example of an "Aggressive Hegemonising Swarm Object" is given as an uncontrolled self-replicating probe with the goal of turning all matter into copies of itself. After causing great damage, they are somehow transformed using unspecified techniques by the Zetetic Elench and become "Evangelical Hegemonising Swarm Objects". Such swarms (referred to as "smatter") reappear in the later novels Surface Detail (which features scenes of space combat against the swarms) and The Hydrogen Sonata. 1998 Moonseed Stephen Baxter In the novel Earth faces danger from a self-replicating nanobot swarm after a rock is returned from the Apollo 18 mission. The rock contains a mysterious substance called "moonseed" (a form of grey goo, whether nanobots, an alien virus or something else) that starts to change all inorganic matter on Earth into more moonseed. 1998 Bloom Wil McCarthy Bloom is set in the year 2106, in a world where self-replicating nanomachines called "Mycora" have consumed Earth and other planets of the inner solar system, forcing humankind to eke out a bleak living in the asteroids and Galilean moons. 1998 Destiny's Road Larry Niven In the novel von Neumann machines are scattered throughout the human colony world Destiny and its moon Quicksilver in order to build and maintain technology and to make up for the lack of the resident humans' technical knowledge; the Von Neumann machines primarily construct a stretchable fabric cloth capable of acting as a solar collector which serves as the humans' primary energy source. The Von Neumann machines also engage in ecological maintenance and other exploratory work. 2000 Manifold: Space Stephen Baxter The novel starts with the discovery of alien self-replicating machines active within the Solar system. 2000–present Revelation Space series Alastair Reynolds In the series Inhibitors are self-replicating machines whose purpose is to inhibit the development of intelligent star-faring cultures. They are dormant for extreme periods of time until they detect the presence of a space-faring culture and proceed to exterminate it even to the point of sterilizing entire planets. They are very difficult to destroy as they seem to have faced any type of weapon ever devised and only need a short time to 'remember' the necessary counter-measures. Also "Greenfly" terraforming machines are another form of berserker machines. For unknown reasons, but probably an error in their programming, they destroy planets and turn them into trillions of domes filled with vegetation – after all, their purpose is to produce a habitable environment for humans, however in doing so they inadvertently decimate the human race. By 10.000, they have wiped out most of the Galaxy. 2002 Evolution Stephen Baxter The novel follows 565 million years of human evolution, from shrewlike mammals 65 million years in the past to the ultimate fate of humanity (and its descendants, both biological and non-biological) 500 million years in the future. At one point, hominids become sapient, and go on to develop technology, including an evolving universal constructor machine that goes to Mars and multiplies, and in an act of global ecophagy consumes Mars by converting the planet into a mass of machinery that leaves the Solar system in search of new planets to assimilate. 2002 Prey Michael Crichton In the novel nanobots were blown into the desert from an isolated laboratory, evolving and eventually forming autonomous swarms. These swarms appear to be clouds of solar-powered and self-sufficient, reproducing and evolving rapidly. The swarms exhibit predatory behavior, attacking and killing animals in the wild and later going as far as to forming symbiotic relationships with humans and even mimicking them. 2002 Lost in a Good Book Jasper Fforde The novel features an alternative pink goo end of the world scenario, where a nanotechnology 'Dream Topping making machine' turns all matter on earth into a pink dessert similar to Angel Delight. The Dream Topping is taken back in time to the beginning of earth, where it supplies the organic nutrients needed to create life. 2003 Ilium Dan Simmons The first part of the Ilium/Olympos cycle, concerning the re-creation of the events in the Iliad on an alternate Earth and Mars. These events are set in motion by beings who have taken on the roles of the Greek gods. In the cycle the voynix are biomechanical, self-replicating, programmable robots. They originated in an alternate universe, and were brought into the Ilium universe before 3000 A.D. 2003 Singularity Sky Charles Stross The Festival, a civilisation of uploaded minds with strange designs on humanity 2004 Recursion Tony Ballantyne Herb, a young entrepreneur, returns to the isolated planet on which he has illegally been trying to build a city–and finds it destroyed by a swarming nightmare of self-replicating machinery. 2005 Spin Robert Charles Wilson In the novel self-replicating artificial life, shot into space to build a huge sentient network in the outer reaches of the Solar System and gather information about the alien "Hypotheticals". It encounters not just other von Neumann machines, but a pre-existing and galaxy-spanning ecology of them. Apparently this vast network of sentient artificial life is responsible for the "Spin" – the placement of an opaque black membrane around the entire Earth. 2005 Olympos Dan Simmons The sequel to Ilium and final part of the Ilium/Olympos series. 2007 Von Neumann's War John Ringo
Travis S. Taylor
In the novel published by Baen Books in 2007 von Neumann probes arrive in the solar system, moving in from the outer planets, and converting all metals into gigantic structures. Eventually they arrive on Earth, wiping out much of the population before being beaten back when humanity reverse engineers some of the probes. 2007 Postsingular Rudy Rucker In Postsingular, nanobots devour the Earth and copy everybody they eat into a simulation... luckily, one of the machine's developers also created a backdoor, and is able to reverse the situation, restoring everybody. Soon after, another set of tiny self-replicating machines are released, which don't devour, merely reproduce until they cover every inch of the Earth, sharing information with each other and the people they're on. They connect humanity like they've never been connected before so that one can watch anyone else by experiencing what the "orphids" on that person's body are experiencing. 2010 Surface Detail Iain Banks The novel depicts self-replicating machines as a universe-threatening infection. 2011 Lord of All Things Andreas Eschbach In the novel (original title "Herr aller Dinge") an ancient nano machine complex is discovered buried in a glacier off the coast of Russia. When it comes in contact with materials it needs to fulfill its mission, it creates a launch facility and launches a space craft. It is later revealed that the nano machines were created by a pre-historic human race with the intention of destroying other interstellar civilizations (for an unknown reason). It is purposed that the reason there is no evidence of the race is because of the nano-machines themselves and their ability to manipulate matter at an atomic level. It is even suggested that viruses could be ancient nano machines that have evolved over time. 2012 The Hydrogen Sonata Iain Banks  2012–present The Machine Dynasty series Madeline Ashby In the novels the protagonists are von Neumann machines, self-replicating humanoid robots. The original proposal for the self-replicating humanoid robots came from a religious End Times group who wanted to leave a body of helpers behind for the millions of unsaved after the rapture. 2014 Creations William Mitchell In the novel biological engineer Max Lowrie gets a job offer of a lifetime that's supposed to pave the way for humanity's future: self-replicating machines that can mine materials from the harshest environments at no cost, opening up as yet unheard of resources in the sea, on land, and ultimately on the Moon.
Many types of self-replicating machines have been featured in the movies.
- The movie Screamers, based on Philip K. Dick's short story "Second Variety", features a group of robot weapons created by mankind to act as Von Neumann devices / berserkers. The original robots are subterranean buzzsaws that make a screaming sound as they approach a potential victim beneath the soil. These machines are self-replicating and, as is found out through the course of the movie, they are quite intelligent and have managed to "evolve" into newer, more dangerous forms, most notably human forms which the real humans in the movie cannot tell apart from other real humans except by trial and error.
- The Terminator is a 1984 science fiction/action film directed and co-written by James Cameron which describes a war between mankind and self replicating machines led by a central artificial intelligence known as Skynet. Machine civilizations are a recurring theme in fiction.
- In the 2008 science fiction film The Day the Earth Stood Still follows Klaatu, an alien sent to try to change human behavior or eradicate them from Earth via grey goo due to humankind's environmental damage to the planet.
- The Star Wars expanded universe features the World Devastators, large ships designed and built by the Galactic Empire that tear apart planets to use its materials to build other ships or even upgrade or replicate themselves.
- The Matrix series features self-replicating nanobots that were deployed by the United Nations to block the rebelling machine population from their energy source.
The concept is also widely utilised in science fiction television.
- The TV series Lexx featured an army of self replicating robots known as Mantrid drones.
- The Replicators are a horde of self-replicating machines that appear frequently in Stargate SG-1. They once were a vicious race of insect-like robots that were originally created by an android named Reese to serve as toys. They grew beyond her control and began evolving, eventually spreading throughout at least two galaxies. In addition to ordinary autonomous evolution they were able to analyze and incorporate new technologies they encountered into themselves, ultimately making them one of the most advanced "races" known. During the course of the series, the replicators assume a human form and pose a huge threat to the galaxy. A more sophisticated version of the human form Replicators, who call themselves Asurans also appear in the spin-off series Stargate Atlantis.
- In Stargate Atlantis, a second race of replicators created by the Ancients were encountered in the Pegasus Galaxy. They were created as a means to defeat the Wraith. The Ancients attempted to destroy them after they began showing sings of sentience and requested that their drive to kill the wraith be removed. This failed, and an unspecified length of time after the Ancients retreated to the Milky Way Galaxy, the replicators nearly succeeded in destroying the Wraith. The Wraith were able to hack into the replicators and deactivate the extermination drive, at which point they retreated to their home world and were not heard from again until encountered by the Atlantis Expedition. After the Atlantis Expedition reactivated this dormant directive, the replicators embarked on a plan to kill the Wraith by removing their food source, i.e. all humans in the Pegasus Galaxy.
- In the Stargate Universe the human adventurers live on a ship called Destiny. Its mission was to connect a network of Stargates, placed by preceding seeder ships, on planets capable of supporting life to allow instantaneous travel between them.
- In Stargate Universe Season 2, a galaxy billions of light years distant from the Milky Way is infested with drone ships that are programmed to annihilate intelligent life and advanced technology. The drone ships attack other space ships (including Destiny) as well as humans on planetary surfaces, but don't bother destroying primitive technology such as buildings unless they are harboring intelligent life or advanced technology.
- Star Trek's Borg – a self-replicating bio-mechanical race that is dedicated to the task of achieving perfection through the assimilation of useful technology and lifeforms. Their ships are massive mechanical cubes (a close step from the Berzerker's massive mechanical Spheres).
- The episodes "A Clockwork Origin" and "Benderama" of the animated science fiction comedy sitcom Futurama.
- The episode "Walkabout" of Gargoyles (season 2, episode 33) is about grey goo.
- The Babylon 5 episode "Infection" showed a smaller scale berserker in the form of the Icarran War Machine. After being created with the goal of defeating an unspecified enemy faction, the War Machines proceeded to exterminate all life on the planet Icarra VII because they had been programmed with standards for what constituted a 'Pure Icaran' based on religious teachings, which no actual Icaran could satisfy. Because the Icaran were pre-starflight, the War Machines became dormant after completing their task rather than spreading. One unit was reactivated on-board Babylon 5 after being smuggled past quarantine by an unscrupulous archaeologist, but after being confronted with how they had rendered Icara VII a dead world, the simulated personality of the War Machine committed suicide.
- In the Justice League Unlimited episode "Dark Heart", an alien weapon based on the idea lands on Earth.
- In Steven Universe, Gems are a race of artificial intelligences composed of gemstones projecting light-construct bodies. These are created by bacteriophage-like Injector engines that drill into a planet's crust and infuse specific gems with the local biota's life energy, animating it; they do not reproduce naturally, and several similarities to computers have been noticed.
In video games
- Grey Goo is a science fiction real-time strategy video game that features a playable faction based on the grey goo scenario.
- Conway's Game of Life
- Tasty Planet, a game released in 2006 by Dingo Games centers around a gray goo eating the universe, starting at the atomic level and progressing to the cosmic level. In the game the player controls a gray goo and eats many objects, such as bacteria, mice, cars, people, Earth, galaxies, and eventually the universe. In the end, the grey goo over-fills, explodes, and starts the universe all over again.
- Hostile Waters: Antaeus Rising
- The Reapers in the video game series Mass Effect are also self-replicating probes bent on destroying any advanced civilization encountered in the galaxy. They lie dormant in the vast spaces between the galaxies and follow a cycle of extermination. In Mass Effect 2 it is shown that they assimilate any advanced species.
- In the computer game Star Control II, the Slylandro Probe is an out-of-control self-replicating probe that attacks starships of other races. They were not originally intended to be a berserker probe; they sought out intelligent life for peaceful contact, but due to a programming error, they would immediately switch to "resource extraction" mode and attempt to dismantle the target ship for raw materials. While the plot claims that the probes reproduce "at a geometric rate", the game itself caps the frequency of encountering these probes. It is possible to deal with the menace in a side-quest, but this is not necessary to complete the game, as the probes only appear one at a time, and the player's ship will eventually be fast and powerful enough to outrun them or destroy them for resources – although the probes will eventually dominate the entire game universe.
- In the Homeworld: Cataclysm video game, a bio-mechanical virus called Beast has the ability to alter organic and mechanic material to suit its needs, and the ships infected become self-replicating hubs for the virus.
- In the computer game Sword of the Stars, the player may randomly encounter "Von Neumann". A Von Neumann mothership appears along with smaller Von Neumann probes, which attack and consume the player's ships. The probes then return to the mothership, returning the consumed material. If probes are destroyed, the mothership will create new ones. If all the player's ships are destroyed, the Von Neumann probes will reduce the planets resource levels before leaving. The mothership is a larger version of the probes. In the 2008 expansion A Murder of Crows, Kerberos Productions also introduces the VN Berserker, a combat orientated ship, which attacks player planets and ships in retaliation to violence against VN Motherships. If the player destroys the Berserker things will escalate and a System Destroyer will attack.
- In the X video game series, the Xenon are a malevolent race of artificially intelligent machines descended from terraforming ships sent out by humans to prepare worlds for eventual colonization; the result caused by a bugged software update. They are continual antagonists in the X-Universe.
- In PC role-playing game Space Rangers and its sequel Space Rangers 2: Dominators, a league of 5 nations battles three different types of Berserker robots. One that focuses on invading planets, another that battles normal space and third that lives in hyperspace.
- In the Star Wolves video game series, Berserkers are a self-replicating machine menace that threatens the known universe for purposes of destruction and/or assimilation of humanity.
- In the Metroid video game series, The massive Leviathans are probes routinely sent out from the planet Phaaze to infect other planets with Phazon radiation and eventually turn these planets into clones of Phaaze, where the self-replication process can continue.
- In the second Deus Ex game, Deus Ex: Invisible War, a videogame features a self-replicating nanomachines bomb in the CGI introduction. A terrorist attack on Boston erased the town and is the beginning of the plot.
- The idea dates back at least as far as Karel Capek's 1920 play R.U.R. (Rossum's Universal Robots). A fundamental obstacle of self-replicating machines, how to repair the repair systems, was the critical failure in the automated society described in The Machine Stops.
- In the comic Transmetropolitan a character mentions "Von Neumann rectal infestations" which are apparently caused by "Shit-ticks that build more shit-ticks that build more shit-ticks".
- In the anime Vandread, harvester ships attack vessels from both male- and female-dominated factions and harvest hull, reactors, and computer components to make more of themselves. To this end, Harvester ships are built around mobile factories. Earth-born humans also view the inhabitants of the various colonies to be little more than spare parts.
- In the role-playing game Eclipse Phase, an ETI probe is believed to have infected the TITAN computer systems with the Exsurgent virus to cause them to go berserk and wage war on humanity. This would make ETI probes a form of berserker, albeit one that uses pre-existing computer systems as its key weapons.
- Storm, the trilogy of albums which conclude the comic book series Storm by Don Lawrence (starting with Chronicles of Pandarve 11: The Von Neumann machine) is based on self-replicating conscious machines containing the sum of all human knowledge employed to rebuild human society throughout the universe in case of disaster on Earth. The probe malfunctions and although new probes are built, they do not separate from the motherprobe which eventually results in a cluster of malfunctioning probes so big that it can absorb entire moons.
- Denial-of-service attacks in the virtual world Second Life which work by continually replicating objects until the server crashes are referred to as gray goo attacks. This is a reference to the self-replicating aspects of gray goo. It is one example of the widespread convention of drawing analogies between certain Second Life concepts and the theories of radical nanotechnology.
- In the manga Battle Angel Alita: Last Order, the surface of Mercury is covered in rogue nanomachines from a Gray Goo event and subsequently spawns a being of dubious morphology known as Anomaly.
- Baldr Sky, a Japanese visual novel features self-replicating nanomachines under the name "assemblers". Their inherent threat and a near-catastrophic grey goo event is central to its plot.
Prospects for implementation
As the use of industrial automation has expanded over time, some factories have begun to approach a semblance of self-sufficiency that is suggestive of self-replicating machines. However, such factories are unlikely to achieve "full closure" until the cost and flexibility of automated machinery comes close to that of human labour and the manufacture of spare parts and other components locally becomes more economical than transporting them from elsewhere. As Samuel Butler has pointed out in Erewhon, replication of partially closed universal machine tool factories is already possible. Since safety is a primary goal of all legislative consideration of regulation of such development, future development efforts may be limited to systems which lack either control, matter, or energy closure. Fully capable machine replicators are most useful for developing resources in dangerous environments which are not easily reached by existing transportation systems (such as outer space).
An artificial replicator can be considered to be a form of artificial life. Depending on its design, it might be subject to evolution over an extended period of time. However, with robust error correction, and the possibility of external intervention, the common science fiction scenario of robotic life run amok will remain extremely unlikely for the foreseeable future.
- Grey goo scenario
- Self-reconfiguring modular robot
- AI takeover
- 3D printing
- Computer virus
- Existential risk from advanced artificial intelligence
- Lights out (manufacturing)
- Spiegelman's Monster
- Freitas, Robert A.; Ralph C. Merkle (2004). Kinematic Self-Replicating Machines. Georgetown, Texas: Landes Bioscience. ISBN 1-57059-690-5.
- "3.11 Freitas Interstellar Probe Replicator (1979-1980)". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- Drexler, K. Eric (1986). "Engines of Abundance (Chapter 4) Clanking Replicators". Engines of Creation.
- Colvin 1947, pp. 6–7.
- Sipper, Moshe; James A. Reggia (August 2001). "Build Your Own Replicator". Scientific American. 285: 38–39. Several other variations on this anecdotal response also exist.
- "1". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- Samuel Butler. "Erewhon, Chapter 24, The book Of the Machines". Nzetc.org. Retrieved 2009-09-16.
- "1". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- Paley, William (1802). "Chapter i, Section 1". Natural Theology: or Evidences of the Existence and Attributes of the Deity, Collected from the Appearances of Nature. E. Goodale. ISBN 0-576-29166-8.; (12th Edition, 1809) See also: Michael Ruse, ed. (1998). Philosophy of Biology. pp. 36–40.; Lenski, Richard (15 November 2001). "Twice as Natural". Nature. 414 (6861): 255. doi:10.1038/35104715. PMID 11713507.
- Bernal, John Desmond (1929). "The World, the Flesh and the Devil: An Enquiry into the Future of the Three Enemies of the Rational Soul".
- "1". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- von Neumann, John (1966). A. Burks, ed. The Theory of Self-reproducing Automata. Urbana, IL: Univ. of Illinois Press. ISBN 0-598-37798-0.
- "2.1 Von Neumann's Contributions". Molecularassembler.com. Retrieved 2009-09-16.
- "2.1.3 The Cellular Automaton (CA) Model of Machine Replication". Molecularassembler.com. Retrieved 2009-09-16.
- Kemeny, John G. (April 1955). "Man Viewed as a Machine". Scientific American. 192 (4): 58–67. doi:10.1038/scientificamerican0455-58.
- Moore, Edward F. (October 1956). "Artificial Living Plants". Scientific American. 195 (4): 118–126. doi:10.1038/scientificamerican1056-118.
- "3.1 Moore Artificial Living Plants (1956)". Molecularassembler.com. Retrieved 2009-09-16.
- Freeman J. Dyson, "The twenty-first century," Vanuxem Lecture delivered at Princeton University, 26 February 1970.
- "3.6 Dyson Terraforming Replicators (1970, 1979)". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- Dyson, Freeman J. (1979). Chapter 18: Thought Experiments. Disturbing the Universe. New York: Harper and Row. pp. 194–204.
- Robert Freitas, William P. Gilbreath, ed. (1982). Advanced Automation for Space Missions. NASA Conference Publication CP-2255 (N83-15348).
- "NASA Institute for Advanced Concepts -- Studies".
- Lackner, Klaus S.; Christopher H. Wendt (1995). "Exponential growth of large self-replicating machine systems". Mathl. Comput. Modelling. 21 (10): 55–81. doi:10.1016/0895-7177(95)00071-9.
- Lackner, Klaus S., and Wendt, Christopher H., "Self-reproducing machine systems for global scale projects," Document LA-UR-93-2886, 4th International Conference and Exposition on Engineering, Construction and Operations in Space/Conference and Exposition/Demonstrations on Robotic for Challenging Environments, Albuquerque, New Mexico, 26 February – 3 March 1994
- "3.15". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- Bass, Thomas (October 1995). "Robot, build thyself". Discover: 64–72.
- "Freitas 2004, pp. 64-67". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- Christopher J. Phoenix (March 21, 1998). "Partial design for macro-scale machining self-replicator". Newsgroup: sci.nanotech.
- "3.20". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- "WebHome". Retrieved 2007-02-18.
- James Jones. CubeSpawn. 2009.
- Lipson, Hod; Evan Malone. "Autonomous Self-Extending Machines for Accelerating Space Exploration" (PDF). Retrieved 2007-01-04.
- Chirikjian, Gregory S. (April 26, 2004). "An Architecture for Self-Replicating Lunar Factories" (PDF). Retrieved 2007-01-04.
- Todd, Paul (30 April 2004). "Final Progress Report on Robotic Lunar Ecopoiesis Test Bed" (PDF). Retrieved 2007-01-04. (phase I report)
- Todd, Paul (July 6, 2006). "Robotic Lunar Ecopoiesis Test Bed" (PDF). Retrieved 2007-01-04. (phase II report)
- Toth-Fejel, Tihamer; Robert Freitas; Matt Moses (April 30, 2004). "Modeling Kinematic Cellular Automata" (PDF). Retrieved 2007-01-04.
- "3.25.4 Toth-Fejel Kinematic Cellular Automata (2003-2004)". Molecularassembler.com. Retrieved 2009-09-16.
- "Researchers build a robot that can reproduce". Physorg.com. 2005-05-11. Retrieved 2010-06-30.
- "Self-Replication Process Holds Promise for Production of New Materials". Science Daily. Retrieved 2011-10-14.
- "Self-replication of information-bearing nanoscale patterns". Nature. Retrieved 2011-10-13.
- "Self-Organization of Template-Replicating Polymers and the Spontaneous Rise of Genetic Information". Entroy. Retrieved 2015-01-31.
- Harris, Zellig (1968). Mathematical Structures of Language. New York, NY: John Wiley and Son. p. 17.
- Wirth, Niklaus. "What can we do about the unnecessary diversity of notation for syntactic definitions?". Commun. ACM. 20: 822–823. doi:10.1145/359863.359883.
- du Castel, Bertrand (15 July 2015). "Pattern Activation/Recognition Theory of Mind". Frontiers in Computational Neuroscience. Lausanne: EPFL. 9 (90). doi:10.3389/fncom.2015.00090.
- Buckley, William R. (2008). "Signal Crossing Solutions in von Neumann Self-replicating Cellular Automata". Automata 2008.
- Buckley, William R. (2008). "Computational Ontogeny". Biological Theory. 3 (1): 3. doi:10.1162/biot.2008.3.1.3.
- "3.11". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- Arbib, Michael A. (1974). Cyril Ponnamperuma, A. G. W. Cameron, ed. The Likelihood of the Evolution of Communicating Intelligences on Other Planets. Interstellar Communication: Scientific Perspectives. Boston: Houghton Mifflin Company. pp. 59–78.
- Freitas, Robert A., Jr. (July 1980). "A Self-Reproducing Interstellar Probe". Journal of the British Interplanetary Society. 33: 251–264. Bibcode:1980JBIS...33..251F. Retrieved 2008-10-01.
- "3.16 The Collins Patents on Reproductive Mechanics (1997-1998)". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- WIPO. "(WO/1996/020453) SELF REPRODUCING FUNDAMENTAL FABRICATING MACHINES (F-UNITS)". Wipo.int. Retrieved 2009-09-16.
- Szabo, Nick. "Macroscale Replicator". Archived from the original on 2006-03-07. Retrieved 2007-03-07.
- "4.11.3 Merkle-Freitas Hydrocarbon Molecular Assembler (2000-2003)". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- "5.9 Brief Mathematical Primer on Self-Replicating Systems". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- "5.1.9 Freitas-Merkle Map of the Kinematic Replicator Design Space (2003-2004)". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- "3.1 Moore Artificial Living Plants (1956)". Molecularassembler.com. Retrieved 2009-09-16.
- "5.11". Molecularassembler.com. Retrieved 2009-09-16.
- "5.5". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- Goldstein, Marc. "Worlds Vast and Various". SF Site Reviews.
- "Assemblers of Infinity". Retrieved 1 January 2016.
- "Absolution Gap (spoilers!)". Retrieved 17 January 2016.
- "Recursion". penguinrandomhouse.com. Retrieved 17 January 2016.
- Raets, Stefan. "Going through the Spin Cycle: Spin by Robert Charles Wilson". tor.com. Retrieved 17 January 2016.
- "Postsingular review by Peter". Retrieved 17 January 2016.
- "Postsingular review by Ben Babcock". Retrieved 17 January 2016.
- Kelly, Stuart. "The Hydrogen Sonata by Iain M Banks - review". The Guardian. Retrieved 17 January 2016.
- Anders, Charlie Jane. "The Most Messed Up Book About Robot Consciousness Ever". io9. Retrieved 17 January 2016.
- "vN by Madeline Ashby". angryrobotbooks.com. Retrieved 17 January 2016.
- Jones, Michael M. "Cracking the Failsafe: iD by Madeline Ashby". tor.com. Retrieved 17 January 2016.
- "Creations by William Mitchell". Goodreads. Retrieved 17 January 2016.
- "1". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- Lemos, Robert (2006-12-24). "Second life plagued by 'grey goo' attack". The Register. Retrieved 2009-12-28.
- Colin Milburn, "Atoms and Avatars: Virtual Worlds as Massively-Multiplayer Laboratories", Spontaneous Generations 2 (2008): 63–89.
- "3.7 Self-Replicating Automated Industrial Factory (1973-present)". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- "5.6 Closure Theory and Closure Engineering". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- "5.1.9.L Evolvability". Molecularassembler.com. 2005-08-01. Retrieved 2009-09-16.
- "5.11 Replicators and Public Safety". Molecularassembler.com. Retrieved 2009-09-16.
- Colvin, Fred H. (1947), Sixty Years with Men and Machines, New York and London: McGraw-Hill, LCCN 47003762. Available as a reprint from Lindsay Publications (ISBN 978-0-917914-86-7). Foreword by Ralph Flanders.
- The Clanking Replicator Project: Bootstrap your own self-replicating, rapid prototyping machine
- Cornell's Self-replicating machine
- The RepRap Project: Creating wealth without money...
- Terraforming Mars and Venus using machine self-replicating systems (SRS)
- Green Goo - Life In The Era Of Humane Genocide by Nick Szabo