Sparse distributed memory
Sparse distributed memory is a mathematical model of human long-term memory introduced by Pentti Kanerva in 1988. It is used for storing and retrieving large amounts ( bits) of information without focusing on the accuracy of the information. It uses patterns to serve as memory addresses, where information is retrieved based on similarities between addresses. Memory addresses are all in a list even if they are not related, and are retrieved based on similar content.
The general formula is where n is the number of dimensions of the space, and is the number of feasible memory items.
The critical distance of a Sparse Distributed Memory can be approximately evaluated minimizing the following equation with the restriction and . The proof can be found in 
- : be the distance to the target;
- : be the number of hard-locations activated during read and write operations (this value depends on access radius values);
- : be the number of total stored bitstrings in memory;
- : be the number of hard-locations in memory;
- : be the number of times the target bitstring was written in memory;
- : be the total of random bitstrings in all hard-locations activated by a read operation;
- : be the mean number of shared hard-locations activated by two bitstrings bits away from each other. One can find some values for a 1000-dimensional SDM in Kanerva's book, Table 7.1, p. 63, or the equations to calculate to any SDM in Appendix B, p. 125 of the same book.
Sparse distributed memory is a mathematical representation of human memory, and uses high-dimensional space to help model the large amounts of memory that mimics that of the human neural network. It utilizes the Hamming distance to measure mismatched bits and read back data between the original write address and one near it. Human memory has a tendency to congregate memories based on similarities between them(although they may not be related), such as "firetrucks are red and apples are red".
Neurons are electrically excitable cells that transmit information to and from the brain. They are used as models for sending and retrieving data in a sparse distributed memory system. Neurons are the cells that recall and send information in a memory system.
Memory inside of a computer is random-access memory(RAM), contrary to sequential access memory. All of the items in a single list, or array, are stored in RAM. The computer has address decoders, similar to the way neurons work in the brain, and return items from the array that match or are similar. Each address in an array points to an individual line in the memory. That line is then returned if it is similar to other lines.
Sparse distributed memory is based on pulling in patterns between different addresses.
Imagine each line as a different memory address, an example from Kanerva's book:
- "Why are fire engines painted red?
- Firemen's suspenders are red, too.
- Two and two are four.
- Four times three is twelve.
- Twelve inches in a foot.
- A foot is a ruler.
- Queen Mary is a ruler.
- Queen Mary sailed the sea.
- The sea has sharks.
- Sharks have fins.
- The Russians conquered the Finns.
- The Russians' color is red.
- Fire engines are always rushin'.
- So that's why they're painted red!"
As a result, all of these addresses are returned to the user, although these may not be the only addresses in that list.
At the University of Memphis, Uma Ramamurthy, Sidney K. D’Mello, and Stan Franklin created a modified version of the sparse distributed memory system that represents "realizing forgetting." It uses a decay equation to better show interference in data. The sparse distributed memory system distributes each pattern into approximately one hundredth of the locations,[clarification needed] so interference can have detrimental results.
Two possible examples of decay from this modified sparse distributed memory are presented
Exponential decay mechanism:
Negated-translated sigmoid decay mechanism:
In the exponential decay function, it approaches zero more quickly as x increases, and a is a constant(usually between 3-9) and c is a counter. For the negated-translated sigmoid function, the decay is similar to the exponential decay function when a is greater than 4.
As the graph approaches 0, it represents how the memory is being forgotten using decay mechanisms.
LIDA uses sparse distributed memory to help model cognition in biological systems. The sparse distributed memory places space is recalling or recognizing the object that it has in relation to other objects. It was developed by Stan Franklin, the creator of the "realizing forgetting" modified sparse distributed memory system.
- Implementation of Sparse Distributed Memory in Github: This implementation has been used for research in Fundação Getúlio Vargas headed by Ph.D. Alexandre Linhares. It was first developed to study Critical Distance and it was tested checking with Kanerva's book theoretical results.
- Kanerva, Pentti (1988). Sparse Distributed Memory. The MIT Press. ISBN 978-0-262-11132-4.
- Brogliato, Marcelo Salhab (2012). Understanding Critical Distance in Sparse Distributed Memory (Thesis).
- Pentti Kanerva (1993). "Sparse Distributed Memory and Related Models". Pennsylvania State University. CiteSeerX: 10.1.1.2.8403.
- M. J. Flynn, P. Kanerva, and N. Bhadkamkar (December 1989). "Sparse Distributed Memory: Principles and Operation". Stanford University. Retrieved 1 November 2011.
- C. George Boeree (2002). "General Psychology". Shippensburg University.
- Mastin, Luke. "NEURONS & SYNAPSES". Retrieved 10 November 2011.
- Uma Ramamurthy, Sidney K. D’Mello, Stan Franklin. "Realizing Forgetting in a Modified Sparse Distributed Memory System". Computer Science Department and The Institute for Intelligent Systems. The University of Memphis. pp. 1992–1997. Archived from the original on 2006. Retrieved 1 November 2011.
- Rocha LM, Hordijk W (2005). "Material representations: From the genetic code to the evolution of cellular automata". Artificial Life 11 (1-2): 189–214. doi:10.1162/1064546053278964. PMID 15811227.
- Rao, R. P. N., & Fuentes, O. (1998). Hierarchical Learning of Navigational Behaviors in an Autonomous Robot using a Predictive Sparse Distributed Memory. Machine Learning, 31, 87-113