Team:USP-UNESP-Brazil/Associative Memory/Background

From 2012.igem.org

(Difference between revisions)
(Biological Mechanism)
(Hopfield Associative Memory Networks)
Line 2: Line 2:
===Hopfield Associative Memory Networks===
===Hopfield Associative Memory Networks===
-
The idea of this project is based on the associative memory network introduced by J.J. Hopfield in the 80’s [http://en.wikipedia.org/wiki/Hopfield_network]. The structure of a Hopfield network is simple, all neurons are interconnected. This system has some interesting memory properties and provide a model for understanding human memory.  
+
The idea of this project is based on the associative memory network introduced by J.J. Hopfield in the 80’s [http://en.wikipedia.org/wiki/Hopfield_network]. The structure of a Hopfield network is simple, all neurons are interconnected, what brings some interesting memory properties and provide a model for understanding human memory.  
-
<!--
+
We have chosen to built a Hopfield network because of its simplicity and robustness. The same methodology can be used to the construction of networks with different architectures, such as the called “perceptrons” [http://en.wikipedia.org/wiki/Perceptron]. In contrast to a Hopfield network, a perceptron is commonly used as a classifier and its structure is feed-forward.
-
On this network, the system tends to converge to a pre-determined equilibrium, restoring the same pattern when exposed to variations of this pattern.
+
-
 
+
-
The architecture, or geometry of the system, is composed in a way that all neurons are connected among them. In math terms, a Hopfield Network can be represented as an “Energy” (E) function:
+
-
 
+
-
[[File:equation1.jpg|center|250px|caption|]]
+
-
+
-
Where “w” values are chosen such that the stored settings are the minima of the function “E”. The variable “x” is the state of the neuron “i”.
+
-
 
+
-
The state of a given neuron “I”(active or silent) can be mathematically  represented as follows: Given that “xi“ is the state of neuron, 1 if is activated or 0 if silent, and a neuron turns active if the sum of all received stimulus (exciting or inhibiting) is more than 0. Mathematically we can represent the state of the neuron xi as:
+
-
+
-
[[File:equation2.jpg|center|400px|caption|]]
+
-
 
+
-
In this equation, “wij” is the weight
+
-
 
+
-
Where "wij" is the weight assigned to the connection from neuron i to neuron j. The summation over j is the sum of all connections made by the neuron i. This dynamics (equation 2) is sufficient  for the network to converge the most similar memorized pattern.
+
-
 
+
-
The so called “learning” of a neural network consists on the choice of “w” weights. There are several ways  to choose them, what, actually, defines different learning methods
+
-
 
+
-
Set "i" and "j" such as the wheight "wij" is defined as:
+
-
 
+
-
[[File:equation3.png|center|400px|caption|]]
+
-
 
+
-
The Figure 1w shows the selection process of weights of connections between adjacent cells. To add more patterns, we have to sum the network of weights of the new pattern to the old network. (as shown in the Figure 2)
+
-
 
+
-
[[File:009.JPG|center|570px|caption|]]
+
-
"Figure 1"
+
-
-->
+
-
We have chosen to built a Hopfield network because of its simplicity and robustness. The same methodology can be used to the construction of networks with other architectures, such as the called “perceptrons” [http://en.wikipedia.org/wiki/Perceptron]. In contrast to a Hopfield network a perceptron is commonly used as a classifier and its structure is feed-forward.
+
-
 
+
-
 
+
-
<!-- One step forward is the way how to deal with continuous biological variables, because the standard model uses discrete ones.
+
-
[[File:0018.JPG|center|620px|caption|]]
+
-
"Figure 2" -->
+
===Biological Mechanism===
===Biological Mechanism===

Revision as of 14:18, 26 September 2012