Team:USP-UNESP-Brazil/Associative Memory/Background

From 2012.igem.org

(Difference between revisions)
(Hopfield Associative Memory Networks)
 
(24 intermediate revisions not shown)
Line 1: Line 1:
{{:Team:USP-UNESP-Brazil/Templates/Header}}
{{:Team:USP-UNESP-Brazil/Templates/Header}}
-
===Hopfield Associative Memory Networks===
 
-
The idea of this project is based on the associative memory network introduced by J.J. Hopfield in the 80’s [http://en.wikipedia.org/wiki/Hopfield_network]. The structure of a Hopfield network is simple, all neurons are interconnected. This system has some interesting memory properties and provide a model for understanding human memory.
+
=Background=
-
<!--
+
===Hopfield Associative Memory Networks===
-
On this network, the system tends to converge to a pre-determined equilibrium, restoring the same pattern when exposed to variations of this pattern.
+
-
The architecture, or geometry of the system, is composed in a way that all neurons are connected among them. In math terms, a Hopfield Network can be represented as an “Energy” (E) function:
+
The idea of this project is based on the associative memory network introduced by J.J. Hopfield in the 80’s [http://en.wikipedia.org/wiki/Hopfield_network]. The structure of a Hopfield network is simple, all neurons are interconnected, and that brings about some interesting memory properties and provides a model for understanding human memory.
-
[[File:equation1.jpg|center|250px|caption|]]
+
We have chosen to built a Hopfield network because of its simplicity and robustness. The same methodology can be used to the construction of networks with different architectures, such as the called “perceptrons” [http://en.wikipedia.org/wiki/Perceptron]. In contrast to a Hopfield network, a perceptron is commonly used as a classifier.
-
+
-
Where “w” values are chosen such that the stored settings are the minima of the function “E”. The variable “x” is the state of the neuron “i”.
+
-
 
+
-
The state of a given neuron “I”(active or silent) can be mathematically  represented as follows: Given that “xi“ is the state of neuron, 1 if is activated or 0 if silent, and a neuron turns active if the sum of all received stimulus (exciting or inhibiting) is more than 0. Mathematically we can represent the state of the neuron xi as:
+
-
+
-
[[File:equation2.jpg|center|400px|caption|]]
+
-
 
+
-
In this equation, “wij” is the weight
+
-
 
+
-
Where "wij" is the weight assigned to the connection from neuron i to neuron j. The summation over j is the sum of all connections made by the neuron i. This dynamics (equation 2) is sufficient  for the network to converge the most similar memorized pattern.
+
-
 
+
-
The so called “learning” of a neural network consists on the choice of “w” weights. There are several ways  to choose them, what, actually, defines different learning methods
+
-
 
+
-
Set "i" and "j" such as the wheight "wij" is defined as:
+
-
 
+
-
[[File:equation3.png|center|400px|caption|]]
+
-
 
+
-
The Figure 1w shows the selection process of weights of connections between adjacent cells. To add more patterns, we have to sum the network of weights of the new pattern to the old network. (as shown in the Figure 2)
+
-
 
+
-
[[File:009.JPG|center|570px|caption|]]
+
-
"Figure 1"
+
-
-->
+
-
We have chosen to built a Hopfield network because of its simplicity and robustness. The same methodology can be used to the construction of networks with other architectures, such as the called “perceptrons” [http://en.wikipedia.org/wiki/Perceptron]. In contrast to a Hopfield network a perceptron is commonly used as a classifier and its structure is feed-forward.
+
-
 
+
-
 
+
-
<!-- One step forward is the way how to deal with continuous biological variables, because the standard model uses discrete ones.
+
-
[[File:0018.JPG|center|620px|caption|]]
+
-
"Figure 2"-->
+
===Biological Mechanism===
===Biological Mechanism===
-
In a biological neural network, the cells occupy a specific location and the information is addressed through a direct physical contact - the neuron axonal projections. In our case, a population of bacteria represents a single neuron and the information is addressed by a quorum sensing molecule (QSM). With different QSM, it is possible to address the information in a specific manner.
+
In a biological neural network, cells occupy a specific location and the information is passed through direct physical contact - the neuron axonal projections. In our case, a population of bacteria represents a single neuron and the information is transmitted by a quorum sensing molecule (QSM). Because of that, each "neuron" has its own QSM and the number of neurons is limitated by the number of different QSM. A comparison between a biological neural network and our design is presented in Fig 1.  
-
A comparison between a biological neural network and our design is presented in Fig 1.  
+
-
<!--
+
-
To replicate this in a cell system that moves constantly it would be also necessary some way to specifically address the information flow between the system’s components, otherwise it would not be possible to attach a meaning to a communication without specificity. Furthermore, is possible to verify the neuronal activity because of the fixed spatial position of the cells, which conserves the signal observation of a neuron in a specific local.
+
-
 
+
-
It is much more complicated to observe the dynamics of this cell communicating system using bacteria. Unlike static single cells communicating with each other (see figure 3), we aim to observe the communication between many genetically distinct bacterial populations. And these cells do not stop moving! In other words: there’s no specific point in space which can be always observed the same phenomenon.
+
-
-->
+
{{:Team:USP-UNESP-Brazil/Templates/RImage | image=Figura0020.jpg | caption=Fig 1. Comparison between a biological neural network and "bacterial neural network" | size=600px}}
{{:Team:USP-UNESP-Brazil/Templates/RImage | image=Figura0020.jpg | caption=Fig 1. Comparison between a biological neural network and "bacterial neural network" | size=600px}}
-
<!--
+
In order to evaluate if one population is active, we have designed the construction of a device that keeps each population of bacteria in a fixed position and enables the communication between different populations via QSM - Figure 2. This device can be built using a plate of 96 wells with membranes attached to their bottom. The membranes allow the diffusion of the quorum sensing substances but prevent the flux of bacterial populations.
-
To solve this information addressing problem, it would be necessary that each bacterial population (like neurons) communicate with themselves in a unique way so a receiving signal can be distinguished between populations. We chose the quorum sensing (QS) communication mechanism for this task, using different quorum systems for each “point” (population) in the network. With different QS molecules, it’s possible to build a communication system with unique signals like neurons in a network, where the information addressing specificity is present by the axonal ligation.  
+
-
Therefore, the influence of inhibition or activation that one point of the network can have under another in the bacterial system would be given by the particular meaning of each signal for each cell population – this point is where the memory programming of the system occurs which is the determination, for each cell population, the meaning of each communication signal of every point of the network.
+
{{:Team:USP-UNESP-Brazil/Templates/RImage | image=Physicalsystemforbacterialnetwork.png | caption=Fig 2. Device that will be used to measure the output. | size=600px}}
-
-->
+
-
<!--As a proof of concept we designed a communication between two types of bacteria. -->
 
-
 
-
Then, to solve the problems of the visualization of the signals’ activation, we planned the construction of a device to make 9 E.coli populations to communicate with each other but still keep their position in space. As an improvement to the example in figure 1, the device enables the communication not only of neighbors but also of all the populations, inhibiting or activating them according to the memory previously inserted in the network. This way, comparing to the communication between neighbors and of the whole system itself, generates a bigger resolution of the output in response to a similar input, once each position has better information about the pattern of activity of all the other positions. Figure 4 shows a construction of the system that can enable an efficient communication between the populations of different positions of the network, even if it is physically separating them.
 
-
 
-
[[File:Physicalsystemforbacterialnetwork.png|center|500px|caption|]]
 
-
 
-
 
-
The device can be easily constructed using a plate of 96 wells with membranes attached to the bottom. The membranes will allow the diffusion of the quorum sensing substances and prevent the flux of bacterial populations between the wells; avoiding, them the mixture of the activation signals (outputs).
 
-
 
====Genetic Construction====
====Genetic Construction====
-
Despite the solution we found to the specificity of the communication, another problem appears when we try to genetically build the bacterial populations: there are not enough quorum sensing systems to create 9 bacterial populations with different ones. In the Registry of Parts there are 4 quorum sensing systems well characterized, and there is a strong activation crosstalk between two of them (Las and Rhl), this fact prevent us from using them, therefore, we end up with 3 systems of quorum sensing that can be used.
+
Each population of bacteria ("neuron") is defined by the way it interacts within the network and by its own QSM. Hence, the number of "neurons" is limited to the number of QSM. As a proof of concept we designed two populations of bacteria that communicate in a repressive manner. In order to make the network visually interesting, we used our 3x3 wells device and designed the population network to recognize two patterns - Figure 3. Since they are complementary, only two different population of bacteria are needed to represent the patterns "X" and "O". In this case, each population placed at the letter "X" inhibits all the ones placed at the letter “O” and activates the positions of its own pattern (and vice-versa).
-
 
+
-
Initially, we intended,only as a concept proof - to create a network associative memory for discrimination activation patterns between the "L" and "T", similar to the one presented in Figure 2. As these memories require that all points of the network have a communication singular, we searched for a case in particular that demanded that memories would require only a small number of quorum sensing substances.
+
-
Asymmetrical memory would be the answer: since only two quorum sensing systems are necessary to accomplish the same recognition memory task as an incomplete memory, similarly to the example showed in Figure 2, the "X" and "O" memories. In these patterns there is no intersection between "letter X" and letter"O”, which greatly simplifies the configuration of communication weights between positions (see Figure 5). Because each position of the “X” inhibits all positions of “O” and activates all positions of its own pattern (and vise-versa), there is no need to distinguish between the kind of signal (quorum sense substance) that each position of the “X” emits. The same works for all positions of the “O” pattern. Still, the positions of each pattern can be programmed with the same genetic code, which allow us to use only two systems of quorum sensing to create a network of associative memory that is capable of differentiate between two memories, given a particular input that would be interpreted as an incomplete memory.
+
<!--.-->
-
[[File:0019.JPG|center|500px|caption|]]
+
{{:Team:USP-UNESP-Brazil/Templates/RImage | image=0019.JPG | caption=Fig. 3. Representation of the input and output in the 3x3 wells device. | size=600px}}
 +
In the Registry of Parts [http://partsregistry.org/Main_Page] there are four well characterized quorum sensing systems. However, there is a strong activation crosstalk between two of them (Las and Rhl). Therefore, we decided to use the system Cin and Rhl.
-
We decided to use two of the four available quorum sensing systems that have substances different enough for the bacterial communication so there would be little cross-talk between them. The quorum sensing systems chosen were Cin and Rhl[1,16].
+
In our system, the QSM signals trigger the transcription of an activator (or an inhibitor) of the transcription of GFP - this is our system activation reporter. Simultaneous inhibitions and activations of a bacterial population will be converted to a molecular competition of activators and inhibitors by the promoter that controls the production of GFP. It is this molecular competition who "chooses" the pattern stored in the system that is most similar to the input.
-
To convert the signals in activation or inhibition, we created a system of transduction of the quorum sensing signal to transcription of an activator or an inhibitor of the transcription of GFP, this will be our system activation reporter. Simultaneous inhibitions and activations of a bacterial population will be converted to a molecular competition of activators and inhibitors by the promoter that controls the production of GFP. It is this molecular competition that promotes the decision between the memories of the communication systems, associating a given input with a more similar memory. As an example, if an input activates more positions of the “X” pattern than the “O”, the competition in the pattern “X” positions will be more favorable to its activation due to the greater number of activators produced by the activated positions, while in the positions of the “O” pattern the opposite occurs, because of its small number of positions activated initially by the given input.
+
As an example, if an input activates more positions at the “X” pattern than those at the “O”, a greater number of "X-activators" will be produced, and it is more likely that other "X" positions become activated, due to this competition for promoters at each position. Meanwhile, in the “O” pattern positions, the opposite occurs: a lesser number of initially activated positions implies less "O-activators", and the outcome is that "X wins over O" - the network reproduces the "X" pattern.
-
The promoter multi-regulated by an activator and an inhibitor is called Prm. Its inhibitor is the transcriptional factor cl434 and its activator is the cl factor. The genetic design of the positions of the patterns “X” and “O” can be seen in Figure 6.
+
This multi-regulated promoter, with an activator and an inhibitor, is called Prm. Its inhibitor is the transcriptional factor cl434 and its activator is the cl factor. The genetic design of the positions of the patterns “X” and “O” can be seen in Figure 4.
-
The construction using the signal transduction containing cl434 and cl allows creating different systems of associative memory, limited only by the quantity of quorum sensingsystems available. Figure 6 shows how this generic system would work and elucidates how this system could be applied to different functions involving a genetic control.
+
This construction that uses the signal transduction containing cl434 and cl allows to create different systems of associative memory, limited only by the number of quorum sensing systems available. Figure 4 shows how this generic system would work and elucidates how this system could be applied to different functions involving genetic control.
-
[[File:0022.png|center|600px|caption|]]
+
<!--[[File:0022.png|center|600px|caption=Fig. 4|]] -->
 +
{{:Team:USP-UNESP-Brazil/Templates/RImage | image=0022.png | caption=Fig. 4. Genetic construction. | size=600px}}
{{:Team:USP-UNESP-Brazil/Templates/Foot}}
{{:Team:USP-UNESP-Brazil/Templates/Foot}}

Latest revision as of 03:47, 27 September 2012