Generally in neural systems of animals, information about behavioural variables such as sensory signals or motor actions are carried by the activity of populations of neurons. Signal decoding of neural system helps researchers to understand brain's functions. Research about hippocampus area of rat brain shows that it is possible to decode position from spiking activity of this area. Recently, with the emergence of deep learning techniques, deep neural networks (D) provide powerful modeling capabilities and achieve state-of-the-art results. Long short-term memory (LSTM), a type of deep neural network, can capture long-range dependencies and nonlinear dynamics, and is widely used in modeling complex dynamical signals like speech. In this research, we propose long LSTM network topologies for decoding 2D movement trajectory of a rat using the neural activities recorded from an ensemble of hippocampal place cells. Despite of wide utilization of D, reliability of result in these modeling approach is not completely understood. To analyze the behavior and accuracy of the networks, we compared the performance of these topologies with point-process filter solution which is used widely in these experiments. The result of the first proposed topology in this thesis, even though it has sufficient decoding accuracy, shows undesired behaviors which are unnatural in rat's movement (like jumps). The second proposed topology restricts output to produce undesired jumps by embedding information of the maze shape. Finally, in the third proposed topology adding information of rat velocity to the model, improves dynamics of rat's movement in decoding. Since the rat's position distribution is not uniform, in training session the cost function is weighted by absolute value of the velocity. We measured least absolute error (LAE) and root mean square error (RMSE) to compare the performance of these methods. LAE metric for LSTM and point-process filter are 8.7 and 6.8, respectively. RMSE metric is 10.3 and 8.86 for LSTM and point-process filter, respectively. We showed that the LSTM topologies and point-process filter provide comparable accuracy in estimating the position. In addition, both the LSTM model and the point-process model can encode the receptive field for each place cell. The LSTM runs 16 times faster than the point-process filter in this research, providing a strong advantage in computational efficiency.
