Computational Neuroscience

\documentclass[11pt,twocolumn]{article}

\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{amsmath,amssymb}
\usepackage{graphicx}
\usepackage{booktabs}
\usepackage{hyperref}
\usepackage{xcolor}
\usepackage[margin=0.75in]{geometry}
\usepackage{natbib}
\usepackage{siunitx}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage{subcaption}
\usepackage{tikz}
\usetikzlibrary{shapes,arrows,positioning}

\title{Deep Learning Models for Decoding Neural Population Activity\\in Primary Visual Cortex}
\author{%
  First Author\textsuperscript{1}, Second Author\textsuperscript{2,3}, Third Author\textsuperscript{1}\\[0.5em]
  \small\textsuperscript{1}Computational Neuroscience Center, University\\
  \small\textsuperscript{2}Department of Neurobiology, Medical School\\
  \small\textsuperscript{3}AI Research Institute
}
\date{}

\begin{document}

\maketitle

\begin{abstract}
Understanding how the brain encodes sensory information remains a fundamental challenge in neuroscience. We present a novel deep learning architecture for decoding visual stimuli from neural population activity recorded in macaque primary visual cortex (V1). Our model, termed NeuroTransformer, combines convolutional feature extraction with self-attention mechanisms to capture both spatial and temporal dynamics of neural responses. Trained on 96-electrode Utah array recordings, NeuroTransformer achieves 94.3\% accuracy in classifying natural images, outperforming traditional linear decoders by 31\% and recurrent neural networks by 12\%. Analysis of learned representations reveals that the model discovers biologically plausible receptive field structures without explicit supervision.
\end{abstract}

\section{Introduction}

The primary visual cortex (V1) performs sophisticated computations that transform retinal inputs into neural representations suitable for higher-level visual processing \citep{hubel1962receptive}. Population-level recordings now enable us to probe these transformations at scale.

\subsection{Neural Decoding}

Neural decoding refers to reconstructing stimulus information from neural activity:
\begin{equation}
  \hat{s} = f(\mathbf{r})
\end{equation}
where $\mathbf{r} \in \mathbb{R}^{N \times T}$ represents the activity of $N$ neurons over $T$ time bins.

\subsection{Contributions}

\begin{enumerate}
  \item Novel transformer architecture for neural decoding
  \item State-of-the-art performance on V1 population recordings
  \item Interpretable attention maps revealing neural coding principles
\end{enumerate}

\section{Methods}

\subsection{Neural Recordings}

We recorded from two macaque monkeys using 96-channel Utah arrays implanted in V1. Experimental parameters:

\begin{itemize}
  \item Sampling rate: \SI{30}{\kilo\hertz}
  \item Spike sorting: Kilosort 2.5
  \item Stimulus set: 1,000 natural images from ImageNet
  \item Trial structure: 250 ms fixation, 500 ms stimulus, 250 ms delay
\end{itemize}

\subsection{Data Preprocessing}

Spike trains were binned into \SI{10}{\milli\second} windows and z-scored:
\begin{equation}
  z_{n,t} = \frac{r_{n,t} - \mu_n}{\sigma_n}
\end{equation}

\subsection{Model Architecture}

The NeuroTransformer consists of three main components:

\subsubsection{Temporal Convolution Block}

1D convolutions extract local temporal features:
\begin{equation}
  \mathbf{h}^{(1)} = \text{ReLU}(\text{Conv1D}(\mathbf{r}; \mathbf{W}_c))
\end{equation}

\subsubsection{Self-Attention Layers}

Multi-head attention captures long-range dependencies:
\begin{equation}
  \text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^\top}{\sqrt{d_k}}\right)V
\end{equation}

\subsubsection{Classification Head}

Global average pooling followed by fully-connected layers:
\begin{equation}
  \hat{y} = \text{softmax}(\mathbf{W}_o \cdot \text{GAP}(\mathbf{h}^{(L)}) + \mathbf{b}_o)
\end{equation}

\subsection{Training Details}

\begin{table}[h]
\centering
\caption{Hyperparameters}
\begin{tabular}{@{}ll@{}}
\toprule
Parameter & Value \\
\midrule
Learning rate & \num{1e-4} \\
Batch size & 64 \\
Attention heads & 8 \\
Hidden dimension & 256 \\
Dropout & 0.1 \\
Weight decay & \num{1e-5} \\
\bottomrule
\end{tabular}
\end{table}

Loss function: cross-entropy with label smoothing ($\epsilon = 0.1$).

\section{Results}

\subsection{Decoding Performance}

Table~\ref{tab:results} compares decoder performance.

\begin{table}[h]
\centering
\caption{Classification accuracy (\%) on held-out test set.}
\label{tab:results}
\begin{tabular}{@{}lcc@{}}
\toprule
\textbf{Model} & \textbf{Monkey J} & \textbf{Monkey K} \\
\midrule
Linear (ridge) & 61.2 ± 2.1 & 65.4 ± 1.8 \\
SVM (RBF) & 67.8 ± 1.9 & 71.2 ± 1.6 \\
LSTM & 79.1 ± 1.4 & 83.5 ± 1.2 \\
GRU & 80.3 ± 1.3 & 84.1 ± 1.1 \\
Transformer & 88.9 ± 0.9 & 91.2 ± 0.8 \\
\textbf{NeuroTransformer} & \textbf{92.1 ± 0.7} & \textbf{94.3 ± 0.6} \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Temporal Dynamics}

Decoding accuracy increases with stimulus duration (Figure~\ref{fig:temporal}).

\begin{figure}[h]
\centering
\fbox{\parbox{0.9\columnwidth}{\centering\vspace{2cm}[Temporal Decoding Curve]\vspace{2cm}}}
\caption{Decoding accuracy as a function of time from stimulus onset. Shaded region: SEM across sessions.}
\label{fig:temporal}
\end{figure}

\subsection{Attention Analysis}

We visualized attention weights to understand which neurons and time points contribute most to decoding.

Key findings:
\begin{itemize}
  \item Peak attention at 80-120 ms post-stimulus
  \item Spatial clustering of attended neurons
  \item Selective enhancement for orientation-tuned units
\end{itemize}

\subsection{Learned Representations}

t-SNE visualization of hidden layer activations reveals:
\begin{enumerate}
  \item Clear clustering by image category
  \item Hierarchical organization similar to IT cortex
  \item Smooth representation of low-level features
\end{enumerate}

\section{Discussion}

\subsection{Biological Plausibility}

The attention mechanism parallels proposed neural gain modulation:
\begin{equation}
  r_i' = g(\mathbf{a}) \cdot r_i
\end{equation}
where $g(\mathbf{a})$ is a context-dependent gain factor.

\subsection{Comparison to Brain-Computer Interfaces}

Our approach could improve BCIs by:
\begin{itemize}
  \item Reducing calibration time
  \item Enabling transfer across sessions
  \item Providing interpretable decoding
\end{itemize}

\subsection{Limitations}

\begin{enumerate}
  \item Limited to V1; generalization to other areas untested
  \item Requires large training datasets
  \item Computational cost for real-time applications
\end{enumerate}

\section{Conclusion}

NeuroTransformer achieves state-of-the-art neural decoding performance while providing interpretable insights into V1 population coding. Our results demonstrate the power of deep learning for understanding neural computation.

\section*{Code Availability}

Code available at: \url{https://github.com/example/neurotransformer}

\section*{Acknowledgments}

We thank the animal care staff and the computing center.

\bibliographystyle{plainnat}
\bibliography{references}

\end{document}