Population sparseness determines strength of Hebbian plasticity for maximal memory lifetime in associative networks

by Naomi Auer, Lars Chen, Jakob Stubenrauch, Benjamin Lindner, Richard Kempter

The brain can efficiently learn and form memories based on limited exposure to stimuli, often even in single trials. Two key factors are believed to support this ability: large synaptic plasticity to strongly encode new memories; and sparse coding, leading to low overlap between memory representations and to small interference. Therefore, increased sparseness can also improve memory capacity. However, it is not well understood how the strength of plasticity of synapses affects capacity. Here, we analyze the combined impact of population sparseness and strength of plasticity on memory capacity. Specifically, we explore how the strength of plasticity that maximizes capacity depends on the sparseness of the neural code. To this end, we study a feedforward network with Hebbian and homeostatic plasticity and a two-state synapse model. The network learns to associate sparse binary input-output pattern pairs. The strength of plasticity is modeled as the probability of synaptic changes. Our results are based on both network simulations and an analytical theory, predicting the expected memory capacity in dependence on strength of plasticity and population sparseness. For both perfect and noisy input patterns, we find that the optimal strength of plasticity increases with increasing pattern sparseness and that this effect is more pronounced for input than for output sparseness. Interestingly, the optimal strength of plasticity remains the same across different network sizes if the number of active units in an input pattern is constant. While the memory capacity obtained at the optimal strength of plasticity increases monotonically with output sparseness, its dependence on input sparseness is non-monotonic. Overall, we provide the first detailed investigation of the interactions between population sparseness, strength of plasticity, and memory capacity. Our findings suggest that differences in sparseness between brain regions may underlie observed differences in how strongly these regions adapt and how quickly they learn.