Many nervous systems develop and form functional circuits through a long period of development involving a myriad of mechanisms. Some of these are determined by genes and molecules, while others depend on neural activity patterns. I will present how these diverse mechanisms work together to set up neural circuits shortly after an animal is born, enabling it to gradually acquire its cognitive and behavioral capabilities. I will focus on the visual system, and demonstrate how neural circuits became established to perform different computations. I will focus on some specific mechanisms such as inhibitory synaptic plasticity and the emergence of excitatory and inhibitory balance, and its role in the detection of novel stimuli.