The medial amygdala (MeA) plays a critical role in processing species- and sex-specific signals that trigger social and defensive behaviors. However, the principles by which this deep brain structure encodes social information is poorly understood. We used a miniature microscope to image the Ca2+dynamics of large neural ensembles in awake behaving mice and tracked the responses of MeA neurons over several months. These recordings revealed spatially intermingled subsets of MeA neurons with distinct temporal dynamics. The encoding of social information in the MeA differed between males and females and relied on information from both individual cells and neuronal populations. By performing long-term Ca2+ imaging across different social contexts, we found that sexual experience triggers lasting and sex-specific changes in MeA activity, which, in males, involve signaling by oxytocin. These findings reveal basic principles underlying the brain’s representation of social information and its modulation by intrinsic and extrinsic factors.