The striatum has been shown to play a critical role in reward prediction. It is composed of two neurochemically and anatomically distinct compartments known as the striosomes and the matrix. The striosomes comprise only about 15% of the striatum by volume and are distributed mosaically therein. Accordingly, it has been difficult to identify striosomal neurons in electrophysiological recordings and it has been unclear whether striosomal neurons, which project to midbrain dopaminergic neurons, engage in reward prediction. In this study, we utilized a mouse line (Sepw1-NP67) selectively expressing Cre in striosomal neurons, combined with endoscopic in vivo calcium imaging to selectively record activities of striosomal neurons during an odor-conditioning task. As mice learned the task, striosomal neurons in the dorsomedial striatum showed predictive activities to odor cues that were associated with water rewards or aversive air puffs. These activities were proportional to the expected reward or air-puff intensity. Intriguingly, repeated recordings of the same striosomal neurons over a period of weeks revealed that predictive activities were learning-stage specific. That is, these activities disappeared after continuous training. Furthermore, presentations of rewards or air puffs activated some striosomal neurons. These findings suggest that the striosomes participate in reward prediction with learning stage-specific neural ensembles, and that they also send reward and aversive signals to dopaminergic neurons.