The cerebellum, long recognized for its role in motor functions, has become increasingly acknowledged for its involvement in a broader spectrum of cognitive functions. Given the cerebellum’s strong neuronal connections with cerebral regions affected by neurodegenerative diseases, this study investigated the cerebellar contribution to cognitive decline in Alzheimer’s disease (AD). This study retrospectively analysed 127 individuals with subjective cognitive decline (SCD, n = 46), mild cognitive impairment due to AD (MCI, n = 24), and AD dementia (ADD, n = 57). Grey and white matter volumes of the whole cerebellum and its lobules were examined and correlated with cognitive scores. Significant grey matter atrophy was observed in posterior cerebellar regions (including crus II, lobule VIIa, VIIIa, VIIIb, and X), particularly between SCD and ADD, but not between MCI and ADD. Additionally, cerebellar volume positively correlated with cognitive function in SCD, MCI and ADD. These findings support the hypothesis that cerebellar atrophy is associated with cognitive symptoms of neurodegenerative diseases like AD.