Impairments in sexual function are common among breast cancer survivors (BCSs), particularly in BCSs receiving adjuvant endocrine therapy (AET). Whether these impairments cause distress, thus qualifying for a more clinically relevant diagnosis of sexual dysfunction (SD), is inadequately described among BCSs and represents an important research gap. Hence, the primary aim of this study was to estimate the prevalence of clinically relevant SD, in this context: impairments with associated distress, and to identify factors associated with SD among BCSs on AET. Secondly, to explore the extent of distress caused by specific impairments in sexual function. In this cross-sectional study of BCSs on adjuvant treatment with endocrine therapy for at least three months, participants completed an online survey comprising standardized measures of sexual and psychosocial function. Female Sexual Function Index (FSFI) and Sexual Complaint Screener – Women (SCS-W) were used to asses clinically relevant SD. Multiple regression analyses were performed to identify factors significantly associated with SD. In total, 333 BCSs with a mean age of 58.7 years were included in the study, of whom 227 were sexually active. Among sexually active BCSs, 134 (59%) met the criteria for having clinically relevant SD, of whom 78 (58%) perceived cancer treatment as the primary reason for their sexual problems. Factors associated with SD included vaginal dryness (adjusted OR= 2.25, 95% CI: 1.52–3.34, <i>p</i> p SD was highly prevalent among sexually active BCSs on AET. Sexual health is important to address independent of the woman’s age.