Environmental stresses such as extreme temperatures, dehydration and food deprivation may have distinct consequences for different age-classes and for males and females across species. Here we investigate a natural population of the model organism Drosophila melanogaster. Males and females at ages 3, 19 and 35 days were tested for stress resistance; i.e. the ability of flies to cope with starvation and both cold and hot temperatures. Further, we tested a measure of metabolic efficiency, namely mitochondrial DNA copy number (mtDNA CN) in both sexes at all three age-classes. We hypothesize that stress resistance is reduced at old age and more so in males, and that mtDNA CN is a biomarker for sex- and age-dependent reductions in the ability to cope with harsh environments. We show that: (1) males exhibit reduced starvation tolerance at old age, whereas older females are better in coping with periods without food compared to younger females, (2) heat tolerance decreases with increasing age in males but not in females, (3) cold tolerance is reduced at old age in both sexes, and (4) old males have reduced mtDNA CN whereas mtDNA CN slightly increases with age in females. In conclusion, our data provide strong evidence for trait and sex specific consequences of aging with females generally being better at coping with environmental stress at old age. The reduced mtDNA CN in old males suggests reduced metabolic efficiency and this may partly explain why males are less stress tolerant at old age than females. We suggest that mtDNA CN might be a suitable biomarker for physiological robustness. Our findings likely extend to other taxa than Drosophila and therefore we discuss the observations in relation to aging and sex specific lifespan across species.