Although genetic factors significantly contribute to variation in body-mass index (BMI), the effects appear to differ with age. To investigate this, we applied polygenic methods to longitudinal data from the Swedish Twin Registry where BMI was available for age categories ranging from 20–35 to above 80. Using GCTA, a polygenic method to estimate heritability, we showed that heritability explains around 20% of the variability in BMI across age categories. However, a polygenic risk score based on the largest GWAS of BMI (PRSBMI) explained 4–6% of the variation in BMI before 65, but less than 0.5% after age 65. This indicates that while there is substantial heritability of BMI across adulthood and late-life, the genetic variants identified in GWAS mainly predict BMI before age 65. Hence, more work is warranted studying the genetics of BMI in late-life, to better understand its biology and what distinguishes it from BMI earlier in adulthood.