Purpose: The purpose of this study was to identify key factors associated with severe stress in families raising a child with autism spectrum disorder (ASD). Methods: Questionnaires were mailed to families with one or more children with a diagnosis of ASD. Data from 543 surveys were analyzed using univariate and multivariate logistic regression. Results: Forty-four percent (n = 241) of the caregivers reported severe family stress related to raising a child with ASD. Severe family stress was associated with (1) reduced ability to socialize; (2) not having accessed individual therapy; (3) negative co-parent relationships; and (4) high out of pockets costs due to the child’s ASD. The specific ASD diagnosis, comorbid conditions, socio-demographic variables, and social support were not associated with severe family stress. Conclusion: The findings of the current study highlight the importance of a systemic approach to family stress, whereby individual, family, and ecological factors are investigated.