The COVID-19 pandemic has highlighted a debate about whether marginalized communities suffered the disproportionate brunt of the pandemic's mortality. Empirical studies addressing this question typically suffer from statistical uncertainties and potential biases associated with uneven and incomplete reporting. We use geo-coded micro-level data for the entire population of Sweden to analyze how local neighborhood characteristics affect the likelihood of dying with COVID-19 at individual level, given the individual's overall risk of death. We control for several individual and regional characteristics to compare the results in specific communities to overall death patterns in Sweden during 2020. When accounting for the probability to die of any cause, we find that individuals residing in socioeconomically disadvantaged neighborhoods were not more likely to die with COVID-19 than individuals residing elsewhere. Importantly, we do find that individuals show a generally higher probability of death in these neighborhoods. Nevertheless, ethnicity is an important explanatory factor for COVID-19 deaths for foreign-born individuals, especially from East Africa, who are more likely to pass away regardless of residential neighborhood.