Résumé : Comparing Galactic chemical evolution models to the observed elemental abundances in the Milky Way, we show that neutron star mergers can be a leading r-process site only if at low metallicities such mergers have very short delay times and significant ejecta masses that are facilitated by the masses of the compact objects. Namely, black hole-neutron star mergers, depending on the black hole spins, can play an important role in the early chemical enrichment of the Milky Way. We also show that none of the binary population synthesis models used in this Letter, i.e., COMPAS, StarTrack, Brussels, ComBinE, and BPASS, can currently reproduce the elemental abundance observations. The predictions are problematic not only for neutron star mergers, but also for Type Ia supernovae, which may point to shortcomings in binary evolution models.