It shouldn't be a problem so long as the other babies don't have hair yet.
I've found if you have newborns born to a nest full of fuzzies or larger, you'll end up with a litter full of starving newborns in a day or so. They're just not strong enough to compete for the nozzles.
Usually when that happens, I try to find a use for the older ones, or move the entire litter of newborns to a colony with babies of around the same size. Most momma mice don't care whose babies they are, just so long as they're babies.