And the reason the rest of us are staying home is because, even if the disease probably won’t kill us, we can still get it and spread it to the vulnerable people.
There are a lot of people in those categories you listed, and you can’t tell them all on sight. Yes, I hope they’re staying isolated even more fervently than less vulnerable people, but I certainly don’t want to be the one who made their chances any worse.
As for those rare cases of younger, healthier people dying … remember that that’s now, with most people doing their best to inhibit the spread of the virus… If we go back to “business as usual” too soon, even something with a tiny chance of happening will happen a lot more often due to the larger numbers of infected.