I would argue that you do not actually morally agree on things nearly as much as you might think. For those who believe in God (Jews, Christians, Muslims) morality flows from God and, thus, is not alterable by circumstance. Those who do not believe in God have morals founded in reason, practice, experience and what they perceive to be good.
When morality comes from God, it is absolute, unchanging and not subject to the whims of man’s desires. All other forms of morality are unable to be tethered to something so firm. They shift as needed to fit a person’s changing life experience.
Life is precious to someone who believes in God because it comes from God. All life, regardless of intellect, physical perfection, circumstance or likely outcome. Is that true of atheists?
I’m not saying a person who does not believe in God cannot be a “good” person, but their morality is completely centered in their own conceptions of right and wrong, which can change over time. There are no absolutes.
This means that the hardest choices that you might ever have to make will be made by people with two different starting points for their concept of morality. Consider it carefully.