Before the theory of the Bang was introduced (by a Catholic priest, by the way, Monseigneur Georges Lemaître) the prevalent theory was “steady state” - that the universe had always existed. This idea is compatible with most pagan religions, in which the gods formed the world from a pre-existing primordial matter.
The Judeo-Christian religion was unique in that it taught that God created the universe from nothing. When Msgr. Lemaître proposed the Bang, many physicists initially scoffed at the idea, thinking it was trying to put a scientific spin on a theological idea of creation. Today, no reputable scientist disputes the Bang. They dispute what happened in the first umpety-umpth of a second after the Bang, but nobody disputes the Bang itself.
Back in the 70’s many physicists had this idea that the Bang was a cyclical event. We know the universe is expanding, but the idea was that gravity would eventually pull everything back together and the Bang would happen all over again, ad infinitum. This would establish an infinite progression of Bangs, and maybe that could be extrapolated to an infinite regression, not requiring a First Cause. But nobody believes this today - Edwin Hubble proved (long before the 70’s) that the universe reached escape velocity at the Bang and would never coalesce. The Bang is a one-time event, and this is now settled science.
It helps to understand the basic idea of the Bang. The universe did not come from an object the size of a golf ball, or a marble, or a pea, or a grain of sand. The universe came from a zero-dimensional singularity. It had ZERO mass (modern science believes that matter acquired mass shortly after the Bang thanks to the Higgs field). It had no size and occupied no space and had no mass. In fact, space itself did not exist - space is created as matter moves into it.
When ancient people say God created the universe from nothing, a zero-dimensional singularity could fit that description, to the best of their ability to understand. It’s not matter - which, by definition, has mass and occupies space. The initial singularity did neither.
But what caused the Bang? It was an event that took place some 14.3 billion years ago. There is no scientific reason to believe anything like it ever happened before, or will ever happen again.
No scientist can say what caused the Bang. But there are only two logical possibilities: either the universe created itself, or it was created by an external entity which exists beyond space and time (which could be rightly called “God”).
Obviously, an atheist prefers the first idea - that the universe created itself. Which is really silly, if you think about it. What OTHER thing can you think of that causes itself to exist? And there’s a logical problem with this idea - for something to create itself, it must simultaneously be in a state of possibility and actuality, but these states are mutually exclusive.
And there you have it.