It is an assumption that the universe had a beginning, so you are not going to convince the atheist by this argument.
Actually, there are two possible versions of the argument.
One version, the Kalam, works if the universe did have a beginning. Big Bang cosmology supports this version.
Basically, the operating principle is that anything that comes into being or existence requires a cause because ex nihilo nihil fit - from nothing nothing comes. In other words, things don’t just “pop” into existence for no reason, without explanation and without proportionate cause. If we assume they do, then be prepared to allow that anything is possible any time.
If an atheist wants to allow that, then he is also prepared to allow anything is possible and science is merely a waste of time.
If the atheist is more reasonable, then he has to allow that if the universe came into existence 13.7 billion years ago according to Big Bang cosmology, then he has to accept a reasonable explanation for its coming into existence. Things like universes do not merely “pop” into existence without explanation and for no reason no matter how much Lawrence Krauss claims they do. Since matter, time, energy and space all came into existence and none of them preceded the existence of the universe, then none of these could be invoked to explain their own existence.
Things don’t bootstrap themselves from nothingness into existence. That “possibility” contravenes the Principle of Proportionate Causality where things that cause other things as effects must contain within themselves the “power” to cause those things as effects. Something that does not exist (matter, space, time and energy) cannot bring something else into existence because “nothing” has no causal power whatsoever. Something has to exist in order to causally bring about the existence of something else.
An atheist who wants to argue otherwise has basically admitted they believe in metaphysical nonsense which means there is no reason to take them seriously when they refuse to believe in proportionate causality - something immaterial with power to bring universes into existence must exist in order for an adequate explanation for the universe to be possible.
Sure, the atheist could be willing to admit nonsense (things can bootstrap themselves into existence,) but then we have no reason to take the atheist seriously.
Okay, so what if Big Bang Cosmology is wrong and the universe had no beginning?
This leads to cosmological argument #2, the Aquinas version. Recall, that Aquinas famously allowed that the universe could have existed through infinite time, so even if the atheist wants to insist the universe had no beginning then Aquinas’ version of the argument covers this eventuality.
In Aquinas’ version, the key premise is based on the principle of sufficient reason. Whatever exists must either have its reason for its existence in the nature of what it is (self-explanatory) or it requires its existence to be explained by some other cause which is sufficient to explain, in a logically adequate way, how it came to be. In other words, it is not sufficient to claim that the universe just has existed through infinite time as a brute fact, because if we accept that as sufficient to explain the entire universe, we ought also be prepared to accept that anything within the universe likewise requires no explanation but we can simply allow that everything and anything can just be for no reason, but just as brute fact. No explanation necessary.
Additionally, the logical problem of traversing actual infinites argues against the universe existing through infinite time, i.e., has “always” existed.
So basically, Aquinas “heads off” the atheist contention by arguing that even things that “simply exist” must have some explanation which sufficiently explains why they do. It infringes the logical principle of sufficient reason to resort to “no reason necessary” merely to avoid having to provide one. This applies to things which exist because they have been caused to exist by other things or things which might be self-explanatory. If things, “just exist” they must contain within them the reason why they do.
The atheist would be compelled to give a plausible explanation for how the universe can sufficiently explain its own existence before he is to be taken seriously. Given that the theist has a plausible explanation for how the universe could come into existence, the atheist would be compelled to give as plausible an explanation (or better) before he or his argument need be taken seriously.
To be clear, if the atheist wants to allow “it just has always existed” with regard to a reasonable explanation for the universe, he needs to be prepared to allow that same retort as an explanation for everything. If the universe needs no explanation, then neither does anything within the universe. The atheist has allowed a logical precedent and must take on the burden of having done so.
Science becomes a meaningless and unnecessary activity since the atheist will allow that the explanations that we currently do have for things are no more necessary, adequate or, indeed, explanatory than merely calling things “brute facts” without need of explanation.