If the space in which the galaxies are embedded is expanding, any observer in any galaxy would perceive every other galaxy to be moving away from him. (In the balloon analogy, the two dimensional surface of the balloon represents the three-dimensional space of the universe.)
It may sound far fetched from a human viewpoint, but no more far-fetched than an infinite flat universe, and no more far fetched than a spherical earth might have sounded to our ancestors. If you set out in a straight line on a sphere from any point on it, you will eventually circumnavigate the sphere and return to your starting point. That applies to the earth. It would also, in a spherical universe, apply to the universe. But the circumference of the universe is a lot larger than earth.
George Gamow once wrote a book called 1, 2, 3… Infinity, which covered a varity of such things, including the expanding universe. It’s outdated now, but still pertinent enough to cover some of the basic concepts.
The universe, from a spatial standpoint, must be either infinite or finite. I believe that current data would show it to be finite, and if it’s finite, it has a geometry, probably spherical. If it turns out to be some other geometry, there’s nothing particular upsetting about that either, from a theological standpoint or a physical standpoint.