Good point. First we must understand that there are two punishments in hell as the Church teaches. The first and primary punishment is the deprivation from the beatific vision of God in heaven, aka pain of loss. For infants that die without baptism, they never had the capacity to know God. If someone never had the capacity to know God, how could they suffer from loosing something they never knew existed? These souls would not gain some perfect or complete knowledge after death but would stay in their ignorant state and in such a state would not suffer from the pain of loss.
The second punishment is the fires of hell, for which souls only suffer for committing actual sins. Since unbaptized infants never commit any actual sins, they would not suffer any of the fires in hell. Therefore, there must be some other place, and this place must also be in hell but it would have to be a part of hell on its outskirts or edge where souls did not suffer. This same place would be the same where the OT just went after death before the coming of Christ when he opened the gates of heaven. This state is known as the limbo of the fathers, a place of perfect natural happiness that is not heaven but also is not in the pits of hell but is rather on the outer edge of hell. Hope this helps see how it fits in with Catholic theology. Hell is a place of fire, etc, but there are degrees of punishments. It would make sense that at its highest level, such a place would exist, as a just and merciful God would require such a place’s existence, since the same God has revealed that souls cannot enter heaven without being born again through baptism.