In reference to heaven, yes and no. The Hebrew word for sky does apply to heaven, but there were also beliefs in multiple heavens or levels of heaven. Some believed in three, others said seven. See 2 Corinthians 12., where Paul makes an explicit reference to third heaven.
As far as the tower of babel… we also have airplanes. Their ability to go up into the sky far higher than any building could should be enough to show you that literal height is not the issue.
The ascensions of Christ, Enoch, Elijah and, possibly, Mary were also miracles. I do not think space time is a problem for God. Furthermore, we must wonder if heaven heaven is a matter of height, or a matter of another dimension or realm.
The Bible also uses poetry and prose to get some of its messages across. It is also written in a cultural context… There is no reason to be a literalness on such issues.
Dear brother, I love you in the Spirit of our lord Jesus Christ. We are brothers and I encourage you to continue seeking the Lord. However, I must also admonish you. I ask that you would thoroughly and sincerely seek out these matters, because you are making our faith look foolish. Do not be deceived by the world, but do not be deluded by a biased, unexamined view of the Scriptures.