God gave this commandment to Moses:
“You shall not have other gods besides me.”
This statement and others like it seem to indicate that, at least for a time, Jews accepted that there were other gods. It would appear that at least until the time of Moses, and likely beyond, the Chosen People recognized that there were more gods, but that their God was the Supreme Lord of all. In other words, the gods that others worshiped may exist, but they don’t match up with ours.
That obviously changed over time, but many of the OT readings, such as the Psalm quoted, still reflect the old view.
The commandment, as well as other texts, can also be referring to the “worship of idols”, as has been noted. But whether they were real or imaginary, in the minds of some people they were still gods.
Keep in mind that in several early Christian groups, the existence of more than one god was accepted doctrine, as well. Gnosticism was one example.