I think there was probably an evolution of understanding among the Jews over time. Obviously the other nations around them worshiped other gods, real or imagined. They spent time in Egypt, where the Egyptians worshiped many gods. Throughout much of the Old Testament the Jews are tempted to (and sometimes do) worship other gods. When Moses disappears on the mountain for too long, they begin to grumble and lose heart, so Aaron makes a god for them (“I tossed the gold into the fire, and shazam, there was this golden calf!”) In the conquest of Canaan, the Israelites are forbidden to intermarry with the other nations, or even to keep the golden “booty” taken from the temples, as this might tempt them to go off and worship the gods of these nations, rather than the God of Israel. Early on there is definitely a sense among the Israelites that maybe there is something to these other gods of the other nations. Especially when things aren’t going so well, they have a tendency to abandon their God and worship one of these others, in case maybe that works out better. Things were much better when we were in Egypt, perhaps we should go back to those ways, etc.
There’s a change in thinking that is clearly apparent by the time of the story of Elijah and the prophets of Baal (I think this is in 1 Kings) - this makes clear that there is no god but God, and that Baal is not really there (or as Elijah says, perhaps he is busy relieving himself…) Even so, the Israelites still turn away from God from time to time; the metaphor of adultery is used, e.g. in Hosea. Probably comes up again and again during the periods of exile, surrounded once again by foreign people worshiping foreign gods.
To sum it up, the covenant that God makes with the Israelites is essentially that you will be my people, and I will be your God. To believe in, trust in, pray to, offer sacrifice to, or otherwise worship other gods is to cease to trust in the one true God, and is thus to break the covenant.
A crude analogy might be that you can sin against your wife by getting caught up in pornography, which is depicting a fantasy, simply by engaging your imagination in it - directing the faculty of erotic love to a figment of your imagination for your own selfish ends, rather than to the proper subject, i.e. your wife, in a self-giving fashion. Clearly it’s not the same thing as idolotry, but there is this element of sinning by going after a fantasy that exists only in your own mind.