Officially, I don’t believe there is anything wrong with this.
Personally, because the Mass is both audible and visual, I feel that I am appreciating the Mass in a fuller way when I keep my eyes on the altar and the Eucharist, or gaze towards the crucifix or the candles.
A certain level of attention to the written word always seemed to be a Protestant tendency, IMO, because many denominations have rejected the visual aspect of worship.
It seems to me to be a stronger image of the theological reality occuring in Mass when everyone is “gazing towards the Lord” rather than everyone staring down at their booklets.
But I think each individual should use their own discretion. There is nothing wrong with following along in a missal or in Magnificat, especially if this is the way that keeps you most focused on the content- rather than gazing off and thinking about things not pertinent to God.