Paul VI had expressed his disappoinment on several occasions. But I think you are right - Priests and Bishops with several post-graduate Theology degrees, inside the Vatican for many years, tend to forget that the average layperson doesn’t think like a Priest or Bishop inside the Vatican. People tend to take the path of least resistance, so, if something is made optional, it’s not going to be done by the vast majority of people. The participants in Vatican II seemed to be ignorant of this aspect of human nature. So, “you can use the vernacular in Mass, but we’d like it if you’d keep using some Latin” became “Mass in the vernacular!” Being able to eat meat on Friday if we substituted another act of penance instead became “we can eat meat on Friday!” and so on…
In other words, the laity saw what the mainline Protestants were doing, saw Vatican II create more options, change the Mass around, relax certain requirements, and thought “cool, we’re doing the same thing! Let’s loosen up!”, i.e., the “Spirit of Vatican II.” Then came Humane Vitae - “whoops! Doesn’t look like the Pope wants to do that after all… Oh well, we’ve already started, let him try and stop us!”