The number of the saved, in the end, will reflect not the number of people God forgives unconditionally (He forgives all of us for everything forgivable): it will reflect the number of people who responded to God’s forgiveness well versus badly.
I imagine the trouble you’re struggling with is believing that humans are really capable of choosing ‘away’ from God at all? (I think it’s common in our culture to believe that humans are basically ‘good’, and that anything humans do ‘wrong’ is basically a mistake or accident; we reckon someone has to be the rarest kind of evil to reject the Holy Spirit, so the idea of many people rejecting God is just bizarre to our cultural conception. But if the choice is in each person’s hands (and the choice is real), then it’s no more plausible to speculate ‘many saved’ than to speculate ‘few saved’. In the end it’ll just be a descriptive question of observing the actual number of people who do choose salvation. And I think the people who argue the ‘few saved’ position do have some legitimate Scriptural ground to stand on, in terms of thinking that Scripture suggests that God already knows (because He sees all times and places at once) that in the end, though He desired all to be saved, few choose it). That may be a tangent though, and a topic for another thread. Anyway, it certainly needn’t be the focus of our private walk (where what matters is the choice we make, not the number of people we imagine to be making choices beyond ourselves).