First of all, I don’t think there’s a single person on the planet who’s life is “on track.” We’re all broken. Even the guy next door that you think has it all. We all need God.
Second, as far as why do tiny things matter, if you think of sin as breaking some random rules, then yeah, it doesn’t make sense why venial sins would matter. Everybody runs a red light now and then, yet we don’t consider them criminals. But really, it’s not like that. It’s about love. That’s why even venial sins matter. Because all sin is fundamentally a refusal to love. To love God who first loved us.
Sometimes we reject his love so completely by rejecting his plan for us that all grace leaves our soul. That’s a mortal sin. But sometimes we reject him only a little. That doesn’t cause grace to completely leave us like a mortal sin does. But it’s still a big deal. Because we shouldn’t want to hurt the one we love at all. And because the more we do these little things against love, the more numb we get to God in general, and the easier it will be to fall into a serious sin.
To see why it matters so much, imagine if someone said they loved you, then constantly bickered with and mocked you and ignored you. Then they told you, “It’s not a big deal. It’s not like I cheated on you or killed you or anything.” You would probably think they didn’t really love you. When we love someone, we don’t want to hurt them even in little ways. Not cheating on someone isn’t really saying much, the relationship is still damaged. And if it keeps going in that trajectory, it’s going to fall apart. That’s how it is in our relationship with God.
It’s also like a cut. (Sorry, I’m full of analogies tonight.

) One isn’t going to do much harm. But fifty will. A hundred will. We keep cutting ourselves over and over and eventually we lose feeling. Keep doing that and your arm will fall off. It’s not something to take lightly or dismiss.
So that’s why all sin matters, even the little ones. It hurts our relationship with God, which in the end is the only thing that matters.