Yes!
You have hit the nail on the head, because all that we do for God, if not from a heart that wants to please him and be conformed into the image of His Son, is for nothing.
For many years, I thought I loved God, but I was not very concerned with those around me, it was a very self centered, self involved, feelings oriented love.
He is changing that, thanks be to Him! He took me through a lot to get me on the right track, but I am noticing that compassion towards others, in an eternal way as well as temporal way, is growing in step with my devotion to pleasing Him in other things. (And I still struggle with sin, but I am more accutely pained by it than before.)
So, I think you are right, love for neighbor as seen through the eyes of Christ, is the true barometer for our love for God.