If you read the Church Fathers and study the early practice of the Church, you will see that praying to the East is absolutely a non-negotiable. The Fathers allow for certain variations in worship, but praying toward the East was essential and could never be varied. St Athanasius wrote that the Blessed Virgin Mary always sat facing East as she never stopped praying. We always prayed toward the East - it is Scriptural, historical and traditional. In fact only since the end of Vatican II has the Priest turned away from the East at the Liturgy in order to ‘face the people’. The rubrics for the Mass still, to this day, assume the position of the Priest to be facing the altar/East at the consecration etc, however, in spite of the fact that the rubrics and GIRM make this clear it is, more or less, universally ignored. Mass ‘facing the people’ has never been officially advocated by the Church. GIRM 299 does state that this is ‘desirable’ but the entire rest of the document assumes the opposite. I am, in fact, waiting for official clarification of this contradiction from the Liturgy Office of my Bishop’s Conference. I believe that the root is a very poor translation from the Latin.
As for only Muslims facing East - this is a practice that they took from us initially. You might also be surprised to know that their use of prayer mats and the prostrations they make upon them are also things they took from us. The Coptic Orthodox still use prayer mats and pray in exactly the same way.
I always carry a compass with me, but, weather permitting, you can always orient (east) yourself by using the sun as your guide.
Don’t be fooled by the utilitarianism and relativism of the modern age - all this ‘as long as I do it in my heart’ business. We are supposed to be living icons; body and soul in unity and harmony. The Fathers were instructed by the Apostles; the Apostles were instructed by Christ. What we have passed to us is that we should pray to the East from where Christ will come again, seated on the clouds of heaven.
If you don’t believe me, go and read the Fathers for yourself - it will change your life. If you want a few tips about where to begin etc, just let me know and I’ll oblige.
Pax.