There lies the misconception. If the Earth was the center of Universe, and if the Sun and the Moon were always rotating around the Earth at the same speed and distance, Gregorian Calendar would be accurate.
But we know the Earth is neither the center of Universe, nor even of our Galaxy. As you might have heard, the outer space is widening, it is not a constant. So does our Galaxy and the distances between the Sun, Earth and Moon. Therefore, Julian calendar is accurate at any point in Galaxy outside the Earth where it can relate itself to the Sun and Moon. Julian time and Julian day are more precise - they are not the same as Julian Calendar, but they just follow its logic, of course, enhanced by scientific discoveries occurred meanwhile.
I hope I was clear enough.