Matthew is almost unanimously testified as the oldest gospel by the church fathers. Clement of Alexandria even supported both Matthew and Luke as before Mark. This is significant because Mark is said to have founded the Coptic branch of Christianity in Alexandria, Egypt. If any place were to argue for Markan priority, Egypt would be the most likely. A sampling of the church fathers' testimony follows:
Papyus “Matthew wrote in Hebrew and others translated.”
Origen said the first gospel was written by Matthew in Hebrew.
Iraneus (grandson in the faith of John by Polycarp of Smyrna) said the first gospel was written written by Matthew in land of Hebrews in their own language.
Eusebias — Matthew had first preached to Hebrews and wrote in their own language
Jerome “Matthew was the first to compose in Hebrew and his text is still available in [library near Bethlehem].” He even challenged his critics to go see it if they doubted.
As the church rose out of the mission to the gentiles, it is interesting that the church fathers supported the Judaic gospel of Matthew instead of Mark. Also consider that they testify that Mark was the companion of the Apostle Peter in Rome which became one of the five sees of the early church (Rome, Constantinople, Antioch, Jerusalem, and Alexandria, Egypt). Unless the tradition of Matthean priority were very early, it is unlikely that they would all arrive at it independently. In fact, the slight differences in their testimonies provide evidence that they came from different sources.
Even though one of the main arguments for Markan priority is that Mark is shorter and "later authors would be more likely to expand than contract," such is not always the case (see, for example, the Reader's Digest Condensced Library). Summarizing a longer work is well known and has been for a long time. There are even ancient works which name their sources and state, "this work will be a shorter, more understandable account of the events than X."
The Didache clearly relies on Matthew. While the date of this document is debated between AD 50 and AD 150, the earlier it is, the earlier Matthew has to be.
When you examine second-century Christian writings, Matthew is quoted far more frequently than Mark. So is Luke. If Mark enjoyed a period when it was the only written gospel, it seems that it should have been more popular. Likewise, Matthew's Gospel enjoys a more central place in the second century liturgy than any other gospel or even Paul's epistles.