The Jewish culture was a very oral culture. Many times in the Gospel Jesus said, “you have heard it it said…” The Scriptures were heard and memorized, oral traditions were passed down and used to settle disputes. So the fact that an illiterate individual was able to debate Scripture, tradition and interpretation would not have been considered shocking. Obviously it would be quite surprising to encounter that in a 12 year old child but they would be been highly impressed rather than dumb founded.
As to whether Jesus was illiterate, that doesn’t seem to be the case. Jesus read from the scroll of Isaiah (Luke 4:17) and no one seem surprised that He was able to read. Secondly the Holy Family were not necessarily peasants. St. Joseph was a craftsman who probably made a decent living and given that the education of boys was a priority in Jewish culture it would not be out of the question for Jesus to have been taught basic literacy. Jesus also often challenges the scribes and pharisees by asking, “have you never read…” In those times no one seems to challenge that Jesus couldn’t have read nor are they shocked that Jesus is able to read.
Given that no one seems surprised that Jesus apparently read later in life and the fact that reading was not required to be able to debate the particulars of faith, Jesus as a child would not have stood out as anything other than a highly intelligent child which would have impressed them greatly but not caused them to keep track of him for the rest of his life.