As a novice in Scripture, I can offer this: It says “He went home”. If you remember, the people in Jesus’s home town did not recognize him. In the very beginning of His ministry, when he announced in the synagogue that “These words are fulfilled in your midst”, the crowd rebelled. They said “Isn’t this Joseph’s son” (or something like that) and it said something about a prophet not being recognized by his own people. Also there is a place where it says that Jesus couldn’t work miracles in his home town because of the lack of belief.
So it follows that those who knew him early on when He was obedient and submissive to his parents thought that he had gone off the deep end when he starts preaching publicly.
I am not sure about the reference to “kinfolk” or whatever the exact word is…perhaps that means people from the same town?