It definitely seems very odd to me.
Still, in trying to give your husband the benefit of the doubt, there are adults who enjoy hanging out with children (such as teachers, coaches of kids’ sports teams, scout troop leaders and others). We generally don’t question the choices of those types of adults to want a profession that involves hanging out with kids. Is what your husband doing much different?
If not, maybe it’s okay. Maybe talk to people who are teachers/coaches and ask them why they are drawn to those professions and whether they could understand an adult man who wanted to hang out with kids in his free time.
Those would be the people to ask. I don’t think it’s helpful to ask the average adult. I personally would have no interest whatsoever in being around children, but then I also would not want a career that involves working with children. Many responders here may be this way too, but someone who is say a coach might understand your husband’s point of view better.