I think I have something in common with your son! I don’t do sports well either, and I would love to just enjoy some beauty and not have a ‘class’ on fitness. It was always hard for me in school. I am a klutz and never could run as fast as the other kids. I loved to play 4 square at school recess, though. I got to make sure that the rules would be followed and fair even for the younger students. Maybe something like this could work out for your family. Is there an activity your son could ‘lead’ the younger children in? Could each child be responsible for planning one physical activity, including your son? Maybe he could come up with something fuctional or fun for him, or even just help the younger children hone their choices. Leadership and seeing the big picture is definitely a plus with some of us more theoretically minded people.
(Personally, I hate the idea of going to a gym or doing an exercise routine ‘just because’. I would much rather do something like dig and maintain a garden. It’s great exercise, you can be proud of what it looks like and what comes out of it, and it is a way to really see the beauty of God’s creation as you nurture it.)