The definition of servile work has been discussed on this site before (too many time to provide links: use the search function to find the one most appropriate to your situation). Here’s my opinion for what it’s worth: if your personal trainer is not a Christian and is happy to work on Sundays, I think that’s for him/her to decide. There are all sorts of people who have to work on Sundays, such as doctors, nurses, pharmacists, paramedics, police officers, firefighters, and people who work in industries such as public transport and the media. If you visit an overwhelmingly Catholic country such as Poland, you’ll find that everything still runs on a Sunday. If public transport shut down on Sunday a lot of people wouldn’t be able to get to church. If the media shut down on a Sunday people would have to wait a whole day before learning of potentially important news such as a natural disaster, terrorist attack, or the outbreak of war. Given that a lot of Catholics do these jobs, I think it would be hard to argue that somebody who is not even a Christian should be deprived or his/her livelihood on a Sunday, especially when it may be the only day on which he/she can do that particular job.
As for exercise itself, I don’t think it’s servile work, even if you don’t particularly enjoy it. Somebody may love his or her job working in a shop, for example, but it’s still servile work to do that job. On the other hand, you may hate cooking, but nobody is going to say that you aren’t allowed to cook lunch for your family on a Sunday. Exercise is something that you should enjoy on some level and it is good for you. I would consider it similar to spending your Sunday reading or visiting a museum. If your hobby was learning Arabic and your Arabic tutor was a Muslim and was happy to give tutorials on a Sunday, would that be a problem for you?
I think there is one fundamental things to remember: Jesus tells us that the Sabbath was made for man, not man for a Sabbath. We have the Sabbath because it is basically good for us to take a day off. If you use part of your Sunday to exercise, that sounds like it’s only a good thing. Also, please do try to think about the spirit of the law as well as the letter. Last Sunday afternoon I met a friend, went to a museum, and got a coffee afterwards, and I did it all using public transport. I got to see my friend (good) and enjoyed the museum (also good). I guess the people who were working at these places and on public transport on a Sunday either aren’t Christians and so don’t worry about it or are Christians but consider that their work is necessary for other people to enjoy their Sabbath.