I might nod or give a little wave to the people around me (because people look at you expectantly during the Sign of Peace), but I don’t actually say anything. imo our attention shouldn’t be directed away from the altar once the Consecration has taken place. The turning to your neighbors thing must be a Vatican 2 add-on; it doesn’t exist in the Latin Mass. Actually, once the priest says, “Peace be with you,” and we have responded, “And with your spirit,” I kneel down. Again, in the Latin Mass, people are kneeling for the Agnus Dei. As far as I know, the proper prayer posture for supplication is kneeling, so it makes sense that when we’re asking God for mercy and to grant us peace, we should be on our knees.
Part of the reason I went to a Latin Mass parish for a long time was that, even during the Ordinary Form Masses, the priest omitted the Sign of Peace. I make a point of smiling/saying hello to people before and after the Mass to the extent that I can because I don’t want to appear unfriendly, but - I do what I can to avoid shaking people’s hands during the Holy Sacrifice of the Mass.