This is different from a situation where chaste affection causes a bodily response. You are a young man and this is a very common thing for young men courting young women they are interested in and attracted to. My own husband took me to dances in high school (we were high school sweethearts) and it happened to him spontaneously a couple times. We were not being provocative, simply dancing near to each other. Of course, since I was very attracted to him, too, I became aware of it and it was flattering but we didn’t encourage anything further. Just finished our dance and went to get some punch. Another situation where chaste affection can cause unexpected arousal in young men.
Similarly, holding hands or chaste kisses on the cheek or lips can be enough to cause this. You do not need to avoid any chaste affection merely because this happens. Just focus on discretion (mostly to avoid embarrassment) and your enjoyment of the company of the young lady. Your body will become more accustomed to her presence, and it will settle down.
I do hope things work out with the young lady. It sounds like you are quite attracted to each other. Good luck!