Before becoming Catholic, showing up to mass once in awhile is not considered a sin. For many is it part of the discernment process to see if you want to become Catholic. So there is no worry there. In fact, many people who convert were profoundly touched by the mass. Now when you decide to enter RCIA to become Catholic, you have taken it upon yourself to enter in communion with the Catholic Church and learn the doctrines, teachings and principles. This is where I think one would want to go to mass as much as possible to have the beauty of the faith unfold in the majestic prayer of the mass.
After one becomes Catholic through the RCIA program, you are now brought (through baptism, if applicable, and confirmation) This confirms your belief in the faith and your openness to accept it. This is where missing mass becomes a sin. In the Catechism of the Catholic Church, it states one must attend mass all Sundays and holy days of obligation.
2180 The precept of the Church specifies the law of the Lord more precisely: “On Sundays and other holy days of obligation the faithful are bound to participate in the Mass.” “The precept of participating in the Mass is satisfied by assistance at a Mass which is celebrated anywhere in a Catholic rite either on the holy day or on the evening of the preceding day.”
I hope this helps, and may God Bless on the journey that you are on.