Of course you can come back to the Church.
Divorcees are not prohibited from receiving the Church’s sacramants, unless they are continuing in an adulterous relationship, which you are not.
Your “second marriage” is almost certainly not one recognised by the Chatholic Church, since the other person was a divorcee and since you married outside of the Catholic Church regardless. So I would discuss this all with a priest, confess of any sins that you and he identify (eg marrying outside the Church, marrying a divocree), and then you will be free to fully participate as a Catholic again. I’m not sure why you are receiving conflicting opinions from different priests, but find one you can trust and discuss this all in detail.
PS Assuming that marriage is considered invalid, the status of your civil divorce or annulment will have no bearing on your marital status in the Catholic Church.