Yes, while it is true that it’s valid, it’s not sacramental. A justice of the peace cannot do that, and as the Muslim refuses to enter a Catholic Church, then the priest can’t as well. .
It is only Sacramental if both parties are Baptized. If two baptized Protestants get married in front of a Justice of the Peace, it is Sacramental Marriage.
The Justice of the Peace, like a priest, is simply a witness to the Sacrament, they are not the ministers. It is neither the Justice of the Peace, nor the priest, who make a marriage sacramental, but the ontological status of the couple themselves.
And as far as the marriage not being Sacramental, neither was the marriage of Joseph and Mary. Joseph was never baptized.
From what I understand, a bishop gives dispensation for a Catholic to marry in another Christisn church. This couple didn’t even marry in a church
That would be called a 'Dispensation of Form", and it just what it says it is, a removal of the requirement to marry under the mandated Form of the Church. Once one is granted, the couple is no longer obligated to marry in a church, but can marry in a Protestant Church, a Synagogue, a Mosque or even a city park in front of a JoP.
And such a marriage would be fully recognized by the Church.
Note the various options on this form (no pun

)
archgh.org/default/marriage-forms/form%20iii-B%20Revised%20Dispensation%20from%20Canonical%20Form%20and%20Report.pdf
Specifically this part
If there has been a dispensation from canonical form, ordinarily the marriage ceremony is celebrated in a non-Catholic church or oratory. In some exceptional circumstances (e.g. certain Catholic-Jewish marriages), it may be necessary that the dispensation be granted so that a civil ceremony may occur. A public form that is civilly recognized for the celebration of marriage is required.