Yes, at the time when it was subsumed back into the diocese of Raleigh, it did compose of just Gaston County, but I was told by the monks at Belmont when I went to school there, they originally had jurisdiction over pretty much everything north and west of the Abbey. They began running into problems because it sometimes took days for the priest-monks who were assigned there to return to the monastery. It was just that, over time, it the early communities sputtered out and they were left with those parishes, slowly ceeding land back to the diocese when they no longer had ministry in those areas. At the time of the dissolution of their territorial abbey status, they were considered the smallest diocese in the world.
I know there were many of these small Catholic Communities in the mountains of Western NC because it used to be part of my own diocese of Charleston SC and from the beginning, they had petitioned the Bishop of Charleston for priests to serve them. Those petitions even continued after the Diocese of Raleigh took over their ministry until the Territory of Belmont Abbey was established because their own diocese was not meeting their needs.
When I said "most of the modern Diocese of Charlotte, I meant that it’s autonomous ministry sprawled across much of the western part of the diocese. There were very little established parishes in the western mountains and the monks traveled almost to the Virginia line ministering in the early days. Canonically, any chapel established by the Abbey (however temporary) was under their territorial authority, even if it was outside of the land area of the Territory. The same principle is used today for the founding of new monasteries. Technically, those communities that the monks regularly visited which did not have a parish or diocesan mission were part of the Territory via the monastic chapel (usually in a barn or someone’s home) rather than the set geographic boundaries. This changed after the 1917 code of Canon Law and the monastic chapels officially became part of the Diocese of Raleigh and the monks were just considered visiting priests.