It isn’t strictly true that Luther took out any books. His Bible contained all the books the Catholic Bible does. He did relegate the dueterocanonical books but did include them as apocryphal. He did express doubt about several New Testament books.
As to why he didn’t more firmly relegate New Testament books who knows. I think this would be pure speculation. Another way to look at it is by what authority did Luther or anyone else have authority to determine the canon, or value or importance of any book.
There were 2 well defined OT canons a person might choose from (if a man felt himself empowered to ****choose ****canons at all!). So it was simpler for Luther to back the shorter OT canon, a canon that had some historic support anyway, rather than the longer OT canon used by Rome.
Re the NT canon, there was no single obvious alternative. Luther expressed doubts about certain books, but it would be much harder to build a consensus around any specific alternative. In fact, once Luther would have dropped a few NT books over here, some other Protestant would have dropped a few other NT books, and a third more daring Protestant would have added a few new NT books that supported his particular Reformation agenda.
Pretty soon there would have been multiple NT Protestant canons. I suspect Luther’s theological advisors pointed it out to him, that he should stick with modifying the OT but retain the Catholic NT canon. The problem for Protestants is that, especially in recent years, Catholics have been challenging more and more, “Where did that NT canon come from?” So recent Protestant apologists devote more time to the NT canon, to show how that particular collection of books was, under God’s guidance, more or less inevitable, not the product of the Church hierarchy.
I think this logical process is faulty, as based on after the fact reasoning. For instance, the argument is that Luke or Colossians are true scripture because their theology is consistent with what we know to be Christian doctrine. But if you believe sola scriptura, then our doctrines are communicated only through scripture, so there is a circular reasoning.
Lutherans and others use “Tradition” and the ECF’s as a back up guidance to sola scriptura, which helps identify Mathew as true scripture, and Gospel of Mary as not; thus no need for a magisterium. But ****someone ****had to pick out 1% of traditions as the reliable Sacred Tradition, and **someone **had to designate **this **list of scholars as ECF’s and ****that ****list of early Christian scholars as heretics.
One can argue that Luther was directly guided by the Spirit (thus not needing a Magisterium) in deeming certain OT books as non scriptural. But then the Mormons could say Joseph Smith was directly guided by the Spirit, or some other formulation, in deigning certain books to be added. Muhammed could claim that same direct guidance.
I can’t see any alternative to the role of the Magisterium in deciding the biblical canon for Christians.
