The earlier posters have the right of it.
While many such works (especially the later ones) are outright heretical, some are in keeping with orthodox Christianity and even give us valuable insights into the teaching and practice of the early Church.
For example, the Didache, a work known as the Teaching of the Twelve Apostles and originally believed to come from them, has been dated anywhere from AD 70 (during the composition of the New Testament books) to AD 150. While it didn’t make the canon, there is nothing heretical in it. It has two parts which may originally have been separate works. The first part is a treatise on Christian morality, echoing much of what we find in the New Testament but with additional specifics. For example, it is here that we find the earliest written Christian condemnation of abortion. The second part is a liturgical manual, which provides a basic outline of early Christian worship that is substantially similar to the Mass we celebrate today.
Such works are valuable witnesses to Christian Tradition, that matrix of common understandings and practices from which Scripture itself arose, and which has been passed on to us today. The New Testament contains no detailed description of a Christian worship service, for none of the letters or gospels had that as its purpose. The various churches that have attempted to “go back to the Bible” to structure their worship have wound up with a dizzying variety of permutations. Works like the Didache and the writings of Justin Martyr show that there was a commonly practiced style of Christian worship early on, that the very Christians who wrote and first read the New Testament would have been aware of, even though it was not detailed in inspired Scripture. The various liturgical forms of worship (our Mass, the different Eastern Divine Liturgies, and even the Anglican service) are recognizably descended from this early structure, which notably featured the Eucharist as a key part of the proceedings. (Indeed, the Greek word underlying “Eucharist” appears several times in the Didache, and though it literally means “thanksgiving,” it has obviously already by that time become a Christian term of art.)
Got off on a tangent there, but that’s just one example of the value that can be found in the orthodox extrabiblical writings of the early Christians. Again, these must of course be distinguished from heretical “Gnostic gospels” and the like.
Usagi