Informational: Jockers/Criddle study not worth using

[Big_Dummy]

Re: Fwd: this may interrest you, reviews added at bottom,

Reply |Sandra Tanner to Big Dummy

Thanks. I am aware of the Jockers/Criddle study but feel there are basic problems with it. I have also read the criticisms. The Jockers study only shows which of a select group of people is the closest to the B of M style, not that that person is the author. Other people not tested could be closer. So you could take any 5 people from the period and do a similar study and it would show you which of those 5 would be the closest to the style–again it would not prove that any of those 5 were the author.

Computer studies certainly have their place, but it is still unclear how well they can determine authorship when there has been a deliberate attempt to mimic another person’s style or to disguise the author’s style. Here is a link to the Twenty-First Conference on Innovative Applications of Artificial Intelligence, July of 2009, in which it was pointed out that when such efforts are made to deceive, computer analysis was no more effective in identifying the correct author than “randomly guessing the correct author of a document.”
cs.drexel.edu/~greenie/brennan_paper.pdf

See my 2010 ed. of ‘Joseph Smith’s Plagiarism of the Bible.’

Sandra Tanner

On 7/8/2010 11:46 AM, Utah Lighthouse wrote:

-------- Original Message -------- Subject: this may interrest you, reviews added at bottom,
Date: Thu, 8 Jul 2010 13:38:09 -0400

Jockers, Matthew L., “Testing Authorship in the Personal
Writings of Joseph Smith Using NSC Classification.”
Accepted and Forthcoming in Literary and Linguistic
Computing, 2010

Jockers, Matthew L. and Daniela M. Witten. “A Comparative
Study of Machine Learning Methods for Authorship
Attribution.” Literary and Linguistic Computing, 2010;
doi: 10.1093/llc/fqq001

Jockers, Matthew L., Daniela M. Witten, and Craig S.
Criddle. “Reassessing Authorship of the Book of Mormon
Using Delta and Nearest Shrunken Centroid
Classification.” Literary and Linguistic Computing,
2008; doi 10.1093/llc/fqn040

“Testing Authorship in the Personal Writings of Joseph Smith
Using NSC Classification.” Digital Humanities
Conference 2008, College Park, Maryland, June 22 - 25,
2009.

 

[LongJourney]

However, when one analyzes and compares the Jockers study with the Broadhurst study, both found Spalding content in similar places. I even did a chi square test on it. When two independent studies come up with similar results (and remember that Delta also came up with similar results) then they validate each other.

That said, yes, I suspect some of the Rigdon results. And yes, there are other authors who can be tested. The Jockers study is only one step in the right direction. It is valuable and useful, but not the final answer to the sources for the BoM.

Sandra T. has done great work towards liberating people from Mormon-think. However, she remains an evangelical Christian with fundamentalist leanings. She is a “Joseph Only-ite”. Such people seem to have a leaning towards JS with satanic inspiration. That stance is weak, given the massive borrowing from other works, when JS was minimally literate. He had human help.

 

[system]

PART 1:

First, I wish to congratulate Mrs. Tanner for displaying a rare moment of skepticism, in that she doesn’t accept claims critical of LDS beliefs merely *because *they are critical of LDS beliefs. However, in this instance, her skepticism is quite misplaced.

The Jockers/Criddle study is actually extremely powerful and compelling and does **not **suffer from the flaws she and others allege. So far, I have found no valid and decisive criticisms of the paper in question anywhere, and I’ve read quite a few. The failings of all these criticisms are due to insufficient knowledge and understanding of the techniques involved in the Jockers/Criddle paper along with profoundly flawed analyses.

Allow me to intersperse my comments with Mrs. Tanners’ as quoted by poster Big Dummy…

Reply |Sandra Tanner to Big Dummy
… I am aware of the Jockers/Criddle study but feel there are basic problems with it. I have also read the criticisms. The Jockers study only shows which of a select group of people is the closest to the B of M style, not that that person is the author. Other people not tested could be closer. So you could take any 5 people from the period and do a similar study and it would show you which of those 5 would be the closest to the style–again it would not prove that any of those 5 were the author.

That is both factually false and highly misleading. The errors in her understanding and logic are vividly revealed in the next part of her comments…

Reply |Sandra Tanner to Big Dummy
… Computer studies certainly have their place, but it is still unclear how well they can determine authorship when there has been a deliberate attempt to mimic another person’s style or to disguise the author’s style. Here is a link to the Twenty-First Conference on Innovative Applications of Artificial Intelligence, July of 2009, in which it was pointed out that when such efforts are made to deceive, computer analysis was no more effective in identifying the correct author than “randomly guessing the correct author of a document.”

cs.drexel.edu/~greenie/brennan_paper.pdf

Virtually every criticism of the Jockers/Criddle paper makes the same or similar erroneous allegations, and the Brennan/Greenstadt paper helps to explain why so many people come to those false beliefs: They are applying valid criticisms of OTHER methods that simply do NOT apply to the methods used by Jockers, et al!

Older and/or other software authorship analysis techniques, as Brennan/Greenstadt correctly point out, are, in fact, too easily fooled by trying to mimic obvious stylistic factors. That’s precisely why Jockers, et al, cautiously decided NOT use any of them! The Brennan/Greenstadt paper, as well as Mrs. Tanner’s comments, are therefore utterly irrelevant to this debate.

Before I proceed, I wish to emphasize my own skepticism regarding one significant aspect of the arguments for including Solomon Spalding as a potential author candidate. Dale Broadhurst and many others have argued that an alleged Spalding work referred to as “[The] Manuscript Found” served as a basis for The Book of Mormon. The Jockers/Criddle paper also argues for this thesis, but I reject this utterly for several skeptical reasons:

(1): There is absolutely no truly credible and trustworthy evidence that the alleged document ever existed; there are hints and scenarios put forth by the theorists that are completely open to much more plausible interpretations, such as with well-established, modern scientific facts regarding human psychology and the enormous fallibility of memory and recall.

(2): The claims are based on *extraordinarily *dubious “affidavits” gathered by an angry individual seeking revenge for being excommunicated by the LDS Church.

(3): Nearly all of the “affidavits” strike me very strongly indeed as having been written or coached by the same individual; they usually employ suspiciously similar odd words and phrases that are quite unrelated to the otherwise understandable commonalities arising from a similar story. It strains credulity that these odd words and phrases would be naturally used by several different writers.

That being said, precisely because of the clever word choice and analyses employed by Jockers, et al., there is no need whatsoever to contend that “Manuscript Found” ever existed! Why not? Because of the simple fact that there are no extant copies of this alleged document, it could not be used in Jockers, et al., study, thus it is totally irrelevant! The study found the strong, clear signature of Solomon Spalding based on other known writings, particularly the Spalding manuscript secured at Oberlin College entitled “Manuscript Story”, and not “Manuscript Found” at all.

The only value that the “Manuscript Found” hypothesis had is that it’s popularity and its presumed plausibility prompted Jockers, et al, to include Spalding as a potential authorship candidate. The study confirmed that Spalding’s words were indeed included in the Book of Mormon, but that in no way serves as evidence that “Manuscript Found” ever existed, contrary to the Jockers/Criddle paper’s own premise. That premise has turned out in the end to be irrelevant, yet it in no significant way alters or diminishes the study’s results and conclusions. The Oberlin “Manuscript Story” contains more than enough thematic similarities to have served as a rough model for both The Book of Mormon and the mis-remembered oral stories that gave rise to the “Manuscript Found” hypothesis. That hypothesis can serve no further use.

To continue, please see PART 2…

 

[system]

PART 2:

Returning to Mrs. Tanner’s main error, it is absolutely false to assert, as Mrs. Tanner did above, that: “The Jockers study only shows which of a select group of people is the closest to the B of M style.” The whole point of choosing the metrics and analyses Jockers, et al, selected is that style is completely irrelevant! There ARE no style comparisons involved in the study’s methods!

Instead, allow me to quote verbatim from the text of the published study: “The method is as follows. First, we compute average word frequency vectors, or centroids, for each known author, on the basis of the text samples of known authorship. Next, we shrink these centroids towards the overall average word frequency vector across all of the authors, in order to make our method more robust to small changes in word frequencies. Finally, we classify a text of unknown authorship by computing its word frequency vector and determining to which of the shrunken centroids it is most similar.”

Thus, style is utterly irrelevant and mimicry is completely useless! For mimicry to defeat such methods, it would be necessary to know – in advance – precisely which truly non-contextual words Jockers, et al, would end up selecting for computer analysis in 2008 -and- perform innumerable iterations of compose-test-repeat by running the exact same algorithm with a modern computer before a sufficiently strong match could be produced!

Such computations would have been necessary during the 19’th century for such mimicry to have succeeded. Consider how still less plausible such would have been in ancient times!

As for Mrs. Tanner’s and others’ assertion to the effect: “Other people not tested could be closer. So you could take any 5 people from the period and do a similar study and it would show you which of those 5 would be the closest to the style–again it would not prove that any of those 5 were the author”, allow me to put that notion to rest:

In the last analysis, it does not matter who the actual authors of the BoM really are, other than that Jockers, et al. compellingly established they were NOT ancient! THAT is what counts! Thanks to this study, the Book of Mormon is now a scientifically demonstrated fraud!

Jockers/Criddle resoundingly settled that question and also unequivocally established that Joseph Smith was not a significant contributor to the BoM. I hold that their work also argues persuasively as to the most probable 19’th century authors, but even though that tertiary question might not have been settled with finality, the fact that the Book of Mormon was not written in ancient times has now been demonstrated far beyond all reasonable doubt.

 

[campeador]

I have absolutely nothing to add, but would love to see what our LDS friends think.

 

[RebeccaJ]

Perhaps Mormons would think they need to test the writings of Mormon or Moroni…only, they’re aren’t any. At all. Zero. Zilch. Nada.