back

by sixhobbits·8y ago·view on hn ↗
I did my master's thesis on authorship verification, which is exactly this problem (deciding if two texts are written by the same author).

I experimented with clustering, SVMs, neural nets, etc for a long time, and got mainly disappointing results.

Even when "modern methods" give very high confidence scores, the problem is very messy and complicated, and usually the training data is different enough from the actual data (in supervised learning scenarios) as to bring the result into question.

I don't have access to the paper, so can't say much more, but I've seen a lot of very good-looking results that are in fact questionable.

Still a fascinating problem!

2 comments
From the abstract[1]:

> After compiling the ‘Jack the Ripper Corpus’ consisting of the 209 letters linked to the case, a cluster analysis of the letters is carried out using the Jaccard distance of word 2-grams. The quantitative results and the discovery of certain shared distinctive lexicogrammatical structures support the hypothesis that the two most iconic texts responsible for the creation of the persona of Jack the Ripper were written by the same person.

So it seems to be a combination of cluster analysis and manual linguistic reading of lexicogrammatical structures. It would be interesting if these studies were done as blind studies, with another letter from the period (or a convincing known fake) was also used as control. Do they find those shared distinctive lexicogrammatical structures in the control too, because they're looking for them now? Does the cluster analysis give significantly higher confidence to grouping the possibly-real letters, compared to grouping one possibly-real letter with the known fake?

Those would make this more interesting and less questionable, especially for a first study into this.

[1] https://academic.oup.com/dsh/advance-article-abstract/doi/10...