A Methodology to Measure the Diachronic Language Distance between Three Languages Based on Perplexity

TítuloA Methodology to Measure the Diachronic Language Distance between Three Languages Based on Perplexity
AutoresJosé Ramom Pichel, Pablo Gamallo, Iñaki Alegria, Marco Neves
TipoArtículo de revista
Fonte Journal of Quantitative Linguistics, Taylor & Francis Online , 2020.
RankProvisionally ranked Q1 in Language and Linguistics by SJR 2018
ISSN0929-6174
DOIhttps://doi.org/10.1080/09296174.2020.1732177
AbstractThe aim of this paper is to apply a corpus-based methodology, based on the measure of perplexity, to automatically calculate the cross-lingual language distance between historical periods of three languages. The three historical corpora have been constructed and collected with the closest spelling to the original on a balanced basis of fiction and non-fiction. This methodology has been applied to measure the historical distance of Galician with respect to Portuguese and Spanish, from the Middle Ages to the end of the 20th century, both in original spelling and automatically transcribed spelling. The quantitative results are contrasted with hypotheses extracted from experts in historical linguistics. Results show that Galician and Portuguese are varieties of the same language in the Middle Ages and that Galician converges and diverges with Portuguese and Spanish since the last period of the 19th century. In this process, orthography plays a relevant role. It should be pointed out that the method is unsupervised and can be applied to other languages.
Palabras chaveNatural Language Processing, Language Distance, Historical Linguistics, Perplexity