From language identification to language distance

TítuloFrom language identification to language distance
AutoresPablo Gamallo, José Ramom Pichel, Iñaki Alegria
TipoArtículo de revista
Fonte Physica A: Statistical Mechanics and its Applications, Elsevier, No. 484, pp. 152–162 , 2017.
AbstractIn this paper, we define two quantitative distances to measure how far apart two languages are. The distance measure that we have identifed as more accurate is based on the perplexity of n-gram models extracted from text corpora. An experiment to compare forty-four European languages has been performed. For this purpose, we computed the distances for all the possible language pairs and built a network whose nodes are languages and edges are distances. The network we have built on the basis of linguistic distances represents the current map of similarities and divergences among the main languages of Europe.
Palabras chaveLanguage Distance, N-Gram Models, Perplexity, Corpus-Based Linguistics, Natural Language Processing, Language Identifcation