基于小型语料库的维吾尔语信息熵估算

Estimation of Information Entropy in Uyghur Based on Small-Scale Corpora

摘要: 信息熵是一种衡量信息量的指标，在信息论中用于度量随机变量的不确定性。语言的信息熵是数学方法和语言学的结合，反映语言中每个字符的平均信息量，可以帮助我们了解语言中某一字符表达能力。本文以维吾尔语为研究对象开展维吾尔语信息熵估算研究。在简要论述香农三大定理与自然语言处理关系基础上，基于120万词的维吾尔语单语语料库开展频率统计，运用信息熵计算方法，将统计结果代入香农信息熵公式初步估算出了维吾尔语的零阶熵，并将估算结果与一些表音文字系统语言进行了对比。

Abstract: Information entropy is a metric for measuring the amount of information, used in information theory to quantify the uncertainty of random variables. Linguistic information entropy combines mathematical methods and linguistics, reflecting the average information content of each character in a language, which helps us understand the expressive power of a particular character. This study focuses on Uyghur as the research subject, conducting an investigation into the estimation of Uyghur information entropy. After briefly discussing the relationship between Shannon’s three theorems and natural language processing, the study performs frequency statistics based on a monolingual corpus of 1.2 million Uyghur words. Using information entropy calculation methods, the statistical results are substituted into Shannon’s entropy formula to preliminarily estimate the zero-order entropy of Uyghur. The estimated results are then compared with some languages of the abugida writing system.