Language is a rich complex structure with the specific biological function of exchanging non-trivial information across human brains. Both the presence of common evolutionary history and cognitive constraints have lead to different languages sharing common features. In some cases this common patterns are so widespread that have become known as linguistic universals. While for languages that have diversified over the past few millennia traces due to common origin are beyond doubt, the question still remains of whether more distant language families still share linguistic structure. In the talk I will discuss a novel measure of relative entropy that can quantify the degree of order of words taking into account the full correlation structure of language. When this measure is applied to text corpora of languages from diverse linguistic families, an almost constant value emerges that can be interpreted as a quantitative statistical universal. Moreover, the analysis of simple models of language together with statistical evidence from real languages suggest that, during its evolution language has diversified under the constraint of keeping a precise balance between vocabulary diversity and the extent of long-range word correlations. It is also argued that the constancy of the relative entropy is probably a consequence of cognitive constraints of the human language faculty.