You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At the moment, we have collections with texts using the latin alphabet and others using cyrillic script, possibly some mixed collections or even texts. This cannot entirely be deduced from the language of the texts, in some cases, but probably has significant impact on analyses using the words as tokens. So it might be worth while noting this somewhere.
Bottom line: it is probably sufficient to update the language codes used as values in "@xml:lang" to have the added information about the script used, like: "de-Latn" (for German in Latin script) or "bg-Cyrl" for Bulgarian in Cyrillic script.
Note that the Serbian texts already do this, with @xml:lang value of "sr-Cyrl", but it might be good to generalize the practice at least for those languages where variation in scripts used is possible.
The text was updated successfully, but these errors were encountered:
That is a good idea. Thanks!
If the schema already allows the modification of "@xml:lang", the only thing we need to do is to update the encoding documentation.
As for the metadata update in the document headers: in the next WG meeting, I could add this to the open tasks for the members responsible for the language collections.
At the moment, we have collections with texts using the latin alphabet and others using cyrillic script, possibly some mixed collections or even texts. This cannot entirely be deduced from the language of the texts, in some cases, but probably has significant impact on analyses using the words as tokens. So it might be worth while noting this somewhere.
TEI recommends using "@xml:lang" to register the language and, optionally, the writing system of a text. For this, it recommends using the IANA.org codes, see here: http://www.iana.org/assignments/language-subtag-registry/language-subtag-registry and, more accessibly, here: https://en.wikipedia.org/wiki/IETF_language_tag#Syntax_of_language_tags
Bottom line: it is probably sufficient to update the language codes used as values in "@xml:lang" to have the added information about the script used, like: "de-Latn" (for German in Latin script) or "bg-Cyrl" for Bulgarian in Cyrillic script.
Note that the Serbian texts already do this, with @xml:lang value of "sr-Cyrl", but it might be good to generalize the practice at least for those languages where variation in scripts used is possible.
The text was updated successfully, but these errors were encountered: