grapheme cluster error rate (gcer) measures differences between two texts using visible characters as the unit. it is useful when a character contains more than one unicode code point.

a grapheme cluster groups the unicode code points that form one visible character, such as a letter and its accent.

how is gcer calculated?

gcer compares the grapheme clusters in a reference text with those in a transcript. the score is the minimum number of substitutions, deletions, and insertions divided by the number of grapheme clusters in the reference.

a substitution changes one cluster. a deletion removes one, and an insertion adds one. a gcer of 0 means every cluster matches.

normalize both texts before scoring them. the same accented character can be stored as one code point or as a letter followed by a combining mark. without consistent unicode normalization, two visually identical strings can produce different counts.

gcer calculation example

this example uses unicode nfd, which stores the diaeresis in "zoë" as a separate combining mark.

Reference code points: [Z] [o] [e] [◌̈]
Reference clusters:    [Z] [o] [ë]
Transcript clusters:   [Z] [o] [e]

Reference code points: [Z] [o] [e] [◌̈]
Reference clusters:    [Z] [o] [ë]
Transcript clusters:   [Z] [o] [e]

the transcript changes the final cluster from ë to e. that is one substitution across three reference clusters: gcer = 1 / 3 = 33.3%.

with the nfd representation shown, character error rate is 25% because the transcript deletes one of four reference code points. under nfc, ë is one code point, so cer becomes 1 / 3 = 33.3%. gcer stays at 33.3% in both forms.

word error rate counts the entire name as one incorrect word, so it produces 100% wer. report the normalization form alongside the scores so another evaluator can reproduce them.

devanagari example

suppose the reference is किताब and the transcript drops the vowel sign ि, producing कताब. using nfc normalization and unicode extended grapheme clusters:

Reference code points: [क] [ि] [त] [ा] [ब]
Reference clusters:    [कि] [ता] [ब]
Transcript clusters:   [क] [ता] [ब]

Reference code points: [क] [ि] [त] [ा] [ब]
Reference clusters:    [कि] [ता] [ब]
Transcript clusters:   [क] [ता] [ब]

cer counts one deleted code point out of five, giving 20%. gcer counts one changed cluster out of three, giving 33.3%. wer counts the single word as incorrect, giving 100%.

the vowel sign belongs to the first cluster even though it appears to the left of the consonant when rendered. this example shows how the unit being counted changes the score; the larger percentage does not make one metric inherently better.

gcer vs wer and cer

word error rate (wer) counts words. one changed vowel sign can make a whole word incorrect.

character error rate (cer) counts unicode code points. it can split one visible character into a base character and several combining marks.

gcer counts the full visible unit. this makes it useful for indic scripts, where consonants, vowel signs, viramas, and combining marks can form one cluster. the three metrics use different units and denominators, so their percentages are not directly comparable.

why does gcer matter for indic tts?

tts teams measure transcript accuracy by running generated speech through a speech recognizer. for indic languages, a code-point score can divide one writing unit into several errors. gcer keeps the unit intact and shows how much of the written output changed at the cluster level.

rumik added gcer to the rumik-oss 1 post-training objective for indic scripts. the generated speech is transcribed, then the reference and transcript are compared as grapheme-cluster sequences. this gives the voice model a training signal based on the structure of the writing system.

gcer still inherits mistakes from the speech recognizer used to create the transcript. it also does not measure timbre, pacing, emotion, speaker identity, or audio quality. use gcer for written fidelity and listen to the audio for the parts of speech that a transcript cannot show.