No need to convert that again for every glyph.
This reduces the size of our Vertex struct from 48 to 32 bytes. It would be nicer if we could store the colors in fp16 format in the rendernodes, and avoid conversion here. But this is still good.