Lyrics in ogg/vorbis and mp3 files

My music library consists of mostly ogg/vorbis files, and I wanted to store the text that is sung directly in the files.

This is mostly uncharted territory - at least for .ogg files. It took three weeks of research to learn about the possibilities, solutions in other file formats and problems.

Lyrics formats

Lyrics can be synchronized or unsynchronized:

Standalone lyrics file formats

There is a huge number of different formats that allow you to store lyrics/subtitles in separate files.

LRC
Easy to read and easy to write text-only format: One line of text per lyrics line. Seems to be standard today for music.
srt
Very popular for movies. More verbose than LRC, but still plaintext.
MP3+G
Binary format used in Karaoke CDs. The karaoke text is ripped into .cdg files which must have the same name as the audio files.

MP3

Let's have a look at MP3 first because tool support is better and there is an official standard for both synchronized and unsynchronized lyrics.

"Official" is not really correct here; the MP3 standard does not define how to store meta data in a file:

The MP3 standards do not define tag formats for MP3 files, nor is there a standard container format that would support metadata and obviate the need for tags.

However, several de facto standards for tag formats exist. As of 2010, the most widespread are ID3v1 and ID3v2, and the more recently introduced APEv2.

Wikipedia: MP3 Metadata

Data storage

The ID3v2 standard has meta tags for both synced and unsynced lyrics:

USLT
Contains the encoding of the text, the language, a description and the actual song text. Multiple entries allowed, which means you can have lyrics in several language.
SYLT
Also contains encoding, language, the time stamp format (MPEG frames or milliseconds), type and time-stamped text in a defined format.

Tools to add lyrics

On Linux, there is kid3 and SYLTeditor (which did not install on Ubuntu 14.04).

For Windows, you have MiniLyrics, SYLTeditor and there once was Window Media Player 11 which could add synchronized lyrics to mp3 files.

kid3-cli can be used on the shell to add lyrics stored in .lrc files to mp3:

$ kid3-cli -c "set SYLT='countdown.lrc' ''" countdown.mp3

Extracting the lyrics via CLI is also possible:

$ kid3-cli -c "get SYLT:/dev/stdout" countdown.mp3

Displaying lyrics

On Linux, there is not a single application that displays synchronized lyrics stored in the file when playing an music track. All players that claim lyrics support either rely on local .lrc files or try to download the lyrics from somewhere on the internet.

On Windows you may use MiniLyrics which reads and displays SYLT data in mp3 files without problems.

Musique at least shows embedded unsynchronized lyrics.

ogg/vorbis

ogg/vorbis is a patent-free audio codec and thus my preferred choice over mp3.

Tool support for lyrics is nearly non-existent.

Data storage

A small list of meta data tags for ogg/vorbis is listed in Vorbis I specification: 5.2.2. Content vector format and standalone in comment field and header specification. In addition, the xiph wiki lists proposed field names and links to three websites that propose some additional ones ( 1, 2, 3). Neither of them mentions lyrics.

The topic was mentioned on the vorbis mailing list in 2006 and 2008 with the conclusion that there is no standard, one should try CMML. I did not find any tools, example files or supporting clients for it, though.

OggKate can be used for synchronized lyrics, and kid3 uses the LYRICS tag to store unsyced lyrics in ogg files.

OggText was brainstormed in 2008 as well, but was never brought any further.

OggKate

In 2008 OggKate, a format for synchronized lyrics in ogg/vorbis files, was invented and implemented.

Kate is an overlay codec, originally designed for karaoke and text, that can be multiplexed in Ogg.

It does not only provide plain text lyrics but also animations, different colors, fonts and other styles.

Stream

It is not some data in a meta data tag field, but a separate stream inside an .ogg file, accompanying the music stream (just like movie files have both video and audio streams):

Lyrics in meta data
+----------+--------------------------+
| Metadata | Audio stream             |
+----------+--------------------------+
Lyrics as stream
+----------+--------------------------+
| Metadata | A u d i o   s t r e a m  |
|          |  L y r i c s   s r e a m |
+----------+--------------------------+

This has several implications:

  1. ✔ When playing the file, the text is available when it needs to be displayed.

    No need to allocate extra memory for the lyrics, and no need for additional timing information in the lyrics data.

  2. ✔ Interweaving the audio and lyrics data makes it suitable for live audio streams that simply cannot have all the lyrics available at the beginning.

  3. ✘ Adding lyrics is not simply setting a tag but breaking up the whole file and re-assembling the whole file.

  4. ✘ Extracting lyrics requires reading the whole file and re-assembling the single pieces.

    Players will need to do this if they not only want to display the current line or syllable, but provide a glimpse to the following ones.

The 99% standard use case - lyrics for music files of max. 10 minutes length - does not benefit from this format. Even operas with a length of several hours only have about 100kiB data of timed lyrics (LRC), which is nothing for yesterday's hardware.

Another argument for streaming was made on the Xiph MIME Types and File Extensions talk page:

If you want it without the timing, you have to store it in headers, as streaming it will get you the text only as its presentation time is reached.

You could do that if you were loading from a file, but that's only a special case, so it's best to leave that text interleaved with other streams.

A player wanting to display the entirety of the lyrics at once would have to, if possible (eg, if not streaming), scan the entire file to recover the text. Parsing Ogg packets is relatively fast, so a threaded player could do this while starting streaming a file and have the text ready in under a second for a typical song I suppose.

I don't believe the "special case" argument; even when you're playing the middle of a ogg/vorbis file via HTTP you still have to fetch the beginning of the file to get information about the header and general structure.

Tool support

Creating and reading OggKate streams is done via kateenc and katedec. oggenc's --lyrics only works if kate support is compiled in, which is not the case on Ubuntu 14.04.

The only tool able to display OggKate lyrics streams in audio files I found is VLC 2.2.2, if you start playing and then select the subtitle. (2.1.6 does not detect the subtitle stream at all).

Creating OggKate streams

kateenc from libkate-tools on Ubuntu 14.04 allows one to convert .lrc and .srt into kate streams, which then can be muxed into ogg audio files with oggz:

$ kateenc -o lyrics.ogg -t lrc -l en countdown.lrc
$ oggz-merge -o countdown-with-lyrics.ogg countdown.ogg lyrics.ogg

Extracting the lyrics as LRC is possible with katedec:

$ katedec -t lrc oggz-kateenc-countdown.ogg

Playing files with Kate streams

Playing kate'd ogg/vorbis files works fine on VLC, mplayer, armarok, xine, audacious and ogg123 (even if they do not show the lyrics).

gstreamer-based applications like Rhythmbox and Totem open a new window that looks for extensions that understand "Kate decoder", but fail. Totem plays the file nontheless, but Rhythmbox does not. Amarok 2.8 sometimes opened that window, but also plays the file.

Patches

The OggKate author - who never left his name anywhere by the way - provided OggKate patches for several media players:

xine
The mailing list discussion did not lead to anything as it seems. Xine does not display kate lyrics.
mplayer
The patch was rejected; the mplayer authors didn't want to support another subtitle format and preferred ASS.
gstreamer

The patch got included in 2009 into gst-plugins-bad 0.10.

Unfortunately it is not included in gstreamer 1.0 (yet), which explains the installation popup.

Thoggen (DVD-Ripper)
The feature request is still open for Thoggen.

Unsynced lyrics in .ogg

There is no officially endorsed vorbis tag/field for unsynchronized lyrics. The only tag in the wild I found was LYRICS which is created by the kid3 editor and lyrico.

We can add it with vorbiscomment from a lyrics text file:

$ vorbiscomment -a file.ogg -t LYRICS="$(cat lyrics.txt)"
$ vorbiscomment --list  countdown-unsync-vorbiscomment.ogg
title=Countdown aligned
artist=cweiske
date=2016
genre=Speech
encoder=Lavf53.21.1
LYRICS=ten
nine
eight
seven
six
5
4
3
2
1
0

oggz-comment can also be used to add unsynchronized lyrics to .ogg files:

$ oggz-comment file.ogg -o file-with-lyrics.ogg LYRICS="$(cat lyrics.txt)"
$ oggz-comment -l file-with-lyrics.ogg
Vorbis: serialno 0033215013
        Vendor: Lavf53.21.1
        title: Countdown aligned
        artist: cweiske
        date: 2016
        genre: Speech
        encoder: Lavf53.21.1
        LYRICS: ten
nine
eight
seven
six
5
4
3
2
1
0

WMA

Windows Media Audio files support synced and unsynced lyrics in their meta tags. The ASF specification unfortunately does not define meta data tag names and thus also no meta data lyrics formats.

kid3 shows the synchronized lyrics in the .wma file with a tag name of WM/Lyrics_Synchronized.

Tools to add lyrics

Windows explorer on Windows XP is able to add unsychronised lyrics, Windows Media Player 11 could write both synchronized and unsynchronized lyrics.

Displaying lyrics

No tool I tested is able to display the lyrics embedded in .wma files while playing music.

exiftool at least shows them on cli, and kid3 notices that there are synchronized lyrics inside - but does not show them.

AAC

The Advanced Audio Coding format is mostly used by Apple's iTunes. I saw screenshots of the iTunes meta data editor that had a (unsynchronized) lyrics field. I did not follow this any further.

MiniLyrics claims to support lyrics in .m4a/.aac files.

Tools

The list of tools I found that could work with lyrics embedded in audio files, alphabetically sorted:

Clementine

Clementine is a music player that is able to show unsynchronized lyrics from mp3 files. Does neither support ogg lyrics nor synchronized text.

eyeD3

eyeD3 is a linux command line application that reads and writes mp3 tags, supporting ID3v1, 2.3 and 2.4.

Unfortunately it is not able to extract a single tag in a way that's readable by other programs, and is also not able to extract synchronized lyrics:

$ eyeD3 --verbose countdown-sync-minilyrics.mp3
 
countdown-sync-minilyrics.mp3   [ 258.24 KB ]
-------------------------------------------------------------------------------
Time: 00:10     MPEG1, Layer III        [ ~0 kb/s @ 44100 Hz - Mono ]
-------------------------------------------------------------------------------
ID3 v2.4:
title: Countdown aligned                artist: cweiske
album:          year: 2016
boo
track:          genre: Speech (id 101)
-------------------------------------------------------------------------------
ID3 Frames:
<Title/>songname/content description (TIT2): Countdown aligned>
<Lead performer(s)/Soloist(s) (TPE1): cweiske>
<Release time (TDRL): 2016>
<Content type (TCON): Speech>
<Software/>Hardware and settings used for encoding (TSSE): Lavf53.21.1>
<Synchronised lyric/text Frame (SYLT)>

It is able to add unsynchronized lyrics to mp3 files:

$ eyeD3 --lyrics="eng::ten
nine
eight
seven
six
5
4
3
2
1
0" countdown.mp3

exiftool

exiftool is the swiss army knife of meta tags. It's able to extract data from most image, document and audio file types.

It shows synchronized lyrics from mp3 and wma files, and unsynchronized from mp3, ogg and wma:

$ exiftool countdown-sync-kid3-id3v2.4.mp3 |grep -i lyr
Synchronized Lyrics Type        : Lyrics
Synchronized Lyrics Description :
Synchronized Lyrics Text        : [00:00.00].ten, [00:01.00].nine, [00:02.00].eight, [00:03.00].seven, [00:04.00].six, [00:05.00].5, [00:06.00].4, [00:07.00].3, [00:08.00].2, [00:09.00].1, [00:10.00].0, [00:11.00].
 
$ exiftool countdown-sync-wmp11.wma |grep -i lyr
Lyrics                          : ten..nine..eight..seven..six..5..4..3..2..1..0
Lyrics Synchronised             : ..vsynchronized lyricstennine�.eight�.seven�.six�.5�.4�.3p.2X.1@.0(#

It does not show OggKate texts.

Foobar2000: show lyrics 3 plugin

The windows-only Foobar2000 supports plugins, and one of the plugins/"components" is Lyrics Show Panel 3 (foo_uie_lyrics3).

While it claims to be able to read both synced and unsynced lyrics from ogg and mp3 files, it does so by using its own (configurable) meta tag, storing LRC in the tag for synced text. It does not support SYLT/USLT tags.

Foobar2000: Show lyrics 3: Lyrics editor Foobar2000: Tag editor Foobar2000: Show lyrics 3: Settings

kid3

The meta data editor kid3 runs on Linux and supports mp3, ogg and wma.

It has a usable synced lyrics editor and documentation for it. Importing LRC is supported.

kid3-qt was the only software that actually supported storing unsynced lyrics inside ogg/vorbis files by setting the LYRICS tag.

kid3-cli is a command line tool that can be used to read and write .lrc files from and to mp3 files.

kid3-qt: Editing synchronized lyrics kid3-qt: Editing unsynced lyrics

lyrico

lyrico automatically finds and downloads unsynchronized lyrics and is able to embed them into mp3, ogg, wma and m4a files.

It does not support synchronized text.

MiniLyrics

MiniLyrics is not available for Linux.

It reads synchronized and unsynchronized lyrics from mp3 meta data, but does not support the WMP11 generated lyrics in wma files, nor does it support .ogg. It claims to support M4A (AAC) files, but I did not test that.

It includes an editor for synchronized lyrics and can import LRC files.

MiniLyrics showing lyrics for foobar2000 MiniLyrics: Lyrics editor Minilyrics save type selection

Musique

Musique version 1.4 is able to read and display unsynchronized lyrics from mp3 files.

Musique showing unsynchronized lyrics from an mp3 file

oggz-comment

oggz-comment from oggz-tools is able to read and write ogg/vorbis meta tags.

See the example how to use it to add unsynchronized lyrics.

SYLT Editor

SYLT editor is Freeware for Windows and Linux, but it did not install on Ubuntu 14.04 correctly.

On windows it worked, but proved to not support SYLT lyrics embedded by other editors. It even once did not read SYLT from a file that it wrote itself previously..

I can't recommend it.

SYLT Editor: Lyrics editor SYLT Editor: Lyrics preview SYLT Editor: Broken import of MiniLyrics tagged mp3 file

VLC - VideoLan Client

VLC was the only application I found able to display OggKate lyrics in .ogg files - when enabling audio visualization, the text is printed on top of that. (version 2.2.2 worked, 2.1.6 did not)

It does not support SYLT, but you can load .srt subtitle files when playing music with audio visualization enabled.

VLC displaying OggKate lyrics VLC displaying .srt subtitles on audio

vorbiscomment

vorbis-comment from vorbis-tools is also able to read and write ogg/vorbis meta tags.

See the example how to use it to add unsynchronized lyrics.

Windows Explorer

Windows XP's Explorer is able to show and edit meta data of .wma files - and it allows you to add unsynchronized lyrics. You cannot set the lyrics language as you could with Windows Media Player 11, though.

Windows XP Explorer tag editor Windows XP Explorer lyrics editing

Windows Media Player 11

Windows Media Player 11 was available on Window XP and Windows Vista. To my knowledge it is the only application able to add synchronized lyrics to .wma files via its Advanced Tag Editor.

This tag editing capabilities were removed in Windows Media Player 12, much to the dismay of users.

Windows Media Player was not able to actually display the synchronized lyrics, which maybe was the reason for its removal. But the removal of the lyrics editing feature from WMP means that Microsoft officially has given up on lyrics in .wma, and maybe even on .wma itself.

Windows Media Player 11: Lyrics editor Windows Media Player 11: Synchronized lyrics editor

If you ever try to install Windows Media Player 11 on an old copy of Windows XP, you'll notice that it will fail because it cannot prove the XP installation as "genuine" anymore.

You can work around this by manually extracting wmp11-windowsxp-x86-DE-DE.exe with 7-zip and manually executing all the wm*.exe files.

Demo files

During the course of my research I created a number of audio files with the various tools. As base I used countdown.ogg from Corsica_S's countdown.wav but had to align it to the exact full seconds with Audacity. Then I created a lyrics .lrc file and used the various tools to add lyrics to it.

Audio files with embedded lyrics
Tool File Note
Source files
ffmpeg countdown.aac
ffmpeg countdown.flac
ffmpeg countdown.mp3
unknown countdown.ogg
ffmpeg countdown.wav
ffmpeg countdown.wma
Synchronized lyrics
kid3 countdown-sync-kid3-id3v2.3.mp3 ID3 v2.3
countdown-sync-kid3-id3v2.4.mp3 ID3 v2.4
MiniLyrics countdown-sync-minilyrics.mp3
oggz-merge countdown-sync-oggz.ogg OggKate
SYLT editor countdown-sync-sylteditor.mp3
Windows Media Player 11 countdown-sync+unsync-wmp11.mp3 Both synchronized and unsynchronized
countdown-sync-wmp11.mp3
countdown-sync-wmp11.wma
Windows Explorer countdown-unsync-explorer.wma
Unsychronized lyrics
eyeD3 countdown-unsync-eyeD3.mp3
Foobar2000 countdown-unsync-foobar2000.mp3
countdown-unsync-foobar2000.ogg
kid3 countdown-unsync-kid3.flac
countdown-unsync-kid3.mp3
countdown-unsync-kid3.ogg
countdown-unsync-kid3.wma
MiniLyrics countdown-unsync-minilyrics.mp3
oggz-comment countdown-unsync-oggz-comment.ogg
vorbiscomment countdown-unsync-vorbiscomment.ogg
Windows Media Player 11 countdown-unsync-wmp11.mp3
Other
Foobar2000: Show lyrics 3 countdown-sync-foobar2000-show_lyrics_3.mp3 Custom tag name
countdown-sync-foobar2000-show_lyrics_3.ogg Custom tag name

Apps missing embedded lyrics support

The following applications miss support for embedded lyrics:

Amarok
Bug 184325: amarok wish: fetch lyrics from the ID3 lyrics tag since 2009
EasyTAG
Bug 769201: Add lyrics tag support since 2016
eyeD3
Issue #103: Display synchronized lyrics since 2016
music player daemon
#3100: Add support for lyrics tag since 2010
Musique
Support for synchronized lyrics in MP3 files since 2016
Read lyrics from ogg/vorbis files since 2016
Quod Libet
#1989: synchronized lyrics plugin: Support embedded lyrics since 2016
Rhythmbox
Bug 769180: Lyrics plugin: Read lyrics embedded in audio file since 2016
VLC
#17207: Display mp3 synchronized lyrics SYLT tag as subtitle since 2016

Conclusion

I'm dissatisfied with the state of embedded lyrics support on Linux.

For ogg/vorbis files - of which my music library consists mostly - I did not find a satisfying solution. OggKate is too complex for most needs; something like SYLT for vorbis would be better suited - or even simply embedding LRC into a SYNCLYRICS tag.

The ID3v2 SYLT format is binary and would have to be base64 encoded, as it is done with album art in vorbis files, too.

And then, tool support would again be missing :(

Written by Christian Weiske.

Comments? Please send an e-mail.