My music library consists of mostly ogg/vorbis files, and I wanted to store the text that is sung directly in the files.
This is mostly uncharted territory - at least for .ogg files. It took three weeks of research to learn about the possibilities, solutions in other file formats and problems.
- Lyrics formats
- MP3
- ogg/vorbis
- WMA
- AAC
- Tools
- Hardware
- Demo files
- Apps missing embedded lyrics support
- Conclusion
Lyrics formats
Lyrics can be synchronized or unsynchronized:
-
unsynchronized lyrics is just the song text as you see it in the booklet of a CD
It's plain text with newlines .
-
synchronized lyrics is the text plus timing information, which allows media players to show the exact line that is currently being sung. Karaoke depends on this.
In addition to the text lines, timing information is needed - be it for lines or even single words or syllables.
Standalone lyrics file formats
There is a huge number of different formats that allow you to store lyrics/subtitles in separate files.
- LRC
- Easy to read and easy to write text-only format: One line of text per lyrics line. Seems to be standard today for music.
- srt
- Very popular for movies. More verbose than LRC, but still plaintext.
- MP3+G
- Binary format used in Karaoke CDs. The karaoke text is ripped into .cdg files which must have the same name as the audio files.
MP3
Let's have a look at MP3 first because tool support is better and there is an official standard for both synchronized and unsynchronized lyrics.
"Official" is not really correct here; the MP3 standard does not define how to store meta data in a file:
The MP3 standards do not define tag formats for MP3 files, nor is there a standard container format that would support metadata and obviate the need for tags.
However, several de facto standards for tag formats exist. As of 2010, the most widespread are ID3v1 and ID3v2, and the more recently introduced APEv2.
Data storage
The ID3v2 standard has meta tags for both synced and unsynced lyrics:
- USLT
- Contains the encoding of the text, the language, a description and the actual song text. Multiple entries allowed, which means you can have lyrics in several language.
- SYLT
- Also contains encoding, language, the time stamp format (MPEG frames or milliseconds), type and time-stamped text in a defined format.
Tools to add lyrics
On Linux, there is kid3 and SYLTeditor (which did not install on Ubuntu 14.04).
For Windows, you have MiniLyrics, SYLTeditor and there once was Window Media Player 11 which could add synchronized lyrics to mp3 files.
kid3-cli can be used on the shell to add lyrics stored in .lrc files to mp3:
$ kid3-cli -c "set SYLT='countdown.lrc' ''" countdown.mp3
Extracting the lyrics via CLI is also possible:
$ kid3-cli -c "get SYLT:/dev/stdout" countdown.mp3
Displaying lyrics
On Linux, there is not a single application
that displays
synchronized lyrics stored in the file when playing
an music track.
As of 2021, only Lollypop supports displaying synchronized lyrics stored in the file when playing an music track. All other players that claim lyrics support either rely on local .lrc files or try to download the lyrics from somewhere on the internet.
On Windows you may use MiniLyrics which reads and displays SYLT data in mp3 files without problems.
Musique and Rhythmbox at least show embedded unsynchronized lyrics.
ogg/vorbis
ogg/vorbis is a patent-free audio codec and thus my preferred choice over mp3.
Tool support for lyrics is nearly non-existent.
Data storage
A small list of meta data tags for ogg/vorbis is listed in Vorbis I specification: 5.2.2. Content vector format and standalone in comment field and header specification. In addition, the xiph wiki lists proposed field names and links to three websites that propose some additional ones ( 1, 2, 3). Neither of them mentions lyrics.
The topic was mentioned on the vorbis mailing list in 2006 and 2008 with the conclusion that there is no standard, one should try CMML. I did not find any tools, example files or supporting clients for it, though.
OggKate can be used for synchronized lyrics, and kid3 uses the LYRICS tag to store unsyced lyrics in ogg files.
OggText was brainstormed in 2008 as well, but was never brought any further.
OggKate
In 2008 OggKate, a format for synchronized lyrics in ogg/vorbis files, was invented and implemented.
Kate is an overlay codec, originally designed for karaoke and text, that can be multiplexed in Ogg.
It does not only provide plain text lyrics but also animations, different colors, fonts and other styles.
Stream
It is not some data in a meta data tag field, but a separate stream inside an .ogg file, accompanying the music stream (just like movie files have both video and audio streams):
+----------+--------------------------+ | Metadata | Audio stream | +----------+--------------------------+
+----------+--------------------------+ | Metadata | A u d i o s t r e a m | | | L y r i c s s r e a m | +----------+--------------------------+
This has several implications:
-
✔ When playing the file, the text is available when it needs to be displayed.
No need to allocate extra memory for the lyrics, and no need for additional timing information in the lyrics data.
-
✔ Interweaving the audio and lyrics data makes it suitable for live audio streams that simply cannot have all the lyrics available at the beginning.
-
✘ Adding lyrics is not simply setting a tag but breaking up the whole file and re-assembling the whole file.
-
✘ Extracting lyrics requires reading the whole file and re-assembling the single pieces.
Players will need to do this if they not only want to display the current line or syllable, but provide a glimpse to the following ones.
The 99% standard use case - lyrics for music files of max. 10 minutes length - does not benefit from this format. Even operas with a length of several hours only have about 100kiB data of timed lyrics (LRC), which is nothing for yesterday's hardware.
Another argument for streaming was made on the Xiph MIME Types and File Extensions talk page:
If you want it without the timing, you have to store it in headers, as streaming it will get you the text only as its presentation time is reached.
You could do that if you were loading from a file, but that's only a special case, so it's best to leave that text interleaved with other streams.
A player wanting to display the entirety of the lyrics at once would have to, if possible (eg, if not streaming), scan the entire file to recover the text. Parsing Ogg packets is relatively fast, so a threaded player could do this while starting streaming a file and have the text ready in under a second for a typical song I suppose.
I don't believe the "special case" argument; even when you're playing the middle of a ogg/vorbis file via HTTP you still have to fetch the beginning of the file to get information about the header and general structure.
Tool support
Creating and reading OggKate streams is done via kateenc and katedec. oggenc's --lyrics only works if kate support is compiled in, which is not the case on Ubuntu 14.04.
The only tool able to display OggKate lyrics streams in audio files I found is VLC 2.2.2, if you start playing and then select the subtitle. (2.1.6 does not detect the subtitle stream at all).
Creating OggKate streams
kateenc from libkate-tools on Ubuntu 14.04 allows one to convert .lrc and .srt into kate streams, which then can be muxed into ogg audio files with oggz:
$ kateenc -o lyrics.ogg -t lrc -l en countdown.lrc $ oggz-merge -o countdown-with-lyrics.ogg countdown.ogg lyrics.ogg
Extracting the lyrics as LRC is possible with katedec:
$ katedec -t lrc oggz-kateenc-countdown.ogg
Playing files with Kate streams
Playing kate'd ogg/vorbis files works fine on VLC, mplayer, armarok, xine, audacious and ogg123 (even if they do not show the lyrics).
GStreamer-based applications like Rhythmbox and Totem open a new window that looks for extensions that understand "Kate decoder", but fail. Totem plays the file nontheless, but Rhythmbox does not. Amarok 2.8 sometimes opened that window, but also plays the file.
Patches
The OggKate author - who never left his name anywhere by the way - provided OggKate patches for several media players:
- xine
- The mailing list discussion did not lead to anything as it seems. Xine does not display kate lyrics.
- mplayer
- The patch was rejected; the mplayer authors didn't want to support another subtitle format and preferred ASS.
- gstreamer
-
The patch got included in 2009 into gst-plugins-bad 0.10.
Unfortunately it is not included in gstreamer 1.0 (yet), which explains the installation popup.
- Thoggen (DVD-Ripper)
- The feature request is still open for Thoggen.
Unsynced lyrics in .ogg
There is no officially endorsed vorbis tag/field for unsynchronized lyrics. The only tag in the wild I found was LYRICS which is created by the kid3 editor and lyrico.
We can add it with vorbiscomment from a lyrics text file:
$ vorbiscomment -a file.ogg -t LYRICS="$(cat lyrics.txt)" $ vorbiscomment --list countdown-unsync-vorbiscomment.ogg title=Countdown aligned artist=cweiske date=2016 genre=Speech encoder=Lavf53.21.1 LYRICS=ten nine eight seven six 5 4 3 2 1 0
oggz-comment can also be used to add unsynchronized lyrics to .ogg files:
$ oggz-comment file.ogg -o file-with-lyrics.ogg LYRICS="$(cat lyrics.txt)" $ oggz-comment -l file-with-lyrics.ogg Vorbis: serialno 0033215013 Vendor: Lavf53.21.1 title: Countdown aligned artist: cweiske date: 2016 genre: Speech encoder: Lavf53.21.1 LYRICS: ten nine eight seven six 5 4 3 2 1 0
Both Lollypop and Rhythmbox display them.
WMA
Windows Media Audio files support synced and unsynced lyrics in their meta tags. The ASF specification unfortunately does not define meta data tag names and thus also no meta data lyrics formats.
kid3 shows the synchronized lyrics in the .wma file with a tag name of WM/Lyrics_Synchronized.
Tools to add lyrics
Windows explorer on Windows XP is able to add unsychronised lyrics, Windows Media Player 11 could write both synchronized and unsynchronized lyrics.
Displaying lyrics
No tool I tested is able to display the lyrics embedded in .wma files while playing music.
exiftool at least shows them on cli, and kid3 notices that there are synchronized lyrics inside - but does not show them.
AAC
The Advanced Audio Coding format is mostly used by Apple's iTunes. I saw screenshots of the iTunes meta data editor that had a (unsynchronized) lyrics field. I did not follow this any further.
MiniLyrics claims to support lyrics in .m4a/.aac files.
Tools
The list of tools I found that could work with lyrics embedded in audio files, alphabetically sorted:
Clementine
Clementine is a music player that is able to show unsynchronized lyrics from mp3 files. Does neither support ogg lyrics nor synchronized text.
eyeD3
eyeD3 is a linux command line application that reads and writes mp3 tags, supporting ID3v1, 2.3 and 2.4.
Unfortunately it is not able to extract a single tag in a way that's readable by other programs, and is also not able to extract synchronized lyrics:
$ eyeD3 --verbose countdown-sync-minilyrics.mp3 countdown-sync-minilyrics.mp3 [ 258.24 KB ] ------------------------------------------------------------------------------- Time: 00:10 MPEG1, Layer III [ ~0 kb/s @ 44100 Hz - Mono ] ------------------------------------------------------------------------------- ID3 v2.4: title: Countdown aligned artist: cweiske album: year: 2016 boo track: genre: Speech (id 101) ------------------------------------------------------------------------------- ID3 Frames: <Title/>songname/content description (TIT2): Countdown aligned> <Lead performer(s)/Soloist(s) (TPE1): cweiske> <Release time (TDRL): 2016> <Content type (TCON): Speech> <Software/>Hardware and settings used for encoding (TSSE): Lavf53.21.1> <Synchronised lyric/text Frame (SYLT)>
It is able to add unsynchronized lyrics to mp3 files:
$ eyeD3 --lyrics="eng::ten nine eight seven six 5 4 3 2 1 0" countdown.mp3
exiftool
exiftool is the swiss army knife of meta tags. It's able to extract data from most image, document and audio file types.
It shows synchronized lyrics from mp3 and wma files, and unsynchronized from mp3, ogg and wma:
$ exiftool countdown-sync-kid3-id3v2.4.mp3 |grep -i lyr Synchronized Lyrics Type : Lyrics Synchronized Lyrics Description : Synchronized Lyrics Text : [00:00.00].ten, [00:01.00].nine, [00:02.00].eight, [00:03.00].seven, [00:04.00].six, [00:05.00].5, [00:06.00].4, [00:07.00].3, [00:08.00].2, [00:09.00].1, [00:10.00].0, [00:11.00]. $ exiftool countdown-sync-wmp11.wma |grep -i lyr Lyrics : ten..nine..eight..seven..six..5..4..3..2..1..0 Lyrics Synchronised : ..vsynchronized lyricstennine�.eight�.seven�.six�.5�.4�.3p.2X.1@.0(#
It does not show OggKate texts.
Foobar2000: show lyrics 3 plugin
The windows-only Foobar2000 supports plugins, and one of the plugins/"components" is Lyrics Show Panel 3 (foo_uie_lyrics3).
While it claims to be able to read both synced and unsynced lyrics from ogg and mp3 files, it does so by using its own (configurable) meta tag, storing LRC in the tag for synced text. It does not support SYLT/USLT tags.
kid3
The meta data editor kid3 runs on Linux and supports mp3, ogg and wma.
It has a usable synced lyrics editor and documentation for it. Importing LRC is supported.
kid3-qt was the only software that actually supported storing unsynced lyrics inside ogg/vorbis files by setting the LYRICS tag.
kid3-cli is a command line tool that can be used to read and write .lrc files from and to mp3 files.
Lollypop
Gnome application Lollypop supports SYLT in mp3 files, at least those that have been added by SYLT Editor, but not those by kid3.
It also reads .lrc files.
lyrico
lyrico automatically finds and downloads unsynchronized lyrics and is able to embed them into mp3, ogg, wma and m4a files.
It does not support synchronized text.
MiniLyrics
MiniLyrics is not available for Linux.
It reads synchronized and unsynchronized lyrics from mp3 meta data, but does not support the WMP11 generated lyrics in wma files, nor does it support .ogg. It claims to support M4A (AAC) files, but I did not test that.
It includes an editor for synchronized lyrics and can import LRC files.
Musique
Musique version 1.4 is able to read and display unsynchronized lyrics from mp3 files.
oggz-comment
oggz-comment from oggz-tools is able to read and write ogg/vorbis meta tags.
See the example how to use it to add unsynchronized lyrics.
Rhythmbox
Gnome's Rhythmbox displays unsynchronized lyrics in .ogg files because I made a patch for it.
SYLT Editor
SYLT editor is Freeware for Windows and Linux, but it did not install on Ubuntu 14.04 correctly.
On windows it worked, but proved to not support SYLT lyrics embedded by other editors. It even once did not read SYLT from a file that it wrote itself previously..
I can't recommend it.
VLC - VideoLan Client
VLC was the only application I found able to display OggKate lyrics in .ogg files - when enabling audio visualization, the text is printed on top of that. (version 2.2.2 worked, 2.1.6 did not)
It does not support SYLT, but you can load .srt subtitle files when playing music with audio visualization enabled.
vorbiscomment
vorbis-comment from vorbis-tools is also able to read and write ogg/vorbis meta tags.
See the example how to use it to add unsynchronized lyrics.
Windows Explorer
Windows XP's Explorer is able to show and edit meta data of .wma files - and it allows you to add unsynchronized lyrics. You cannot set the lyrics language as you could with Windows Media Player 11, though.
Windows Media Player 11
Windows Media Player 11 was available on Window XP and Windows Vista. To my knowledge it is the only application able to add synchronized lyrics to .wma files via its Advanced Tag Editor.
This tag editing capabilities were removed in Windows Media Player 12, much to the dismay of users.
Windows Media Player was not able to actually display the synchronized lyrics, which maybe was the reason for its removal. But the removal of the lyrics editing feature from WMP means that Microsoft officially has given up on lyrics in .wma, and maybe even on .wma itself.
If you ever try to install Windows Media Player 11 on an old copy of Windows XP, you'll notice that it will fail because it cannot prove the XP installation as "genuine" anymore.
You can work around this by manually extracting wmp11-windowsxp-x86-DE-DE.exe with 7-zip and manually executing all the wm*.exe files.
Hardware
List of hardware music players with lyrics support:
- iPod classic
- iPod nano (4th gen)
- Clicking 3 times on the wheel button shows the lyrics. source
Demo files
During the course of my research I created a number of audio files with the various tools. As base I used countdown.ogg from Corsica_S's countdown.wav but had to align it to the exact full seconds with Audacity. Then I created a lyrics .lrc file and used the various tools to add lyrics to it.
Apps missing embedded lyrics support
The following applications miss support for embedded lyrics:
- Amarok
- Bug 184325: amarok wish: fetch lyrics from the ID3 lyrics tag since 2009
- EasyTAG
- Bug 769201: Add lyrics tag support since 2016
- eyeD3
- Issue #103: Display synchronized lyrics since 2016
- GStreamer
- Bug 784634: unsynchronized lyrics not extracted from mp3 files since 2017
- music player daemon
- #3100: Add support for lyrics tag since 2010
- Musique
- Support for synchronized lyrics in MP3 files since 2016
- Read lyrics from ogg/vorbis files since 2016
- Quod Libet
- #1989: synchronized lyrics plugin: Support embedded lyrics since 2016
- Rhythmbox
- Bug 463978: Lyrics reading embedded tags since 2007. I added support for ogg/vorbis lyrics tag support in 2017.
- VLC
- #17207: Display mp3 synchronized lyrics SYLT tag as subtitle since 2016
Conclusion
I'm dissatisfied with the state of embedded lyrics support on Linux.
For ogg/vorbis files - of which my music library consists mostly - I did not find a satisfying solution. OggKate is too complex for most needs; something like SYLT for vorbis would be better suited - or even simply embedding LRC into a SYNCLYRICS tag.
The ID3v2 SYLT format is binary and would have to be base64 encoded, as it is done with album art in vorbis files, too.
And then, tool support would again be missing :(