My music library consists of mostly ogg/vorbis files, and I wanted to store the text that is sung directly in the files.
This is mostly uncharted territory - at least for .ogg files. It took three weeks of research to learn about the possibilities, solutions in other file formats and problems.
- Lyrics formats
- Demo files
- Apps missing embedded lyrics support
Lyrics can be synchronized or unsynchronized:
unsynchronized lyrics is just the song text as you see it in the booklet of a CD
It's plain text with newlines .
synchronized lyrics is the text plus timing information, which allows media players to show the exact line that is currently being sung. Karaoke depends on this.
In addition to the text lines, timing information is needed - be it for lines or even single words or syllables.
Standalone lyrics file formats
- Easy to read and easy to write text-only format: One line of text per lyrics line. Seems to be standard today for music.
- Very popular for movies. More verbose than LRC, but still plaintext.
- Binary format used in Karaoke CDs. The karaoke text is ripped into .cdg files which must have the same name as the audio files.
Let's have a look at MP3 first because tool support is better and there is an official standard for both synchronized and unsynchronized lyrics.
"Official" is not really correct here; the MP3 standard does not define how to store meta data in a file:
The MP3 standards do not define tag formats for MP3 files, nor is there a standard container format that would support metadata and obviate the need for tags.
However, several de facto standards for tag formats exist. As of 2010, the most widespread are ID3v1 and ID3v2, and the more recently introduced APEv2.
- Contains the encoding of the text, the language, a description and the actual song text. Multiple entries allowed, which means you can have lyrics in several language.
- Also contains encoding, language, the time stamp format (MPEG frames or milliseconds), type and time-stamped text in a defined format.
Tools to add lyrics
$ kid3-cli -c "set SYLT='countdown.lrc' ''" countdown.mp3
$ kid3-cli -c "get SYLT:/dev/stdout" countdown.mp3
On Linux, there is not a single application that displays synchronized lyrics stored in the file when playing an music track. All players that claim lyrics support either rely on local .lrc files or try to download the lyrics from somewhere on the internet.
ogg/vorbis is a patent-free audio codec and thus my preferred choice over mp3.
Tool support for lyrics is nearly non-existent.
A small list of meta data tags for ogg/vorbis is listed in Vorbis I specification: 5.2.2. Content vector format and standalone in comment field and header specification. In addition, the xiph wiki lists proposed field names and links to three websites that propose some additional ones ( 1, 2, 3). Neither of them mentions lyrics.
The topic was mentioned on the vorbis mailing list in 2006 and 2008 with the conclusion that there is no standard, one should try CMML. I did not find any tools, example files or supporting clients for it, though.
Kate is an overlay codec, originally designed for karaoke and text, that can be multiplexed in Ogg.
It does not only provide plain text lyrics but also animations, different colors, fonts and other styles.
It is not some data in a meta data tag field, but a separate stream inside an .ogg file, accompanying the music stream (just like movie files have both video and audio streams):
+----------+--------------------------+ | Metadata | Audio stream | +----------+--------------------------+
+----------+--------------------------+ | Metadata | A u d i o s t r e a m | | | L y r i c s s r e a m | +----------+--------------------------+
This has several implications:
✔ When playing the file, the text is available when it needs to be displayed.
No need to allocate extra memory for the lyrics, and no need for additional timing information in the lyrics data.
✔ Interweaving the audio and lyrics data makes it suitable for live audio streams that simply cannot have all the lyrics available at the beginning.
✘ Adding lyrics is not simply setting a tag but breaking up the whole file and re-assembling the whole file.
✘ Extracting lyrics requires reading the whole file and re-assembling the single pieces.
Players will need to do this if they not only want to display the current line or syllable, but provide a glimpse to the following ones.
The 99% standard use case - lyrics for music files of max. 10 minutes length - does not benefit from this format. Even operas with a length of several hours only have about 100kiB data of timed lyrics (LRC), which is nothing for yesterday's hardware.
If you want it without the timing, you have to store it in headers, as streaming it will get you the text only as its presentation time is reached.
You could do that if you were loading from a file, but that's only a special case, so it's best to leave that text interleaved with other streams.
A player wanting to display the entirety of the lyrics at once would have to, if possible (eg, if not streaming), scan the entire file to recover the text. Parsing Ogg packets is relatively fast, so a threaded player could do this while starting streaming a file and have the text ready in under a second for a typical song I suppose.
I don't believe the "special case" argument; even when you're playing the middle of a ogg/vorbis file via HTTP you still have to fetch the beginning of the file to get information about the header and general structure.
Creating and reading OggKate streams is done via kateenc and katedec. oggenc's --lyrics only works if kate support is compiled in, which is not the case on Ubuntu 14.04.
Creating OggKate streams
kateenc from libkate-tools on Ubuntu 14.04 allows one to convert .lrc and .srt into kate streams, which then can be muxed into ogg audio files with oggz:
$ kateenc -o lyrics.ogg -t lrc -l en countdown.lrc $ oggz-merge -o countdown-with-lyrics.ogg countdown.ogg lyrics.ogg
Extracting the lyrics as LRC is possible with katedec:
$ katedec -t lrc oggz-kateenc-countdown.ogg
Playing files with Kate streams
Playing kate'd ogg/vorbis files works fine on VLC, mplayer, armarok, xine, audacious and ogg123 (even if they do not show the lyrics).
gstreamer-based applications like Rhythmbox and Totem open a new window that looks for extensions that understand "Kate decoder", but fail. Totem plays the file nontheless, but Rhythmbox does not. Amarok 2.8 sometimes opened that window, but also plays the file.
- The mailing list discussion did not lead to anything as it seems. Xine does not display kate lyrics.
- The patch was rejected; the mplayer authors didn't want to support another subtitle format and preferred ASS.
The patch got included in 2009 into gst-plugins-bad 0.10.
Unfortunately it is not included in gstreamer 1.0 (yet), which explains the installation popup.
- Thoggen (DVD-Ripper)
- The feature request is still open for Thoggen.
Unsynced lyrics in .ogg
$ vorbiscomment -a file.ogg -t LYRICS="$(cat lyrics.txt)" $ vorbiscomment --list countdown-unsync-vorbiscomment.ogg title=Countdown aligned artist=cweiske date=2016 genre=Speech encoder=Lavf53.21.1 LYRICS=ten nine eight seven six 5 4 3 2 1 0
$ oggz-comment file.ogg -o file-with-lyrics.ogg LYRICS="$(cat lyrics.txt)" $ oggz-comment -l file-with-lyrics.ogg Vorbis: serialno 0033215013 Vendor: Lavf53.21.1 title: Countdown aligned artist: cweiske date: 2016 genre: Speech encoder: Lavf53.21.1 LYRICS: ten nine eight seven six 5 4 3 2 1 0
Tools to add lyrics
No tool I tested is able to display the lyrics embedded in .wma files while playing music.
The list of tools I found that could work with lyrics embedded in audio files, alphabetically sorted:
Clementine is a music player that is able to show unsynchronized lyrics from mp3 files. Does neither support ogg lyrics nor synchronized text.
Unfortunately it is not able to extract a single tag in a way that's readable by other programs, and is also not able to extract synchronized lyrics:
$ eyeD3 --verbose countdown-sync-minilyrics.mp3 countdown-sync-minilyrics.mp3 [ 258.24 KB ] ------------------------------------------------------------------------------- Time: 00:10 MPEG1, Layer III [ ~0 kb/s @ 44100 Hz - Mono ] ------------------------------------------------------------------------------- ID3 v2.4: title: Countdown aligned artist: cweiske album: year: 2016 boo track: genre: Speech (id 101) ------------------------------------------------------------------------------- ID3 Frames: <Title/>songname/content description (TIT2): Countdown aligned> <Lead performer(s)/Soloist(s) (TPE1): cweiske> <Release time (TDRL): 2016> <Content type (TCON): Speech> <Software/>Hardware and settings used for encoding (TSSE): Lavf53.21.1> <Synchronised lyric/text Frame (SYLT)>
It is able to add unsynchronized lyrics to mp3 files:
$ eyeD3 --lyrics="eng::ten nine eight seven six 5 4 3 2 1 0" countdown.mp3
It shows synchronized lyrics from mp3 and wma files, and unsynchronized from mp3, ogg and wma:
$ exiftool countdown-sync-kid3-id3v2.4.mp3 |grep -i lyr Synchronized Lyrics Type : Lyrics Synchronized Lyrics Description : Synchronized Lyrics Text : [00:00.00].ten, [00:01.00].nine, [00:02.00].eight, [00:03.00].seven, [00:04.00].six, [00:05.00].5, [00:06.00].4, [00:07.00].3, [00:08.00].2, [00:09.00].1, [00:10.00].0, [00:11.00]. $ exiftool countdown-sync-wmp11.wma |grep -i lyr Lyrics : ten..nine..eight..seven..six..5..4..3..2..1..0 Lyrics Synchronised : ..vsynchronized lyricstennine�.eight�.seven�.six�.5�.4�.3p.2X.1@.0(#
Foobar2000: show lyrics 3 plugin
While it claims to be able to read both synced and unsynced lyrics from ogg and mp3 files, it does so by using its own (configurable) meta tag, storing LRC in the tag for synced text. It does not support SYLT/USLT tags.
kid3-qt was the only software that actually supported storing unsynced lyrics inside ogg/vorbis files by setting the LYRICS tag.
lyrico automatically finds and downloads unsynchronized lyrics and is able to embed them into mp3, ogg, wma and m4a files.
It does not support synchronized text.
It reads synchronized and unsynchronized lyrics from mp3 meta data, but does not support the WMP11 generated lyrics in wma files, nor does it support .ogg. It claims to support M4A (AAC) files, but I did not test that.
It includes an editor for synchronized lyrics and can import LRC files.
oggz-comment from oggz-tools is able to read and write ogg/vorbis meta tags.
On windows it worked, but proved to not support SYLT lyrics embedded by other editors. It even once did not read SYLT from a file that it wrote itself previously..
I can't recommend it.
VLC - VideoLan Client
vorbis-comment from vorbis-tools is also able to read and write ogg/vorbis meta tags.
Windows XP's Explorer is able to show and edit meta data of .wma files - and it allows you to add unsynchronized lyrics. You cannot set the lyrics language as you could with Windows Media Player 11, though.
Windows Media Player 11
Windows Media Player 11 was available on Window XP and Windows Vista. To my knowledge it is the only application able to add synchronized lyrics to .wma files via its Advanced Tag Editor.
Windows Media Player was not able to actually display the synchronized lyrics, which maybe was the reason for its removal. But the removal of the lyrics editing feature from WMP means that Microsoft officially has given up on lyrics in .wma, and maybe even on .wma itself.
If you ever try to install Windows Media Player 11 on an old copy of Windows XP, you'll notice that it will fail because it cannot prove the XP installation as "genuine" anymore.
During the course of my research I created a number of audio files with the various tools. As base I used countdown.ogg from Corsica_S's countdown.wav but had to align it to the exact full seconds with Audacity. Then I created a lyrics .lrc file and used the various tools to add lyrics to it.
Apps missing embedded lyrics support
The following applications miss support for embedded lyrics:
- Bug 184325: amarok wish: fetch lyrics from the ID3 lyrics tag since 2009
- Bug 769201: Add lyrics tag support since 2016
- Issue #103: Display synchronized lyrics since 2016
- music player daemon
- #3100: Add support for lyrics tag since 2010
- Support for synchronized lyrics in MP3 files since 2016
- Read lyrics from ogg/vorbis files since 2016
- Quod Libet
- #1989: synchronized lyrics plugin: Support embedded lyrics since 2016
- Bug 769180: Lyrics plugin: Read lyrics embedded in audio file since 2016
- #17207: Display mp3 synchronized lyrics SYLT tag as subtitle since 2016
I'm dissatisfied with the state of embedded lyrics support on Linux.
For ogg/vorbis files - of which my music library consists mostly - I did not find a satisfying solution. OggKate is too complex for most needs; something like SYLT for vorbis would be better suited - or even simply embedding LRC into a SYNCLYRICS tag.
The ID3v2 SYLT format is binary and would have to be base64 encoded, as it is done with album art in vorbis files, too.
And then, tool support would again be missing :(