Home Uniquely NZ Travel Howto Pauline Small Firms Search
Mint and Ubuntu Linux making Music


Contents

Introduction

The main page covering my experiences in making the transition from Microsoft Windows to Ubuntu Linux - Fun with Ubuntu Linux quickly become excessively long.  This is the latest in the series and was initially created to contain my experiences in archiving all my old LP records and cassette tapes onto the computers in MP3 format in the same way as I have been transcribing all by old VHS tapes and early analogue camcorder tapes. It has now been extended to include CD ripping and Music Players and Libraries

Ripping CDs

Introduction to Ripping

Ripping means extracting audio or video content typically from a CD to a useful form on your computer. Ripping is distinct from simple file copying and in most cases the output will be split into individual compressed files corresponding to each track. CD audio has two major design constraints that make it difficult to obtain accurate transfers to standard files. The CD system was designed to provide audio in real time in order to ensure continuous playback without gaps and the designers felt that it would be preferable for damage to the disc to be covered up rather than resulting in total failure. Normally, an error correction system such as Reed Solomon would provide either a perfect copy of the original error-free data, or no result at all. However, CD audio's Cross-interleaved Reed-Solomon coding includes an extra facility that interpolates across uncorrectable errors. This means that the data read from an audio CD may not in fact be a faithful reproduction of the original. Ripping involves specialized software that will attempt to correct errors, and will use a variety of techniques, such as making use of error correction information, knowledge of the peculiarities of different drives, and ripping multiple times and comparing the results. Ripping programs often go a lot further than just extracting a series of files - they usually encode them into a compressed format such as MP3 and put them into a suitable folder structure and with suitable tags added.

Tags and use of CD Data bases.

Most music file formats such as MP3 can and almost always have Metadata (Tags) containing information on Artist, Album Title, track Number, track Title, Genre, Year etc which is used by the music playing programs. Subsets of this data are often used to create the names of the files and folders in the structure of a music library. This information can be inserted by hand from the CD cover but over time large collaborative databases have been built up so once the first person has transcribed the information from a new CD and uploaded it everyone else can look it up and download it. Most Rippers have the facility to access one or other of the main CD Data Bases (CDDBs). Various matching criteria are used including track lengths. The information is often not perfect and may need editing and compendiums of popular music are often offered under different labels at different time in different languages but overall it saves a lot of work. Freedb was the original large database but has not been kept up as well since 2006 and MusicBrainz is now taking over and has tools such as Picard if the simple matching in your ripper does not work.

More about ID3 tags

Before we go to far we need to know enough about how MP3 files have metadata associated with them to be able to choose which versions to add when we rip files to balance utility with support by common media players.

ID3 is a metadata container most often used in conjunction with the MP3 audio file format. It allows information such as the Artist, Album Title, track Number, track Title, Genre, Year and other information about the file to be stored in the file itself. There are two largely unrelated versions of ID3: ID3v1 and ID3v2 but both work by adding a small chunk of data to the audio file.

The earlier ID3v1 tag occupies 128 bytes placed at the end of the file to maintain compatibility with older media players. This tag allowed 30 bytes each for the title, artist, album, and a "comment", four bytes for the year, and a byte to identify the genre of the song from a predefined list of 80 values. This is now enhanced with an extra data block before an ID3v1 tag, which extends the title, artist and album fields by 60 bytes each, offers a freetext genre, a one-byte (values 0-5) speed and the start and stop time of the music in the MP3 file.

The new standard called ID3v2 was created in 1998. ID3v2 tags are of variable size, and occur at the start of the file, to aid streaming media. They consist of a number of frames, each of which contains a piece of metadata. Frames can be up to 16MB in length, while total tag size is limited to 256MB. Internationalization is solved by allowing the encoding of strings in UTF-16 or UTF-8 in version ID3v2.4. In the latest ID3v2 standard there are 84 types of frame, and applications can also define their own types. There are standard frames for containing cover art, BPM, copyright and license, lyrics, and arbitrary text and URL data, to name a few.

ID3v2.4 is the latest version of the standard, dated November 1, 2000. Notably, it allows textual data to be encoded in UTF-8, which was a common practice in earlier tags. Microsoft however is however lagging the standards even more than usual and apparently Windows can only understand ID3v2 up to and including version 2.3. Windows Explorer and Windows Media Player still cannot handle ID3v2 version 2.4 tags in any version, up to and including Windows 7/Windows Media Player 12. If you want to know more about ID3 tags and their support The Wikipedia article is, as usual, a good starting point.

File and Folder Structures

I would like to be able to say that the folder structure is of no importance in how modern collections of music are organised and that they only depend on the metadata. Most music players organise (index) music their music collection based primarily on the metadata and allow you to sort, display and search on - at a minimum - the main fields which are contained in the metadata (album , artist, track, genre, track number and year) . You will note I have not included 'Cover Art' in the list. The folder structure is much less well defined and is largely determined by the ripping software and the details can often be configured within the ripping software for CDs by the user making it of little use in organising a music collection.

In practice there are a number of constraints and conventions. The concept of albums has a physical basis. An album was based on a record or CD, sometimes sets of records or CDs. Each Album needs to have a unique title in the music collection and comprises a number of tracks which are individual audio files which have the same album title in their metadata and by convention are in a single folder. It is slightly less clear how sets of CDs should be handled but each CD normally has its own folder. The concept of individual folders for each album is a requirement for the most common way of implementing album art which was not included in the original ID3 metadata standards.

The folder and file names are usually derived from the album title, artist, track title and track number and may be at a single or double level depending on the way the CD has been ripped or a download has been put together. The file and folder names are rarely (if ever) used in organising a collection of music by a player. There are some problems in the automatic creation of these names as they may be very long and can contain characters not allowed in files and folder names, in particular, under windows where colons and slashes are obvious problems as well as overall path lengths being exceeded. I have also had problems on android where the standard file manager has prevented me renaming files to have over 50 characters.

File types recognised: The folder containing the music files often contains other files as well but only a small number will be used by any music player in creating the collection.

Other file types are ignored by most players

MP3 and LAME

MP3 (short for MPEG-1 or MPEG-2 Audio Layer 3), is a patented digital audio encoding format using a form of lossy data compression. MP3 is the most common format for consumer audio storage, as well as a de facto standard for transfer and playback of music on digital audio players. It was designed by the Moving Picture Experts Group as part of its MPEG series of standards. The audio subgroup was formed by several teams of engineers at Fraunhofer University of Hannover and others before becoming part of the ISO/IEC standards. MP3 uses a very sophisticated lossy compression algorithm which is designed to greatly reduce the amount of data required to represent the audio recording and still sound like a faithful reproduction of the original uncompressed audio for most listeners. An MP3 file that is created using a setting of 128 kbit/s will result in a file that is about 11 times smaller than the original. MP3 is based on use psychoacoustic models to discard or reduce precision of components less audible to human hearing, and then records the remaining information in an efficient manner which can include Variable Bit Rates (VBR) so most data can be given to sections which are audibly important.

The most common and best OpenSource encoder and decoder is called LAME (LAME is a recursive acronym for "LAME Ain't an MP3 Encoder") and was created by Mike Cheng along with a new psychoacoustic model (gpsycho) for research. Like all MP3 encoders, LAME implements some technology covered by patents owned by the Fraunhofer Society and other entities. The developers of LAME do not themselves license the technology described by these patents so distributing compiled copies of LAME, its libraries, or programs which are derivative works may be considered infringing on the relevant patents but source code for research is acceptable. In some countries and for commercial sales a licence is needed from the patent owners. The whole area is sufficiently complex that Ubuntu does not include LAME by default and it has to be installed separately. Interestingly Wikipedia reports that the notorious Extended Copy Protection rootkit included on some Sony Compact Discs included portions of the LAME library without complying with the terms of its licence conditions!

LAME has huge number of variable parameters and modes of operation and much acoustic research is carried out to decide what is best for the majority of people. Higher bit rates give better results at the expense of file size so the Variable Bit Rate solutions are best and there are a number of 'presets' available. If you want to get into it the http://wiki.hydrogenaudio.org/index.php?title=LAME is good and up to date. Some of the rippers give you an ability to change some parameters whilst still maintaining a GUI interface although some use Gstreamer to give a common access to many formats in a standardised way. GStreamer is a pipeline-based multimedia framework which allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. The pipeline design serves as a base to create many types of multimedia applications such as video editors, streaming media broadcasters, and media players. It is cross-platform and is known to work on Linux, Mac OS X and Microsoft Windows. The GNOME desktop environment is the primary user of GStreamer technology and encourages GNOME and GTK+ applications to use it. This all sounds perfect except that the default way it implements MP3 via LAME is an inconsistent disaster which has not been sorted after many years.

System Software Packages required for MP3 and other media support.

Using the Synaptic Package Manager to add programs

The Synaptic package manager is similar to Add/Remove but at a lower level and used more for adding services than main stream programs although it will do so just as well - we could have used it for Thunderbird we are going to be adding a number of library routines to for playing and ripping media files. The package manager is clever and every program in Ubuntu carries the information on its dependence on any other services and automatically downloads them, if required.

Open the Synaptic Package Manager by System -> Administration -> Synaptic Package Manage.

When the Synaptic Package Manager has opened use Search to find the package/program you want, click the box and click Mark for Installation each one. When they are all marked, click Apply. The following are from the list I use when setting up a basic machine.

This is a list of useful packages to install using the Synaptic Package Manager

Ripping Software

Sound Juicer

Screenshot

I have used Sound Juicer to simply rip most of my CDs but it seems that the options it uses are far from optimum and that it is almost impossible to set it up to do a better job. It uses the Musicbrainz CDDB and everything runs automatically. You will however need to have installed the LAME support for MP3 as above before you start as it is not automatically loaded due to legal considerations in some countries. if it is not in place you will not see it as an option for the output format. The current version has less options and only encodes to .ogg format which reduces compatibility with Windoz.

RipperX

Screenshot

This is in theSoftware Centre does an excellent job of high quality conversions to MP3 files so it is worth loading. You need must install the original version lame using synaptic, the library versions pulled in by ubuntu-restricted-extras is not sufficient.

Ripperx is set up to access the freedb CDDB but it is easy to change to a Gateway to the MusicBrainz database which gives access to an alternative and sometimes more up-to-date database. You can see the information which it has found for this old CD is more complete than MusicBrainz found for Sound Juicer in this case. The information you need to swap is:

host is: freedb.musicbrainz.org
access via: http
on port: 80
with: path /~cddb/cddb.cgi


In practice this is quite easy Click Configuration at the top -> CDDB tag -> and editing text box marked URL containing the line freedb.freedb.org/~cddb/cddb.cgi to freedb.musicbrainz.org/~cddb/cddb.cgi and it switches over.

Screenshot

Currently the original freedb database has ben taken down and has a new home at Gnudb which can be accessed by:

URL: gnudb.gnudb.org/~cddb/cddb.cgi
Port: 80
Use HTTP
Proxy Server: proxy.gnudb.org
Proxy Port: 3128

See this if you need any more info: https://www.gnudb.org/howto.php

It is not quite as automatic as on Sound Juicer and you have to make sure the CD is in the correct drive then click Scan, CDDB and Go in turn. The folder you use is set up in Configuration -> General and you may want to set up the way the file names are created in the files tab. Once it is all configured it is very quick Scan and look up the CDDB - if you do not find the CD you can change between freedb and MusicBrainz as above. You can also change the quality and resulting sizes on the MP3 tab.

Conclusions on choice of Ripper for CDs

I have used Sound Juicer (on menus as Audio CD Extractor) for most of my CDs so far without realising the problems it had. RipperX uses the older freedb for CD lookup but easily be changed to give a choice of MusicBrainz which is a positive. So perhaps the answer for a newbie is to experiment initially using Sound Juicer but be prepared switch and to redo some CDs if you need higher quality in the future - that is my situation and apathy, midrange equipment and an undemanding nature probably mean nothing will be done.

Problems with Filenames and Windows Filesystems when using freedb and musicbrainz

Most of the ripping software places the output in a file structure based on the meta information delivered by freedb and musicbrainz. These can result in file names which are valid in Linux filesystems but not in Windows filesystems such as FAT32 or NTFS. This becomes a problem if you transfer to a memory stick or synchronise with Unison to an external drive or Windoz machine. Obvious problems occur with colons (:), slash / and backslashs \ and * and ? are best avoided. The answer is to look very closely at what is delivered by freedb, musicbrainz and any other services and edit them before you rip the CD - that way the metadata added to the tracks is correct as well as having sanitised file names which will not give trouble in the future. You also have the advantage of having the CD to check the metadata is correct. Sound Juicer has options to strip special characters which should be set in any case. Ripperx only has the option to convert spaces to underscores which is less useful.

I have been unable to find a general purpose routine to sanitise filenames in bulk but this article http://stackoverflow.com/questions/2709458/bash-script-to-replace-spaces-in-file-names gives a lot of information which should enable you to convert it to remove colons which were my main problem. You can check and correct a few problems using the search in the Nautilus file browser to look for : ? * \ etc in turn. That way you will not get too many surprises latter.

What I have actually done is to use the simplest approach to remove : with separate commands for folders and filenames using rename which is a Perl script which was on my system already. I did it in two steps replacing : with space -

cd Music
find -name "*:*" -type d | rename 's/:/ -/g'
find -name "*:*" -type f | rename 's/:/ -/g'

 

Transferring Cassette Tapes and LPs to MP3

This has turned out to be both easier and more difficult than I had initially expected. I have found an excellent program for recording, editing and converting to MP3 and many other formats called Audacity which is available through the Ubuntu Software Centre and The Synaptic Package Manager. This has good documentation with lots of tutorials, faqs and reference documents to help. The difficult part has been setting up the system side although again there are some very complete tutorials even showing pictures of the connectors you use on a normal computer. The sound input is normally through a 3.5 mm 'miniature' jack which the built in sound cards or motherboards support on most desktops (blue socket) but not on any of my laptops. You can monitor this in Audacity but it is best to connect it directly through your sound system. Ubuntu now presents one with a very simple main interface to the sound system via an Applet in the tool section of the panel. This gives you access to a slider for volume and some very simple preferences which are not sufficient to set up you full sound system for recording and simultaneous monitoring so you need to install a full mixer with access to all the features of ALSA (Advanced Linux Sound System).

Setting up the Sound System for Audacity

Gnome ALSA Mixer

The panel Applet and that through System -> Preferences -> Sound gives minimal access and control of the sound system so you may need to access a proper Mixer and Control Panel for controlling the ALSA sound system - the obvious is the Gnome ALSA Mixer which is available through the Software Centre as ALSA Mixer and provides a nice GUI interface to a huge number of parameters. Unfortunately it gives an error message when you start it or try to set its preferences but this does not seem to effect its basic operation. This is a known bug which has come and gone with various versions for many years which nobody seems to think worth fixing. The full panel stretches the whole width of my screen so I have only been able to show the area of most importance to sound recording.

Screenshot

The two controls we are most interested in are the 'Line' input and this needs to have the mute checkbox cleared so we can listen to the input stream and we need to control the recording level with the 'Capture' controls where we set up the level and the balance. You also need to make sure the Microphone - and other possible sound sources - are muted and not set to be recorded. The cursor on the picture above is on the balance - for some reason almost all Cassette recordings are not balanced and it is best to get this about right although you can do a final adjustment in Audacity. Likewise you want the best signal to noise but not to saturate so again the overall level should be set so that the highest level is about or just above -3dbs but never clipping - this may mean a test run or quick abort on each cassette. LPs seem more consistent and have a better balance between channels. Audacity has record meters built in which show the peak as well as instantaneous levels and also have the two tracks displayed. You can drag the end of the meters to increase the size which is nice and likewise drag the bottom of the tracks down to increase their size to aid adjustment.

Alsamixer (terminal program)

The alternative to the Gnome ALSA Mixer if you find error messages non-acceptable is the terminal version alsamixer which takes some getting used to - when you start it use it on the keyboard immediately to get a help screen or you will take hours to work it out! See below for the help screen (note the windows have been reduced to the minimum practical below).

 

Screenshot

This is the Help display you get with the h or H key - you will need it!

Screenshot

We have done an F3 to get to Playback settings, Left/ Rights to get to the adjustments for the 'Line' input, M to toggle Mute off so we can listen as well as capture it, and Up/Down to change the volume.

 

Screenshot

This shows how to set the window to be Always on Top

Screenshot

We have reduced the size of the window even more, F4 for Capture view and then used the arrow keys to get to the 'Capture' adjustments. We can mute/unmute with M and enable/disable with the space bar. We can now adjust the Left and Right channels independently with the q, z, e and c keys.

What I normally do first is to start Audacity and Gnome Alsa Mixer both running so I can watch and get the levels and balance correct on a test recording started by the red button. I Find it is often best to keep the mixer program I am using always running 'Always on Top' in as small a window as possible. You can do this for any program by clicking on the little button at the very top left (above file) or right clicking for a menu on the program on the bottom panel. You can then watch record meters built in to Audacity which show the peak as well as instantaneous levels and the tracks which are displayed whilst you are setting the level and balance. You can drag the end of the meters to increase the size which is nice and likewise drag the tracks down to increase the size. You can use Ctrl 1 and Ctrl 3 (or one of the zoom icons which look like little magnifying glasses on the toolbar) to zoom in and out during the recording to get an idea of the balance and levels as the trial progresses.

Gamix ALSA Mixer

More recently I have found yet another mixer available from the Ubuntu Software Centre namely Alsa Mixer (Gamix). This has no problems when being started but crashes if you try and access the configuration menu under Prog. Its display is similar to Gnome ALSA-Mixer but the sliders are jerky in operation so I have ended up using Gnome ALSA-Mixer most of the time. For completeness its window is shown below:

Screenshot

Pulse Audio Mixer (pavucontrol)

I am using this as part of the controls for input from a USB cassette player.

Capturing the sound from the Cassette or Record using Audacity

Once I have the settings it is time to capture the full cassette tape.

Processing the sound using Audacity

I have only used a fraction of the facilities so far but for cassette tapes I:

Creating an MP3 in Audacity

I am assuming that the LAME encoder/decoder is already loaded- it may not be by default as it is counted as proprietary and protected in some countries. LAME (LAME Ain't an MP3 Encoder) is a research project for learning about and improving MP3 encoding technology. You can find out by looking in Edit -> Settings -> If it is not installed try Synaptic and Search for lame and install ubuntu-restricted-extras libmp3lame0 and libtwolame0 - These LAME packages include an MP3 encoding library, a simple front-end application, and other tools for sound analysis, as well as convenience tools.

Now make the MP3 by File -> Export and select MP3 in the File Type drop down and then click on the Properties button.

Screenshot

Creating an MP3 in Audacity

I have entered the name of the file and chosen the folder where it will be put. I have selected MP3 files and now need to click Options to set up the MP3 quality.


Screenshot
The default is a fixed encoding at 128 Kbps which is predictable but we can do better by using one of the presets recommended by the writers of LAME as shown in the screenshot.
Screenshot
When we click OK a couple of times we are taken to a screen where we set up the metadata for the file which will be displayed when we are choosing it and playing it in Rhythmbox or your favourite player.

Then you just have to wait. Each of the Effects and the final writing take 5 to 10% of the length of the tape so allow an elapsed time for a conversion of about 150% of 'tape' time much of which you can be doing other things.

Saving Time

When you have had some practice you can investigate chaining the commands to save interaction and tags and exporting multiple files to separate your tracks.

Cautions

Audacity uses a lot of disk space especially with lots of undo depth. I have a drive with 17 Gbytes free and I run out of space unless I exit the program after every couple of cassette tapes - it takes a long time when you exit as it is recovering all the temporary space etc. so do not respond to any messages saying it is not closing and do you want to do a forced quit. So save your project often and then you can recover if you run out of space. Watch the status bar as it will tell you how much time you have left when recording.

Mixer confusion problems leading to no sound being recorded

Sometimes I find that I am getting no capture at all - it is worth then checking the volume and control settings which are access from the volume control in the panel as the setting for sound preferences -> input sometimes seem to get out of step with those from Alsamixer and the input is muted. This is the only place I have found where I can select my new USB cassette player and the setting is not 'sticky' one has to select it every time I plug it in. Only after that has been done can one see the USB input in the other mixers. The correct settings for a normal analog sound input are below:

Screenshot

Music transfers and my 1991 Panasonic RX DS660 Boombox

Once I started on the task of converting a large number of cassette tapes to mp3s using my USB cassette player using old machines with systems using Lubuntu or Cinnamon I found I really needed to be able to listen to the music during the long process so I retrieved my old Panasonic RX DS660 music system from the loft. I used it for many years on our narrowboat - it was bought largely because it had an option of running off 12v and had auxilary inputs as well as tape, FM radio and CD. It was often sold as a Boombox but actually produced good if not HiFi quality sound and was very advanced for 1991. It had only been replaced by a more modern Sony car system because it intermittently lost one channel and until recently I was still using the speakers which are removable. Last time I looked at trying to fix it I failed to find any way of opening the box without damage so this time I started with a big search of the internet looking for a service manual and finally came on a site offering free manuals which had a copy but not uploaded yet. The site was www.freeservicemanuals.info and I emailed them and within a couple of days they emailed back to say it had been scanned and uploaded. The manual had all the information to open the box and identify what was actually a very simple fault in a jack plug for headphones which was dirty and blanking the sound from one channel even when not pluged in - solved with the miracle cure WD40 in minutes after a five year search for manuals! I can not thank the guys at www.freeservicemanuals.info enough for their quick response and thoroughly recommend that it should be your first stop for any old manuals you need. It is a matter of principle with me not to replace but repair and the Panasonic RX ds660 is now a collectors item as a classic Boombox which also produces high quality sound which is now filling our library whilst I am transfering all my tapes. I will send them any old manuals I find to add to the collection.


Screenshot

I had forgotten how good it was and I dread to think how much a modern system of the same quality would be. So I now have an early 1990s Boombox linked by Bluetooth to a network of Linux computers - a classic mix of new and old repaired thanks to the internet and Opensource.

Editing MP3 Tags (ID3 MetaData)

Ex Falso

Audacity does an excellent job of capturing and cleaning up Cassette Tapes and LPs and converting into MP3s and you can set the Tag information when you save them. I have however made a number of mistakes and one does not want the loss of bringing the files back into Audacity so I searched for a simple tag (metadata) editor and found Ex Falso which can be loaded by the Synaptic Package Manager. It is all very self explanatory and I have not even looked to see if it has help files! It allows one to make changes to multiple files which you need when changing every track in a recording.

Quod Libet

Quod Libet is a Music Player which uses, what seems to be a very close relative of Ex Falso internally to edit the Metadata. It is in many ways easier to use at you have the tracks etc in front of you when you are editing.

Rhythmbox

Rhythmbox is my chosen music player for Linux and it also has a perfectly adequate tag editor intergrated in so it is my first choice for editing MP3 tags, I am not sure how it copes with other formats. It is easy to group tracks to do global edits and it is easy to move from individual tracks to adjacent ones making edits quick.

Puddletag

Puddletag is a dedicated and powerful tag editor for many formats similar to Mp3tag for Windows.

I have it installed it from the WebUpd8 PPA but have not used it so far. It as facilities to handle album art and can lookup and collect tag data from a number of sources such as:

This should enable one to collect tags in the same way as rippers do but in a much more flexible way and from multiple sources and also search for album art.

To quote the author:

[Puddletag supports] the usual tag editor features like extracting tag information from filenames, renaming files based on their tags by using patterns and basic tag editing.

Then there arere Functions, which can do things like replace text, trim it, do case conversions, etc. Actions can automate repetitive tasks. Doing web lookups using Amazon (including cover art), Discogs (does cover art too!), FreeDB and MusicBrainz is also supported.

Supported formats include: ID3v1, ID3v2 (mp3), MP4 (mp4, m4a, etc.), VorbisComments (ogg, flac), Musepack (mpc), Monke's Audio (.ape) and WavPack (wv).

The downside of such a powerful tool is that it is not intuitive and has a steep learning curve. I have had some success in trials on test data but it can have a lot of unintended consequence including adding a lot of extra tags when you do web lookups. Exercise caution, work on single folders and keep lots of backups.

Basic utilities such as eyeD3 run from the command line.

A great way for a power user to do actions across your whole music collection or destroy it completely. See below in the Album Art section on how to embed Album Art in ID3 tags for your whole music collection in a single command using eyeD3. Also useful if you really want to see what you have as tags.

Music Players and Library Managers

Rythmbox

Screenshot

Rythmbox is the default music player in Mint and can play directly from the 'filtered' library or by dragging tracks or an album onto a playlist or into the playqueue. Combined with the Browse mode - shown above - the selection is quite powerful. You normally set it to watch your Music Library location and automatically update the Library. It should be integrated with Sound Juicer to rip CDs for your Library.

Clearing the Rhythmbox Music Database

I have been moving the location of my music library from one drive to another and onto mount points for remote file systems (see below) and I have noticed that there isn't any clear way to change library locations once you have set them - you still have all the old locations in your music library as well as the new ones.

Deleting the file ~/.local/share/rhythmbox/rhythmdb.xml should just remove the music database and when you reopen Rhythbox it should rescan the new location.

rm ~/.local/share/rhythmbox/rhythmdb.xml

The remainder of the configuration can be cleared by removing these folders - they will be replaced next time Rythmbox is started:

~/.local/share/rhythmbox
~/.cache/rhythmbox
~/.gconf/apps/rhythmbox.

by:

rm ~/.local/share/rhythmbox/ ~/.cache/rhythmbox/ ~/.gconf/apps/rhythmbox/ -r

Exaile (Currently not available in Mint/Ubuntu Repositories)

Screenshot

Exaile has a number of nice features with a good display for the collection on the left which can display based on a number of headings. The search can be very powerful as you can specify where the key words are ie artist=beethoven album=9 album=symphony although you only seem to be able to get to the artist when it is at album and not different for every track You always play by dragging an album or track onto a playlist - double clicking also adds it at the bottom of the current playlist. You can have multiple playlists so it is easy to set new ones up whilst you are playing. You can also refine what you play within the playlist. The playlists are retained between opening and closing. There are a lot of plugins one of which is an equaliser which would be very useful but I could not get it to work.

Exaile is just leaving a two year upgrade cycle and only a beta is available - watch this space in the future.

Gmusicbrowser

This is the default Music Browser under the 'light' distributions of Ubuntu Xubuntu and is very versatile. Its configurations can be set up to mimic Rhythmbox and Exaile as well as many other ones of its own. So far I have found it less easy to use when searching for music than Rhythbox but that may just be lack of familiarity and it does not use appreciably less memory or processor power on my legacy machine running Xubuntu.

Clearing the Gmusicbrowser Library

I have had the same trouble with Gmusicbrowser as Rhythmbox when moving my music library from one drive to another and onto the djmount and I have noticed that there isn't any clear way to change library locations once you have set them - you still have all the old locations in your music library as well as the new ones.

The Music database and other settings are held in ~/.config/gmusicbrowser. If you delete this folder all the settings should be restored to that of a fresh install when you restart. The database file is called gmbrc and it may be possible to just edit (delete all the lines in the section called [songs] and retain your settings. But the simple way is to completely halt gmusicbrowser and:

rm ~/.config/gmusicbrowser

Rocket Player for Android

Strictly I should not be including players for Android but there is so much overlap in preparing a music library which will be usable on multiple devices that one needs to consider players other than Linux based.

This is the best player I have found for Android and I now play music almost entirely from my pad using Bluetooth to link to the Hifi in the house or on the boat. All the ripping and preparation of the collection is done on my Laptop and transfered using a special 32 Gbyte USB stick which has both standard and micro USB plugs. The normal view is a matrix of album covers but it has good searching facilities to locate the music one is looking for as tracks, artists or albums. In many respects it is more sophisticated 'out of the box' than Rythmbox even when all the Rhythm box extensions are installed. The support has proved excellent.

Conclusions on Music Players

To follow

Music Over a Network

Introduction to DNLA and UPnP Media Servers for Music, Video and Pictures

We started looking seriously at networking music and other media between machines after buying a fancy High Definition Video Recorder which offers many additional features including the ability to access video, photo and sound files over a network. I am using MediaTomb as a server for this data. MediaTomb is an open source Universal Plug and Play (UPnP) MediaServer compliant with the required sections of the V 1.0 specification. A UPnP server allows one to stream digital media, usually from a local hard disk on the server, through ones home network for listening to/watching on a variety of UPnP compatible devices. Humax is advertised as being DLNA (Digital Living Network Alliance) compliant MediaRenderer (player). In practice these 'standards' are very similar so most DLNA compliant player, ought to be able to browse, request and play audio, video and photos over a network from a UPnP server such as MediaTomb.

The reason one uses such Media Servers is that many media files, in particular video ones are very large so that it is impossible or impractical to transfer them in their entirety between machines before playing them. The information is therefore requested as required and streamed from the server to the player. Depending on the server/player and type of file/stream it may, in order of complexity, be possible to pause, step forwards and backwards or fast forwards and back. Again, as a generalisation, it is easier to control a local file than one provided from a server. Most video media contains a number of streams, the audio, the video and sometimes other streams with, for example, subtitles or alternative audio streams. Almost all media is heavily compressed and there are many alternative formats for the audio and video streams in the same way that pictures may be in jpeg, png gif, etc formats. Within, say a video format, there are a huge number of options - some obvious like the size 720x578 or 1920x1080, the frame rate and whether the stream is interlaced. Others are more complex such as average, maximum and minimum bandwidths when variable bandwidth encoding is used and some which a normal person has great difficulty in understanding at all. If you want to get an idea of the complexity of the modern standards have a look at http://en.wikipedia.org/wiki/H264 for the latest video format H264. Again there are different audio formats and parameters for the streams, Dolby, Stereo, MP3, OGG etc with variable and fixed bandwidths. The audio and the video streams are not, of course, independent and need to be played in synchronisation so they are in a container - what you normally see is just a container such as AVI, MPG, TS, VOB. In contrast to Video, Audio should be relatively easy and once one has set up the server for video the audio is a minor extension at the server end.

Very little support is however built into the popular music browsers and the extensions to provide UPnP streaming audio support and those available currently do not work well or at all. The alternative is to 'mount' the UpnP server so it looks like local file system or go network straight to the folder containing your music. The advantage of UPnP is that the database is on the server an access can be limited to just the files in that database. Mounting UPnP as a file system negates the first advantage as the music files still have to be added to a database in the music player although this itself allows much more powerful searching. In the case where one goes directly one has no restrictions to the areas which can be accessed with security implecations. Some alternatives I have investigated follow:

djmount - a UPnP Client for Linux which mounts as a FUSE File system.

I had difficulty in identifying a general purpose client for UPnP for Linux. The best seems to be a very old utility called djmount which mounts as a Linux filesystem (using FUSE (File System in User Space)) the media content of compatible UPnP AV devices. The Audio and Video content on the network is automatically discovered, and can be browsed as a standard directory tree.It is available through the Ubuntu Software Center or Synaptic and needs the FUSE utilities to be installed. Mine seemed to be installed by default on 10.04 and 11.04 but check. You need to run in a terminal and to specify a mount point which needs to be set up with permissions to access for all users. I created a new folder /media/UPNP_APV and set up the permissions. This is best done in a terminal.

sudo mkdir /media/UPNP_APV
sudo chmod 777 /media/UPNP_APV

I could then mount the UPnP server by

djmount /media/UPNP_APV

The server should now be visible in /media/UPNP_AVP and as we are in a terminal it is worth checking:

xxxx@xxxx:~$ ls -l /media/UPNP_APV
total 1
-r--r--r-- 1 root root 7 2000-01-01 11:00 devices
dr-xr-xr-x 7 root root 512 2000-01-01 11:00 Triton
xxxx@xxxx:~$:~$ ls -l /media/UPNP_APV/Triton
total 2
dr-xr-xr-x 9 root root 512 2000-01-01 11:00 Audio
dr-xr-xr-x 4 root root 512 2000-01-01 11:00 PC Directory
dr-xr-xr-x 7 root root 512 2000-01-01 11:00 Photos
dr-xr-xr-x 5 root root 512 2000-01-01 11:00 Video
xxxx@xxxx:~$

We can see the server called Triton is mounted and the standard folders from MediaTomb are below it and have sensible permissions.

It is possible to add a program to the autostart the mounting with a delay to ensure the network is up.

Mounting a Remote File System securely using SSH and SSHFS (Advanced)

There are ways to add security when accessing the remote file system which contains our Media Files. The usual way is to use SSHFS but this is an involved process to set up and the following is extracted from Ubuntu All Together - Sharing, Networking, Backup, Synchronisation and Encryption with some additions specific to this application.

SSHFS (SSH File System) uses SSH to mounting a remote filesystem on a local machine. SSHFS uses FUSE (Filesystem in Userspace) which is included in Ubuntu and authenticates connections so only those who are authorised can mount them. SSH connections are encrypted, no one can see your files as they are transferred over the network and once mounted the FUSE filesystem is owned by the user that created it and other users on the local machine can not access them.

Firstly we must install sshfs using the Synaptic Package Manager by System -> Administration -> Synaptic Pacakage Manager -> Search for sshfs, Right Click and Mark for Installation -> Apply.

Secondly you need to add yourself to the fuse group. Under Ubuntu 12.04 the settings via Users and Groups has been castrated and you can no longer set up or change individual Groups so it is back to the terminal. You can check the groups you are in (and get some other useful information), in a terminal, by typing:

id

The output from id on my system which works with gnome-PPP looks like:

pcurtis@vortex-ubuntu:~$ id
uid=1000(pcurtis) gid=1000(pcurtis) groups=4(adm),20(dialout),21(fax),24(cdrom),
25(floppy), 26(tape),29(audio),30(dip),44(video),46(plugdev), 104(fuse),
105(lpadmin),112(netdev),119(admin),122(sambashare),1000(pcurtis)
pcurtis@vortex-ubuntu:~$

The following commands will add YOURUSERNAME to the fuse group which is required to use SSHFS:

sudo adduser YOURUSERNAME fuse

Once you have added yourself to the fuse group, you need to either log out and log back in again or reboot for the change to take effect. I suggest checking again with id

There is a risk that your ssh session will automatically log out if it is idle. To keep the connection active add the following parameter to ~/.ssh/config or to /etc/ssh/ssh_config on the client.

ServerAliveInterval 15

This will send a "keep alive" signal to the server every 15 seconds.

Using sshfs in a Terminal

A local mount point must exist and have permissions set for you to use it before you connect. If you have it in your home directory you can use the file manager to create the new directory which will have the correct permissions.

Assuming that you have ssh installed and the server running on the remote machine you now just need to run the sshfs command to mount the remote directory. In a terminal type:

sshfs -o idmap=user username@remotemachine:/remotefolder ~/localfolder

The idmap=user option ensures that files owned by the remote user are owned by the local user. If you don't use idmap=user, files in the mounted directory might appear to be owned by a different user.

You can not unmount a FUSE file system mounted this way in a terminal using eject or unmount from a right click or nautilus; To unmount you again have to use the terminal and do:

fusermount -u ~/localfolder

Using ‘Connect to Server’ in the Nautilus file browser with the SSH option

An Alternative way to mount a secure remote file system is to open the Nautilus file browser then File -> Connect to Server and select SSH from the drop down menu. Fill in the server name and other parameters as required. This way will give you a mount on the desktop and you can unmount by a right click -> Unmount Volume. You still need SSH on both machines and the fix to the Hostname resolution problem using winbind covered above. I am not sure if you also need the keep alive signals as I already had the fix in place.

If this fails with a timeout error try deleting or renaming ~/.ssh/known_hosts - you will get a message saying that it is a new host you are connecting to and everything will then work. It seems to be caused by the IP address changing for the remote hostname and you not getting the warning message.

Using Public Key Authentication with ssh and sshfs to avoid password requests

It is often inconvenient to have to provide a password when one logs into a remote machine using ssh and impossible if you are automatically mounting a file system at login. The way round this is to use public/private key authentication instead. The key is encrypted and passed between the machines but there is obviously a security implication as anyone who gains access to one machine has access to the other. There is a good introduction to ssh at http://kimmo.suominen.com/docs/ssh/ which covers this.

The following proceedure allows one to set up access via a public/private key pair between two machines running OpenSSH. You should first check that you can log in to the remote machine and are authenticated on it. When running ssh-keygen you will be prompted for a file name and a passphrase. Simply hit the return key to accept the default file name, and enter no passphrase at this time - you can and should enter one a latter stage to improve security. scp is a command for a secure copy using ssh which is part of the ssh package. The command examples below need to be edited to use your own machines and usernames. It assumes you are logged in as username1 on host1 and want remote access to username2 on host2. Run the following commands in a terminal.

cd .s
ssh-keygen -t dsa
scp id_dsa.pub username2@host2:/home/username2/.ssh/host1_username1_id_dsa.pub
ssh username2@host2:
cd .ssh
cat host1_username1_id_dsa.pub >> authorized_keys

You should find that you no longer have to provide a password when using ssh sshfs scp etc. If it ever stops working try ssh in terminal as it is possible the ip addresses have changed and you need to reset things by deleting ~/.ssh/known_hosts and doing a new ssh login. This gets tedious after a few IP address changes (due to DCHP) and you can make the following change to configure the secure shell (ssh) on the local machine not to check the IP, again with a reduction in security. You find the ssh configuration file in /etc/ssh/ssh_config

sudo gedit /etc/ssh/ssh_config

and the section to change is:

####### CheckHostIP
#
# If this flag is set to "yes", ssh will additionally check the
# host ip address in the known_hosts file. This allows ssh to
# detect if a host key changed due to DNS spoofing. If the option is
# set to "no", the check will not be executed.
#
CheckHostIP no

Scripts to Mount and Unmount the Media on Matrix (my server machine) using sshfs (or djmount)

I have written two scripts to mount and unmount the media. The mounting script has delays to ensure that the Wifi is up when it is added to the startup program list activated at login. Both scripts check whether the file system was already mounted and has been successfully unmounted and put up alert boxes. This script requires zenity to be installed to display the alert boxes.

#!/bin/bash
# Script to mount server on matrix ~/mediamount using sshfs.
# Assumes that password is not needed.
# Assumes that local mount point has been set up in ~/mediamount
# and has correct permissions
# Script lives in ~/bin/sshfs_mediamount where it must be given execute permissions
# and ~/bin should have been be added to to #PATH in ~/.bashrc
# by adding the line
# PATH=$PATH:~/bin
#
# Check if Media Folder already has contents
if [ "$(ls -A ~/mediamount)" ]; then
  zenity --info --text "Media Folder was already available for Media Players to use"
else
# Allow time for Wifi to come up when script automatically run at login
  sleep 35;
  sshfs -o idmap=user pcurtis@matrix:/media/DE_MEDIA ~/mediamount;
# Alternative for djmount
# djmount ~/mediamount;
#
# Check mediamount folder now has contents
  if [ "$(ls -A ~/mediamount)" ]; then
    zenity --info --text "Media Folder has been successfully Mounted for Media Players to use"
  else
    zenity --warning --text "Media Folder mounting failed - is Matrix turned on? If not turn on and try again using the Desktop Icon";
  fi
fi

#!/bin/bash
# Script to unmount ~/mediamount
# This script requires zenity to be installed to display alert box.
# Lives in ~/bin/unmount_localmedia where it must be given execute permissions
# and ~/bin should have been be added to to #PATH in ~/.bashrc
# by adding the line
# PATH=$PATH:~/bin
fusermount -u ~/mediamount
# Confirm folder is now empty
if [ "$(ls -A ~/mediamount)" ]; then
  zenity --warning --text "Media Folder Un-mounting failed - is it in use?"
else
  zenity --info --text "Media Folder has been successfully Un-mounted"
fi

The same basic scripts can be used for djmount by changing a single line.

Album Cover Art.

This has proved to be one of the biggest problems I have had when ripping CDs and trying to get a consistent music collection together which is suitable for use on both Linux and my Android phones/tablets. Whilst the automatic identification of in all the 'metadata' for albums tracks etc is almost perfect from MusicBrainz there seems to be no reliable way of getting cover art for classical music. The matching is done for albums based on the start and end times of each track but no similar mechanism exists for finding cover art and the album names etc are used - this works acceptably for popular music but not classical and one is left with complex and frustrating internet searches with a poor overall success or scanning covers oneself which in practice was by far the quickest way. On the internet Amazon and Discogs were the major sources of Cover Art.

There are many constraints in choosing the format for album art for use on multiple players which are not initially apparent. In my search for background information I again and again came across one site which has a lot of invaluable information on setting up and maintaining a music collection. It is associated with a commercial music library management system called Bliss set up by Dan Gravell, an enthusiast, for his own collection and Dan's blog and his free music library management ebook are packed with useful insights. His information has clarified many of my ideas and if I was starting again I might well have paid for the Bliss system which is now available on many platforms and seems to be very flexible.

Returning to album art. There are two ways that you can store album art in your master master collection which can be used by many players on different machines and operating systems. You can store a single file in each folder containing all the tracks (audio files) in an album or you can embed an image in the metadata in each track or you can do both as some players handle one or the other. Either way there are further important choices:

Name and type of the file - the most standard names are folder.jpg and cover.jpg for the files in a folder but iTunes now entirely uses the PNG format when artwork is embedded which is not accepted by many players. They do recognise folder.jpg however.

Size of the album art both in pixel resolution and data size. There has been a tendency to steadily increase the size as the size of displays on devices, especially mobile devices have increased in size and resolution. There are however constraints, for example, most current in-car entertainment systems impose limits on resolution and some on the size of data stream, e.g. images no larger than 256KB. The same applies to Sonos systems used around the house. any constraints on file size pushes one even more towards JPEG, although it uses a lossy compression, as it is much more efficient than PNG and is much more commonly accepted by players. I have standardised on JPEGs of 500 x about 500 pixels which is well under any limits for the music players I do or might use.

To embed or not to embed: My initial thinking has been that embedding is wasteful of space as the file is embedded multiple times and less flexible as it is more difficult to change. So currently I just have a file called cover.jpg in each folder. But there seem to be many advantages in embedded artwork in the files types whose metadata support it as it seems more players support it and more consistently. The overhead is not as large as one might expect. In my case I have 914 albums with 11290 tracks for an average of 12.35 tracks/album with a total of 45 mbytes of artwork which would increase to 550 mbytes. This is an increase of 0.8% which is hardly an issue.

It is possible to write a terminal commands which will embed the whole library a one go using find and eyeD3 which is a very power tag tool for mp3 files.

find . -name "cover.jpg" -size -150k -execdir eyeD3 --add-image 'cover.jpg:FRONT_COVER' . \;
The above command must be pasted as a single line into the terminal. It finds all folders with acover.jpg file under 150kB and embeds it into all the mp3 files in that folder.

You can find the large files to edit if necessary.

find . -name "cover.jpg" -size +150k -exec ls -sh {} \;

It is also possible to remove them by:

find . -name "cover.jpg" -execdir eyeD3 --remove-all-images . \;

Note 1: This checks that there is a cover.jpg before removing the images

Note 2: see https://ubuntuforums.org/showthread.php?t=2268072 for a warning that the space is not automatically recovered when ID3 tags are removed. The solution requires a more recent version of eyeD3 that I had on my system to recover space. The latest version should be available in Mint 18.3 but I installed it separately to test

find . -name "cover.jpg" -execdir eyeD3 --remove-all-images --max-padding 1 . \;

Health Warning: The commands above can do a lot of damage! Above all make sure you are in the correct folder and back up first - I take no responsibility.

How to tag Classical Music

More to follow but the following posts from the Bliss blog by Dan Gravell will give you some food for thought.

What does come out from the above and my experiences are that it is all about compromises and your best solution will depend very much on the Music Player(s) you use. One certainly can not depend entirely on the tagging you get from a Rip - I have a box set where every CD is tagged in a significantly different way.

I have tried a number of Media Players but I come back to Rhythmbox on Linux and Rocket Player on Android so my tags are chosen to cover them. Rocket Player can use more tags but I find the Search less powerful in practice.

My 'solution' breaks most of the rules but to enable easy searching I have something like Composer - Orchestra, Conductor in the Artist tag and as much as possible of what is on the spine of the CD in the Album tag. Details left over are in the Track tag.

Where there is a soloist or the prime reason for the purchase that come out of order ie James Galway may be before Flute Concerto or Karijan before the Orchestra - a bit like a lead author being out of alphabetic order on a paper.

What Impact does Android have?

You may well ask why one should consider Android when this series of papers is about Linux - the answer is that we are using Linux to manage a Music Collection and one has to know what metadata is of value in the Media players which are used. The majority of the time the music will not be played on a Linux Computer. Most music is going to be played on mobile devices of some form either dedicated like the iPlayer or Android or Apple phones and pads and most people using Linux and going to use Android rather than Apple devices. We must also take into account music systems such as Sonos, in-car entertainment systems and Smart TVs. Android is however different and needs special consideration because of the MediaStore.

The MediaStore is a central database that holds information on every media file on the system: images, videos and audio files. This is intended to remove the need for every program to have to index the machine and to share changes. For example a dedicated program could seek album art which can be accessed by all other programs but without modifying the original files.

Overall Conclusions

To Follow

Before You Leave

I would be very pleased if visitors could spare a little time to give me some feedback - it is the only way I know who has visited, if it is useful and how I should develop it's content and the techniques used. I would be delighted if you could send comments or just let me know you have visited by Sending a quick Message to me.

Link to W3C HTML5 Validator Copyright © Peter & Pauline Curtis
Fonts revised: 28th April, 2021