iTunes 8.0 (part 2)

Another day, another post.  The iTunes Genius’s analysis ended up taking about fifty minutes to run, with another ten or so of swapping data with the iTunes store. After that it all seems set to go.

One last setting adjustment: If you don’t want iTunes to open in a window (i.e. not full-screen) every time it starts, you’ll also need to right-click the iTunes icon you use to start it.  Either on the Desktop or the Start Menu – select Properties, and change the Run box from Normal Window to Maximised. Instant full-screen.

Couple of other things to note.  First, iTunes takes a lot longer to open now – I assume Genius playlists count as smart playlists, too many of which always slowed it down.  Its not bad, but there’s a noticable delay, perhaps a couple of seconds longer.  Its probably a good idea to leave the Genius panel closed unless you’re specifically using it, otherwise iTunes takes (comparitively) forever to get started and ready, like closing it on the iTunes Store window does.  Related, the time iTunes takes to respond to task switching when your working on something else has also slowed down, now and then there’s a noticable pause.  There was a slight stall once, and a couple of times a slight jitter.  This seemed to coincide with Firefox running flash-heavy webpages, something I’ve noticed is a problem on other, slower systems.  Itunes has always been a resource pig, and it looks like it just got worse.  The question is whether Genius, and reported better performace on larger libraries, makes it worth it.

Second, be aware there’s a couple more files to back up for the Genius playlists, iTunes Library Genius.itdb and iTunes Library Extras.itdb. They’re in the iTunes folder and clearly labelled, so you should catch them automatically if you’re just copying the whole folder.

On to the Genius.  Genius did okay.  The first track I tried, Claire Guerreso’s Matador, got no results.  The second, Uh Huh Her’s Say So, worked pretty well, a list of music from my library and some suggestions from the store.  Presumably Genius is more geared to mainstream music, and perhaps to the traditional boomer iTunes demographic, but it did okay here.  I wonder if the label makes a difference, if there’s a skew towards bigger labels (Uh Huh Her are on Nettwerk, and I think Guerreso might be unsigned – and presumably Genius, at the end of the day a sales tool, is better on tracks the iTunes store carries).  Playing around with some other tracks, Genius usually makes a list which have some connection, and although its not perhaps the one I’d be thinking of, I can see a connection between A and B.  Its not perfect, but its not meant to be.  Its a smarted-up party shuffle.  It gets past the most annoying aspect of shuffling – jarring transitions and randomness – pretty well, while preserving the best aspect of shuffling – randomness.  One interesting thing would be whether different stores give different or better results for people with multiple accounts.  Logically, if different stores have different patterns of use, and the US store is largest, that one might give better suggestions on more obscure artists.

WHAT I’M LISTENING TO: Genius playlists, for now.  I won’t duplicate whole lists since half of the tracks are CDs and others are from the iTunes store, so I can’t do the usual links to labels and indie online stores.

iTunes 8.0 (part 1)

Well, I was going to hold off on this a while – especially after reading some of the bad press on the web – but I cracked.   Yeah, I wanted to play with the Genius thing, I admit it.  According to reports, iTunes 8 and Vista might not play nice.  Very not nice.  There’s commentary all over the web, including the Apple message boards, about obscure – apparently HP related – software conflicts that cause BSOD crashes in Vista when you plug in an iPod.  I’m on XP, which is why I’m risking it, and for reference, no crashes and iTunes did find the iPods when they were plugged in.

First thing about the installation is it behaves – mostly.  The installer gives you the option of leaving automatic updates and the like off, and seems to preserve your options, although you will need to go into the XP Software Installer and turn Quicktime’s autorun off again - like you always have to.   The installer didn’t crash.  It didn’t take too long.  Everything else went fine.  No nightmare scenarios.

Don’t assume that, though. Back up your library files (in your user profile, iTunes Music Library or similar .xml and .itl) and your preference file (in your user profile, Application Data/Apple Computer/Itunes/iTunesPrefs.xml)

Second thing is some of the options have moved.  If lost, Google will provide the answer, though in short, the import options are now on the General tab, and the backup and export options are on a submenu called Library on the File menu.  Settings appear to be preserved across the upgrade, as are the logged-in and authorisation status on your iTunes store account – I seem to remember the last upgrade didn’t and you needed to reauthorise the computer, which meant remembering to ask your partner for her password again, which was a bit tiresome.

When you start iTunes it’ll spend a while checking album art.  This seems to be converting the existing art for the new grid view, not redownloading.  Don’t panic – it isn’t copying over the corrections you already made.  After that its done, although if you turn on the genius function (which you are obviously nagged to do), it then spends a long time analysing your library.  The genius thing obviously sends data to Apple.  If you care about privacy blah blah – although if you’re worried you should probably care more about Google than Apple.

When Genius has done its thing, the last little piece of housekeeping is to turn those annoying arrow things that point to the store back off.  They get turned on by default, and the option to turn them off is disabled.  Odd, since these seem especially redundant in light of the genius function.

To get rid of these, on a PC, close iTunes and open your prefs file in a proper text editor (not notepad, a programmer’s editor like EditPad Lite).  This is in your XP user profile, in Application Data/Apple Computer/Itunes/iTunesPrefs.xml.  This file should look odd, full of blocks of As and things.  Search for User Preferences and go to the place in the file that looks like:

<key>User Preferences</key>
<dict>

Beneath that add the text:

<key>show-store-arrow-links</key>
<data>
AA==
</data>

On a mac, open your terminal and do:

defaults write com.apple.iTunes show-store-arrow-links -bool FALSE

Be aware that apparently on a Mac, in iTunes 7, you could swap functionality between these arrows linking to the store and to your local library, a functionality which has gone in iTunes 8.  Something to consider before you upgrade if that sounds appealing.  In a similar vein, the other missing option is turning off the genre list in the browser view.  I don’t, but people seem to be missing this.

On a mac, in the terminal, do:

defaults write com.apple.itunes show-genre-when-browsing -bool FALSE

On a PC, add the following below the text you just added above, same place in the file:

<key>show-genre-when-browsing</key>
<data>
AA==
</data>

Both of these with thanks to Rodalpho.

Back to Genius.  I’m writing this while iTunes does its analysis thing.  Its been forty minutes and it looks to be two-thirds complete.  I’ll post this now and leave it to it, then make some comments about Genius tomorrow.

WHAT I’M LISTENING TO: All India Radio (mentioned yesterday). Downtempo trip-hop, reminiscent of Portishead, especially the album The Fall.  Although since Portishead don’t like being called trip-hop, maybe these guys don’t either, but its hard to explain without using the words, if you see what I mean.

Moving iTunes from one computer to another

I just bought a new computer – I’m using it now. The old one was a bit flaky.  It had one small problem – it was made by HP. Not trying to be nasty, but Compaq used to be a great brand, then HP bought it and now its not. Overpriced, shoddy quality control, and half the time they just don’t work. Of course, HP also used to be a good brand and now HP are also overpriced, shoddy, and don’t work, so that’s what you get. Anyway, I got sick of those problems and got a new machine – the brand of which I won’t mention as this is intended purely as negative product placement, not endorsement.

Thing is, with a new computer, I need to move my iTunes library. All of it.

I could just move the music files, and re-import it, but a quick hunt of the interwebs tells you this is a bad idea. It seems iTunes may lose your play counts, last played, and other such metadata, which I kind of like having. (One iPod playlist is ‘not played recently’ – shuffling would be too easy – another is ‘purchased and never played’. I need my metadata.)

In short, what I want to do is copy all the music files onto the new computer, then copy all the iTunes metadata, then start iTunes on the new computer without it noticing its been moved.

In addition to the actual music, movies, and so on, iTunes has three metadata files. In Windows, these are usually at:

[user profile]\My Documents\My Music\iTunes\iTunes Library.itl

[user profile]\My Documents\My Music\iTunes\ iTunes Library.xml

[user profile]\Application Data\Apple Computer\iTunes\iTunesPrefs.xml

The iTunes Library files are the actual library – apparently duplicates, iTunes makes one from the other if it only finds one. I had a nasty moment back in the day when I was bad and didn’t have proper backups, and my iTunes Library.xml file got deleted – but iTunes rebuilt it from the .itl file when it was next started, so all was well. The iTunesPrefs.xml file contains, well, preferences. You might need to change some Windows view settings to see the Application Data folder.

The problem is that my music collection on my old computer has grown – and spilled over into an external hard drive.

I could keep using that drive – but why, when I have a shiny new hard disk inside the new computer’s case. I want to consolidate everything into one physical disk without iTunes noticing. Now, sure, I could probably let iTunes do this. Except it would take a long, long time, and I’d rather do things myself than turn any software lose on a large and important (to me) collection of files. I could also try and search and replace one directory string for another inside the iTunes xml file, but I have no idea how consistently this would work and don’t want to pick up the pieces of ten thousand lost songs if something goes wrong.

There’s a much easier way: replicate the old disk structure on the new computer and don’t tell iTunes anything changed. Again, there’s two ways to do this. I could repartition the new hard disk and create the same logical disk structure as was on the old disk. But that takes time and effort – new computers come with everything installed, and I expect to be buying a new, bigger disk in a year, so why make more work if I can kludge it.

Which I can.

The old DOS subst command still lurks beneath Windows XP. (Yep, I got the downgrade rights. I use iTunes – obviously – why do I care about Vista’s slight benefits when its had so much horrendously bad press. I’d almost say Bill, come back, we miss you.)

subst is there. The syntax is subst drive folder. It’s a map network drive in reverse.

So I create a folder on the C: drive to hold all the external drive’s old data. The drive used to be called F:, so imaginatively I’ll call that folder F_drive. I could use a space and quote marks in the subst command, but underscores are that much geekier.

Then I create a batch file called, say, logon.bat. (A batch file is a text file – in Notepad – with the extension .bat rather than .txt). The single line in the batch file is:

subst f: c:\F_drive

Or if you prefer not to underscore, this will also work:

subst f: “c:\F drive”

I want the logon.bat script to run each time Window starts, so I can just put its name in one of the registry keys that Windows uses for this purpose – like autoexec.bat under DOS. Oh yeah, don’t play with the registry, back it up, don’t blame me if this goes wrong, blah blah.

Seriously.

Don’t do this if you don’t know how to use the registry editor. Messing this up can be very bad (although I’m not quite sure how you’d mess up editing a single key – but random deletions due to mistyping would in fact be extraordinarily bad).

One key that will autorun whatever’s stored there – and this is just the one I happen to use, there are others – is:

HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Windows

The important one of the several values of this key is load. If it isn’t there, just add it as a String value. Put the full path to your logon.bat file in the load value, for example, c:\logon.bat

That’s all. Log out and back in. You should see an F: drive in My Computer.

If you don’t, if you see one of your external drives in place of the fake F: drive you expect, this is because Windows assigns drive letters earlier in the boot process than it runs the logon.bat script. You might need to reassign your drive letters with the Disk Management utility in Administrative Tools – Computer Management.

And after all that I did all the copying. Which took a long, time time.

When I started iTunes everything worked perfectly, completely seamlessly – my local iTunes didn’t seem to notice and still opened onto the last playlist I’d used. I had to log back into the iTunes store – it obviously picked up that this was a different computer – and obviously I also had to authorize my purchases on the new computer, but other that, no problems at all.

Like last time, sorry I’m not more help to Apple people, but I assume you can do something similar – and probably more easily, since you have unix running under the GUI. I assume you can mount the folders in a similar way and iTunes wouldn’t notice, but I couldn’t tell you how to actually do it.

WHAT I’M LISTENING TO: Very little right now as I just finished watching my music copy from one drive to the other.

An iTunes bug… um feature

iTunes is a pretty good music library.  Its nice to look at, it seems stabler than some of the other alternatives with very large libraries (sorry WinAmp fans) and its never, in the time I’ve used it, managed to corrupt its databases or lose files or do anything else really nasty.

However, there’s an annoying quirk in the way iTunes handles its ‘add folder to library’ feature that means you can end up with the same file being added multiple times.

Say you have thousands of tracks in your library.  You probably have the ‘copy music to…’ option switched off.  If you don’t, you should – unless you’re buying those tracks from the iTunes Store, adding them to the library will take a lot longer with this option on, as iTunes is physically copying the file when you add it to the library, rather than just adding a reference to an internal database.  This option being on also means your library takes up twice as much disk space since there’s one copy in the iTunes folder and one in the file’s original location.

You have a bunch of folders.  Folders for albums, then for artists, then – in my case – folders for the stores I got the music from.  (I usually organise by the store I bought it from, because that’s the one piece of information a lot of mp3 tags omit and you sometimes really need.  And because with a 60 000 song library you need to stricture it somehow – opening that folder is going to take a lot of time to happen with 10 000 or so albums right there to display.)  So I have an Amie Street folder which contains a bunch of folders named for artists and bands, which in turn contain folders named for albums, which contain mp3s.  All very normal and familiar, right?

So I buy some music, three albums, say.  I want to add the new music to my library and do this the quickest way I can.  I download three new albums and put them in the Amie Street folder.  In iTunes, I go File – Add Folder To Library and iTunes rescans the whole Amie Street folder, picks up those albums, and adds them to the iTunes library and I go and find them in Recently Added (because I’m me, so I’ve forgotten the names in the thirty seconds it takes to do this).  Sounds complicated?  Not really, and it saves time once you have the routine.  I could add the three albums one at a time, as individual album folders, but I don’t, because that takes time, three times as many clicks, and iTunes is perfectly able to scan the entire Amie Street folder and only add those tracks it hasn’t seen before.

Seems like a good idea, right?

Um, no. Unfortunately iTunes sometimes adds tracks more than once.  It picks up the old tracks and adds a second copy.  Sometimes music added to the library months ago, so it takes me a while to notice – usually I don’t see it until I next search for a particular album and play it through and notice the duplicates.  And strangely, it’s usually duplicates, not three or four copies.  This is a strange problem, and its been annoying me for months.  I couldn’t see why it was happening, and couldn’t make the error happen consistently (which, as we should all know, is what makes it a bug and not just a user error).

So, duplicate songs, no apparent reason, making my library all untidy.  This is bad.

Today I got annoyed enough to spend an afternoon trying to fix it, and turns out this is what it is: iTunes thinks an mp3 with lyrics or album art added is a different file to the one with no lyrics or art.  So you add a song, add art, rescan the folder, and iTunes sees the file as a different one and re-adds the track again.  You end up with two files, one with the art, one without.  If this is the source of your problems, its easy enough to check – do Find Duplicates in iTunes and see if the only difference in your duplicates is the dates added and presence or absence of art and lyrics.

Oh, and welcome to the nightmare land of Find Duplicates and manual deleting.  iTunes handles this well.  Really well.  Is it so hard to assume the same track on different albums – with different lengths, from different files – aren’t duplicates?  You must have done this sorting game before, going through the list: That one’s a live version, this one’s an actual duplicate.  A thousand times over.  Oh fun.

That’s the problem.  That’s what iTunes was doing.  It was consistent, it was replicable – and it was an absolute monster of a problem because I’ve run the iTunes Get Album Art command and I’ve run an add-on lyrics finder multiple times and I have no idea when.  No idea any more what’s been changed and what hasn’t.

See the problem here?  Any new album I add anywhere on the computer now has to be added manually, only the album folder, because I don’t know where the ‘changed’ mp3 files are.  Or I can go through every so often and fix iTunes’ duplicates after I add art.

Anyway, if you’ve got to this blog from google looking for a fix, sorry, I don’t have one.  No fix, but there is a workaround.  It takes a bit more time than rescanning whole folders, but not as much as manually adding folders one by one.   Instead of letting iTunes add everything and then deleting duplicates, you can add to the library only the folders that contain new music by dragging them.  At least in Windows world.

Go to the music folder.  Add your music – unzipping or copying or whatever.  Write down the names of any artist folders you ‘this file already exists’ errors on.  In the music folder, change the view to Detail view.  Sort by Date Modified – click it twice and the most recent folders will come to the top of the list.  These are the ones with the new music in.  Open iTunes.  Select the new music folders and drag them to iTunes (by dragging down to the iTunes taskbar icon which will open the iTunes window), then dragging to the Library item (the capitalised one at the very top) in the navigation panel on the left.  If you drag to the Library item, the new albums get added to the library.   If you drag to a playlist, the albums get added to the playlist – which may or may not be what you want.  Lastly, go to the artist folders you wrote down earlier and add the individual albums the same way.  These won’t be included among the ones you just added as the folder was created the other day, when you first bought that artists’ work, not today.

That’s the only way I know of to get around this bug/quirk.

This is on a PC, but I assume Macs have much the same options.  Seems to me Apple didn’t think enough before they did this design.  Like maybe iTunes has been far more successful than they ever imagined.  Like maybe they just didn’t think people would have 60 and 100 GB libraries and would have directory hierarchies in place to organise those, and so would always have the ‘copy music to…’ turned on.

Nah.

Hopefully they fix it in iTunes 8.  Here’s hoping.

(On the subject, and as an aside, I shouldn’t need to add that once you’ve imported the song into iTunes you never move it or iTunes will lose track of its location and you’re going to be spending a lot of time clicking on the More Info screen to tell iTunes where it is now.  Just don’t.)

WHAT I’M LISTENING TO: Julie London, actually – beautiful smoky-smooth old-school jazz from way back when.  She’s here on emusic.  The Essential album is a good starting place – its littered with standards, and the first track, Cry Me A River, is probably her best-known song.