sounds familiar

Today was a bad day.

Work made me mad in the morning, Sharaun made me mad at lunch, and work continued to make me mad after lunch. The sun broke through on two occasions: First, the hour I had with Keaton while Sharaun was at the gym. She sat on my lap for nearly the whole time and we played. She was super huggy and talkative. Second, being able to do some brief listening to the Sgt. Pepper’s Lonely Hearts Club Band multitracks on my new iPod. I wrote about them before, over here, but getting some time to actually hear them closely makes them seem so much more awesome. They are indeed amazing. I’d recommend you check them out, if you can manage to locate them, that is…

Anyway, it was a long, stressful, and frustrating day. But, I eventually made up with work… choosing to be the bigger man and continue to accept my paycheck in return for the occasional abuse. Incidentally, I also made up with Sharaun… apologizing for yelling at her and stomping around the house in a fit. So, all’s well that ends well, I suppose.

But… this does not end well. Not by a long shot. Later on, at night, I decided to try out a program I’ve heard other audiophiles praise. It’s called MediaMonkey and it’s one-stop music management application, and can even sync with iPods and other portable devices. On a whim, I decided to install it. Unknowingly, I had my brand new iPod Classic plugged in when I installed and launched it. Turns out, the damn piece of software “automatically” sought out my iPod, read the contents, and somehow completely corrupted the iTunesDB file. So, when I fired up iTunes again, I got the familiar heartbreaking message: “iTunes cannot read the contents of iPod…” Sigh, all my music lost. What a waste… I’m so upset. Why can’t I just learn to leave well enough alone?

Right now I’m frantically trying all sorts of recovery methods… and, even as I write this, I’m using a promising piece of software that touts one-click iPod-to-iPod cloning, with support for all generation iPods, including Classic/Touch. The cool thing is, it supposedly also will transfer all the iTunes/library data like “played count,” “rating,” etc. – which is exactly what I was looking for in the first place. If this works, I’ll make a quick writeup and post tomorrow. Wish me luck, OK?

Really sorry about all the iPod talk, but, as if you couldn’t tell, I’m a bit obsessed with this new toy. Hope you’ll understand as I continue to fawn over it. Thanks for sticking it out. Goodnight.

Don’t miss the latest updates to this story, click here.

Being in Oregon – the land of no sales tax – I decided there was no better time than now to go ahead and drop the dosh on one of those new-fangled iPod “Classics.” The 160GB-ness of the thing has been taunting me since the things were announced, and I knew it was just a matter of time before I upgraded. And, not only did I save the ~$25 sales tax I would’ve paid in California, but I also took advantage of the Costco price, which is already $10 less than Apple’s suggested retail price (iPod’s are rarely sold under this suggested price, especially by major retailers). So, all in all, I think I made out just about as good as can be. Took the thing home for $339 out the door. May sound expensive to those who could care less about 60GB vs. 160GB, but for musicophiles with platter-busting collections it really is worth it to carry around that much more variety.

So, last night I began trying to figure out just how in the world I was going to get all my music off my old iPod, and subsequently on to my new iPod. See, I manually manage my music on my iPod – I don’t use the iTunes library features to manage my music collection, and that means I also don’t use the “automatically sync” feature to get music on and off my device.

I realize this may put me in a minority of users, but, because 1) I don’t use iTunes to manage my music in general, and 2) my collection is larger than my iPod’s capacity, it’s just more convenient for me to drag-and-drop what I want when I want rather than relying on some selective sync functionality. Don’t get me wrong, I can see the plus-side to using the auto-sync functionality, like not having to completely delete and re-add things in situations where I’ve “updated” an MP3 folder/album on my hard drive, perhaps adding artwork or updating ID3 tags. But still, I prefer to do it manually.

Anyway, back to my problem. Surely someone has faced this before. Say you’re a manual iPod user like me (there has to be more of us out there, right?), and you outgrow your old iPod, or just figure the new one is plain “cooler,” and decide to upgrade to Apple’s latest offering. If you’re like me, you’ve likely spent a lot of time pruning, tending, and shaping the music (and other content) on your iPod, and you probably want to “start” with your new iPod having the same content as your old one, and then add new content from there. In other words, it’d be nice if the “baseline” content on your new iPod was a mirror image, or snapshot, of what all is on your old iPod. Makes sense, right? Think of it as “migrating” the entirety of your old iPod onto your new one.

Note: I’m aware that techniques exist for doing this using both iPods in disk mode, but I’m pretty sure that will only work on two iPod’s of the same generation (or, at the least, having the same database version). Since Apple changed the iTunesDB structure completely (version 25) going to the Touch/Classic sixth generation devices, I don’t think I can get away with doing something as easy as this when moving from a less-than-sixth generation one. But, honestly, I also didn’t give it a try.

Again, before you criticize me in the comments and tell me this would all be “easy as pie” if I just used iTunes to manage my music collection – I know. Trust me, I know. I just don’t want to use iTunes to manage my music, OK? Now that we’re done with that…

Now, even though I still have all the content that’s on my iPod stored on my PC, I don’t exactly know offhand what’s on there and what’s not. I have a general idea, but my collection is huuuge, and the best and easiest way to be sure I get an exact copy of my existing iPod content would be to simply “copy off” everything that’s on there and then move it to my new iPod. Sure, worst case, I could simply re-pick all the right content from my larger collection, but it would likely take me weeks to get just the right items (honestly, you may opt to do this… give yourself a “fresh start” and relive the fun that is filling your iPod with the perfect collection of music for the first time). So, again, we need to do the following things:

• Copy all the music off my old iPod (and onto the PC, as an intermediary location)
• Put the resulting music from step one onto the new iPod, ideally retaining the neat track-level metadata like “last played” dates, ratings, play count, etc.

I’m gonna tell you how I did this, even though it was an imperfect process in the end (meaning, I didn’t exactly meet my own requirements above, as I lost the iTunes/iPod metadata). Ready? Here it is; it’s actually quite simple:

1. Download SharePod and copy the folder to your old iPod (your iPod must be plugged in, and must be enabled for “disk mode” use).
2. Run SharePod from your old iPod (if you get an error, make sure you have the .NET framework installed).
3. Select all your songs from SharePod’s righthand pane, and click the “Copy to PC” button.
4. Choose your preferred location and foldername/filename structure (make sure you have enough hard drive space to hold what you plan to copy off).
5. Kick back and relax while SharePod copies every last track off your iPod (if you had a full 60GB model like I did, this can take over an hour, even at USB2.0 speeds).
6. After SharePod is done, close it out and unplug your old iPod.
7. Plug in your new iPod.
8. Fire up iTunes (or, alternately, fire up whatever you normally use to add songs to your iPod), and drop all the extracted tunes on your new iPod.
9. Relax while iTunes (or whatever you use) copies all your songs onto your new device.
10. Oh, and remember, if SharePod saved the day for you… go ahead and PayPal the author a few bucks… it was certainly worth it for me!

To be clear, SharePod does create an XML file which you can use via iTunes’ “Import” function. I originally thought that, since this XML file contained the metadata mentioned above, I may be able to preserve things like “play count” and rating, etc. when importing. Turns out, iTunes didn’t “get” the metadata, even when using the import feature and SharePod’s XML file (at least, for me it didn’t). So, the end result is that you’ll effectively reset all that data. You’ll lose play counts, ratings, last played dates, etc. It kinda sucks, but it’s better than nothing. And, as far as I was willing to muck with it, it was the best I could come up with.

There may be other ways to do this, using different software to achieve the same thing (especially since I’m only talking Windows here, and not Mac). But I think it all amounts to the simple process of 1) extracting the data from the old device and 2) putting back on the new device. I would love to find a way to maintain the metadata (I know… you’re going to say, “Use iTunes to manage your music…”), but for now I’m happy enough to have a cloned copy of my old iPod as a starting-point for my new one.

But Dave, what about my playlists?

Look, I know that a lot of people have spent a big amount of time and effort making super-awesome playlists from the content on their iPods. For that reason, I can see why you’d want to make sure your playlists come over to your new iPod intact as well. Unfortunately, I’m not entirely sure how to do this, as I didn’t have any treasured playlists I wanted to bring over for my upgrade. I think, however, that you can use SharePod’s playlist export feature in much the same way as described above for tracks to accomplish the same thing. Export the playlist, which, I believe, amounts to exporting the songs that the playlist contains, and then import the playlist through iTunes using the XML file that SharePod creates.

Well, that’s it. That’s I how managed to transfer all the music from my old iPod onto my new iPod, and start with what’s essentially a carbon-copy of what I already had. Hope it helps someone who may find themselves wanting to do the same thing. Oh, and, by the way, if anyone has any suggestions or tweaks for this process (especially if you know how to do the above while keeping the metadata for the tracks), please leave a comment on this entry and let the world know! Thanks!

Update – 11.27.07

After writing this, I decided to try a bunch of commercially available alternatives to the method I describe above. For what it’s worth, all the software I tried was able to get the MP3s off the device. However, none were able to successfully preserve the iTunes metadata (things like playcounts, preset-EQ, date-added, skipcounts, etc.). My findings:

• CopyTrans: This application is supposed to copy music from your iPod and into your iTunes library. The webpage advertises full iPod Classic compatibility. Out of the four, CopyTrans got closest to what I want as it was able to import the playcount and date-added metadata into iTunes.
• iCloner : Part of the CopyTrans suite, this is like Ghost for iPods. It promises to make a “clone” backup of one iPod which can be restored onto another iPod. The functionality is there: I “cloned” my 5G iPod and then restored the image onto my new 6G iPod. However, the 6G with the restored 5G image was still unrecognizable to iTunes, and the music would not play. I also noticed that, in disk view, the post-restoration 6G iPod was truly a “clone” of the 5G (which is a bad thing, since Apple changed the folder and database structure from 5th to 6th generation). Even though the website claims full iPod Classic compatibility, the version of iCloner I used couldn’t do 5G-to-6G. Interestingly, I have to think that iCloner may indeed be a “real” solution to my problem if I was trying to move from one same-generation iPod to another. And even though I haven’t tested it, I would expect iCloner to make a carbon-copy, including all meta-data, if the target iPod is of the same generation as the source.
• Music Rescue (formerly PodUtil): Music Rescue also successfully exported the raw MP3 files, but was unable to preserve the metadata when importing the tracks iTunes.
• Amarok (Linux only): Although I didn’t run the test myself, I had a Ubuntu-loving buddy confirm that the latest version of Amorak (at the time of writing, at least) can export MP3s from iPods (all generations), and copy them back onto iPods (all generations). However, when testing specifically to see if the metadata can be preserved, nothing was imported.
• iTSfv (iTunes Store File Validator): This tiny piece of free and versatile software can do lots of neat things with your iTunes music library, and has an extremely promising tab called “Backup/Restore.” At first glance, this tab appears to contain the holy grail: The ability to export an XML file containing only the not-in-ID3 metadata for your tracks, along with another option to “restore” the data in the resulting XML file back “over top” of another library (iTSfv looks for matching song titles, and writes the saved metadata on a hit). Unfortunately, it looks like iTSfv can only export this data from tracks that already “live” in an established iTunes library, and not directly off of an iPod. Too bad, because it looks nearly perfect for what I want. However, being that iTSfv is open-source, I’m actually tinkering with the idea of modifying it to work directly off the iPod… keep your fingers crossed.

So, again, overall it seems like there are a lot of solutions to reclaim your actual MP3 files from an iPod and move them to a new one (regardless of iPod generation). However, based on my research, I cannot find any reliable application to reclaim files and metadata – specifically going from a pre-6G device to a 6G one. So all you people who manually manage your music, and want to upgrade your iPod Photo to an iPod Classic… get ready to lose your metadata.

Again, commentary welcome!

Oh, and hey, before I go… and also iPod related…

On another note, I’m with this poster… I’d love it if Apple allowed you to group the tracks within a playlist by album or artist. This way, you could open a “New Music” playlist and see, not a huge list of songs, but instead, a nice tidy list organized albums or artists, to which you could further drill-down to track level. I know this isn’t useful for most folks who don’t view their music from an album-level of granularity (rather than just a “loose” collection of tracks), but since I tend to listen to music as albums moreso than a string of singles – it would indeed be useful to me. Anyway, would be nice to see in a future firmware… is all I’m sayin’.

Until later, take care.