Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
Tinderbox Users >> Tinderbox applications >> Tinderbox and iTunes

Message started by Alastair on Jul 9th, 2011, 5:56am

Title: Tinderbox and iTunes
Post by Alastair on Jul 9th, 2011, 5:56am


Has anyone any experience with importing their iTunes data into Tinderbox?  It seems like a good way to get my hands on a large amount of data that I could use as a test file for Tinderbox - dashboards, exporting, that type of thing.

Cutting and pasting directly from iTunes will give you a spreadsheet with a limited amount of data (Song name, artist, album, song length and playcount).  I had assumed that, once formatted, I would be able to cut and paste directly into Tinderbox. Unfortunately this doesn't work -at least not for large amounts of data.  I have managed to import 3500 songs, in 10 discrete folders of varying size - the most I have copied in one go is 999 - but this has taken 50 attempts.  Sometimes nothing happens, sometimes a note arrives with the full library listed in the title but no child notes - all in all, it's not a simple process.  Is there a limit on pasting spreadsheet data?

A second method could be exporting the music library to an .xml file, then tidying it up in a text editor and importing, exploding etc. I'm loath to do this however, as I'm not sure how to then process the exploded data into attributes, rather than discrete notes.

So, any ideas?

Title: Re: Tinderbox and iTunes
Post by Mark Bernstein on Jul 9th, 2011, 6:44am

Interesting idea!  I've never tried this myself.

Try pasting the iTunes table into Numbers or Excel or whatever else is convenient.  Look for empty cells and other formatting problems; Tinderbox is fairly conservative in wanting clean tabular data.

That should take care of the problem.  If not, send the table to support (info@eastgate.com) and we'll investigate.

Title: Re: Tinderbox and iTunes
Post by Mark Anderson on Jul 11th, 2011, 7:34am

OK, I assume we're looking at iTunes spreadsheet-like display here, i.e. the 'song list' view. I believe the reason drag/drop doesn't just work in primarily because the selection data has no column head (attribute) names.  Ergo, the data needs to be copied pasted elsewhere and column names added before use.

iWork's Numbers is not ideas as bizarrely it lacks tab-delim export. Spreadsheet lacking key format. Hmm. However if you copy/paste to Numbers, add column titles then select and drag drop to TB it works.

Pasted the same source data to Excel 2011 and the data imported OK. I added column head data. I could save this to Tab-delim. Both selection drag to TB and Tab-delim file drag to TB worked OK, except non-Latin text (I've a lot of world music) came out as underscores.

I think the best route is:
  • Set up iTunes song list view with the desired fields displayed. Make column #1 - likely track name - the column you'll be importing as TB $Name
  • Select all rows and copy.
  • Paste to BBEdit (or some other proper text editor - TextWrangler is free [sic]).
  • Add a new row #1 with column header names reflecting the desired TB attribute names. Don't use $ prefixes. Column 1 should be 'Name'. If you want a track-name attribute, clone the data from $Name after import using an agent.
  • Either select and drag to TB or, save the data in UTF-8 or UTF-16 (Unicode) form and drag-drop the resulting data.
A known limitation of TB is Unicode is not supported in key attribute display so for instance a Japanese artist's name will appear as "???" if $Artist is displayed as a key attribute. However, the underlying $Artist data will be correct if if you can't see it via that means.

Nice as it would be if all apps used consistent formats, they don't. In my general (work) experience moving large amounts of text data between apps often involves some form of cleaning/fettling en route. Indeed, such is why apps like BBEdit and TextEdit exist.

With the exception of the (known) key attribute issue, TB actually copes pretty well given the messy data - albeit with the need for some minimal fettling en route. However, having experimented for the above notes, I'm not sure I'd concur that iTunes is a good data block to use for TB experimentation, but your mileage may vary...

[Post-edit: reading around some more UTF-16 is probably not ideal in the spreadsheets. Certainly TB uses UTF-8 itself so that's a better end to end match. also, if you've Excel handy don't bother with Numbers - as the later's missing a lot of features you might expect from a spreadsheet, not least being unable to export a major standard interchange format (tab-delimited).]

Title: Re: Tinderbox and iTunes
Post by Alastair on Jul 11th, 2011, 12:33pm

I initially tried this with both Excel and Numbers and had the same problem with each.  I did have a limited amount of success in the end, although it was a frustrating process - I certainly couldn't get Tinderbox to accept a 13,000 track cut-and-paste, 3000 songs was the largest chunk of data that I could transfer.  The crucial fix was to replace all " characters in the song titles with single quote marks, which TBX coped with more successfully.  Thanks for your advice on spreadsheets though.

Given the limited data set that iTunes will let a user cut and paste, I'm toying with the idea of using a different route.  As the library can be exported in .xml format with considerably more data per track, I would like to work out a way of bringing everything into Tinderbox.

Removing the xml tags in TextWrangler and adding delimiters allows me to import and explode a full song list, each track containing it's own data in the $Text.  Is it possible to allow a note to set its own attributes by reading a list contained within itself, line by line?  Would it be easier to perform a secondary explode*, populating each track with separate notes named Album, Year, SongLength, etc, each one containing a discrete value which could be fed back up the food chain to the 'Track' note?

*This would probably involve automating the explode process in some way, which may not be possible.  I may need to think this through!

Title: Re: Tinderbox and iTunes
Post by Mark Anderson on Jul 11th, 2011, 1:59pm

Why note go to OPML, which TB now support well. See OPML import. There are 3 'unusual' attribute mappings - due to meeting existing OPML conventions:
text        Name
_note     Text
_status   Checked

Otherwise attributes names are added (case0sensitively) to TB. The best method is to deicide and set up the attributes in TB and then use those names when cleaning the XML to OPML.

I can definitely say this works as I used the method to help a user parse some really messy 24k+ line XML file exported from a badly configured Access database and got a clean OPML import into TB. Sadly I can't share the before/after data as it was private consulting, but it did work - indeed much better than I'd expected. I've been impressed with TB's consumption of OPML.

Title: Re: Tinderbox and iTunes
Post by Alastair on Jul 22nd, 2011, 4:39pm

Sorry, this thread was delayed by a sudden outbreak of childbirth.

OPML seems like the way forward,  and the cleanup was relatively simple (I can post a swift guide if anybody needs one.)  This resulted in a 360,000+ line file, which seems too big for import (is there a limit?). However, great chunks of the info, dragged in separately, are importing fine.

Quick question:  The date format of OPML leaves me with strings of the type "2010-01-11T16:22:19Z". What's the best way of processing this to a recognisable date in tinderbox? Alternatively, can TBX distinguish between strings/integers/dates during OPML import?

Title: Re: Tinderbox and iTunes
Post by Mark Bernstein on Jul 22nd, 2011, 5:20pm

Tinderbox can indeed make an effort to distinguish strings, numbers, and dates on import. I'm not sure the importer will recognize that date format, though. If it's imported as a string, try an action like


Title: Re: Tinderbox and iTunes - AppleScript
Post by Sumner Gerard on Oct 19th, 2012, 12:37am

AppleScript is great for this kind of thing with a minimum of fuss. It can easily send the data to the clipboard in the "tab-separated-value" (tsv) format that Tinderbox's powerful "spreadsheet import" can handle with ease. Often no need to fiddle around with saving to a file and then importing that.

The following takes less than 5 minutes to set up.

Using AppleScript Editor to repurpose and simplify the many iTunes scripts available online gives this. Paste into AppleScript Editor to run:

--puts tab-separated-data information about tracks in selected iTunes playlist on clipboard for pasting into TB
--in your data row you can also use properties named: lyrics, rating, year, comments, description. See AS Dictionary for others.
tell application "iTunes"
     set this_playlist to the view of browser window 1 --view contains user playlist id and source id of currently selected playlist
     --these "headers" import as TB attributes--if they do not exist TB creates them automatically:
     set tsv_data_string to "Name" & tab & "Artist" & tab & "Album" & tab & "Length" & tab & "PlayedCount" & tab & "TrackID" & tab & "TimeOffset" & return
     tell this_playlist
           repeat with i from 1 to the count of tracks -- the number/order of columns in "data row" must match the "header" row above:
                 tell track i to set tsv_data_string to tsv_data_string & quote & name & quote & tab & artist & tab & album & tab & time & tab & played count & tab & persistent ID & tab & "" & return
           end repeat
     end tell
     set the clipboard to tsv_data_string as text
end tell
tell application "System Events" to display dialog "Information on tracks in playlist " & quote & name of this_playlist & quote & ¬
     " ready to paste into Tinderbox" buttons {"OK"}

All this script does is put the desired iTunes data into a tab-separated-values (tsv) format on the clipboard for pasting onto a Tinderbox view. To use it, in iTunes drag the tracks you want for Tinderbox into a playlist, select that playlist, run the script, and wait (depending on how many tracks there could be a half a minute delay or so) for the "ready to paste" message.

Tinderbox "spreadsheet import" is great on my machine. It has no problem swallowing my entire collection in one fell swoop, Chinese names and all. It automatically creates new notes adding new attributes if they don't already exist.

I did have to surround the data in the Name column with quotes, though. Any columns that have line returns in their content, say lyrics or comments, also must be surrounded by quotes. Unfortunately Tinderbox doesn't strip those quotes after import like some other applications, but Mark A's aTbRef explains how you can clean up with "string trimming."

So why the extra Track ID (persistent ID) column and TimeOffset (no data in it) columns? On the Tinderbox side those permit you to play a track starting in the middle somewhere if you want.

To do that, paste this into a Tinderbox code note named PlayiTunesBmarkAS (this name can be changed but must match the name in the stamp below).

on run argv
     set {trackID, timeOffset} to {item 1, item 2} of argv
     tell application "iTunes"
           play location of item 1 of (file tracks of playlist 1 whose persistent ID is trackID) as text
         set player position to timeOffset as string
     end tell
end run

And create a stamp named, say, "Play iTunes bookmark" and paste this in the Action box and save:

runCommand("osascript -e '"  +$Text("PlayiTunesBmarkAS") +"' '" +$TrackID +"' '"+ $TimeOffset +"'")

All the code in the stamp does is tell the system to run the AppleScript using the values from the two Tinderbox attributes. The AppleScript receives them as item 1 and item 2 of the "run handler", here named argv.

Now, whenever you want to play that track just select the note (no need to open it) and choose "Play iTunes bookmark" in the Stamps menu. That in itself isn't too exciting, of course, because if you want to play it you can just do that directly in iTunes (after you find it). But what if you want it to start playing, say, five minutes in because that's where a really interesting part starts?  To do that open the note and enter 300 in the $TimeOffset key attribute (the first time, you may have to change the attribute type to number first as TB may think it is boolean), and then choose the stamp to play.

You can of course duplicate these "bookmark" notes to make multiple time offsets in the same track.

Using a similar approach of short AppleScripts and TB's spreadsheet import capability, and one line of TB action code in a stamp, there are easy ways to further automate the creation of bookmarks to your current spot in a playing audio or video, not just in iTunes but in QuickTime Player, literally one click and a paste and you have a new bookmark ready to go. Very handy for analyzing and annotating and interlinking long audios or videos, say an interviews or a conference call recordings or lectures or language lessons or ...

Welcome feedback, problems, questions, corrections, improvements.

Title: Re: Tinderbox and iTunes
Post by Sumner Gerard on Oct 21st, 2012, 9:30pm

ways to further automate the creation of bookmarks to your current spot in a playing audio or video

Among the track information you bring over from iTunes, suppose there are some tracks where you want multiple bookmarks in Tinderbox that you can use for annotation and linkages of ideas, with the ability to replay as needed, for example when you're analyzing and comparing different parts of interviews or telephone call recordings, looking for linkages and common themes. You can set up each bookmark manually as described above, by duplicating an existing bookmark note and entering a different time offset in seconds in the duplicate... Or you can let AppleScript do the bookmarking for you.

tell application id "com.apple.iTunes"
     if player position > 0 then
           -- get the track info and player position from iTunes, and make hh:mm:ss time stamp:
           set {trackName, trackID, trackGenre} to {name, persistent ID, genre} of current track
           set timeOffset to player position as integer
           set timeStamp to my secsToHMS(timeOffset)
           -- put a tab-delimited header row and the one and only data row on the clipboard:
           set the clipboard to "Name" & tab & "TrackID" & tab & "Genre" & tab & "TimeOffset" & return ¬
                 & timeStamp & " -" & quote & trackName & quote & tab & trackID & tab & trackGenre & tab & timeOffset & return
           tell application "System Events" to display dialog "Click a Tinderbox outline view and hit <command-v> to paste iTunes bookmark." buttons "OK" default button "OK"
           tell application "System Events" to display dialog "Oops! Choose a start point in a selected iTunes track first, then run script again." buttons "OK" default button "OK"
     end if
end tell

on secsToHMS(secs)
     --convert seconds to hh:mm:ss <http://macscripter.net/viewtopic.php?id=27203>;
     -- to suppress hh: delete:  text 2 through 3 & ":" & "
     tell (1000000 + secs div hours * 10000 + secs mod hours div minutes * 100 + secs mod minutes) as string to return text 2 thru 3 & ":" & text 4 thru 5 & ":" & text 6 thru 7
end secsToHMS

Paste this into AppleScript Editor and save wherever convenient. In iTunes whenever you're at a spot you want to bookmark, run the script (pausing iTunes first often works best) and then paste onto a Tinderbox view. Tinderbox then creates a new time-stamped bookmark note automatically. The bookmark can be played whenever needed by applying the stamp/short AppleScript combination described upthread.

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.