Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi
Tinderbox Users >> Exporting from Tinderbox >> Exporting Tagged Files?
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi?num=1307305847

Message started by Alex Strick van Linschoten on Jun 5th, 2011, 4:30pm

Title: Exporting Tagged Files?
Post by Alex Strick van Linschoten on Jun 5th, 2011, 4:30pm

Is Tinderbox able to export files that are tagged (with the OpenMeta format standard that Macs can, for now, handle)?

Title: Re: Exporting Tagged Files?
Post by Mark Anderson on Jun 5th, 2011, 5:43pm

I don't believe so.

Title: Re: Exporting Tagged Files?
Post by Paul Walters on Jun 5th, 2011, 5:56pm

AFAIK, Tinderbox does not have an export action that would tag a file with OpenMeta tags.  OpenMeta uses a custom command line interface (CLI) that sets and retrieves tags by modifying the extended attributes of a file.  The OpenMeta CLI is available for download from Google Code at

http://code.google.com/p/openmeta

In theory, if the CLI is installed, one could use the Tinderbox runCommand action, call the CLI, and tag a file.  To do this at export time, the file would have to be created, closed (i.e., no longer controlled by Tinderbox), the path known, and then runCommand executed.  

This seems like a lot of heavy lifting.  The simpler approach would be to export from Tinderbox, and then use Punakea or some other OpenMeta-capable application to tag that file.

Title: Re: Exporting Tagged Files?
Post by Alex Strick van Linschoten on Jun 6th, 2011, 1:28am

Many thanks. Would have simplified my life somewhat (export of several thousand notes) but realise it was a long-shot.

A

Title: Re: Exporting Tagged Files?
Post by Mark Anderson on Jun 6th, 2011, 3:14am

What is the next app in the line needing the metadata, perhaps there's a different way (format)?

Title: Re: Exporting Tagged Files?
Post by Alex Strick van Linschoten on Jun 6th, 2011, 3:23am

Ok so I have a database in Tinderbox of individual statements (several thousand in number) organised by year and by month (i.e. nested containers). Each individual statement I'd like to tag (using a 'set' attribute called metatags or sometghing.

At the same time, I'd like to have those files in DevonThink to be able to have them alongside my other documents there (just sort of as a read-only reference copy).

So I'd like to export these notes (in their respective folders) pre-tagged so I don't have to tag them each all over in DevonThink. The database will be updated quite regularly for the first year or two, so I'd be exporting all the notes about once a month (too often to consider doing the tagging manually, esp if I'd have to redo it all next month)...

DevonThink reads OpenMeta tags, which are the de facto tagging solution for OSX at the moment (although people fear Apple might pull the carpet out under their feet at some point and disable support for this.

Title: Re: Exporting Tagged Files?
Post by Mark Anderson on Jun 6th, 2011, 4:43am

Reading up on OpenMeta, it's a crafty hack on Apple data so definitely unsupported. THe data storage method also strikes me as only usable locally and thus is prone to breakage. As such, given the likely time (cost) of engineering such a feature into TB for the benefit of 'talking' to just a few apps, it's perhaps pertinent to look at whether existing tools can't close the loop.

Doesn't DT have any tab-delim import? I don't have to app but looking at the (rather terse) PDF manual online, tab-delim (a.k.a. 'TSV') is listed as an import method. That way you just export a list of (unique) resource names assuming DT allow data merge to existing records.

Failing that, DT is AppleScriptable, it seems so you could export a data list from TB with existing tools and simply run an AppleScript to update the DT tags of the appropriate DT notes.

Title: Re: Exporting Tagged Files?
Post by Alex Strick van Linschoten on Jun 6th, 2011, 4:44am

Hmm ok. As I thought, no easy way to do this. Ok I'll have a think as to which of these two methods would work.

Title: Re: Exporting Tagged Files?
Post by Martin Boycott-Brown on Jun 6th, 2011, 12:25pm

But doesn't a folder in DT (what DT refers to as a group) correspond to a tag anyway (unless you have switched off tagging for the group)? I realise it's not the same as an OpenMeta tag, and maybe I've misunderstood what you are trying to do. Then again, DT supports Spotlight comments, and there are scripts available through their website that will convert Spotlight comments to tags -- I don't know if there would be any mileage in that. Is Tinderbox capable of writing Spotlight comments? Apologies if I haven't grasped what you are trying to do, but I'm also interested in working between Tinderbox and DTPO.

Martin.

Title: Re: Exporting Tagged Files?
Post by Mark Anderson on Jun 6th, 2011, 1:14pm

TB can't write Spotlight comments. FWIW, OpenMeta is a hack to make Spotlight comments available in other apps.

I'll admit to never having seen the point of Spotlight - it's rarely found what I'm looking for - you clearly need to use data in a very specific way for it to work.

One issue I do see with metadata of this type is you (or all bar the most techie users) can't see this data, it becomes magic data hidden away. Fine if it works, a real pain if not; now not only is it not turning up in the next app but you can't even find it to figure out what's wrong. But maybe I'm being a bit glass half empty here  :)

Title: Re: Exporting Tagged Files?
Post by Alex Strick van Linschoten on Jun 6th, 2011, 1:18pm

The previous comment does make a useful suggestion though. Indeed, DevonThink tags are exactly the same as folders (albeit not in certain functional ways). So if we could get tinderbox to export as per the tags:

I.e.

We have 2 notes.

note1 - tagged with "china, football, space travel"
note 2 - tagged with "russia, tennis, space travel"

The export would give us the following folders and contents:

China - note 1
Football - note 1
Russia - note 2
Space Travel - note 1 and note 2
Tennis - note 2

I presume Tinderbox could do that? then as far as DevonThink is concerned, these are pre-tagged (i.e. it assigns a tag of the parent-foldername as the default setting).

Title: Re: Exporting Tagged Files?
Post by Martin Boycott-Brown on Jun 6th, 2011, 1:25pm

I'm not a great fan of tagging, either. Too much of a faff to enter all of the tags. I'd rather search the actual data for key words (one of the advantages of DTPO is the complex searches you can build "foo" within twenty words of "bar", etc.). And I've also found Spotlight unreliable -- yet another reason for using DTPO.

Incidentally, I've just been looking at the DT forums, and there are various interesting scripts submitted by people that might be useful for exchanging material between programs. Also an interesting technique for using Notational Velocity, indexing its folder in DT, and synchronising via Simplenote. I wonder if that might be a way of exchanging some kinds of information with Tinderbox. See:

http://www.devon-technologies.com/scripts/userforum/viewtopic.php?f=2&t=11899&p=55817&hilit=simplenote#p55817
http://www.devon-technologies.com/scripts/userforum/viewtopic.php?f=4&t=10659

Title: Re: Exporting Tagged Files?
Post by Alex Strick van Linschoten on Jun 6th, 2011, 1:48pm

The only reason I ask is that with tagging I can do quite easily what it would take a fair amount of manual coding to do in Tinderbox, and for certain types of query it is 10,000 times easier to do it via tagging.  But agreed, tagging is only as useful as the tags you assign...

Title: Re: Exporting Tagged Files?
Post by Mark Anderson on Jun 6th, 2011, 1:51pm

@alex, this is do-able...sort of. The problem is you want to do an export of each item in an arbitrary list - i.e. the current set of Tag values. Today it might be 6 items, next week 60.  Tinderbox doesn't have a "for each item in an unknown size list construct, nor is there a for(i=0;I<count;increment) type syntax.


Quote:
The export would give us the following folders and contents:

China - note 1
Football - note 1
Russia - note 2
Space Travel - note 1 and note 2
Tennis - note 2

How do you see that building from TB data? Do you have a working (as in imports to DT) example of the sort of file we're making? It helps to be able to map what goes where. For instance, in 'space travel' above, is the bit after the hyphen [$Name]+" and "+[$Name]...etc. To do formatted export we need to know the 'rules' of the format.

Title: Re: Exporting Tagged Files?
Post by Paul Walters on Jun 6th, 2011, 6:48pm

@alex, I crafted an AppleScript for you that might help.  If you export your notes into a comma-delimited tab-delimited file with a structure such as this:


Quote:
Name Tags Text
"First document"   "red;blue;yellow"  "This is the first document text"
"Second document"   "pink passion;green"  "This is the second document text"


Put your tags into the second element of the export as a semicolon-delimited string (a quoted string is recommended).

Then, import that file into DEVONthink, it will become what DEVONthink calls a sheet - a table of rows and columns

Select that sheet in DEVONthink and run this script, you will get individual RTF records for each of the rows of the sheet.  The name, tags, and text will be assigned per their respective columns of the sheet.


Code:
(*
This script will create a new RTF document from each row of a "sheet".  The sheet should have three columns.

Column 1: the name of the new document
Column 2: the tags for the new document, delimited by semicolons
Column 3: the text for the new document

The column header names are not relevant, but it is useful to name them "Name", "Tags", and "Text", respectively.

Select the sheet and run the script.  The script will prompt for the target location of all the documents to be
created.

*)
tell application id "com.devon-technologies.thinkpro2"
     
     try
           if not (exists think window 1) then error "No Window is Open"
           set this_window to think window 1
           tell this_window
                 
                 copy number of rows to rowCount
                 copy number of columns to colCount
                 
                 set targetGroup to display group selector "Destination group?"
                 
                 set currRow to 1
                 
                 repeat while currRow ≤ rowCount
                       set theName to get cell at row currRow column 1
                       set tagList to get cell at row currRow column 2
                       set theText to get cell at row currRow column 3
                       
                       create record with {name:theName, tags:(my getTags(tagList)), type:rtf, rich text:theText} in targetGroup
                       
                       set currRow to currRow + 1
                       
                 end repeat
                 
           end tell
     end try
end tell

on getTags(tagList)
     set {od, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ";"}
     set tagList to text items of tagList
end getTags


I can modify this script to read a fourth element (column) of the table which corresponds to the path for each note, and the script will put the documents into a hierarchy of folders in DEVONthink.  However, this is a bit more complex and it will take me a while to code and test it.

As DEVONthink parses tags and automatically de-duplicates them, you shouldn't need to do any preprocessing of your tag set attribute to eliminate duplicates.

I trust you can do the export template in Tinderbox pretty easily.  Or see Mark Anderson's post, following this one.

Edit: I've later found that DEVONthink does not reliably recognize .csv (comma-separated values) files as sheets, but will reliably convert .tsv (tab-separated values) files to sheets.  I've asked Mark Anderson to amend his export recipe accordingly, which is, I believe, a simple adjustment.

Title: Re: Exporting Tagged Files?
Post by Mark Anderson on Jun 7th, 2011, 2:57am

Post-edit. Following Paul's update (last post above) I'm updating the templates to Tab-delim format. I've moved the CSV stuff to a new post (next one below) as it might be of general use in exporting CSV data.

For those less familiar with export, here's the TB end. OK, some assumptions:
  • Export will happen via an agent
  • The tags are stored in a user Set-type attribute $TagSet. Use a set rather than a List so the per-item values are always de-duped (i.e. a given tag doesn't appear more than once in any given item's TagSet)
  • If your tag set has a different name substitute reference to Tagset in code with your set's name
To set up export, first add two HTML export templates. In both cases there is a line return at the end of the last line of code, i.e. you can place the cursor at the beginning of the line after the last line of code (Code mark-up in the forum can make this aspect ambiguous). In the output text file you want each item's data on a separate line and no blank lines.

1. DT-tags-Wrapper:

Code:
Name      Tags      Text
^children(DT-tags-item)^

N.B. in the first line there is a Tab character between each word.

2. DT-tags-item:

Code:
^value($Name)^      ^value($TagSet)^      ^value($Text)^

N.B. there is a Tab character between each ^value()^.

Create an agent. Open it's HTML view and select "DT-tags-wrapper" as the template.  Change the export extension from ".html" to ".txt". close HTML view. Via Rename add this query to find all notes with $TagSet data:

  $TagSet   (yes, it's that simple a query!)

You can of course add extra terms to the query to limit the scope of search, date range, whatever.

Now select the agent, open HTML view and hit the Export button (top right). Nominate an export location (you only get asked for this once per TBX), and export ready for use in DT.

If, as in Alex's original example, you're not exporting $Text then simply amend both templates to remove the last Tab+item, i.e. delete the last column from the export.


Title: Re: Exporting Tagged Files?
Post by Mark Anderson on Jun 7th, 2011, 8:49am

This was my original follow up to Paul's answer (#14  above) before it proved necessary to change from CSV to tab-delim data.

~~~~~~~~ Original CSV based data ~~~~~~~~~~~~
1. DT-tags-Wrapper:

Code:
Name,Tags,Text
^children(DT-tags-item)^

2. DT-tags-item:

Code:
"^value($Name)^","^value($TagSet)^","^value($Text)^"


Create an agent. Open it's HTML view and select "DT-tags-wrapper" as the template.  Change the export extension from ".html" to ".txt". close HTML view. Via Rename add this query to find all notes with $TagSet data:

  $TagSet   (yes, it's that simple a query!)

You can of course add extra terms to the query to limit the scope of search, date range, whatever.

Now select the agent, open HTML view and hit the Export button (top right). Nominate an export location (you only get asked for this once per TBX), and export ready for use in DT.

If, as in Alex's original example, you're not exporting $Text then use:

Code:
Name,Tags
^children(DT-tags-item)^

and

Code:
"^value($Name)^","^value($TagSet)^"


Notes re use of Comma Separated Variable (CSV) format:
  • If values contain commas, they must be enclosed in straight double quotes.
  • If values contain line breaks (paragraphs) or leading/trailing white space they must be enclosed in straight duoble-quotes.
  • I've quoted all 3 fields above as $Name or $Text might contain commas.
  • Quoted values can contain straight double quotes but such characters must be doubled. So 'I said "Hello" to him' must - for CSV value usage become 'I said ""Hello" to him'. Tinderbox has no built-in method for this. Suggestions: either use curly double quotes inline within your $Text instead of straight noes, or write a command line to double any quotes. The latter would need to be done via runCommand pre or during export. Don't use $HTMLExportCommand as we don't want the quotes enclosing values to also be doubled
  • This page seems a useful reference on the 'rules' of CSV.

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.