Welcome, Guest. Please Login
Tinderbox
  News:
IMPORTANT MESSAGE! This forum has now been replaced by a new forum at http://forum.eastgate.com and no further posting or member registration is allowed. The forum is still accessible via read-only access for reference purposes. If you wish to discuss content here, please use the new forum. N.B. - posting in the new forum requires a fresh registration in the new forum (sorry - member data can't be ported).
  HomeHelpSearchLogin  
 
Pages: 1 2 
Send Topic Print
Linking into DEVONThink items (Read 39846 times)
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Linking into DEVONThink items
Reply #15 - Sep 07th, 2012, 12:44pm
 
By a 'bare' link I just meant the normal one, bare of any query suffix (time/page/search). I'm just heading off on leave, but in due course I can review that article and perhaps improve the sense of it.

Re OPML. You'd need to write a custom OPML export script for DT - there seem plenty of examples in the DT scripting sub-forum that would help you get going and people there seem pretty responsive and helpful.

By default, in DT's built-in export the OPML URL attribute contains the DT item's URL info, i.e. to what it links or from where it came (whatever - I'm not 100% sure, but certainly not the DT link info you're after). Therefore, to use the DT 'item link' URL value you need to get the AppleScript reference URL property for the item and insert it into the OPML code.

You can write it into the URL OPML attribute to map to $URL in TB, or use a custom OPML attribute name. To ensure the latter data goes into a TB URL-type attribute, ensure that you make an attribute of the correct data type in your TBX before you import the OPML data. Then, if the custom OPML attribute name matches an existing TB attribute name (case-sensitive!), TB will use the existing attribute. Otherwise TB will make a String-type attribute for your custom OPML attribute. That's fair enough as TB has no way to know, from the OPML, what data type a custom OPML attribute's value should be. Only a few OPML-TB attribute mappings are hard-coded, such as OPML's _note to TB's $Text and text to $Name.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Linking into DEVONThink items
Reply #16 - Sep 7th, 2012, 2:11pm
 
It was the reference to 'icon' that made me think I was missing something. Probably a typo to be fixed in due course. Enjoy the leave!

I now know a bit about custom OPML export script from DEVONthink to Tinderbox, of course, learned in the course of modifying and posting the script up-thread, which works well for chopping a long document in DEVONthink into paragraphs for exploration in Tinderbox, maintaining the links back to specific passages in the DEVONthink document.  

What I am interested in now is how to detect and grab highlighted text only, not paragraphs. The mention of 'highlight detection' made me think maybe you'd already come across something useful on that. Looking for it, I got lost in the thicket of threads over on the DEVONthink scripting forum. It's somehow related to 'annotation' I think, but I haven't been able to narrow the search down enough.
Back to top
 
 
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Linking into DEVONThink items
Reply #17 - Sep 7th, 2012, 2:24pm
 
'icon'? Ouch, how did that creep in? Sorry!.  Now corrected.

Highlight detection? I think it's possible.  Spun my wheels yesterday trying this (inadequacies of AppleScript language definition rather than DT, in fairness). I think it is possible, but it's not going to be blink-of-the-eye fast.

I'd float the concept under a topic of that name in the DT scripting forum., IOW, export only highlighted text to OPML. From what i've seen, I'm pretty sure it's possible though for more than the odd doc or two the process will likely give you time to relax while you wait (again, speed of AppleScript not a knock at DT). I suspect this is faster simpler to do for a single DT item (doc) than for your whole database. some of the knowledgable hands in the DT scripting forum also turn up here so they might have some idea what you're about.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Linking into DEVONThink items
Reply #18 - Sep 7th, 2012, 4:49pm
 
From what I can tell from this thread I just found, grabbing just the text from highlights may not be possible in DEVONthink. Or if possible, not (as intimated by your comments about having time to relax) practical. For pdf's the way to ago apparently is via Skim. For rtf documents not sure.
Back to top
 
 
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Linking into DEVONThink items
Reply #19 - Sep 7th, 2012, 5:33pm
 
Mining a bit deeper, it seems DT can't script read PDF annotations (in fairness this stuff isn't simple under the hood - lots of gnarly Postscript). I do think the task is do-able with RTF and TXT (and DOC, etc.) files but I suspect PDF is where most people's DT data lives.

Skim (free!) does seem to be AppleScriptable. In reality, we're very often doing something similar  but not the same as others for which a one-tool solution simply doesn't exist and thus a degree of flexibility is required. Tools that do one (or several tasks) very well and have configurable import export are the things to looks for.

TB may not be AppleScrip-table, it does consume formats like UTF-8 encoded OPML and tab-delim text fairly well so is pretty good at importing formated plain text data. Ergo, if the upstream app has built-in configurable export or is scriptable (to make such a data file) it then becomes a matter of trial and error testing to get it into TB. User forums exist to take up the slack - not least as some of the building blocks of a workflow solution can be quite arcane. Also, if it's a really big task maybe it's maybe worth someone with XML skills to enable simple read-write to the source TB data. Sometimes having a UI blinds us to the possibilities (even if it means other people/cost in the loop).

[edited for typos]
Back to top
 
« Last Edit: Nov 27th, 2012, 1:09pm by Mark Anderson »  

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Linking into DEVONThink items
Reply #20 - Sep 7th, 2012, 11:38pm
 
Quote:
I do think the task id do-able with RTF and TXT (and DOC, etc.) files but I suspect PDF is where most people's DT data lives.

Right you are that it is in fact doable with TXT and RTF. Via the DTPO Scripts/More Scripts menu one can download 'extras'. These 'extras' happen to include two 'Merge Highlights' scripts. These scripts sweep up the highlighted text in selected plain text or RTF documents and dump them into a newly created DT plain text or RTF note. Presumably those scripts can be modified to output an OPML file for Tinderbox with links back to the relevant passages in the original DT document using the same search-string-appended-to-the-DT-link approach discussed upthread.

For PDFs, though, the best route seems to be through Skim, which out of the box apparently can export highlights and annotations... Or first convert the PDF to RTF.
Back to top
 
« Last Edit: Sep 7th, 2012, 11:40pm by Sumner Gerard »  
  IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Linking into DEVONThink items
Reply #21 - Oct 11th, 2012, 11:35pm
 
Here are some enhancements to Mark A's script here for exporting multiple selected DEVONthink records to the clipboard for pasting into Tinderbox, retaining the links back to DEVONthink.

Code:
-- exports selected DEVONthink records to clipboard for pasting into Tinderbox
-- http://www.eastgate.com/Tinderbox/forum//YaBB.pl?num=1346083940/7#7
-- http://forum.devontechnologies.com/viewtopic.php?f=20&t=15873
-- expanded to include "quotified" multi-paragraph text

set pTab to tab -- "tab" has a specific meaning to DEVONthink so use "pTab" in tell block

--Set up tab-separated-values table "header" with Tinderbox attribute names:
set dataString to "Name" & pTab & "URL" & pTab & "Text" & return

tell application id "com.devon-technologies.thinkpro2"
	set itemList to selection
	if itemList is {} then
		tell application "System Events" to display dialog "Oops! ... First select some DEVONthink records and try again." buttons "OK" default button "OK"
	else
		repeat with anItem in itemList
			set aText to rich text of anItem
			--replace any straight quote with curly/smart, and surround with straight double quotes:
			set quotifiedText to quote & my quotify(aText) & quote
			--add a tab-separated-values "data row" to the pile:
			set dataString to dataString & (name of anItem) & pTab & (reference URL of anItem) & pTab & quotifiedText & return
		end repeat
		set the clipboard to dataString as Unicode text
		tell application "System Events" to display dialog "DEVONthink data placed successfully on clipboard! … Now click Tinderbox outline view (not an open note) and hit <command-v> to paste." buttons "OK" default button "OK"
	end if
end tell

on quotify(txt)
	-- replace straight quotes with curly/smart versions
	--"quoted form" of mwra quotify() <http://www.eastgate.com/Tinderbox/forum//YaBB.pl?num=1265006946/0#4>
	set scrptCom to "sed 's:\"\\([^\"]*\\)\":“\\1”:g' | sed \"s:'\\([d|ll|re|s|t|ve]\\):’\\1:g\" | sed \"s:'\\([^']*\\)':‘\\1’:g\""
	return do shell script ("echo " & quoted form of txt & " | " & scrptCom)
end quotify

return dataString -- (optional) display in AppleScript Editor Result pane
 



This brings over text as well as the name and the link.

Initially, it would choke on text with multiple paragraphs and/or quotes. Surrounding the text with straight double quotes and "quotifying" it using an AppleScript-friendly version of this gem solved that problem.

But, sadly, Tinderbox doesn't discard the surrounding quotes after the import as does, say, Excel...and probably other programs that accept tab-delimited data. And it *seems* to insert an extra line return before the closing quote. Harmless (I think) but not sure it should be there. Any workaround?

Back to top
 
« Last Edit: Oct 12th, 2012, 3:50pm by Sumner Gerard »  
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Linking into DEVONThink items
Reply #22 - Oct 12th, 2012, 3:27am
 
Re the stripping of enclosing quotes, the problem is contextual. It wouldn't be useful if any quote enclosed data was stripped of those quotes when pasting. It's only in certain defined contexts (as definable from an app's code perspective, not a human one) where we might want TB to strip quotes. Not sure how easy the latter is - though I agree it would be nice if everything 'just' worked as intuited!

Meanwhile, whilst the app may be unable to guess, the user's easily able to see if quotes have strayed in. Why not make a stamp that uses String.substr() to trim the quotes? Thus:

$Text=$Text.substr(1,($Text.size-2))

Code above tested in v5.11.2.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Linking into DEVONThink items
Reply #23 - Oct 12th, 2012, 5:47am
 
Also see this thread which links to more examples building on my last answer above.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Linking into DEVONThink items
Reply #24 - Oct 12th, 2012, 12:15pm
 
Quote:
It's only in certain defined contexts (as definable from an app's code perspective, not a human one) where we might want TB to strip quotes.

Pasting of tab-separated-value data in table form definitely, I think, qualifies as one of the defined contexts in which we would expect TB to strip surrounding double straight quotes. Stripping them seems to be the de facto standard in Excel, Google Spreadsheets, Bento, and probably many (most?) other programs that accept (and export) tab-separated-value format.

That would avoid, I think, the problems with import from Aeon Timeline discussed here as well as with import of multi-paragraph text from DEVONthink.

Meanwhile, thanks for the useful stamps to add to the collection!  (Someday, maybe, regular expressions will seem less inscrutable.)
Back to top
 
 
  IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Linking into DEVONThink items
Reply #25 - Nov 27th, 2012, 12:38pm
 
Here's an AppleScript that grabs chunks of highlighted text from DevonTHINK rich text records for pasting onto a Tinderbox view, using TB's "spreadsheet import" capability. This imports just the chunks of text wanted for analysis in Tinderbox, with each chunk retaining a link back to its specific location in the DT source document, making it easy to review the original context whenever needed.

I've found it useful for making sense of long, meandering conference call and interview transcripts and the like. I read through, highlighting what seems most relevant, then with the DT document(s) selected, run the script, and paste the results onto a Tinderbox view, where the notes can be manipulated in the usual ways.

This grabs all highlight colors, though it could easily be modified to grab only highlights of specific colors if one is using a highlight color-coding system. The subroutines with one-liner shell scripts may be of interest to those working with applications other than DT too.

Code:
-- Gathers highlights in selected DEVONthink Pro rich text documents and copies them to the clipboard for pasting into Tinderbox
-- Appends search string to URL$ that links back to the specific passage in the source DT document
-- Handles Chinese characters, and other multi-byte languages
-- A highlight must be followed by a space or punctuation mark for the DT search string to work
-- Loop logic simplified from "Merge Highlights" scripts by Christian Grunenberg
-- To use: select one or more DT records containing highlights, run this script, and paste onto a Tinderbox view

-----------Initialization-----------
set data_str to "Name" & tab & "Text" & tab & "URL" & return --header row column names map to TB attribute names
set start_length to length of data_str as string --for error handling
set p_tab to tab --tab has specific meaning in DT so use p_tab instead in the DNtp tell block
set n to 10 --maximum number of words desired for Tinderbox attribute $Name and $URL search string
-----------Main routine-----------
tell application id "DNtp"
	try --the try block allows error processing
		set the_selection to the selection --list of all DT records selected by user
		if (not (exists think window 1)) or (the_selection is {}) then error "Please select some rich text documents."
		set the_window to missing value
		set the_main_window to think window 1
		show progress indicator "Gathering highlights..."
		repeat with a_record in the_selection -- loop through the selected DT records
			if type of a_record is rtf or type of a_record is rtfd then
				if the_window is not missing value then
					set record of the_window to a_record
				else
					set the_window to open window for record a_record
				end if
				tell text of the_window
					repeat with an_attribute in attribute runs --loop through an "attribute runs" list of strings grabbing those with a non-white "background" property
						try
							set the_properties to properties of an_attribute
							set the_background to background of the_properties
							if the_background is not missing value and the_background is not {65535, 65535, 65535} and length of (an_attribute as string) > 1 then
								set raw_highlight to an_attribute as text
								set cleaned_highlight to my tsvify(raw_highlight) --replace any tabs or line feeds with spaces, for $Name and $Text
								set dt_uri to "x-devonthink-item://" & uuid of a_record & ¬
									"?search=" & my pct_encode(my first_n_wrds(raw_highlight, n)) --DT search string--already encoded so do not clean
								set data_str to data_str & my first_n_wrds(cleaned_highlight, n) & p_tab & cleaned_highlight & p_tab & dt_uri & return --add data row
							end if --> highlighted
						end try
					end repeat -->until no more strings from "attribute runs"
				end tell
			end if -->DT record type is rtf or rtfd
		end repeat -->until no more DT records in selection
		set the clipboard to data_str --put the tab-separated-values table on the clipboard
		hide progress indicator
		if the_window is not missing value and the_window is not the_main_window then close the_window
		if length of data_str is not greater than start_length then error "No highlights found."
		display dialog "Highlights placed on clipboard… Now click a Tinderbox view (not an open note) and hit <command-v> to paste." buttons "OK" default button "OK"
	on error error_message number error_number
		if the error_number is not -128 then display alert "DEVONthink Pro" message error_message as warning
	end try
end tell

----------- subroutines ------------------
on tsvify(str) --replace any tabs and line feeds with spaces - syntax:  tr 's' 'r'
	do shell script "echo " & quoted form of str & " | tr  '" & tab & "' ' ' | tr '" & linefeed & "' ' '"
end tsvify

on pct_encode(str) -- "percent encode" search string for uri
	do shell script "echo " & quoted form of str & " | php -r \"print urlencode(trim(file_get_contents('php://stdin')));\""
end pct_encode

on first_n_wrds(str, n) --return first n words - syntax: cut -d' ' -f1-n
	do shell script "echo " & quoted form of str & " | " & "cut -d' ' -f1-" & n
end first_n_wrds
 

Back to top
 
« Last Edit: Nov 28th, 2012, 11:21am by Sumner Gerard »  
  IP Logged
Simon Smailus
Full Member
*
Offline



Posts: 141
Tipton, UK
Re: Linking into DEVONThink items
Reply #26 - Jan 21st, 2014, 9:51am
 
I'm no coder and don't really understand all this. I have a simple question:

How do I link multiple DT items to a single TB Note? I noticed that keynote files can be dragged onto the "Drop Files and Folder" section out of DT. However, this does not work with pages files. I can create on one URL attribute to paste an item link.
Back to top
 
 

Simon Smailus
WWW   IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Linking into DEVONThink items
Reply #27 - Jan 21st, 2014, 10:58am
 
Pages 'files' aren't actually files - they're folders that the OS shows as files. That might be one problem.

Separately, if you want to have only note links to, for example ,3 discrete URLs then you need to have at least 3 such attributes. TB has $URL configured by default, so you'd need to add 2 URL-type user attributes. Also, although $URL gets a button on the sidebar, to interact with the other two URL attributes you'd ned to show them as key attributes.

You can also make web links from selections of text. So I guess (not tested) you could make a web link from a selection in the note's $Text add paste the local DT pseudo-protocol URL into that.

Bear in mind that whatever you do, these DT URLs only work if TB and the DT database are on the same Mac.
Back to top
 
« Last Edit: Jan 21st, 2014, 12:44pm by Mark Anderson »  

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Ellen Madono
Full Member
*
Offline



Posts: 90

Re: Linking into DEVONThink items
Reply #28 - Jun 11th, 2015, 10:31pm
 
Use of codes is out of my experience.  I have Keyboard Maestro, DevonThinkOfficePro and TBx.  I want to go between DTOP and TBx.  

I am looking at Sumner Gerard's code to bring text into TBx.  I am wondering where to use it.  It is apple script, so I could put in a Keyboard Maestro macro.

I imagine there are multiple approaches, but I am looking for the most handy.  I know how to make KM macros.

Best,
EllenM
Back to top
 
 
  IP Logged
Pages: 1 2 
Send Topic Print