Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi
Tinderbox Users >> Agent, Actions, Rules & Automation >> Similar To
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi?num=1466867220

Message started by jtaekman on Jun 25th, 2016, 11:07am

Title: Similar To
Post by jtaekman on Jun 25th, 2016, 11:07am

I am a relative novice user of Tinderbox, but have become increasingly interested in its use in my academic writing.

I read lots of manuscripts and extract individual highlighted passages from each article I read. I currently use Devonthink's "AI" to find similar passages to the one I am reading. I use this feature extensively while writing grants and manuscripts. I have thousands of manuscripts and hundreds of thousands of extracted notes.

While developing a simple agent in Tinderbox, I stumbled across the "Similar To" selection in the GetInfo area of a note. I explored this and the suggested notes seem relevant, but Tinderbox only returns the title of each relevant note, not the embedded text.

Two questions:
1. Is there a way to automate viewing of the text of the note (rather than just the title)?
2. How would the ninjas of the forum suggest I set up an agent or adornment for this? I'm concerned about grinding my computer to a halt given the large number of notes that exist in my library. Is it possible to do this on demand only?

Thank you!
Jeff Taekman

Title: Re: Similar To
Post by Mark Bernstein on Jun 25th, 2016, 4:14pm

Obviously, similarTo() is computationally costly for large documents, though Tinderbox does a fair amount of indexing to speed it along.  

I'd try it and see how it goes and the scale you require.

Title: Re: Similar To
Post by jtaekman on Jun 26th, 2016, 12:14pm

Thanks. When I say I'm a novice, I really mean it. I've only just discovered the power of text searches within Agents. I have LOTS to learn.

i have no idea how to set something like this up.  Typically, when writing, I select similar notes and import them into Scrivener. In order to be able to do this, I'm interested in reading the text of the selected text in Tinderbox as well as that of the SimilarTo notes.

Intuitively, I believe this could save me a trip to Devonthink, but can't figure out how to do this given my newbie skills.

Title: Re: Similar To
Post by Mark Anderson on Jun 26th, 2016, 1:33pm

The action code for testing similarTo() is described here.

Agent approach. Assuming you want to look for notes similar to "Brian Griffin" you'd make the agent query:

similarTo("Brian Griffin",10)

The agent will create aliased of the 10 most similar notes.  By selecting any of these aliases you'll see their text in the right page.

However, that method doesn't scale easily as to test "Stewie Griffin" you'd need to edit the agent query.  A way to ease that is to first set the the agent to show the built-in attributes $MyString  and $MyNumber as key attributes. Now, make he agent's query:

similarTo($MyString(agent),$MyNumber(agent)

The query uses the note name you set in the agent $MyString (using the agent's key attributes table) and the number of matches to the agent's $MyNumber value. Thus you can easily re-focus the agent. But, as Mark B notes, do experiment and be mindful of signs of excessive query activity (e.g. by having lots of agents and a high similar count number).

A point re the get Info 'similar' tab.  If you tear-off the pop-up by dragging it to a stand-along window it will remain focused on matches of the source note regardless of how you change the selection in the main window (and, if needs be you could have several tear-offs open). Do experiment as there's no 'right' way other than the one that best suits your work style and task at hand.

Title: Re: Similar To
Post by jtaekman on Jul 17th, 2016, 9:04am

Thanks...I've been trying to figure out how to do this given your guidance. Rather than do this on every note, is there a way to do this only on selected notes? Is there a way to open the referenced note directly rather than having to navigate to the outline to find it?

Here is what I'm trying to do:
1. When I read a manuscript, I extract individual pertinent passages. Each of these becomes its own text file.
2. When I'm writing, I choose one of these text files to support an argument
3 I would like to be able to see AND select all the other relevant passages similar to the one I choose. I currently use Devonthink to do this....but am discovering the power of Tinderbox and suspect , using TB, I can completely forego Devonthink at some point.

Title: Re: Similar To
Post by Mark Bernstein on Jul 17th, 2016, 11:20am

I'm not sure that forgoing the use of DEVONthink is a particularly important goal.  Tinderbox and DEVONthink do quite different things, after all; DEVONthink is a place to store masses of stuff, while Tinderbox is meant for organizing and analyzing.

The current design of similarTo is chiefly interested in locating potential links.  We'll take a look at providing more information about the proposed destination, but we're not working off summaries or deep semantic analysis. It's just tf-idf!  Still, it's remarkably powerful in the right context.

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.