Welcome, Guest. Please Login
Tinderbox
  News:
IMPORTANT MESSAGE! This forum has now been replaced by a new forum at http://forum.eastgate.com and no further posting or member registration is allowed. The forum is still accessible via read-only access for reference purposes. If you wish to discuss content here, please use the new forum. N.B. - posting in the new forum requires a fresh registration in the new forum (sorry - member data can't be ported).
  HomeHelpSearchLogin  
 
Pages: 1 2 
Send Topic Print
Mass-creating agents for unique values of a set (Read 18333 times)
Les Orchard
Full Member
*
Offline



Posts: 29
Livonia, MI
Mass-creating agents for unique values of a set
Jun 15th, 2010, 12:20am
 
Consider an attribute $Tags and lots of notes with various permutations of values as lightweight categorization. I wanted to easily and repeatedly build a set of searches for tags collecting notes by unique tag values.

Well, it seems like I've got something working, though it's very hacky:

http://decafbad.com/2010/06/set-agent-generator.tbx

I've wanted this off and on for a few years, but have never really seen a solution until now. In fact, this demo combines a slew of Tinderbox features that I've only just started understanding in the past month or so.

If anyone can point me to a better way to do this, I'd be happy. But, I'd also be just as happy if anyone finds this puzzle solution handy.

Back to top
 
« Last Edit: Jun 15th, 2010, 12:29am by Les Orchard »  

--
l.m.orchard@pobox.com
{web,mad,computer} scientist
http://decafbad.com
WWW deus_x deusx23   IP Logged
Charles Turner
Full Member
*
Offline



Posts: 180
New York, USA
Re: Mass-creating agents for unique values of a set
Reply #1 - Jun 15th, 2010, 7:10am
 
Nicely done, Les!

So when's the Wrox "Programming Tinderbox" book going to come out?

Best wishes, Charles
Back to top
 
 
WWW   IP Logged
Les Orchard
Full Member
*
Offline



Posts: 29
Livonia, MI
Re: Mass-creating agents for unique values of a set
Reply #2 - Jun 15th, 2010, 8:43am
 
Quote:
So when's the Wrox "Programming Tinderbox" book going to come out?


Heh, not any time soon. But, if I come up with another dozen or so hacks like this, I might just self-publish something. It's been awhile since I had a book in progress, so I might get the itch again. The few I've done so far all started in Tinderbox, so it's only fitting. Smiley
Back to top
 
« Last Edit: Jun 15th, 2010, 8:44am by Les Orchard »  

--
l.m.orchard@pobox.com
{web,mad,computer} scientist
http://decafbad.com
WWW deus_x deusx23   IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Mass-creating agents for unique values of a set
Reply #3 - Jun 15th, 2010, 8:43am
 
Les, I enjoyed this. As there's no explode() action it's hard to automate this particular route further. I'd looked at self-configuring agents a while back and it's good to see this moved forward.

To my surprise, I still haven't figured where the 'delay' referred to is set - all agents appears to be set to normal priorities. In due course a logic walk-through of the explode process & agent generation would probably help more users get what this is about.  That's not critique - I know documentation for the general reader is more difficult to write than most suppose. Thanks again for this neat demo.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Les Orchard
Full Member
*
Offline



Posts: 29
Livonia, MI
Re: Mass-creating agents for unique values of a set
Reply #4 - Jun 15th, 2010, 9:05am
 
Quote:
To my surprise, I still haven't figured where the 'delay' referred to is set


The overall process is:

* The container note's Rule continually collects unique $Tags set values in $Text, one per line.

* Performing Explode on the container note produces an Exploded Text child, and the container note's OnAdd applies the *tagSearchExplosion prototype to it.

* The children of Exploded Text are all given the prototype *tagSearchTmpFolder, thanks to the OnAdd of *tagSearchExplosion.

* Each *tagSearchTmpFolder comes along with an agent of prototype *tagSearchAgent, thanks to $PrototypeBequeathsChildren

* Once it sees that its parent is not a prototype, the *tagSearchAgent Rule assigns an AgentQuery based on its parent's name (the set value); renames the agent after its parent; moves the agent up to its great-grandparent (the parent of "Exploded Text"); and finally self-cancels.

* The Rule for *tagSearchExplosion replaces itself with a subsequent self-cancelling Rule to flag itself for the trash.

* An agent elsewhere moves flagged notes to the trash.

* By this point, all the *tagSearchAgent agents should have found their great-grandparent and escaped the exploded structure before it gets tossed away.

Lots of Rube-Goldbergian hacks here, but I think the worst part of this thing is the Rule-that-sets-a-Rule delay in "Exploded Text" sending itself to the trash. It seems to work reliably, and I think I understand why (ie. multiple Rules across many update cycles), but it's a bit of smoke and mirrors.

The other thing buried in there is the trash mechanism, which I've found handy elsewhere:

* An agent moves notes with $IsTrashed=true to a trash folder, preserving the original location in $ContainerBeforeTrash.

* Setting $IsTrashed=false causes another agent to move the note back to that container.

* Stamps to set $IsTrashed work as handy menu commands to trash/restore sets of notes.
Back to top
 
« Last Edit: Jun 15th, 2010, 9:18am by Les Orchard »  

--
l.m.orchard@pobox.com
{web,mad,computer} scientist
http://decafbad.com
WWW deus_x deusx23   IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Mass-creating agents for unique values of a set
Reply #5 - Jun 15th, 2010, 11:03am
 
Excellent, I saw all this in the code but having written down sure helps.  The set syntax shortcomings, stumbled over here are another factor of which to beware. In summary, syntax you'd intuit from use with other attribute types fails (silently) when used with sets. In the examples below blue = valid and red is invalid syntax:

To first set, or completely replace, a set's value:
  $MySet = "dog"
  $MySet = $MyString
  $MySet = $MySet(other note)


A different syntax is used to add/subtract/toggle a set value reuiring the set to be stated both sides of the assignment:

  $MySet = $MySet + "dog"  (add)
  $MySet = $MySet - "dog"  (remove)
  $MySet = $MySet ~ "dog"  (add or remove if already present)
...but, hazards await:
  $MySet = $MySet + $MyString
  $MySet = $MySet + eval($MyString)

For a a $MyString value of "dog"

  if($MySet==$MyString){...

...is true if $MySet is the single value "dog", but false if any other values are present, e.g. "dog;cat". But even where this is true:

  if($MySet==$MyString){...
...this is always false...
  if($MySet==$MyString(some note){...

IOW, equality tests on sets only return false where the entire set's string equals the other side - as opposed to completely matching a single value. So, testing sets in if() conditions using attribute-stored values is very difficult. It is slightly easier using an agent and the AttributeName(pattern) operator but even then we have to precompile the query. So:

AgentQuery:  MySet($MyString)
AgentQuery:  MySet(eval($MyString))
...but...
Agent's Rule:   $AgentQuery="MySet("+$MyString+")"

Still the need for any given syntax isn't necessarily obvious until someone actually needs it  Wink
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Jody Foo
Full Member
*
Offline



Posts: 60
Linkoping, Sweden
Re: Mass-creating agents for unique values of a set
Reply #6 - Mar 15th, 2011, 11:42am
 
Thanks for posting this!

When I tried it however, the result was that one agent was created for each instance of the tag used.

I suspect that this is due to some change in Tinderbox between your post and my trial (5.8.0).

I had to change the $Rule of *tagSearchFolder from

if (!$IsPrototype) {$Text=format(collect(all,$Tags), "\n") }

to the following:

if (!$IsPrototype) { $tmpSet = collect(all,$Tags); $Text=$tmpSet.format("\n") }

The difference is that I added a $tmpSet variable to the Tinderbox of the type Set. I am guessing that collect(<Set>) no longer outputs a <Set>, but a <List>. By directing the output to a Set, duplicates are eliminated.

I also changed the formatting syntax from format() to <Set>.format(). What syntax is preferred?
Back to top
 
« Last Edit: Mar 15th, 2011, 11:43am by Jody Foo »  
WWW   IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Mass-creating agents for unique values of a set
Reply #7 - Mar 15th, 2011, 12:15pm
 
collect() was switched to a List from a Set in v5.6.0 (see more). As at v5.8.0, the aTbRef note is correct but TB Help lists collect() as returning set data; I've submitted a change for the latter.

Your solution is quite the right one. To get a de-duped list pass the List-type output to a Set-type attribute. With aSset you can use either the old style format() or the new Set.format() dot operator. At present both have the same effect.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
J Fallows
Full Member
*
Offline



Posts: 418

Re: Mass-creating agents for unique values of a set
Reply #8 - Mar 22nd, 2011, 3:54am
 
I am still learning the innards of TB, but I have a basic question. I collected this (useful) file for the "Set Agent Generator," and then I made the change mentioned above, to avoid having duplicate agents appear for each occurrence of a tag. (Ie, to go from "list"-like to "set"-like behavior.) But even after that change, exactly copied from the screen here, I still get a long set of duplicate results. Behavior is the same before and after the change. What might I be missing here? Am using the latest release of Tbox.   Thanks  J Fallows / Beijing 

For reference, this is the change previously mentioned, which I made:
>>I had to change the $Rule of *tagSearchFolder from

if (!$IsPrototype) {$Text=format(collect(all,$Tags), "\n") }

to the following:

if (!$IsPrototype) { $tmpSet = collect(all,$Tags); $Text=$tmpSet.format("\n") }      <<
Back to top
 
 
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Mass-creating agents for unique values of a set
Reply #9 - Mar 22nd, 2011, 4:45am
 
OK, I downloaded Les' files, added a Set attribute $tmpSet and a note with this rule:

$tmpSet = collect(all,$Tags); $Text=$tmpSet.format("\n")

...and the result is a de-duped list, one item per line.

A Set will de-dupe a list. So if the output of $tmpSet has dupes, your attribute may not be the correct data type.  When creating a new attribute, it is very easy to type the name and forget to set the data type pop-up; I regularly trip up on this especially when my attribute name includes the intended data type (I must intuit TB will guess the needed type <g>).  So, I'd open User Attributes (Cmd+2) and confirm your $tmpSet is a Set and not the default String type.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
J Fallows
Full Member
*
Offline



Posts: 418

Re: Mass-creating agents for unique values of a set
Reply #10 - Mar 23rd, 2011, 2:17am
 
Thank you. I figured out how to apply these tips and feel as if I have taken another step forward along the learning curve. Appreciate it jf.
Back to top
 
 
  IP Logged
Marcos Ramos
Full Member
*
Offline



Posts: 1

Re: Mass-creating agents for unique values of a set
Reply #11 - Oct 16th, 2014, 8:15am
 
The overall process is:

* The container note's Rule continually collects unique $Tags set values in $Text, one per line.

* Performing Explode on the container note produces an Exploded Text child, and the container note's OnAdd applies the *tagSearchExplosion prototype to it.

* The children of Exploded Text are all given the prototype *tagSearchTmpFolder, thanks to the OnAdd of *tagSearchExplosion.

* Each *tagSearchTmpFolder comes along with an agent of prototype *tagSearchAgent, thanks to $PrototypeBequeathsChildren

* Once it sees that its parent is not a prototype, the *tagSearchAgent Rule assigns an AgentQuery based on its parent's name (the set value); renames the agent after its parent; moves the agent up to its great-grandparent (the parent of "Exploded Text"); and finally self-cancels.

* The Rule for *tagSearchExplosion replaces itself with a subsequent self-cancelling Rule to flag itself for the trash.

* An agent elsewhere moves flagged notes to the trash.

* By this point, all the *tagSearchAgent agents should have found their great-grandparent and escaped the exploded structure before it gets tossed away.

Lots of Rube-Goldbergian hacks here, but I think the worst part of this thing is the Rule-that-sets-a-Rule delay in "Exploded Text" sending itself to the trash. It seems to work reliably, and I think I understand why (ie. multiple Rules across many update cycles), but it's a bit of smoke and mirrors.

The other thing buried in there is the trash mechanism, which I've found handy elsewhere:

* An agent moves notes with $IsTrashed=true to a trash folder, preserving the original location in $ContainerBeforeTrash.

* Setting $IsTrashed=false causes another agent to move the note back to that container.

* Stamps to set $IsTrashed work as handy menu commands to trash/restore sets of notes.

Awesome you shared out the entire process.. I was bit confused but you sort out my confusion and I am able to get the menu now..
Back to top
 
 
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Mass-creating agents for unique values of a set
Reply #12 - Oct 16th, 2014, 9:06am
 
Some follow-up points.

Collect/collect_it changed from returning sets to returning lists in v5.6.0, released 15 Sep 2010. If you want a set back from collect, just chain .unique to the source list or pass output to a set:

$MyList = collect(children,$SomeAttr).unique;  (Result is list, but no dupe values)
$MySet = collect(children,$SomeAttr);  (Passing a list to a set creates a set, inherently de-duping input)

If you'll explode regularly you may find it useful to use 2 cascading prototypes to set the exploded items' prototype. However, if you just want the latter to receive a protoype in a one-off explode, set the $OnAdd of the to-be-exploded note to:

$OnAdd='$Prototype="pSomeProto";';

Note the single quotes wrapping the whole OnAdd string to be placed in Exploded notes.  As the inner string uses double quotes, we use single quotes for the outer pair. You can nest quotes either way around - as long as the nesting is correct.

If the idea is for each exploded note to find values of itself and hold a reference to it, we don't need agents, but can use find(). So the prototype for exploded notes might have a rule:

$ListOfPathsToNotesUsingThisTag = find($Tags.contains($Name(that)));

Of course, it's likely the find() will have extra query terms to limit search scope (always good for performance). Also, rather than than just get the paths to matching notes you could perhaps store their titles:

$SetOfNames = collect(find($Tags.contains($Name(that))),$Name)

Or link back to each one:

linkTo(find($Tags.contains($Name(that))),"uses tag")

If you do use a custom linkType name (e.g. above) as at v6.0.4 you must define the link type in the TBX before using the link type name in code like that above.

Consider performance (important!). Don't simply set all this code up and leave it running continuously. All this back-referencing via .contains()** pushes a large document quite hard.  Design your code such that you run the code once, or as required. That way you get all the upside of action code automation but don't wonder why the docs running a bit slow.

** Operators like .contains() use a log of regex searching and this is an inherent overhead. Use when needed except in trivial size documents.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
J Fallows
Full Member
*
Offline



Posts: 418

Re: Mass-creating agents for unique values of a set
Reply #13 - Oct 16th, 2014, 9:27am
 
Maybe I am missing something here, but I believe that in most cases, the need to mass-create agents has been overtaken/obviated by the new Attribute Browser feature in TB6.

That is, my purpose in joining this discussion several years ago was to be sure that if I created a new value for an attribute, I would know that a new agent would be created to collect all notes with that value. That is what the Attribute Browser now does.

Not sure if that covers the exact use case being discussed here. But for the basic task of making sure that notes will be grouped according to all values of a certain attribute, that now happens automatically.
Back to top
 
« Last Edit: Oct 16th, 2014, 9:58am by J Fallows »  
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Mass-creating agents for unique values of a set
Reply #14 - Oct 16th, 2014, 12:28pm
 
@JF, yes probably so for most strictly internal UI based stuff, like just looking at the value split in an attribute. But for some more action code/network work and in bigger docs, I'm less sure. But this is edge case stuff - I'm doing heavy lift about 30 attributes and some have c.500 discrete values which make the app chuff a bit (it's having to work hard!). From a quick test, I think the above is probably easier/more stable for *big* data sets than Attribute Browser which is a really cool feature and which you're quite right to mention.

Also having an actual note per value in an attribute (AB view just reorganises existing notes based on an attributes values  - nicely, too!) then allows you to add notes about that value.  So, in my discourse analysis, each subject data post records $UserName (the poster's ID), the topics record a $UserNameSet of all posters in that thread and the root container's $UserNameSet aggregates a set off all discrete user names. I then set another note to read that set, sort it, and split it into value per line .format("\n") and pop it in the note's $Text. Explode, and now I have a note for each user and store info about them. I can't o the latter in Attribute Browser. So, it's a case of what you're trying to do. For just viewing the allocation of the values of $SomeAttribute across all/some existing notes, the Attribute Browser is the way to go. I hope that helps clarify the choices use cases. If anyone's confused, do ask, as finding out the right approach from several possibilities can be a boon to making progress with your work.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Pages: 1 2 
Send Topic Print