Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
Tinderbox Users >> Agent, Actions, Rules & Automation >> import attributes from "plain text"

Message started by Derek Van Ittersum on May 11th, 2015, 11:24pm

Title: import attributes from "plain text"
Post by Derek Van Ittersum on May 11th, 2015, 11:24pm

I'm thinking of doing double-duty with some plain text documents. I'd like them to live in the file system (where they can be managed by a variety of programs) and in TBX (where I can manage them using agents etc.). I'm wondering about the feasibility of "importing" attributes in some fashion.

The files are formatted like so:

Category1: cow
Category2: pig
Category3: sheep

Some text about animals

In other words, a MultiMarkdown metadata block at the top and regular old text after the line break. I'm wondering about how to turn the metadata key/value pairs at the top of the file into attributes in TBX.

Anyone else have experience with this?

I kind of feel like I'm halfway there: create an agent with query:


But I'm not sure what the action would be. Some kind of regex to get the "value" part of the pair and strip the "Category1" part away?

I'm totally fine with pre-populating the TBX user attributes--I'm not expecting a way to automate their creation (although, I'm happy to hear if there is one).

Title: Re: import attributes from "plain text"
Post by Mark Anderson on May 12th, 2015, 2:55am

Taking the last bit first, attribute auto-creation only occurs with tabular data - i.e. text files with tab-delimited columns with the first row being the (case-sensitive) attribute names. In the latter case, 'Name' should always be column #1 for everything to work correctly. The auto-creation process will generate a String type unless the first data row for than column is a number in which case Number type is used. For things like URL or Boolean I find it pays to make these in advance in which case the data maps correctly.

'Spreadsheet' import also splits each row (after the first) into a record and sets the new attributes as Key Attributes. This works really well so you might at least consider if it's worth re-structuring your external data to fit this import mode - though I suspect that might conflict with the existing Markdown layout.

Thus importing your text files won't generate attributes, for the reason given above.

Are the metadata blocks consistent? IOW, are there always the same number of/named attributes in the first section. If there are, it offers an easier route.

For your queries, if using $Text.contains() always endeavour to scope that call so as few as possible notes are matched before $Textis tested. for instance:

inside("Imported Texts") & $Text.contains("Category1:")

Now only notes that are children of container "Imported Texts" are tested for $Text.contains() rather than every note in the document.

Now for your action to set $Category1, $Category2, and $Category3. I'll assume the three attributes are any single value type (String, Number, etc). There's more than one way to do this but I'll use a var to avoid the need for intermediary attributes to hold in-process values.

var X;
X = $Text.split("\n").at(0).replace("Category1: ","");
$Category1 =  X;
X = $Text.split("\n").at(1).replace("Category2: ","");
$Category2 =  X;
X = $Text.split("\n").at(2).replace("Category3: ","");
$Category3 =  X;

We don't need to clear the var at the end as it only exists for the duration of the action. What is the three entries are actually discrete values for the same multi-value attribute. For example , 3 tag value we want to add to a Set $Tags:

var X;
X = $Text.split("\n").at(0).replace("Category1: ","");
$Tags =  $Tags + X;
X = $Text.split("\n").at(1).replace("Category2: ","");
$Tags =  $Tags + X;
X = $Text.split("\n").at(2).replace("Category3: ","");
$Tags =  $Tags + X;

[edit]I initally used a var called 'varX' but on testing that confused TB's parser. The var(iable) name can be whatever you like - just be sure to use a value that would have a literal meaning within the rest of your action.[/edit]

Title: Re: import attributes from "plain text"
Post by Derek Van Ittersum on May 12th, 2015, 12:50pm

Thanks, as always, for the comprehensive explanation. This looks promising. Will report back after I've a chance to test it in the wild for awhile.

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.