Welcome, Guest. Please Login
Tinderbox
  News:
IMPORTANT MESSAGE! This forum has now been replaced by a new forum at http://forum.eastgate.com and no further posting or member registration is allowed. The forum is still accessible via read-only access for reference purposes. If you wish to discuss content here, please use the new forum. N.B. - posting in the new forum requires a fresh registration in the new forum (sorry - member data can't be ported).
  HomeHelpSearchLogin  
 
Pages: 1
Send Topic Print
import attributes from "plain text" (Read 1239 times)
Derek Van Ittersum
Full Member
*
Offline



Posts: 168

import attributes from "plain text"
May 11th, 2015, 11:24pm
 
I'm thinking of doing double-duty with some plain text documents. I'd like them to live in the file system (where they can be managed by a variety of programs) and in TBX (where I can manage them using agents etc.). I'm wondering about the feasibility of "importing" attributes in some fashion.

The files are formatted like so:


Category1: cow
Category2: pig
Category3: sheep

Some text about animals


In other words, a MultiMarkdown metadata block at the top and regular old text after the line break. I'm wondering about how to turn the metadata key/value pairs at the top of the file into attributes in TBX.

Anyone else have experience with this?

I kind of feel like I'm halfway there: create an agent with query:

$Text.contains("Category1:")

But I'm not sure what the action would be. Some kind of regex to get the "value" part of the pair and strip the "Category1" part away?

I'm totally fine with pre-populating the TBX user attributes--I'm not expecting a way to automate their creation (although, I'm happy to hear if there is one).
Back to top
 
 
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: import attributes from "plain text"
Reply #1 - May 12th, 2015, 2:55am
 
Taking the last bit first, attribute auto-creation only occurs with tabular data - i.e. text files with tab-delimited columns with the first row being the (case-sensitive) attribute names. In the latter case, 'Name' should always be column #1 for everything to work correctly. The auto-creation process will generate a String type unless the first data row for than column is a number in which case Number type is used. For things like URL or Boolean I find it pays to make these in advance in which case the data maps correctly.

'Spreadsheet' import also splits each row (after the first) into a record and sets the new attributes as Key Attributes. This works really well so you might at least consider if it's worth re-structuring your external data to fit this import mode - though I suspect that might conflict with the existing Markdown layout.

Thus importing your text files won't generate attributes, for the reason given above.

Are the metadata blocks consistent? IOW, are there always the same number of/named attributes in the first section. If there are, it offers an easier route.

For your queries, if using $Text.contains() always endeavour to scope that call so as few as possible notes are matched before $Textis tested. for instance:

inside("Imported Texts") & $Text.contains("Category1:")

Now only notes that are children of container "Imported Texts" are tested for $Text.contains() rather than every note in the document.

Now for your action to set $Category1, $Category2, and $Category3. I'll assume the three attributes are any single value type (String, Number, etc). There's more than one way to do this but I'll use a var to avoid the need for intermediary attributes to hold in-process values.

var X;
X = $Text.split("\n").at(0).replace("Category1: ","");
$Category1 =  X;
X = $Text.split("\n").at(1).replace("Category2: ","");
$Category2 =  X;
X = $Text.split("\n").at(2).replace("Category3: ","");
$Category3 =  X;


We don't need to clear the var at the end as it only exists for the duration of the action. What is the three entries are actually discrete values for the same multi-value attribute. For example , 3 tag value we want to add to a Set $Tags:

var X;
X = $Text.split("\n").at(0).replace("Category1: ","");
$Tags =  $Tags + X;
X = $Text.split("\n").at(1).replace("Category2: ","");
$Tags =  $Tags + X;
X = $Text.split("\n").at(2).replace("Category3: ","");
$Tags =  $Tags + X;


Edited:
I initally used a var called 'varX' but on testing that confused TB's parser. The var(iable) name can be whatever you like - just be sure to use a value that would have a literal meaning within the rest of your action.
Back to top
 
« Last Edit: May 12th, 2015, 3:05am by Mark Anderson »  

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Derek Van Ittersum
Full Member
*
Offline



Posts: 168

Re: import attributes from "plain text"
Reply #2 - May 12th, 2015, 12:50pm
 
Thanks, as always, for the comprehensive explanation. This looks promising. Will report back after I've a chance to test it in the wild for awhile.
Back to top
 
 
  IP Logged
Pages: 1
Send Topic Print