Welcome, Guest. Please Login
Tinderbox
  News:
IMPORTANT MESSAGE! This forum has now been replaced by a new forum at http://forum.eastgate.com and no further posting or member registration is allowed. The forum is still accessible via read-only access for reference purposes. If you wish to discuss content here, please use the new forum. N.B. - posting in the new forum requires a fresh registration in the new forum (sorry - member data can't be ported).
  HomeHelpSearchLogin  
 
Pages: 1
Send Topic Print
Exporting to Aeon Timeline (Read 25087 times)
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Exporting to Aeon Timeline
Jun 1st, 2012, 2:30pm
 
Aeon Timeline, a neat looking timeliner that seems to have emerged form the Scrivener community has at last left beta. Before some one posts asking if it can import TB data the answer is maybe. Basically AT's import filters are a bit fragile. It aims to support CSV or tab-delim but neither (yet) works properly, having spent some time trying today. Still, it's v1.0.0 so I'm sure that will be resolved. If you're tempted to try yourself, don't attempt tab-delim and in either format never include the standard line #1 with the target field names.

When it is, you should be able to export the following:
  • $StartDate -> Event Start Date
  • $EndDate -> Event End Date
  • $Name (or some String) -> Event Title
  • The relevant timeline band label name  -> Arc. IOW $TimelineBandLabels(parent).at($TimelineBand)
  • A colour ($Color?) -> Label
I was going to post some export templates but i'll wait until AT import is more stable. I'll update this in due course.
Back to top
 
« Last Edit: Jun 25th, 2015, 2:38am by Mark Anderson »  

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Exporting to Aeon Timeline
Reply #1 - Jul 24th, 2012, 2:00pm
 
Now that Aeon Timeline seems to be at 1.0.7 or so, am wondering whether it imports better.
Back to top
 
 
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Exporting to Aeon Timeline
Reply #2 - Jul 24th, 2012, 4:54pm
 
Still a little work to do. AT (I used v1.0.8) exports UTF-16, which TB can't understand. Changed to UTF-8, it does import like so:



#1. This is the data in BBEdit. Note all columns (field/attribute) values are quote enclosed. This is perfectly valid for the format but TB doesn't strip them. I've lodged a TB support request that it does as it's the only way TB can import tab delimited values with tabs or line breaks in them.

#2.  The data in the outline. note the effect of TB not stripping the enclosing quotes. However this is quite easily fixed in TB by an agent:

Query: $AttributeName.substr(0,1)== '"' & $AttributeName.substr(-1,1)== '"'
Action: $AttributeName=$AttributeName.substr(1,($AttributeName.size-2))

..noting that you'll need to do this for each attribute brought in. actually as the source attributes never change you could simply write one big action.

#3. The unstripped quotes on field names (row #1 of the import data) and spaces in the names result in some funky field names. I'd suggest that in the source data TXT file you strip both the spaces and quotes from "Start Date" and "End Date" and the data as provided will become valid $StartDate and $EndDate data (TB in this case will ignore the quotes). A further issue is AT's uses commas as list delimiters.  More action code can fix this.  Add 2 sets $Tags and $Participants. then use this code:

$_Tags_ = $_Tags_.replace(",",";");
$_Participants_ = $_Participants_.replace(",",";");
$Tags= $_Tags_.substr(1,($_Tags_.size-2));
$Participants = $_Participants_.substr(1,($_Participants_.size-2));


So, not quite flawless. Neither app is wrong in its assumptions so I guess it's down to users persuading either/both app vendors to make changes. FWIW, I've already passed this data on to AT's author




Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Exporting to Aeon Timeline
Reply #3 - Jul 24th, 2012, 10:20pm
 
Thanks for the illustration. So AT to TB does look workable already, unfortunately with a bit of fiddling. Is TB to AT better now, i.e. do AT's import filters behave more as expected?  A relatively seamless TB<-->AT export/import both ways coupled with AT's sync with Scrivener would seem to open up very interesting possibilities.
Back to top
 
 
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Exporting to Aeon Timeline
Reply #4 - Jul 25th, 2012, 5:20am
 
To test export, I had to finish importing. So we need to add String attributes $MyString, $Arc and $Label and then run this code:

$Text= $_Notes_.substr(1,($_Notes_.size-2));
$Arc= $_Arc_.substr(1,($_Arc_.size-2));
$Label= $_Label_.substr(1,($_Label_.size-2));
$MyString |= $Name.substr(1,($Name.size-2));
$Name = $MyString;


Note the need to pass cleaned $Name data to another attribute via a |=. Otherwise, the $Name would clipped by 2 characters each agent cycle. One could try to use a $Name.replace() but formally TB doesn't allow escaping quotes (e.g. \") and regex patterns in contains/replace calls must be duoble-quote enclosed string (so you can't regex search for double quotes!). I hope this anomaly will go away but likely not in TB v5.x

Now all data is in and clean. Well, not quite as AT will escape double quotes in values by doubling them , i.e. " -> "". This follows the de facto standard for quotes in delimited TXT data values. Interestingly this query works (and finds one note):

$Name.contains('""')

But this action results in $Name being "":

$Name = $Name.replace('""','"')

The curse of the unescapable quote strikes again! I tried the above as I've seen this replacement method work sometimes in action code but not consistently.

Lastly, AT's arcs. AT's arcs are single-value and there are no aliases.  so, if an event is in 3 arcs the event must be replicated twice and each - in theory - could have different other attribute values, such as differing Notes. AT doesn't export its arc list so this needs to manually transcribed. If going the whole hog for replication of AT data you'd manually transcribe the arc list to the events container's $TimelineBandLabels and somehow map $Arc data to $TimelineBand. I've not done the latter, here is AT's demo 'Murder on the Orient Express' (same data as grabp previously) in a TB timeline:



Export to AT follows.
Back to top
 
« Last Edit: Jul 25th, 2012, 5:24am by Mark Anderson »  

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Exporting to Aeon Timeline
Reply #5 - Jul 25th, 2012, 5:56am
 
TB to AT goes fine. AT has an import glitch so it won't import Participants- but it fails using its own tab-delim data so that's an AT bug.**

** Not quite so - see my next post in the thread.

My templates. First, the page/file to export the agent (NB tabs between items and the trailing line return):
Code:
"Event Title"	"Start Date"	"End Date"	"Label"	"Arc"	"Participants"	"Tags"	"Notes"
^children(AT data item)^
 



Per-item (NB tabs between items and the trailing line return):
Code:
"^value($Name)^"	"^value($StartDate.format("y-M0-D h:mm:s"))^"	"^value($EndDate.format("y-M0-D h:mm:s"))^"	"^value($Label)^"	"^value($Arc)^"	"^value($Participants.format(","))^"	"^value($Tags.format(","))^"	"^value($Text)^"
 


Edited:
date format string originally had 'ss' for 's'. It didn't cause failure but was incorrect.

~~~~~~~~~
My take on the process overall.

TB. Needs to deal with quoted values and escaped quotes in value on export. Being generic to the format, if added this will benefit imports
from other sources and has no negative impact (that I can see).

AT. Needs to fix the 'Participants' field import bug - see my next post in the thread.

Between two stools:
  • Mapping AT field names to TB attributes.
  • AT exports comma delimited lists, TB uses semi-colons
  • AT doesn't export a list of Arc names - needed for timeline band/arc assignments
  • Map AT dupes to TB aliases?
  • AT exports UTF-16. TB reads UTF-8.
Only things I can't work around as at AT v1.0.8 / TB v5.11.2 is AT's import bug (we lose one field only) and TB's problems doing things with double-quotes (I didn't even attempt quote-doubling on export). Still at present there's a fair deal of self-assembly (and TB skill) needed to 'just' import At -> TB. TB has the reverse well covered.

Is it worth the vendors doing more to ease this? My hunch is that depends on the numbers of active users of both. These inter-app smoothings all take dev time/budget for other features.

I'll update this if changes occur.
Back to top
 
« Last Edit: Jul 26th, 2012, 6:24am by Mark Anderson »  

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Exporting to Aeon Timeline
Reply #6 - Jul 25th, 2012, 5:30pm
 
Thanks for guidance.

With some fiddling (I removed quotes in the suggested 'AT data item' template so they wouldn't show up in AT, and exported $TimelineBand in the "Arc" column) the export from TB to AT went more smoothly than expected. In AT it turned out to be easy to manually rename the exported $TimelineBand numerical values to the corresponding members of $TimelineBandLabels. I suppose if one were exporting often and had lots of bands, the population of an $Arc attribute in TB could be automated quite easily.

I didn't have the added complications of events in multiple TimelineBands (aliases), participants, etc.  

Can see, as things stand now, how going from AT back to TB could turn into a project in itself.  Maybe that will become easier over time.

Of course the number of active users of both applications is likely to be greater if the the applications play nicely with each other, just as (I assume) is the case with Scrivener. Chicken and egg sort of issue, I suppose.
Back to top
 
« Last Edit: Jul 25th, 2012, 8:54pm by Sumner Gerard »  
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Exporting to Aeon Timeline
Reply #7 - Jul 26th, 2012, 6:02am
 
Further TB -> AT findings re AT's import...

Format. AT is happy to consume TB's UTF-8 file with Classic Mac style line breaks (i.e. carriage return).

Value formats: quoted values. Not sure the issue Sumner had, I find AT strips enclosing quotes just fine. Values containing double-quotes/tabs/line-breaks must be quote-enlcosed.

Value formats: double-quotes in values.  AT import will fail if straight-double quotes in values aren't escaped by doubling:
He said "Hello" to me --> "He said ""Hello"" to me"
..though unfortunately TB struggles to do this during export (in in action code within export code). Helpfully, or not(!), my test case has a note with straight double quotes in the title.

Value formats: list values. Doesn't mind if last comma (such as it exports) is omitted from list. As long as there is a comma between each list value, that's OK.

Value formats: dates. No need to provide the era (e.g. 'AD') tag on the end. Format is 4 digit year/zero-padded month number/zero-padded day number/space/zero-padded 2 digit hours 01-24/colon/zero-padded 2 digit minutes/colon/zero-padded 2 digit seconds. The TB format string for that is "y-M0-D h:mm:s"

Tags. As with other lists, if correctly formatted these are added to note and to the AT files Settings->Tags pane.

Arcs. AT has no way to bulk import/export Arc values. All imported Arc data is set to 'Global' unless the imported arc values are already added to the AT document.  IOW, you need to manually set up all arcs before import. On the AT import pane, under "Create new if there are no existing matches", tick "Arc" and the Arc list will auto-populate. If not ticked, all imported events are assigned to the 'Global' arc. AT emits a dupe event for each arc it is in

Participants ( = 'Entity' within AT). As for arcs - these must be set up first & and theres no way to bulk import a list of Participant values. Otherwise, participant data is blank.  On the AT import pane, under "Create new if there are no existing matches", tick "Entity" and the Entity list will auto-populate with Participant values. AT Participant status (None/Participant/Observer/Birth/Death) is not exportable from AT nor can it be supplied to it via import. Participant colour is not catered for but that's only used in AT anyway and easily altered.

Labels. As for arcs - these must be set up first & and theres no way to bulk import a list of Label values. On the AT import pane, under "Create new if there are no existing matches", tick "Label" and the Label list will auto-populate. Otherwise, all imported events are assigned the default Label.

Note that AT helpfully remembers the last used state for the label/arc/entity tick boxes so if only going between TB and AT you'll only need to set these once. Plus, you'll have a good hunch of the cause if all events end up on the Global arc as these boxes got unpicked for some reason.

AT colours. These aren't exportable or displayable as colour value.  For the detail obsessive, I guess you can dig out your Mac's (Apps/Utilities/) Digital Color Meter so you can read the colour values from AT's UI and apply them in TB.

Duplicate events. To a TB user's eye these may look effectively like aliases. They aren't.  In AT, for an event to appear in more than one arc, the event must be duped. In doing so, the dupe becomes an event in its own right and can have discrete Arc/Label/Tag/Note info (which wouldn't be possible for an alias).

So a round trip is possible, but really only as a proof of concept at this stage! However, if you set up your TB and AT with some pre-staging and have a decent text editing tool to hand (e.g. the free TextWrangler), repeat import/exports are less effort if not exactly painless.

Should the two apps decide to have closer interchange, I've one other thought and that is to emit a separate data file for the doc level info: timeline's name. overall start/end dates, tag/label/participant lists,etc.  This is less for totally automated import but so as to have a concise reference to configuring the 'other' app. Sure full integration would be swell, but practically this little step would make current interchange less painful.
Back to top
 
« Last Edit: Jul 26th, 2012, 1:45pm by Mark Anderson »  

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Exporting to Aeon Timeline
Reply #8 - Jul 26th, 2012, 1:03pm
 
Thanks for the helpful additional details.

Quote:
AT has no way to bulk import/export Arc values. All imported Arc data is set to 'Global' unless the imported arc values are already added to the AT document.

Arcs imported well for me "in bulk" into a blank AT document (with no pre-staging) after I noticed that in AT's 'Import Events' pane I could tick 'Arc' under 'Create new if there are no existing matches:'  After import I found I didn't have to fiddle with individual events. All I had to do was go to the 'Manage Arcs' pane and rename each imported Arc name (I had six Arcs from six TB TimelineBands, with which I had associated several dozen events) from the numerical $TimelineBand value exported by TB to whatever I wanted in English.  The events in AT immediately appeared in the appropriate (and now appropriately named) Arcs.

Supposedly same for Labels and Entities, though I haven't tried those, or Participants.

Quote:
if you set up your TB and AT with some pre-staging and have a decent text editing tool to hand (e.g. the free TextWrangler), repeat import/exports are less effort if not exactly painless

Or if you happen to have a copy of Excel on hand.  I've found Excel lives up to its name when importing tab-delimited or comma-separated files for visual inspection, cleanup and reexport.

Overall, taking a moderately complex timeline from TB into AT is already pretty easy, especially if an export template is provided for TB.  If made a little easier, it would open up some interesting new possibilities for sophisticated analysis by writers and researchers using TB and Scrivener that other tools simply can't match.
Back to top
 
 
  IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Exporting to Aeon Timeline
Reply #9 - Jul 26th, 2012, 1:35pm
 
Ah, I completely missed those settings. Or rather, i tried them early on then forgot completely whilst checking other stuff.  I'll correct my errors up thread. I can see now how your $TimelineBand -> (number) Arc will work nicely.

FWIW, Participant == Entity.  AT uses the latter term internally but exports/imports said data using the former term. I suspect the former got superseded by the latter during the design process but only the the UI labels changed, not the schema beneath.

Agreed that Excel is a good tool. For anyone handy with AppleScript i suspect Numbers might suffice too.

Thanks for the correction - now to go fix those errors up-thread.
Back to top
 
« Last Edit: Jul 26th, 2012, 1:48pm by Mark Anderson »  

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: Exporting to Aeon Timeline
Reply #10 - Jul 26th, 2012, 2:32pm
 
For those not comfy spelunking in Excel, the following query & action work to strip opening/closing straight double quotes:

Query: $Text.contains('"')
Action: $Text = $Text.replace('^"','').replace('"$','')).replace('""','"')

I do realise the docs categorically state a regex pattern for .contains() and .replace() must be enclosed in double-quotes but for now I'll not quibble, especially as neither of these work, unlike those above:

$Text.contains('""','"')
$Text.contains('"{2}','"')
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Sumner Gerard
Full Member
*
Offline



Posts: 359

Re: Exporting to Aeon Timeline
Reply #11 - Jul 26th, 2012, 4:06pm
 
Quote:
FWIW, Participant == Entity.

Ah, thanks for that pointer. I'd seen Entities in a quick look at AT videos but was wondering what a Participant was. Haven't had time to work with Entities in AT but assume they and relationship mode and the ability to display ages at the time of an event may be features currently without (easy) equivalents in a TB timeline.
Back to top
 
 
  IP Logged
Pages: 1
Send Topic Print