Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi
Tinderbox Users >> Exporting from Tinderbox >> Need help exporting to BeeDocs Timeline
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi?num=1295194256

Message started by iGregor on Jan 16th, 2011, 11:10am

Title: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 16th, 2011, 11:10am

I have a project with several hundred historical event notes in a history container.  I'd like to export them to BeeDocs TL3D for presentation purposes.  How do I set up a template to export a tab-delimited text file of selected attributes and note text (Name, Start Date, End Date, URL, note text) in a format that TL3D can read (Label    Start Time     End Time    Link     Notes)?

Thanks.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Jan 16th, 2011, 11:44am

Create 2 template notes (tip - use the built-in prototypes). Template 1, "beedocs-page":

Quote:
Label      Start Time      EndTime
^children(/TEMPLATES/beedocs-item)^

Template 2, "beedocs-item":

Quote:
^if(Name(,))^"^title^"^else^^title^^endIf^TAB^if(StartTime!="never")^^get(StartTime,"D/M/Y")^^endIf^TAB^if(EndTime!="never")^^get(EndTime,"D/M/Y")^^endIf^

N.B, include a line return (new line) after the closing ^endIf^. Replace TAB with a Tab character in the template. The first test ensures that is the title has a comma text, the title is quoted otherwise Beedocs malfunctions.

Now, set the container holding your events (as immediate children*) to use template "beedocs-page" and not to export it's children (an option on the HTML view). Open HTML view for the event's container and click the 'Export' button.

* If the events - the notes with dates - have a more complex nested structure, use an agent to find them and export the agent via the method described.

Title: Re: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 16th, 2011, 9:59pm

Thanks Mark, but I'm still pretty clueless about how to properly use almost all of the TBx command expressions.  So bare with me, please, because I'm still stumped what to do here.  I need explicit instructions.  

1) How does using a built in prototype help?  I cannot locate a built-in text prototype.  Are you suggesting the use of the html template?  If so, what to do with the boilerplate expression in the note text field?

2) What does N.B. in your message mean?

3) I couldn't figure out what to do with the prototype templates, so I created two non-html template notes titled BeeDocs-page and BeeDocs-item. But why need two?  

4) Where do I place your Quoted material--directly into the template note field, the expression field, the rule field or other?  

5) How do I generate a Tab character to replace TAB?

6) Once the notes are crafted, then what???

Thanks.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Jan 17th, 2011, 3:48am

OK, lets try again.  

1. After adding the 'HTML Template' prototype, and before using it as a prototype in your document, open the prototype 'HTML Template' note, select all $Text and delete it.  Boilerplate is gone.

2. My 'N.B.', is as it says, to make sure that in your note's copy of the code there is a line return after the last bit of code.  In other words, if you place the cursor at the end of the note, it is on the beginning of the line after the code and not immediately after the last code. This is because when this template spits out the data for each of your events, we want each item in the output text document to be on its own line. Without the extra line return, they'd all be squashed on one line and Beedodcs would not be able to spot where one item stops and the next begins.

3. The two documents hold the code - the prototype you apply to the note ensures it is set up appropriately for the task. You need to set the prototype (e.g. 'HTML Template', as discussed at #1 above) before you add the code to the actual notes; thus the order of the instructions.
  • Set up the necessary prototypes so they are available to use.
  • Add notes that will hold the export code and act as the templates. There are two different templates involved so we need 2 notes.
  • Set both notes to use the 'HTML Template' prototype
  • Copy the relevant code up-thread to the correct template note
  • You now have the templates ready for export use
4. An export template note is a note whose $Text holds code whose only purpose is to supply the 'framework' for all or part of an exported file (HTML, text, XML, whatever format you design). So past the code into the note's $Text. If you've chosen the correct prototype you should see no sidebar and - if copying/pasting from here - use Edit ->Paste and Match Style to maintain the monospace font of the prototype.

5. In your note(s), select the TAB text and click the Tab key. A Tab character will replace the selected text. Ass HTML doesn't reflect hidden characters like Tabs very well and because they're critical to the Beedocs import I figured it better to make sure you know they're there. Beedocs uses Tab-delimited data files for import. It's a general text format you'll likely have used with lots of other apps - whether knowingly or otherwise.

6. Export them, using the method described in my earlier post. One other thing omitted earlier. Open the HTML View of the note you'll use for the actual data export (the one using the beedocs-page template). In the 'HTML Extension' box change '.html' to '.txt'.


Title: Re: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 17th, 2011, 11:18am

Thanks, but still can't get this to work.  Here's what I've done.  It's likely I've misinterpreted something.  Please advise which steps in this list are incorrect/incomplete, and/or what I might have overlooked from your instructions.

1.  file>built-in>html prototype.  opened note and deleted ALL text in text field.
2.  opened the html template note via rename note.  noticed the prototype box was NOT checked.  checked it.
3.  created two new plain regular notes (not yet set to template).  named one beedocs-page and the other beedocs-item.  from pulldown menu set prototype for each note = html template.
4.  copied your text from above and pasted into text field in beedocs-page and beedocs-item respectively.
5.  located two ^tab^ references in text field in beedocs-item.  selected tab and pressed tab key.  NO tab symbol appeared.  instead got two ^^ separated by a tab-length blank space (won't reproduce in this post for some reason).  left it like that.
6. selected the container to export in outline view.
7. opened view>new html view. unchecked export children. changed file extension to .txt.  set export to TBxport folder on desktop.  clicked export.
8. TBxport folder contained a folder named for the selected container's parent.  inside the parent folder was container's exported file named history.txt.
9. opened history.txt.  got a long list of each note within history container with a lot of formating symbols and other stuff.  does not look like format acceptable to beedocs import, at all.  here's the first few lines from history.txt:

<!--   ** Standard Tinderbox Template [plain] **  -->
<h1>HISTORY</h1>

<!--   ** Standard Tinderbox Template [plain] **  -->
<h1>Maddox Reportedly Attacked in Tonkin Gulf</h1>
<p>The captain of the U.S.S. Maddox reports that his vessel has been fired on and that an attack is imminent. Though he later says that no attack took place, six hours after the initial report, a retaliation against North Vietnam is ordered by President Johnson. American jets bomb two naval bases, and destroy a major oil facility. Two U.S. planes are downed in the attack.</p>
<!--   ** Standard Tinderbox Template [plain] **  -->
<h1>Tonkin Gulf Resolution Passed</h1>
<p>The U.S. congress passes the Gulf of Tonkin Resolution, giving President Johnson the power to take whatever actions he sees necessary to defend southeast Asia.</p>


What next???

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Jan 17th, 2011, 11:44am

#5. No select TAB not ^TAB^. Otherwise correct.  You can't see a tab character, because...it's a tab! The tab here is no different to a tab in any word processing/text edit tool. As this is tab-delimited data we're making, we add a tab to tell Beedocs where a new field's data starts/ends.

#9. The specimen output indicates the error is at step #7 - we shouldn't see any HTML code at all. At #7, did you set the view selected export template to you the 'page' template (i.e. the first of the two given above)? TB can't guess which (non-default) template you may want to use, you have to tell it. Thereafter the document will remember.

Title: Re: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 17th, 2011, 12:25pm


Quote:
At #7, did you set the view selected export template to you the 'page' template (i.e. the first of the two given above)?


No. Didn't set it to anything.  Just went to view>new html view, and used what showed up.  How to set proplerly?

Thanks.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Jan 17th, 2011, 12:41pm

So, open the the HTML view, export tab (which is the default tab on opening). Now click the template button (top right). It will show a list of your new template notes and and the bottom the built-in default templates. You're looking for an item called "beedocs-page" and likely listed as a full path name and not just the note name. Click the "beedocs-page" item. Delete the old exported content and try again.

If you're exporting a page using a non-default template choice - as we are here, you must tell TB which template it should use. There are other ways to set this, but for now HTML view is the most user-friendly location/method for doing this.

You might ask, "Why don't I have to set a template for the child notes?". In this case it's not required because the ^children^ code in the 'beedocs-page' template includes the name of the template to be used.  This means you don't have to set it per child note.  It also means those notes can continue to use a different template for normal note-as-file HTML export as might be required.

Title: Re: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 17th, 2011, 3:14pm

OK.  I went back to view>new html view.  clicked the template drop down to select /beedocs-page from the menu.  clicked export.  got the following [column headings only.  no data in any of the columns]:


Label      Start Time      EndTime



Next?




Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Jan 17th, 2011, 3:51pm

Do you have a template at path "/TEMPLATES/beedocs-item"? (as at code example further above). That is the template being looked for to export the child data.

(I'm stepping through this slowly as this way there's a chance to learn from mistakes.)

Title: Re: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 17th, 2011, 4:00pm

yes.  both templates for beedocs are on the list in the export tab.  one is titled  /Templates/beedocs-page and the other  /Templates/beedocs-item.  the template that I have set for export is the former.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Jan 17th, 2011, 5:03pm

Are they? You cite a different path to the one I mentioned - they are cases sensitive. The path is case sensitive so "/Templates/beedocs-item" is not the same as "/TEMPLATES/beedocs-item". If you're using the wrong path in beedocs-page to call beedocs-item, TB won't find he note holding the the template.

Title: Re: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 17th, 2011, 8:44pm

Well, Mark, I surely can't tell you where it comes from, or why it says what it says.  All I can do is report what the screen shows.  And, sorry, but it shows Templates, not TEMPLATES.  I tried to attach a screenshot below, but can't get this board's image insert to work either.

Jeez, this is fun.  What next?




Title: Re: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 17th, 2011, 8:55pm

OK.  I went back to check that I copied the text as you quoted it into each note.  Done.  And the beedocs-page note has the word TEMPLATES in caps.

I then noticed that I did not capitalize the name of the container with the two template notes (Templates).  Changed it to TEMPLATES.

Then tried to export.  Template pull down in view>new html view now reads  /TEMPLATES/beedocs-page.  But same result as before.  History.txt only shows Label   Start Time    End Time   with NO data in any column.

Title: Re: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 17th, 2011, 10:50pm

Just tried it again.  Did not change a thing.  Closed and opened TBx.  This time I got some data in the Label/Name column, but still NO Start/End times (several lines below)

Label      Start Time      End Time
Maddox Reportedly Attacked in Tonkin Gulf            
Tonkin Gulf Resolution Passed            
China Tests A-Bomb            
Rolling Thunder            
"LBJ boosts troop levels to 60,000"


Why no Start/End times?  

Also, notice the quotes around LBJ boosts troop levels...  The actual note name has NO quotes.  Several other note labels in the exported list have quotes, while the actual note names do NOT.  What gives?

And, in the view>new html view window the column headings (Label, Start Time, End Time) are black while all of the note labels appear red.  Why?




Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Bernstein on Jan 17th, 2011, 11:32pm

The text colors in HTML view help distinguish the template from thisnote, and from materialin included or childnotes.

Can you send a copy of your document to info@Eastgate.com?  We can forward to markA, and perhaps that's the easiest way to see the sticking point.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Jan 18th, 2011, 8:07am

The missing dates are most likely because you''re not storing your dates in the same attributes as the template uses. The template is looking for start dates in a StartTime attribute and end dates in an EndTime attribute. I'd assumed you'd use the latter as you didn't specify any particular attribute names at outset. The actual attribute names themselves don't matter as long as you use the same for both source data and the template. My hunch is you're using different attributes - possibly $StartDate and $EndDate, because they're already there? If so, simply change either the attribute names used in the beedocs-item template or use date attributes for your TB data that are the same as those used in the template.

I've emailed you a functional demo file. I think you may find it quicker/easier to get to a working result. In checking the demo, I notice the Beedocs notes had changed from a few years back when the demo (on which my notes above were based) has changed slightly.  As I had the hood up I also re-worked the export code to avoid some working but now deprecated usage. Note: my Beedocs demo has expired so I can't test the actual input (it worked back when first tried some while back!).

Title: Re: Need help exporting to BeeDocs Timeline
Post by iGregor on Jan 18th, 2011, 8:43am


Quote:
My hunch is you're using different attributes - possibly $StartDate and $EndDate, because they're already there?


Bingo -- at least for this step (see below)!  I changed the body of the beedocs-item note to:

^if(Name(,))^"^title^"^else^^title^^endIf^TAB^if(StartDate!="never")^^get(StartDate,"D/M/Y")^^endIf^TAB^if(EndDate!="never")^^get(EndDate,"D/M/Y")^^endIf^

Upon export I now get a file with the table of data expected.  :)  

Unfortunately this file will NOT import into BeeDocs (ugh!).  And I'm overextended on time available to devote to learning to fit TBx into my workflow.  So, will commence manual transfer of data out of TBx (copy & paste), today, and put further work in TBx on hold indefinitely.  :(

Thanks for your generous attention and support.  

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Jan 18th, 2011, 8:55am

No problem - do let me know if the file I send (via email) works or not; the output to text should work but I'm now less certain about the Beedocs end. I don't have Beedocs with which to test (I've a system full of expired demos as it is!). However, this moring I have emailed the app's author pointing out the lack of clarity in his import FAQ page. There's no indication as to the d/m date order required and bizzarely- all examples are for '1/1' (I Jan) so don't help. Also, it seems that any text field (Label from $Name, Notes from $Text) must be double-quote enclosed if it contains a comma. Implicit in that is that a double-quote in the field (attribute) value is either not allowed or must somehow be escaped (the doc doesn't mention how to escape anything).

Anyway, Beedocs lack of text sophistication isn't TB's fault; in fairness I think iBeedocs is more for screen eye candy than something trying to support data heavy-lift (it's very good at what it does do).  If Beedocs can correctly - unambiguously - define its format it should be possible to import data to it - I was certainly able to test data when I originally looked at this in 2009. If you use Beedocs a lot, it might be worth dropping a line to them pointing out the problems their lack of documentation is causing you.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Bernstein on Jan 18th, 2011, 12:45pm

It is often very helpful, when exporting to a program with complex data needs, to hand-code an example file and import it into that program. This both confirms that you understand its input requirements and gives you a base template which then becomes your export target.

Then, set up a Tinderbox experiment to export a replica of that hand-coded test file.  This can usually be done in a minute or two, and that, too, builds confidence.

Finally, try the "real" data. This might work first time. If it doesn't, you might receive partial data or an error message to help locate the discrepancy. If the failure is silent, edit the exported data and try importing just the first few lines.  Does that work? If so, divide and conquer will quickly locate the problem.

Here, the difficulty may well lie (as Mark Anderson speculated) in escaping quote marks, commas, or perhaps in character encoding.  Wikipedia appears to have the answer to the problem of escaping quotation marks, incidentally.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Jan 18th, 2011, 1:20pm

Re Mark's wikipedia link, where the solution seems to be the old-school double-the-quotes trick (takes me back to Visual Basic - remembrance of the Dark Side <shiver>), the sort of method below might work. An agent might:

Query:   $Text.contains('\"')
Action:   $BeedocsText=$Text.replace('\"','\"\"')

Then export something like this for the export file's "Notes" field:

  ^if($BeedocsText)^^value($BeedocsText)^^else^^text(plain)^^endIf^

... assuming of course that inline double quotes are (one of) the issue(s) Beedocs is having.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Alex Strick van Linschoten on Aug 18th, 2011, 3:12am

Any idea if the Beedocs export (as part of the Simile timeline template) is no longer working? I haven't used for a while, but some strange events seem to be created...

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Aug 18th, 2011, 3:22am

Seems unlikely. In the scenario cited, I'd look at the data that's mis-translating in BBEdit and see if you spot anything odd.  I know you work with quite complex text (Roman/Cyrillc/Arabic text, etc.) so there's scope for odd construct creeping in.

In fairness to TB, the issue is more likely weaknesses in Beedocs data import abilities than TB's export.  TB ends up in the frame over it being used to manipulate export form to work around BeeDocs limitations - presumably because the latter can't/won't improve their import (the fact it's a rather hidden feature is somewhat telling). If you haven't asked them, it might be worth it. If BeeDocs had more capable import you might not need edge case export template stuff in TB. After all, it's 'just' text we're passing!

Drop me a line direct if needs be - it might help to see the 'bad' data.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Aug 18th, 2011, 6:24am

[Later after chat off-forum with Alex]

BeeDocs has moved along a bit but their FAQ page on import is a bit vague. So far we've figured that:
  • BeeDocs uses the user's current OS locale for dates. Thus for Alex & I, using UK locale, the dates want to export like ^value($StartDate.format("D/M0/y"))^ rather than in month/day order.
  • Text file import is only possible via drag/drop - into the app and not a current project. This is not necessarily self-evident, and bypasses some BeeDocs project set-up steps that my subsequently not be possible to apply.
  • It is not clear if text import is restricted to ASCII, basic ASCII or is Unicode capable (or at least UTF-8 capable).
  • Line breaks (in $Text data and such) definitely break BeeDocs import
Alex (as a BeeDocs user) is going to follow up with BeeDocs support and see what they say and will add anything of use here if it turns up.

Title: Re: Need help exporting to BeeDocs Timeline
Post by Mark Anderson on Aug 18th, 2011, 8:46am

With the help of good old BBEdit (or use the free sibling app TextWrangler) I've figured the conundrum.

BeeDocs uses tab-delimited import. the only valid delimiter for field start end is a Tab Character.  Any quotes single/double/curly/straight are treated as literal text characters. Any line return is a record (line) delimiter. So, $Text, or any attribute with line breaks needs to encode its line (paragraph) breaks. A literal '\n' or '\r' in a string is read by BeeDocs as a line break. From experiment, for each TB paragraph break, you want 2 in BeeDocs.  You do this via:

^value($Text.replace("\n","\\n\\n"))^

BeeDocs will also accept '\t' and insert a tab stop, e.g. for an indented paragraph start or indented list items.

It appears Beedocs will accept any valid unicode character (in UTF-8 encoding as output via TB). I had no problem with accented letters and various characters off the Mac character viewer. For non-Roman alphabets (Cyrillic, Arabic, etc.) you might need to test but apart from left-right vs right-left display issue I think you should be fine.

Alex's ex-TB data looks sweet in BeeDocs' 3D display on a large monitor!

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.