Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi
Tinderbox Users >> Questions and Answers >> Tinderbox to Omnioutliner to Scrivener (OPML)
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi?num=1312035655

Message started by John Kendrick on Jul 30th, 2011, 10:20am

Title: Tinderbox to Omnioutliner to Scrivener (OPML)
Post by John Kendrick on Jul 30th, 2011, 10:20am


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Admin note: Thread resolved and locked. After a trip around the houses, a working OPML export example is here.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I would like to be able to move material from Tinderbox to Scrivener, without writing scripts. So I've been reading both forums to see whether it can be done and, if so, how. Here's my understanding at the moment, after trying several illustrations. If this is not accurate, please let me know.

1. Tinderbox outline formats can be moved from TBX to Omnioutliner via copy and paste. Just the headlines; not the contents of the containers.

2. Omnioutliner outlines can be moved to Scrivener, both headlines and contents of "containers." That's done via OPML. Save the file as OPML, then import into Scrivener.

I can see no easy way, short of scripts, to move the contents of TBX containers to Scrivener, save manual copy and paste of the contents of each container.

Is this the state of moving TBX info to Scrivener at the moment?

Thanks in advance for any help,

John Kendrick

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Bernstein on Jul 30th, 2011, 10:31am

You could easily use OPML templates, much discussed here, to effect the export. No scripting is needed.

And it's quite easy to write any export template you like. Again, no scripting. (templates can seem complex at first, but most are very simple)


[edit: opal -> OPML]

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Jul 30th, 2011, 2:45pm

You could easily use OPML templates, much discussed here, to effect the export. No scripting is needed.

Greetings, Mark. Can you direct me to some information on how to do that. Is there a template already prepared? If so, where is it and how to I use it to export? If not, are there some directions somewhere as to how to prepare a template to export via OPML?

Thanks in advance,

John Kendrick

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Lucas D on Jul 30th, 2011, 3:17pm

Hi John,

I cannot provide any expertise on how to do this, but as a starting point, there is the following relevant information (including templates) in Mark Anderson's aTbRef:

http://www.acrobatfaq.com/atbref5/index/ImportExportFormatting/OPMLExport.html

I think this is also crucial:

http://www.acrobatfaq.com/atbref5/index/ImportExportFormatting/Notescanbeexporttemplate.html

However, I had a go at it and was unsuccessful, so perhaps this is a good opportunity to clarify the exact steps needed. In order to get the resulting OPML files to open in OO, I had to manually add some missing slashes. (I figured this out by creating an identical outline in OO, exporting to OPML, and then comparing the two OPML files.) But anyway,  the problems I ran into may have been a result of not properly understanding Mark's instruction regarding "taking care to encode for XML standards and to encode paragraph breaks in $Text."

If I make further progress in troubleshooting this I can post back.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Jul 30th, 2011, 4:56pm

Lucas,

Thanks for the experimental work and for passing it along. That's precisely, however, what I'm hoping to avoid. And I'm sufficiently eager to avoid it that I might not use TBX for the early, data gathering portions of the project. It's far and away the most powerful tool for such but it remains terribly difficult to get the information to my writing platform, Scrivener, without lots of manual copying and pasting.

I had hoped for something easier. I haven't given up and will take a look at the links you've provided.

However, to step back to my original post, I gather that stands as the basic way to move material from TBX to Scrivener unless one steps into the template creation business.

John Kendrick

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Lucas D on Jul 30th, 2011, 6:49pm

For the record, although I'm guessing you've already seen it, this thread over at the Scivener forums appears to have good information on getting data from Tinderbox to Scrivener:

http://www.literatureandlatte.com/forum/viewtopic.php?f=2&t=7244&hilit=Tinderbox+export

As I said, if I figure anything else out, I can post here, and I'm sure there are others who know much more.

Also, if you go ahead and try the method Mark A describes (in aTbRef),  one the one hand it may simply work, and on the other hand, if you run into trouble, I'm guessing someone here could help with the details.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Bernstein on Jul 30th, 2011, 6:50pm

It's far harder to get the information you need than it is to spend an afternoon copying and pasting -- and that's the worst case scenario, if you absolutely don't want to take the trouble to learn a handful of export codes.

We've got some interesting Scrivener export features coming in the next few months.

But there are innumerable ways you might want to arrange a project in Tinderbox, and almost as many ways to arrange one in Scrivener. The more you do yourself -- either manually or using the power tools of Tinderbox export -- the more flexible you can be about arranging everything just the way you want it.


Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Jul 30th, 2011, 7:59pm

Lucas, thanks for the Scrivener link. You're right; I have read it.

It's one of the sources for my bit of kluge--TBX to Omnioutliner to Scrivener with some manual copy and paste of TBX contents into Scrivener. And is my plan for the moment.

I gather from this conversation so far that my little kluge is the best there is, short of pulling out the coding tools. I'll take a look but would much rather spend my time working on my content than writing code. At least at this stage of my life.

As for Mark, I understand your point and it's been a point you've made to me several times. Output from TBX is difficult--printing, exporting to other programs, etc., without learning your code. It's one of the things that accounts for my on-again off-again relationship with TBX.

But I'm still here. Because of the power of TBX and because, despite a bit of diffidence at times, you guys supply among the very best customer service in the business. And I appreciate that.

John Kendrick

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Lucas D on Jul 30th, 2011, 10:14pm

Hi John,

I did a little tweaking to Mark A's OPML code, and I've got it working now. So, here are the basic instructions:

1. In your TBX file, create a top-level note with the title "TEMPLATES"

2. Create two children of "TEMPLATES" with the following names: "OPML-wrapper" and "OPML-item". Make both of these notes into templates by opening the rename window of each and clicking the checkbox "Template".

3. Open the text window of "OPML-wrapper" and add the following
Code:
<?xml version="1.0" encoding="UTF-8"?>
<opml version="1.0">
<head>
<title>OPML-wrapper </title>
<expansionState></expansionState>
</head>
<body>
^children(/TEMPLATES/OPML-item)^
</body>
</opml>

(This part is unchanged from Mark's original.)

4. Open the text window of "OPML-item" and add the following
Code:
^if(ChildCount)^^indent("\t",2)^<outline ^if(Text)^text="^value(attributeEncode($Name))^"_note="^value(attributeEncode($Text))^"^endIf^ ^if(!Text)^text="^value(attributeEncode($Name))^" _note="^value(attributeEncode($Text))^"^endIf^^if($Checked)_status="^value(attributeEncode($Checked))^"^endIf^>
^children(/TEMPLATES/OPML-item)^^indent("\t",2)^ </outline>
^else^<outline ^if(Text)^text="^value(attributeEncode($Name))^"_note="^value(attributeEncode($Text))^"^else^text="^value(attributeEncode($Name))^"^endIf^^if($Checked)^ _status="^value(attributeEncode($Checked))^"^endIf^/>
^endIf^


5. Open the document preferences (command-8), click on the "Text Export" tab, and choose "/TEMPLATES/OPML-wrapper" as the default template.

6. In Outline View, create a single parent note for everything you want to export (so that for what you export there will only be one top-level note).



7. Click on this top-level container note, then click the File menu and select "Export as text".

(7b: In the "Export" window, there are four buttons on the top left. Make sure the one for "Selected notes" is selected.)

8. Click the Export button and save the file, manually adding ".opml" as a file extension.

9. You should now be able to open the resulting OPML file in OmniOutliner.

Note: I don't have an expert understanding of Tinderbox export codes or of OPML, but so far the above seems to be working here. Starting with Mark's "OPML-item" code, I added a missing slash near the end and also a clause for notes that don't contain text. It wasn't clear to me that I did this correctly, and I can't make any guarantees, but it seems to work.

I've also uploaded a sample TBX with the export templates installed:

http://www.mediafire.com/file/0vr6fzeeeoge4vu/OPML%20Export%20Template.tbx

You could always just copy everything you want to export and paste it into the sample TBX.

Hope that helps!

Lucas

(Later: If you try to export a very large volume of notes it may take a while.  So I suggest trying it first with a smaller group of notes to make sure you get the desired result. If it works, then you could try something larger.)

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Jul 30th, 2011, 10:36pm

Lucas,

This all looks terrific. I'm going to take a careful look at all of it, hopefully, tomorrow. Thanks again for the terrific help.

I'll even try tweaking some code. That is the sort of thing I needed to get me started.

As I said, terrific support.

John Kendrick

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Anderson on Jul 31st, 2011, 5:12am

Lucas' problem had me scratching my head but I've found the error in the source reference.  OPML 'outline' tags follow XML convention and are (sould be!) written in one of two ways:
    <outline></outline>
    <outline />


The first form is used to nest descendants' oultine tags. Items with no children use the second form. In the aTbRef code reference cited further above, the closing '/' in the second form of tag had somehow gone missing. It definitely was correct when originally tested and posted so I guess got broken in subsequent editing (liely when the support for checkbox data was added). Anyway, the original aTbRef is now corrected - i.e. meeting Lucas' correction. Whilst the hood was up I also updated the ^if()^ syntax to current standards whereby all short-form attribute queries use the $-prefix. Thus ^if(Text)^ becomes ^if($Text)^. Both forms currently work but eventually the non prefixed version won't when legacy code support is dropped.

For John, the result of the last paragraph's change is here, "OPML-Item":


Code:
^if($ChildCount)^^indent("\t",2)^<outline ^if($Text)^text="^value(attributeEncode($Name))^"_note="^value(attributeEncode($Text))^"text="^value(attributeEncode($Name))^"^endIf^^if($Checked)_status="^value(attributeEncode($Checked))^"^endIf^>
^children(/TEMPLATES/OPML-item)^^indent("\t",2)^ </outline>
^else^<outline ^if($Text)^text="^value(attributeEncode($Name))^"_note="^value(attributeEncode($Text))^"^else^text="^value(attributeEncode($Name))^"^endIf^^if($Checked)^ _status="^value(attributeEncode($Checked))^"^endIf^ />
^endIf^


I've also updated my starter.tbx doc to reflect this fix (right-click link and 'save as').

@Lucas - thanks for flagging up the error in the reference.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Jul 31st, 2011, 6:30am

Thanks, Mark. I'll try to find the time later today (Sunday) to work on this.

John

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Jean Goodwin on Jul 31st, 2011, 9:00am

Hi, Mark:  You may want to check--your file doesn't seem to quite work.  It leaves "parent" items nameless.  Lucas' works, though, at least for what I've thrown at it.  Thanks!

This sort of thing is why I've never invested the time to go beyond very basic export.  It's not that export is hard--it's that it's fussy. That's why it's great when people donate their time and share their work product!

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Anderson on Jul 31st, 2011, 11:54am

Hmm. That's odd. @Lucas' code doesn't make sense, insofar as it adds a cluase that says "if there is no $Text set the value of OPML attribute '_note' to value of $Text.  So I don't think that the issue.  it may be a whitespace thing.

Try this 'OPML-item' code:

Code:
^indent("\t",$OutlineDepth(parent)-1)^^if($ChildCount)^<outline^if($Text)^ text="^value(attributeEncode($Name))^" _note="^value(attributeEncode($Text))^"^else^ text="^value(attributeEncode($Name))^"^endIf^^if($Checked)^ _status="^value(attributeEncode($Checked))^"^endIf^>
^children("OPML-item")^^indent("\t",$OutlineDepth(parent)-1)^</outline>
^else^<outline^if($Text)^ text="^value(attributeEncode($Name))^" _note="^value(attributeEncode($Text))^"^else^ text="^value(attributeEncode($Name))^"^endIf^^if($Checked)^ _status="^value(attributeEncode($Checked))^"^endIf^ />
^endIf^

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Jul 31st, 2011, 12:08pm

Six cheers for Lucas, whether his code makes sense or not. It definitely worked. Though it took me a bit to fill in the blanks.

I needed this kind of very explicit, kindergarten level hand holding to make it work: click on this button, paste this code, check this box, etc. However, I forgot that any templates note had to be checked as such. Spent some time trying to correct my errors at that stage. But the proverbial light finally went off in my head.

Once Mark Anderson gets his code cleared up, I'll try it as well. But, at least for the moment, Lucas' work "works" perfectly.

I gather the coding is elegant but the process definitely is not. It would be much better to have this built in to the process. At least for non coders. Coders could always modify the code to fit their circumstances.

Why not have this opml code or some other as one of the built in text options?

Thanks again, everyone, particularly Lucas. I would say your code made it possible but, actually, more than the code, it was the kindergarten instruction set.

John Kendrick

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Jul 31st, 2011, 12:23pm

I should also add a thought or two more.

After working my way through this and then creating some sort of help files, I see, Lucas, that you suggested this be done on a small file first. I see that you added it later. I always try to do experiments that way at any rate. But thanks for the thought.

I will try it later on a larger file, though only as an experiment, just to see how it works.

I will also need to learn to put more material in the containers and less in the titles. Scrivener will like that. But so long as the export only worked with titles, they were longish.

John

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Paul Walters on Jul 31st, 2011, 12:45pm

I do a lot of OPML exporting from Tinderbox, to Scrivener and elsewhere.  I'd suggest the code that Lucas and Mark designed (for the "OPML-Item" template) can be simplified, as follows:


Code:
<outline text="^value(attributeEncode($Name))^" _note="^value(attributeEncode($Text))^" _status="^value(attributeEncode($Checked))^">
^children(/Templates/OPML-item)^ </outline>


There's no need to put the indents in an OPML file.  It's the nesting of <outline /> elements that handle indenting, and Tinderbox, OmniOutliner, Scrivener, etc., understand that when they parse the file.   There's no need to have the existence tests for attribute content (the "if..." statements), because empty attributes in OPML are just passed over by the parser.  And there's no need to check for children, because the above code will capture children from zero to infinite depth.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Anderson on Jul 31st, 2011, 5:41pm

Further to Paul's points, it is also not a requirement to use <tag/> for a tag with no children, <tag></tag> can be used as well (indeed that is what his example does). But, as anyone who's tried debugging broken OPML can attest, using indented code makes it much easier to see the closures.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Jul 31st, 2011, 7:56pm

In my case, my test file only works with Lucas' code. Perfectly.

The children and the container material don't make the trip with Brad's code.

I entered Brad's code in the OPML-item file, then tested the results by passing it through Omnioutliner and on to Scrivener; and a second time taking the file directly to Scrivener. Lost the children and the container contents both time.

I recall Jean found that Lucas' code worked and Mark's did not. Someone observed that opml coding is famously varied. And I think I read somewhere that Omnioutliner's opml code is a bit specific. Since Scrivener says they want to make certain outlines from Omnioutliner can be imported into Scrivener, perhaps somewhere in there lies some sort of an answer.

I'm now trying to find ways to streamline the process so I don't have to copy and paste those template notes from file to file. Or, if I have to, I find a way to cut down the steps.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Anderson on Aug 1st, 2011, 7:16am

I think the real underlying problem for @John was an unclosed <outline> tag since when we've been going round in circles. After a morning spent tinkering with OPML I've found some other things (e.g. here re checkbox info) but I think they aren't the issue. I now want to get a code test that I know works verified by John so we can move forward. As a result, I've updated my previous "OPML-for-Scrvener" TBX (right click link and Save as). I've also uploaded the OPML output that it exports: NewOPMLTest.opml (right click link and Save as). The TBX looks like this:



The file's set up so only "New OPML Test" exports, so it's just a mater of opening the file and using File -> Export as HTML. The use of accents, angle brackets and such us deliberate - they're there to test encoding works correctly. I've also added as an alternate, a new pair of templates based on Paul's suggestion further above although please be aware that to use the latter you'll need to make the template assignment changes described in the TBX's help note "The 'Simple' templates".

John, could you please try the above (specifically the files linked to in this post not earlier ones) and tell me if it works for you? I'd welcome another Scrivener user giving a report on whether this works.

I want to ensure we've got export working for John and then , in my admin 'hat' I'll prune or possibly simply delete this thread and write up the successful result. In doing so, I'm not dismissing the help of various forum members here but I'm just conscious that subsequent readers need to work through a page and a half of what doesn't work to get to a solution.  Thought I'd flag the latter as coming - I don't want anyone thinking they're in some way censored.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Jean Goodwin on Aug 1st, 2011, 8:58am

"Simple" still isn't working for me (for OmniOutliner).

Mark, I have no complaints about "censorship."  But sometimes it is very educational to read a thread developing--not just the end result.  At a minimum, could you save LucasD's fine set of instructions?

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Paul Walters on Aug 1st, 2011, 9:02am

Jean, what flavor (pro, not pro) and version of OO are you using?  Does it give an error message?  If you export the OPML from Tinderbox, and then try to open it as a new Tinderbox document, does it work?

I'd agree that I'd rather see this thread left intact, and maybe the final answer posted on a locked, sticky thread - or the Cookbook (is the Cookbook actually maintained these days?).

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Anderson on Aug 1st, 2011, 10:01am

Is the CB maintained? Yes. I, amongst others, certainly add new tests as new features are added.  The only thing missing perhaps is a way to automate publishing of the changes as I notice CB source is at v5.9.2 but the online pages are still for v5.8.0.

A harder issue to work around is that date/date coercion based tests rely on the user's locale settings. Currently many of the tests - because I wrote them - use UK settings so I can check things work. It has to be that way as I have other important processes running that won't cope with me me switching my Mac's locales. I've stated looking at a way to at least make the CB tests have, say US and GB locale version though one set will always fail in the HTML depending on the users locale - indeed both might fail if run out elsewhere.  I don't see a way round that as the test strings are hard coded (deliberately).

As to the other stuff, comments noted. Anyway, that's later. Has anyone tested the file I posted and successfully imported into Scrivener. I'm pretty happy that TB<->OO works fine (barring the checkbox side-issue noted in a separate thread). We still need to prove there aren't glitches in the Scrivener OPML import using a common test done by several people. I'm not suggesting a fault, but think it sensible to allow for the possibility before happily telling users who don't want to fiddle with export to "just do X and all will work".

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Bernstein on Aug 1st, 2011, 10:05am

A difficulty with some threads (I've not been following this one closely) is that they fill up with confusing, irrelevant, or obsolete information.  The result is that people today decide not to use Tinderbox because of something that someone erroneously claimed was impossible in 2007.

As I believe I said earlier in the thread:

(a) OPML export from Tinderbox is should not be difficult
(b) We're looking into even better export tailored to Scrivener

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Paul Walters on Aug 1st, 2011, 10:11am


Quote:
Mark Anderson wrote
Has anyone tested the file I posted and successfully imported into Scrivener[?]


Yes and Yes.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Anderson on Aug 1st, 2011, 10:18am

Many thanks. Just hopefully get a yes from John and I can sort this thread out.

P.S. A polite request from your moderator that others than John do not add to this thread now as I'll be re-factoring it once I know it he's got the correct code working.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Aug 1st, 2011, 3:16pm

Mark,

I just saw a reminder to this thread in my email but can't get to work on Tinderbox until this evening. I'm on the road and will have not time to do anything.

As for working, I'm quite comfortable with Lucas' solution. It worked perfectly, not only on the fairly short tbx file I exported yesterday but a longer one last night.

I do appreciate your effort, however. As for keeping the thread, let me offer another vote to do so. It helps to see how folk work through these issues. And it also helps to see the level both of the support tbx provides but also the work of a vibrant user community.

Back later,

John Kendrick

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by John Kendrick on Aug 1st, 2011, 8:16pm

I'm back later with mixed results, Mark. The export file you linked in your text imported directly into Scrivener just fine. No use of Omnioutliner. The children showed fine and the text in the containers as well. Good work.

Not so good with the second material, the full TBX file that included your code. I did an export with it via html, as you suggested. Scrivener didn't like it as a file import. Error message was:

Some files could not be imported because they were of an unsupported file type or contained data that could not be read.

If, however, I exported that file as text using the Simple Wrapper template, it worked just fine. Perhaps you mistyped those instructions.

So, in summary, the code works fine. So long as you use the "text" export button on the templates associated with it. But doesn't work with the html export option.

Thanks again for all the attention to this.

John Kendrick

(Later edit) I continue to want to vote to keep this thread as is. It's a terrific illustration of good customer support and, as I said, a vibrant user community.

Title: Re: Tinderbox to Omnioutliner to Scrivener
Post by Mark Anderson on Aug 2nd, 2011, 4:10am

Resolved at last. It appears that the TBX had somehow got its template assignments messed up (and wasn't exporting OPML) and was using the 'plain' built-in template which is understandably why the Scrivener import failed in John's test. I had been worried there might be an issue the Scrivener end but finding this export configuration glitch, which was user - me not John - and not app error, all seems to fall into place given that the specimen OPML export did work.

The glitch didn't show in testing (for reasons perhaps for a thread another day) before upload but was immediately obvious on re-downloading it and looking at the export in HTML view and in the exported source code. Of course, for John who's not looking at source code the glitch simply wouldn't be apparent, so confusion isn't surprising on his part. The same TBX file, re-tested and re-uploaded again(!), now produces the expected OPML which already I know from John's report does work. Test complete!

John, please throw away your existing OPML-to-Scrivener.tbx and re-download it. You will find it will now work.

Wow, all this from one missing '/' ... and in misdiagnosing that a lot of false trails. But there at last  :)

Anyway, this thread is now locked. Please start any new OPML discussion in a new thread (the "Exporting from Tinderbox" sub-forum seems the most logical place).

Meanwhile, I'm working on some OPML export example files for both inexperienced and more seasoned users. It may be a few days to get done; 90% of the time is the explanatory notes, etc. Until then, today's revised TBX should suffice.

Title: Re: Tinderbox to Omnioutliner to Scrivener (OPML)
Post by Mark Anderson on Aug 2nd, 2011, 4:15am


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Admin note: Thread resolved and locked. After a trip around the houses, a working OPML export example is here.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.