Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
Tinderbox Users >> Exporting from Tinderbox >> Exporting Tinderbox journal notes for MacJournal

Message started by Greg Korgeski on Nov 23rd, 2012, 12:37pm

Title: Exporting Tinderbox journal notes for MacJournal
Post by Greg Korgeski on Nov 23rd, 2012, 12:37pm

Sorry if this has been covered elsewhere, but I can't quite find this one.  I've got a master Tinderbox doc with about 11,000 notes, including about three years' worth of daily journal notes that use a common prototype (including the title term "Journal:" followed by a display expression for the date of each note.)  I'd like to export copies of just the journal notes (several hundred) with dates, into a MacJournal file that both predates my use of Tinderbox by about four years, and will become my main journal file again, starting next year. (Mostly this is because the MacJournal collection is quite extensive and it'd be nice to have all this material combined in one master journal, but also the Tinderbox file has become quite slow to open and isn't performing so well any more. I'm not sure I want to keep adding to a file that already takes 10-15" to open every time I use it.)

I can't quite find a clear set of instructions for exporting, particularly if I want to have each note save its display expression data on the specific date. So I've been trying a workaround test, involving exporting a smaller set of about 45 notes from an alias container for something else (my books read file for the year - the "journal" notes alias is so slow to export that doesn't lend itself to experimenting) The test worked sort of all right, though only via importing them first into Scrivener and from there, exporting the Scrivener-converted OPML notes to a file that I could successfully import into MacJournal. (That's a long way around, but it worked for a small set.)  

But trying to set up an export of the "journal" notes using an agent to gather the journal notes, has not worked so well. I get the spinning beachball, using the Tinderbox "View/HTML view" export  on that agent container (it would be perhaps daunting to do it without the agent, as I use journal notes as containers themselves for odds and ends I accumulate each day.)  I had previously tried to export the whole Tinderbox document, figuring I'd sort it out later, but that was left to run all night and froze up early on  and after an all night export, there was nothing completed or even moved along very far.

Are there some simple instructions here? While there are all sorts of Tinderbox guides that say exporting Tinderbox notes is flexible and easy, the fact is, when you only need to tackle this kind of project once in a blue moon, it's pretty daunting finding the instructions you need and making them work. It can resemble mastering the art of French cooking, where the first instruction in the cookbook is "learn to speak French."



Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Mark Bernstein on Nov 23rd, 2012, 1:25pm

a) Your journal takes fifteen MINUTES to open?  Something strange is going on. My weblog (ten years, 11,000 notes) takes less than fifteen SECONDS to open on a MacBook Air.  So, something is definitely odd.

  • This might be a good prospect for contacting support: info@eastgate.com

b) Again, the problem with exporting your agent suggests something is not right.  Here, I'd suspect either the agent export template or, if it exports children, the export template for the children.  In particular, it would be a bad thing (for example) if each child exported its parent again, which would export the child, and then the parent, eternally.  Something like this is probably the problem here.

  • Your impulse above is probably the right one: make a smaller Tinderbox document just for a test case. Try out and improve your export templates there; everything will be small and fast, and errors in the output won't be buried in masses of detail.

c) I think the "learn to speak French" comment is undeserved. The manual is not written in French, nor does it expect much background. The aTbRef resource is thorough and systematic; the manual proceeds by example; the people on this forum have been more than generous with their time and advice to questions -- including, if memory serves -- your own questions.

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Mark Anderson on Nov 23rd, 2012, 7:26pm

Indeed, I just opened a TBX here with 33k+ notes (and 145 agents) in 13 secs. The slow response is likely one or more of:
  • rules
  • agent queries
  • display or hover expressions
A bit of tinkering and some unnoticed errors can add up. In the above-mentioned file I help a user with, and which I received with agents turned off and a cycle time of 85,916 msec. A bit of tidying up of code and that got down to 4033 msec and the user now runs it with agent updating on. The point being, if things run slowly, check your action code. Ways to do that: $AgentPriority, $RuleDisabled, $DisplayExpressionDisabled. These may help make the file more responsive while you find the code errors.

What you see in a major view, for a note with a display expression is $DisplayName, not $Name. So exporting that 'text' in the context of a template is as trivial as using ^value($DisplayName)^.

However, it's likely you've 3 years worth of cruft and failed attempts lurking in your TBX. Some cleaning out of code with help. I'd rip all the rules out and insert clean ones via prototypes so you can edit-once-correct-many.

If you're willing to share your file - or a version of it that displays the problems you describe - I'm sure fellow users here can help. Or, send your TBX to support (info@eastgate.com) so they can take a look. Whatever, I'd seek some help before frustration builds further.

It's easy to badmouth the documentation but you don't give any examples of where they are wrong. It matter - if it's shown to be wrong it can be corrected. If it's less clear than intended it can be re-written. If you've simply misread it, someone can help explain it.

[edit: typos]

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Mark Anderson on Nov 24th, 2012, 5:34am

Can you post the template(s) you're using for the journal export? My hunch is you're using some form of include that is working in a way you'd not presumed. There's no one correct way to do these sorts of exports beyond the most trivial because - as experience shows - everyone is doing slightly different. Normally this is accommodating the limitations of import into some other app.

There certainly should be no need to go via Scrivener to create OPML; Tinderbox gives you more control of formatting and can write cleaner OPML as a result. The latter suggests it may be the export code syntax with which you may be spinning your wheels.

@Greg, if you're data's too private to show in the forum, you're welcome to email me and discuss off-forum if it helps. I'm sure your problem is fixable.

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Greg Korgeski on Nov 28th, 2012, 1:06pm

Thanks for your replies and offers of assistance.  Really, no intent to offend in mentioning "French." But I think it's kind of obvious, from the perspective of a "lay" user and not a software engineer, that Tinderbox is basically a very different "language" than most other Mac software. Despite having used it for probably eight or ten years, I continue to find it very complicated. It's not the issue of the documentation; I didn't even remember there was the documentation you mention: I went to the menu bar to export, did a lot of unsuccessful trial and error testing, looked in the Help menu and didn't see a "manual," but mostly did like most of us do with other apps: keep trying things that seem like they should work. (I have tons of notes and reference material on using it, but didn't start any kind of systematic search because well, I keep assuming that "this time" or trying some small variant, I'll figure it out.  

The problem with "steep learning curve" softwares is that there is also a "steep forgetting curve," and so when after a year or two of just entering notes day to day, I do need to do something unusual like export (which I've never done before in any serious way), I feel like I no longer remember the things I spent time learning to do and did maybe once, many months ago. Since Tinderbox requires a large number of skills that are quite different than most or all other Mac apps, and its interface is so different than all other Mac apps I use... there is a big "refresh your memory" curve to do something out of the ordinary. That's what I mean by "French." (I know enough French, way below fluent but workable to read a magazine, so that comparison would actually be pretty close for me.)

I did err in stating it takes 15 minutes - it's fifteen seconds - to open the doc. (Note to self: stop using the apostrophe sign, spell that out.) But fifteen seconds to open and eight or nine to save the doc SEEMED long to me, and made me nervous, because in part it's longer than other large database type apps I use, I think, but moreso because of a gradual accumulation of problems I've had with either this doc or Tinderbox over the past few one or two years, which have not resolved. So I start to wonder if there's a bug. I've written in about each of these but not gotten any resolution despite your advice and attempts to help: maps not behaving correctly (adding new notes to a map, moving notes on the map or resizing notes resulted in their not rendering on the map till I clicked out of the item and clicked the cursor on the background of the map; this happened across two docs, two build versions of Tinderbox, and two different macs; not sure it's still an issue on my newer Macbook Air, as I sort of migrated away from using maps very much after that.)  Second issue was grayed out and so inaccessible text inside notes such as checkboxes in the main attributes; feedback was this happens if an explorer view is open but I've had it happen if there is ONLY the single note open; have to close and reopen the doc to get it to stop. Third and most problematic has been the contents of a note sometimes vanishing when note is closed, if I have not re-saved the doc first.

All of these things going on have been difficult; the last in particular, since every note has to be saved before it's closed, and it's a 9" (second) save.  So any sign of something working "wrong" has made me more skittish.

Now, if the export is not working right, that's four. That's critical mass for me. It's made me start to really think about whether I want to put more time into learning about whatever I need to learn to fix it, and that's left me reassessing whether Tinderbox is frankly the best app for me anyway. It's a lot more work, doesn't feel reliable enough at the moment.  I have been leaning toward concluding that I'm going to go on a Tinderbox hiatus for a bit. I think I don't trust the software (nothing else has so many glitches and hassles), and that makes me nervous. I don't think I really get as much use out of my 11,000 notes as I probably would with better focus in a single app that worked more easily and made the data more findable.  I am wondering if dividing my work between this and DEVONthink has resulted in not really using either tool to its best advantage, and I think it's not such a good idea to continue dividing my mental energies so much.

So.... am putting this problem on hold. I've already spent more time trying to export a few hundred journal notes than it would have taken to have hand copied/pasted them.  I need to think this through, try some things. But thanks again for your help.  I'll let you know if I want to pick it up and work on a fix.  I do appreciate your help and encouragement.


Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by russ lipton on Nov 28th, 2012, 3:07pm


I applaud your persistence. Clearly, you do like many things about TbX. I have used TbX since  V1.0 and second some of your experiences. Particularly, I find that if I fail to use an underlying feature set for a while, it all looks like Fren ... Greek to me when I return. Odd, that, and I think you are correct that TbX differs markedly from much other Mac software. Whether one considers that a feature rather than a 'bug' is something else. I distinguish that from the apparent product 'bugs' which concern you.

Even if you put TbX aside, I hope you take up Mark's suggestion and send him your files ASAP.

TbX's design is one thing; bugs and performance another.  I have found TbX to be remarkably robust and bug-free over the years. It is the only product where I routinely use beta versions as production versions. I will wager that Mark A. can spot and correct the 'buggy' stuff quickly. You will do all of us a favor as well - if, as is possible, you have hit unreliable portions of code, please give Eastgate a chance to fix them ASAP by inspecting your work.

If (worst case), you designed your notes in some presumably lame manner, well, join the club. No harm, no foul. There are dozens of ways to skin our rabbits (thinking French cusine here and Mark B may correct me). I'm not sure a single, optimally designed TbX document can exist, in principle. I'm quite convinced, rightly or not, that my documents would make any real TbX expert laugh out loud.

Anyway, should the causes be clumsiness on your part, no need for that to be publicized to the world. Keep it all between you, Mark A and Mark B, but let us know the bottom line result. Please. The premium support of both Marks has plugged much of that hole you mention between customary Mac software and .... ceci (?), so let's make them work, eh?

The issue, of course, always boils down in the end to the trade-offs between your cost (more time even than money) and your benefit. I adore TbX, yet do not find it contradictory to say that I review my current and future usage regularly in light of other tools, some of which are also quite unusual, powerful and relevant to my projects. TbX is only as good as its latest release and that in comparison to all possible competitors, something I suspect Eastgate understands all too well.

Whichever way you go, good luck!

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Greg Korgeski on Nov 28th, 2012, 4:38pm

Thanks for your encouraging reply.  

Apparently I'm the only person to have ever had these particular bugs or glitches or problems, according to Mark, so I don't suspect anyone else's files are about to explode.  Like I say, I'm still thinking... not sure whether I'll conclude it's worth the work and time to keep going on about this problem at this point.  This was the kind of "wouldn't it be neat?" export project that came up because I was on vacation and so had time to tinker; I'm back at work and can't spend another hour or two for a few weeks trying to repair an ambivalently loved relationship.  Maybe later...

Best, G

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Mark Anderson on Nov 28th, 2012, 5:18pm

@Greg, I think the problem you're facing is that  - unintentionally - your date 'house' is built on sand. One or more infelicitously constructed rules is eating up horsepower and making everything run slow. If you haven't found the cause by now, likely you won't. Your best bet is to let support or myself take a look at (a copy of) it. No downside for you in that - if nothing else it's not more of your time being used as someone else will fix things. At worst you might need to field the odd question about what certain agents are for or what rules were intended to achieve. If you have sensitivities about the data therein, that's for a direct conversation off forum.

Although code in export templates isn't used except during export, I'd get the basic speed issue resolved before trying anything new. Otherwise it's building on sand and likely doomed to further failure. Trying "just one more" experiment yourself is definitely not the ideal approach at this point. I'd concentrate on getting an experienced TB user looking at the file as soon as possible so you can get back to doing things with the data. If for whatever reason you can't send the TBX to support , my offer of taking a look at the data still stands…

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Greg Korgeski on Nov 28th, 2012, 8:31pm

Thanks, Mark.  Will consider.  Appreciate your offer.  G

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Sumner Gerard on Nov 29th, 2012, 2:28pm

I've gotten a kick out of reading this thread, even the "learn to speak French" quip, since the French sometimes say, when encountering something difficult, "ç'est du Chinois." And written Chinese is indeed a challenge, even for Chinese. So it's all relative.

Eastgate and its community certainly have no reason to be defensive. Relative to other sophisticated Mac software, Tinderbox to me seems reliable, exceptionally well supported, well documented, and user friendly for many tasks, though to me export, while endlessly flexible, is undeniably harder to learn than many other aspects.  That's not to say I've found Tinderbox naturally suited for all tasks. It doesn't (yet…it's coming) fully support multibyte languages throughout its interface. I've noticed that editing text in an open note can get a bit funky at times. And, given a ready alternative, I'm not sure I'd want to keep 11,000 notes there. MacJournal seems a better repository, with TB as a place to analyze the relationships between things.

One option to keep in mind here is that MacJournal is AppleScriptable. That means you can send a selection of Tinderbox notes directly to it rather than messing about with exporting, saving and importing files, or going through Scrivener and such. All it takes for me to send a big bunch of TB notes directly to DEVONthink, for example, is a four-line AppleScript (two lines within an "on run" handler) plus a stamp in Tinderbox with one line of simple TB action code to pass the desired Tinderbox attribute values to the AppleScript. I select the notes in Tinderbox, apply the stamp from its Stamp menu, and the notes appear in DEVONthink.  I believe the process with MacJournal would be very similar.

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by lisah2u on Nov 29th, 2012, 8:19pm

Hi Sumner, this is neat. I'm new to Tinderbox but love DT. What is the AppleScript you used plus the action code? Thus far, I'm using Tinderbox specifically to explore ideas and construct summaries across multiple sources, but otherwise storing and organizing data in DT.

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Mark Bernstein on Nov 29th, 2012, 10:29pm

Lisa -- I think we might want to fork this thread out and start a new topic about Applescripting and DT.

Title: Re: Exporting Tinderbox journal notes for MacJournal
Post by Sumner Gerard on Nov 29th, 2012, 11:27pm

I've described my DEVONthink method here. It works for me, and look forward to hearing how it does for others.

For MacJournal the corresponding syntax is something like
make new journal entry with properties {name: p_title, plain text content: p_text, date: p_date}

Some details on MacJournal here.

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.