Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
Tinderbox Users >> Questions and Answers >> General advice re performance in large files?

Message started by Rob Forsyth on Sep 26th, 2012, 12:07pm

Title: General advice re performance in large files?
Post by Rob Forsyth on Sep 26th, 2012, 12:07pm

Continuing to work with the TBX referred to in http://www.eastgate.com/Tinderbox/forum//YaBB.pl?num=1348135946 I'm hitting very significant usability/performance issues: spinning beachballs for several seconds with each and every mouse click. The TBX file is large but simple (~1200 notes with a $Name about sentence or so long, a boolean custom attribute and $Text an article abstract that can be several paragraphs of free text) Total file size is 2.2MB. I only have one agent (currently) searching for a single word in the abstracts or titles but have automatic updating of agents turned off. Are there any other settings/preferences options that I could adjust in order to improve usability?



Title: Re: General advice re performance in large files?
Post by Mark Anderson on Sep 26th, 2012, 12:38pm

Might I politely suggest this be followed up directly via support (info@eastgate.com)? There's nothing obviously 'wrong' from the setting you describe, but there are many factors (the TBX, the OS' resources on which is runs, etc.) that are difficult to pick over - unless you're happy to share your TBX publicly. I would say though that you seem to have ticked the obvious boxes in terms of easy levers that might have affected performance. If you tend to use one main view type, I'd try a different one (e.g. Outline vs Map vs Exlorer) to see if that alters anything. Also if you keep many windows from the TBX open, you could try closing a few. If turning off/deleting the agent(s) completely fixes the issue than that gives some pointer as to cause but not an answer, though the agent query is a good place to start. Overall it sounds like there's some window updating/redraw issue but bottom line, someone else probably needs to see your file to diagnose further. Sorry if that seems unhelpful - it's certainly not intended as such.

Re size, and at a tangent, I've been consulting on some TBX files with >150,000 notes ($Name only, Outline view only) and whilst they push some aspects of TB such as complex agents, working with care I've been happily surprised at what TB will allow. FWIW, were it not for my clients particular needs I'd actually break the data into smaller files … but needs must. A more sensible size would be <50,00 notes.

There are no fixed limits per se but as a rule of thumb;
  • Many small notes are better than fewer large ones (IOW, don't put a whole novel in one $Text!).
  • Agents work better if the query is scoped; i.e. start by collecting as few items as possible needing to undergo a more complex query. thus a descendedFrom() or inside() test should come before a more complex .contain() regex test.
  • Agents can search inside other agents so this can be another way in which to simplify otherwise complex queries. IOW, Agent B's query can act just on inside("Agent A").
  • Fine-tune agents. Consider using $Searchable.
  • In v5 (though likely fixed from v6) Some views caanot display vast amounts of data. It's a technical limitation - the container can hold X items, but TB doesn't have a virtual canvas big enough to draw them all on. The drawing aspect is all being re-written for v6 so should make that go away.
Lastly, as I know Mark B's on the road right now with patchy connectivity, if it's urgent, I'll happily take a look at the file (data in confidence) in case I can see anything obvious. If so, email me.

Title: Re: General advice re performance in large files?
Post by Mark Bernstein on Sep 27th, 2012, 2:59am

As Mark Anderson said, the document you describe shouldn’t cause that sort of performance problem.  (Mark’s 150,000 note Tinderbox document, on the other hand, astonished me because I would never have expected it to work!)

I've got lots of files I use frequently that are larger than this; my weblog has 10,000 notes

So, yes, Something Is Not Right.

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.