Welcome, Guest. Please Login
Tinderbox
  News:
IMPORTANT MESSAGE! This forum has now been replaced by a new forum at http://forum.eastgate.com and no further posting or member registration is allowed. The forum is still accessible via read-only access for reference purposes. If you wish to discuss content here, please use the new forum. N.B. - posting in the new forum requires a fresh registration in the new forum (sorry - member data can't be ported).
  HomeHelpSearchLogin  
 
Pages: 1
Send Topic Print
"How Many Notes?": A Usage Report (Read 4040 times)
Alex Strick van Linschoten
Full Member
*
Offline



Posts: 61
Holland
"How Many Notes?": A Usage Report
Aug 15th, 2011, 5:48am
 
It seems the 'how many notes per document' question is often asked on this forum (or at least, with some regularity) so I thought I'd post some figures for a large text-crunching project I'm nearing the end of.



As you can see, lots of words, lots of notes, and a fair few agents. The number of agents means that I have to turn off the auto-processing of agents; it also means that the update time isn't exactly instant. But that's all fine. I don't need instant feedback for this document.

I tried to sidestep some of the agent update processing by using display expressions of containers to calculate various values from the notes contained inside it. For example, a figure for how many of a certain type of event took place, or a calculation of how many people something happened to.

I'm slowly using agents to calculate numbers for various variables and transferring them out into a spreadsheet application to generate some charts.

Any problems or lessons learned? Not really. The good old ideal of 'incremental formalisation' worked a treat -- even one time when I realised I had forgot to add details for a certain variable across the whole database, a quick agent query made it very easy for me to find the notes in question -- and what was originally quite a simple design slowly shaped into something far more complex.

And once again, I'm not sure how I would ever have completed this project work without Tinderbox.
Back to top
 
 
WWW strickvl   IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: "How Many Notes?": A Usage Report
Reply #1 - Aug 15th, 2011, 7:10am
 
One incremental tip with agents. It often happens that you have an agent with a complex-ish query (or wide scoped, i.e. testing many/all notes). you then create a variant of it. It might well the the same query with one extra term targeting a different attribute. As this yields benefit, you add some more.

At this point consider making an agent with the original 'base' query. Now the other agents - the variants, if you will - can just look inside() the base agent at a single 'pre-compiled' listing. That way one agent may be polling the whole TBX but 30 others are mainly looking inside the first agent's matches and filtering on other attributes.

Such a solution is easy to test and retro-fit. A good nudge for when to look for this sort of approach is if your TBX performance is slowing down. Inefficient design won't matter with a few hundred small notes.  Pump the number of note and degree of content up and some test and tune can make a different.  It's nice to know, though, that you don't have to guess the right solution at the start!

If you do use nested agent, do be aware that some change may take a while to trickle through- especially if you've put some agents on a lower priority. If in doubt, use File -> 'Update now' to flush everything through
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Alex Strick van Linschoten
Full Member
*
Offline



Posts: 61
Holland
Re: "How Many Notes?": A Usage Report
Reply #2 - Aug 15th, 2011, 8:54am
 
Right. Yes. That's useful. One of the main reasons why there are so many agents is that I need some way of disaggregating events on a per province basis. There are 34 provinces in Afghanistan, so each one needs its own agent.
Back to top
 
 
WWW strickvl   IP Logged
Mark Anderson
YaBB Administrator
*
Offline

User - not staff!

Posts: 5689
Southsea, UK
Re: "How Many Notes?": A Usage Report
Reply #3 - Aug 15th, 2011, 11:02am
 
If you implement the above suggestion, it will be interesting to hear if your Agent Update cycle time comes down.
Back to top
 
 

--
Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Pages: 1
Send Topic Print