Welcome, Guest. Please Login
IMPORTANT MESSAGE! This forum has now been replaced by a new forum at http://forum.eastgate.com and no further posting or member registration is allowed. The forum is still accessible via read-only access for reference purposes. If you wish to discuss content here, please use the new forum. N.B. - posting in the new forum requires a fresh registration in the new forum (sorry - member data can't be ported).
Pages: 1
Send Topic Print
Performance and complex Display Expressions (Read 2550 times)
Mark Anderson
YaBB Administrator

User - not staff!

Posts: 5689
Southsea, UK
Performance and complex Display Expressions
Oct 26th, 2012, 7:05am
I'll share an interesting thing I just learned in diagnosing a 'slow' TBX for someone. This initial guess of it being related to display expressions (DE) was correct but not for the reasons we assumed. The $DE being used was this:

$String1+', '+$String2+' '+$String3+', '+$String4+', ['+$Number1+': '+$Number2+'.'+$Number3+'-'+$Number4 + ']'

Note that none of the attributes were themselves calculations so it just looked like a simple concatenation. Whilst there were lots of concatenations there didn't seem much complex calculation.

I then noticed that $Sort's doc-level default was set to "DisplayName", so every container was sorting on an evaluated value. I reset that and only used the original value for the one container where this was actually needed. Still, as this affected some 950 notes (most of the total note count), as I'd guessed it had little effect.

The TBX already had an agent that collected the items in the container of interest (using a simple inside() query). I added new String attribute $DisplayString and got the agent action to store the existing $DE's evaluation in the new attribute like so:

$DisplayString=$String1+', '+$String2+' '+$String3+', '+$String4+', ['+$Number1+': '+$Number2+'.'+$Number3+'-'+$Number4 + ']'

…whilst altering the affected note's $DE to just: $DisplayString**. As $DisplayName was now the value if $DisplayString, I also altered the container sort to "DisplayString".

**These had been set per-note via the parent's $OnAdd. By making the latter set a prototype instead and setting the DE there. Edits became much easier - editing one note updated 000s of notes.

The result was a significant change in performance of the TBX on my fast Mac - doubtless more so on the document owners less powerful laptop.

The lesson:  if you've a complex DE and you want to use it in things like sorts - or do work based on $DisplayName, then it's better to do the calculation elsewhere and set DE to use the resulting value than to calculate in the DE itself. These effects pass unnoticed in very small TBXs but as the note count rises (and/or the calculation is more complex) you may find some optimisation will help a lot.

Aside, another performance improvement came from better scoping of agent queries. A number of agents needed to work on all items in Container 'X'. The queries started:


This query worked, but in doing so polled every note in the TBX. However, action code already provides a specific query code for this task - inside(). Thus:


At the same time, there were several agents looking at notes in X plus some other query term, in order to effect different actions.  So, I made one agent called "AllOfX" with the above query. It simply made a match to every child of 'X'. The other agent then started their queries:

inside("AllOfX") + the extra query terms needed.

Not as dramatic as the main fix, but this too, improved things a bit.

Back to top
« Last Edit: Oct 27th, 2012, 5:26pm by Mark Anderson »  

Mark Anderson
TB user and Wiki Gardener
aTbRef v6
(TB consulting - email me)
WWW shoantel   IP Logged
Pages: 1
Send Topic Print