Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
Tinderbox Users >> Tinderbox applications >> TB vs File System

Message started by Simon on Jun 23rd, 2014, 5:52pm

Title: TB vs File System
Post by Simon on Jun 23rd, 2014, 5:52pm

In a discussion at http://www.eastgate.com/Tinderbox/forum//YaBB.pl?num=1388249314/0 Mark Anderson encouraged me to look a little deeper at the design of what I'm looking to achieve. This questions some of my current practices.

At the moment all my files (including notes) are written in (Multi)Markdown, saved using a descriptive title (yymmdd-category-name) then tagged and placed in a 30 day folder. Hazel (http://www.noodlesoft.com) then moves the notes if they have not been modified in the last 30 days into a filing folder, which files them by date into a folder hierarchy thus YYYY/Q1/ (through Q4). This means all my notes are chronologically stored. They can be easily found with Houdaspot (http://houdah.com/houdahSpot/) or Leap (http://www.ironicsoftware.com/category/leap/). This system has worked well for me for the past 4 years, although I must admit that notes on subjects don't fit well in this system because they are only chronological in terms of creation date not in terms of use. They may also be needed at any point. Tags has helped in finding this stuff.

Here's my problem. I have thousands of documents (.doc/.docx) with sermons, teaching, study, courses, literally thousands of pages of information that is sitting not being used, because it spread in different folders and files (most of this comes from stuff I did pre 2000). I'd like this in some usable framework. Voodoopad was a nice idea, but just didn't work for me. TB gives me the impression it could do it. However, before I start on this mammoth task I need to be assured it can.

Most likely I will start only using freshly created stuff and adding old stuff that is relevant. I would like all this in a single TB or if I can link TBX's it can be spread about. I'd like notes to be short and specific although some will run into thousands of words referencing other notes. The whole point will be that I can find everything I have on a specific topic in one place and see how it relates to other topics/subjects.

Originally I was going to do this with multimarkdown files well named and tagged in finder. Is TB a better option? If so, where do I start and has anyone done something similar that I can learn from?

Title: Re: TB vs File System
Post by James Regan on Jun 23rd, 2014, 7:49pm

Have you ever considered using DevonThink Pro Office?

Title: Re: TB vs File System
Post by Simon on Jun 24th, 2014, 2:50am

Yes, I have DTPO. The problem is that it's not a good note taker. It deals in files. For doing what I want it seems to be overkill. Although this would be possible.

Title: Re: TB vs File System
Post by David Bertenshaw on Jun 24th, 2014, 4:01am

On the face of it, your situation is made for DevonThink for storing your data and notes, supplemented by TBX for analysing the data, and if you prefer, taking the notes themselves.

DTP will easily store all your documents and provide the search and serendipity features you're looking for: auto classification, related documents, concordance and so on. There are various strategies for this (e.g. importing everything into the DTPO database, or just 'indexing' your documents so that they remain outside the database but with their content read by the AI) but many users have large (several Gigabytes) databases without issue. It's certainly not overkill for this role -- it's what it was designed to do.

Actually, DTPO's AI works better with lots of small notes, and as in daily use you never see the fact that each note is a separate file, I personally don't think this would be a problem. The quick note capture and clipping facilities from other programs are very useful, too. Yes, the standard Apple text editor in DTP is clunky compared to a decent editor, but it's the same one that is now used in Tinderbox 6. DTP also has native support now for markdown files so you don't need to use RTF if you don't want to.

Where TBX shines is the ability to display your data visually (impossible in DTPO) and automate and manipulate the data through attributes and agents, so what many people do is maintain everything in DTPO, and then when the time comes to work on a subject, use its excellent AI to gather together relevant documents, then either export the suitable ones to TBX, or just work from both programs at once.

You don't have to write new notes in DTPO if you don't want to, of course. You could, for example, continue to take them in TBX and export them to DTPO for long term reference, or just index the TBX files themselves in DTPO.

Unfortunately, getting notes between DTPO and TBX isn't as smooth as it could be, but it is possible and there are several posts on this forum giving ways of doing so. E.g. you can export a set of DTPO notes to a plain text file, complete with tags and other metadata (including a reference back to the original DTPO note), which can be imported into TBX, exploded into individual notes with their attributes filled automatically. The reference back to the original DTPO means of course that it is available at a click from TBX.

Title: Re: TB vs File System
Post by Mark Anderson on Jun 24th, 2014, 4:15am

From the first above, I don't think you problem is finding stuff so much as figuring out the workflow for use. For instance, Markdown makes is easy (for the few who know MD) to set styles into plain text. But, if you want to search the text, unless you've an MD adware search tool it will see all your MD mark-up which may lead to incorrect results. That's no a fault of the tool, but rather a flawed assumption on usage.

TB can certainly handle lots of notes (0,000s of notes). BY judicious use of attributes you don't need to force things like dates into title simply for Finder-style sorting. Clearer titles enable less cluttered use. Notes can link to files (these are fixed links but you could also link to a finder alias of a file). So if you have 5,000 items, you could have a note for each as well as supporting notes about them - or other topics with no file assets. In fact, a note can link to more than one file**. Those notes can be scored as you like and agent can re-query the data to show all notes matching a topic, without affecting the content or location of the original note.

** you need a File-type attribute per linked file in a note. so if your TBX has $File and 4 user File-type attribute, any note can link to up to 5 discrete files, etc.

I think questions like "Does tool X work?" or "is export method Y the best?" are moot until the user has a clear understanding of (a) what they expect the process to deliver and (b) any 'sacred cows'. By (b) I refer to a self-imposed desire for a certain naming style or scripting method, etc., such that it may make otherwise attractive workflows unusable.

Title: Re: TB vs File System
Post by Simon on Jun 24th, 2014, 5:19am

@David. Many thanks for the response. I can see how that could work. I think my reticence with DTPO is that I've had it for years but never really used it. It's also not a programme I enjoy using and sadly this is important for me, especially something I need to use every day. TB does win on that as does its customisable querying engine.

@Mark. I think you've hit the nail on the head. It's my workflow that is an issue. I'm a text worker and realise that I can actually achieve a great deal in TB. It does require some work though.

By wiki I was hoping for some form of [linked word] system in the text, but imagine this is not available in TB?

I like the way I can set my own attributes and my own queries. This really pulls me towards TB.

I'm hoping to slowly remove the doc/docx files and move all the text into plain text files.

How powerful is the TB query? Is it possible to search for a particular word and show not just all the notes with the word in but pull the the line with the search term. Rather a bit like a concordance. I find with text that cycling through the text jumping to each mention of the word extremely unhelpful. I'm more after a index of lines with the word in to give context to the word.

Title: Re: TB vs File System
Post by David Bertenshaw on Jun 24th, 2014, 5:41am

Simon, I understand your issue with DTP -- it does have some rough edges that can grate, although it has been improving over the years. I tend not to work in it every day but just keep it open and dump everything that could be of use into it for later.

Have you looked into DevonSphere Express (on the App Store for about 7)? It's essentially the AI of DTP but without the interface, so it could help with getting the best out of your existing data.

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.