Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com)
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi
Tinderbox Users >> Exporting from Tinderbox >> run script on export
http://www.eastgate.com/Tinderbox/forum//YaBB.cgi?num=1348225664

Message started by Derek Van Ittersum on Sep 21st, 2012, 7:07am

Title: run script on export
Post by Derek Van Ittersum on Sep 21st, 2012, 7:07am

Hi all,

I'm trying to work way above my expertise and understanding, so you'll have to forgive me if I'm not able to express my goals as clearly as necessary.

Basically, here's what I'd like to do:

In my syllabus TBX file (shared he before), I have agents that collect reading notes and lesson plans and then an export template that spits the whole thing out for me to print nicely and use as notes in class.

The template is an envelope-style one. The agent gets this template:


Code:
^value($DisplayName)^
^text^
^if(^get(ChildCount)^)^
^children(LessonPlanItem)^


While the "LessonPlanItem" merely consists of
Code:
^text^


This results in me having to open up the export file in a word processor and apply light formatting.

I've been experimenting with running the whole output through Pandoc (essentially similar to a Markdown script). I couldn't figure out how to do that, mostly because I'm not really all that proficient at command line stuff--I just cut and paste from examples and cross my fingers!

I did, however, manage to cobble together a solution via Automator. I created a workflow that:

1. Gets specified finder file (the export file created by TBX)
2. Runs shell script:

Code:
PATH=$HOME/.cabal/bin:/usr/local/bin:/usr/texbin:$PATH

for file in "$@"
do
   output=${file%%.*}.docx
   pandoc --smart "$file" -o "$output"
done

echo "$output"

PS: I copied this code from elsewhere, so I only have a limited understanding of how or why it works. I understand all the Pandoc options (e.g., --smart sets all the typography to smart quotes) but not the scripting.
3. Opens Finder item (which results in MS Word opening the newly created "export file.docx" in Word.

This lets me apply markdown formatting in TBX and end up with a properly formatted Word file--no more manual messing with things. I'm guessing I may have been able to create properly formatted HTML pages all along, but at this point I'm more comfortable with the whole Markdown-->Word processor workflow.

At any rate, I'm wondering whether I've added an unnecessary step by using Automator. Is there some way to call the Pandoc process through the runCommand and include that in the template itself, so that TBX itself will spit out the docx file?

Title: Re: run script on export
Post by Mark Anderson on Sep 21st, 2012, 8:11am

I'm not a command line expert, but...  As you're export is one block of text, why not use $HTMLExportCommand. That runs a script in the exported data - IOW the data being written to disk after all ^export^ code has been run.

You could try pasting your shell script code into the exporting note's $HTMLExportCommand or, better, use a code note to hold the script and use the note's rule to set the $HTMLExportCommand for the exporting note.

Title: Re: run script on export
Post by Mark Bernstein on Sep 22nd, 2012, 3:21am

$HTMLExportCommand contains a string that specifies a command line to be applied to your exported file.  So this will do exactly what you want.  (Sorry to be terse; this internet connection in Kolkata times out quickly!)

Title: Re: run script on export
Post by Derek Van Ittersum on Sep 24th, 2012, 4:42pm

Ok, so I've not been able to successfully get my command to run through with $HTMLExportCommand, so I'm hoping for a bit more guidance.

Here's what I've done:

1. Created an agent that collects certain notes.

2. Created export templates (envelope/letter) that exports the $Text of the agent followed by the $Text of any child notes (in one single file).

3. Turned off HTML markup for the agent and the notes, so that plain text is spit out (best for Pandoc/markdown).

4. Set $HTMLExportCommand to:

pandoc --smart test.txt -o test.docx

5. Opened the HTML export window. Set file name and extension to spit out a test.txt file

6. Press export

What I get is an empty test.txt file (zero bytes).

I tried wrapping the "pandoc ..." in quotes, to no visible effect.

The command works in the terminal (i.e. it creates the appropriate docx file if I use the nakakoji view to create the appropriate test.txt file).

Am I formatting the command wrong or missing some other attribute to tweak?

Thanks!

Title: Re: run script on export
Post by Mark Anderson on Sep 24th, 2012, 4:53pm

Your CL is expecting an input file 'test.txt' that hasn't yet been created. The $HTMLExportCommand runs after all other activity and just before the output file is written.

I think you need to re-write your CL to use the shell's stdin as the input instead of 'test.txt'. Perhaps one of the more CL-savvy folk here can fill in the gaps?

Title: Re: run script on export
Post by Derek Van Ittersum on Sep 25th, 2012, 5:35am

Ok. It seems that Pandoc accepts stdin without any fancy commands:


Quote:
If no input-file is specified, input is read from stdin. Otherwise, the input-files are concatenated (with a blank line between each) and used as input. Output goes to stdout by default (though output to stdout is disabled for the odt, docx, and epub output formats). For output to a file, use the -o option:

pandoc -o output.html


So, I updated my $HTMLExportCommand, but still getting the empty file on export and not the docx pandoc result.

Here's what I put in there:  pandoc --smart -o test.docx

Title: Re: run script on export
Post by Mark Anderson on Sep 25th, 2012, 7:42am

My hunch (I'm outside my expertise area here) is that pandoc may be installed in a location/configuration such that when called from TB, it's not recognised.  IIRC, the pwd (current directory) for TB CL calls is your account's home directory (i.e. ~). Using Terminal, from your home directory can you run a pandoc test successfully? What you're trying to confirm is that if pandoc is called from your ~ directory that it can read (specimen) stdin. If that works, your code should work from TB.

My experience with other open-source CL tools is the creator assumes users have the necessary knowledge re file names/locations/permissions. For instance, you might need to include the path to pandoc in your CL. From your quote it looks like there's a pandoc mailing list. If so that might also be a good place to ask.

(Side note, Mark B's currently at a conference in India & on a slow/flaky connection for a few days, but it might be a good idea to folow up via support)

Title: Re: run script on export
Post by Mark Bernstein on Sep 25th, 2012, 9:44am

Try replacing "pandoc" with the complete path to the pandoc application (e.g. /usr/bin/pandoc)

Title: Re: run script on export
Post by Derek Van Ittersum on Sep 25th, 2012, 9:19pm

On my machine its /usr/local/bin/pandoc, but same result (zero kb file) with:

/usr/local/bin/pandoc --smart -o test.docx

Will follow up with support sometime in the near future. Not urgent. Just playing. Could be useful for future project though.

Title: Re: run script on export
Post by Mark Bernstein on Sep 27th, 2012, 9:06pm

Doesn't the flag

    -o test.docx

ask pandoc to save its output to the fild "test.docx"?  You want to save the output to the standard output, stdout.

And the ".docx" extension is a Microsoft Office format, which is unlikely to be right....

Title: Re: run script on export
Post by Mark Anderson on Sep 28th, 2012, 4:16am

pandoc's documentation (I don't have the app itself) indicates that if you need to use stdout - e.g. in the $HTMLExportCommand context - you can't use DOCX ouptut format as that doesn't support stdout:


Quote:
-o FILE, --output=FILE
Write output to FILE instead of stdout. If FILE is -, output will go to stdout. (Exception: if the output format is odt, docx, or epub, output to stdout is disabled.)


If the aim is an MS Word DOCX document you may need to use pandoc after exporting from TB, as you were doing previously. I may be misunderstanding the docs, so I'd suggest asking on pandocs mailing list. The possible ambiguity is whether you can get DOCX formatted output via stdout (it may be, for instance, that you can but can't specify the file name/path).

Neither TB not pandoc are doing anything wrong here, it's just that their design assumptions don't mesh. If you set your syllabus TBX(s) to export to a particular folder, you could set that OS folder up with a folder script or droplet making the running of the pandoc transform fairly simple to trigger.

Title: Re: run script on export
Post by Mark Anderson on Sep 28th, 2012, 6:33am

Why specifically is DOCX format needed? Would another Word-compatible   format be OK, such as RTF?  For instance, via pandoc docs alone, I figure this should work inline during export, assuming your export templates are exporting markdown without HTML:

pandoc -f markdown -t rtf -S

The '-S' is for curly quotes instead of straight ones. FWIW, some of pandoc's main assumptions:
  • If using stdin for input, the input format is considered html.
  • Input format can be specified via the -f flag.
  • If an input file is specified without an -f flag, input format is guessed from input file's extension.
  • If using stdout, the default output format is markdown.
  • Output format can be specified via the -t flag.
  • If a file output is specified without a -t flag, the output format is guessed from the output file's extension.
  • For reasons not explained in the documentation, the odt, docx and epub ouptut formats can not be sent to stdout.

Title: Re: run script on export
Post by Derek Van Ittersum on Sep 28th, 2012, 6:39am

Aha. This looks to be the problem. I'll investigate more. DOCX is not really necessary, but it let me set a reference file style sheet to get things styled up automatically. Not sure if RTF supports that. Will continue to look into it and report back.


Quote:
If you set your syllabus TBX(s) to export to a particular folder, you could set that OS folder up with a folder script or droplet making the running of the pandoc transform fairly simple to trigger.


This seems like the best advice and easiest to implement, given the weird restrictions from Pandoc on DOCX or ODT, which both support reference style sheets.

Title: Re: run script on export
Post by Mark Cubberley on Jul 15th, 2014, 12:17pm

I checked and Derek was never able to run pandoc via TBX.  I'm not sure why this won't work.

Let's assume that we want to convert a single note with Markdown into a pdf.  I have been trying this with two notes in TBX: one note containing text with Markdown and one note serving as a template.  The template has two attributes: IsTemplate and HTMLExportCommand. (Does the note need anything else?)

If anyone's interested, what next?

Title: Re: run script on export
Post by Mark Anderson on Jul 15th, 2014, 12:54pm

If I recall $HTMLExportCommand runs on the note being exported - IOW setting it a template has no effect. If you want lots of notes to use the same command, seed it via a prototype.

A good while back, in v5, I needed to use a complex sed call in an export command.  As you only get a small input box in Get Info of as a ket attribute, i made a code note and made that not'e rule set the other note's $HTMLExportCommand.

As for the wider pandoc process, I'm afraid it's a rabbit hole I've not the spare time to explore for a while - in case you were hoping - but likely someone else with a need will step forward. No antipathy, just not enough time.

Tinderbox User-to-User Forum (for formal tech support please email: info@eastgate.com) » Powered by YaBB 2.2.1!
YaBB © 2000-2008. All Rights Reserved.