145 lines
11 KiB
Plaintext
145 lines
11 KiB
Plaintext
<p>Regular readers will know how much I enjoy a good tinker with my system. I have
|
|
been playing with <a href="https://tiddlywiki.com">Tiddlywiki</a> recently, partly as a result of admiring Jack
|
|
Baty’s <a href="https://rudimentarylathe.wiki/">rudimentarylathe.wiki</a> instance of Tiddlywiki, and partly because of Soren
|
|
Bjornstad’s <a href="https://groktiddlywiki.com/read/">Grok Tiddlywiki book</a>, which I think I also found out about via Jack.
|
|
I had tried out Tiddlywiki before but never quite <em>got</em> it. Soren’s book helped me
|
|
to see how flexible it could be and how I might be able to use it in a similar
|
|
fashion to the way I have been using <a href="https://www.rousette.org.uk/archives/extending-org-roam/">org-roam</a>. While I still enjoy org-roam,
|
|
things feel (to me anyway) a bit up in the air with it at the moment, as there
|
|
are big changes coming in version 2 which will probably involve a bit of
|
|
backwards incompatibility. I couldn’t decide whether to wait to make the
|
|
changes, or transition to the new version now, and that indecision made me
|
|
reluctant to add to my collection of notes. In addition — for reasons too long
|
|
and boring to go into here — I have also moved (reluctantly) from Bookends to
|
|
Zotero. I like the flexibility that Zotero offers to those of us having to live
|
|
in a dual Word/Pandoc citations world, but I really miss Bookends’ speed and UI.</p>
|
|
<p>Anyway, this change in the tools I depend on left me with a puzzle: how could I
|
|
export references (with metadata) from Zotero to Tiddlywiki so that I could make
|
|
notes (known as ‘tiddlers’) on each journal article of interest? There’s a
|
|
vanishingly small possibility that anyone else might want to solve a similar
|
|
problem in exactly the same way as me, but in case anyone is curious, this is
|
|
how I made it work.</p>
|
|
<p>First, a health warning: this is big, messy hack, involving multiple bits of
|
|
software and a lot of the coding equivalent of gaffer tape. It works for me, but
|
|
there’s a lot which is specific to my setup. Nevertheless, I thought it might be
|
|
useful to explain what I did, in case it inspires anyone else to set up their
|
|
own system using bits of this solution. I’ll also indicate what you can do with
|
|
each separate component, in case you want to pick and choose which parts you use.</p>
|
|
<p>The overview of this system is as follows:</p>
|
|
<ul>
|
|
<li>Zotero can export metadata of selected references in various forms using
|
|
Javascript files called ‘translators’. We can therefore export metadata in
|
|
JSON format, making sure that we include the fields that Tiddlywiki expects to
|
|
find in a JSON-format import.</li>
|
|
<li>If you use the <a href="https://tiddlywiki.com/static/TiddlyWiki%2520on%2520Node.js.html">node multi-file setup</a> of Tiddlywiki and run it as a server
|
|
(locally), you can make use of the <a href="https://tiddlywiki.com/prerelease/static/WebServer%2520API%253A%2520Put%2520Tiddler.html">Tiddlywiki API</a> to create tiddlers by
|
|
passing some parameters plus a JSON body, which includes the content of the
|
|
tiddler and its fields.</li>
|
|
<li>The part we need to connect the two is a script of some kind to take the JSON
|
|
exported from Zotero, construct a <code>PUT</code> request and submit it to the API.
|
|
Since I have been learning Go recently, I decided to make a Go script to do
|
|
this, and call that script with Keyboard Maestro to glue together a few other
|
|
useful steps that I will explain below.</li>
|
|
</ul>
|
|
<h3 id="zotero">Zotero</h3>
|
|
<p>First you need a translator file which gets placed in <code>~/Zotero/translators</code>. I
|
|
based mine (which I’ve put in this <a href="https://gist.github.com/bsag/638f3380ac7fba12cb7e1abb5e5f272f">gist</a>) on one I found (<em>somewhere</em>,
|
|
unfortunately I can’t remember where) which also exported JSON. I then looked at
|
|
the JSON format that Tiddlywiki needs for imports and used that to construct the
|
|
correct keys and values. <a href="https://github.com/wshanks/Zutilo">Zutilo</a> is a very useful Zotero plugin that I was using
|
|
anyway, but which offers a way to set up particular translators so that they are
|
|
accessible from a contextual menu or keyboard shortcut. The goal I had with
|
|
getting this pipeline to work was to export the JSON from Zotero to the
|
|
clipboard, then use the macOS <code>pbpaste</code> command to pipe the clipboard contents
|
|
into the Go script. To make this work, you add the <code>translatorID</code> UUID string
|
|
(line 2 of the script) in Zutilo to identify the translator to call for the <code>alt1</code>
|
|
action. This is set up by going to Preferences > Advanced > Config Editor, then
|
|
entering a value for <code>extensions.zutilo.quickCopy_alt1</code> of <code>export=3c2ea…</code> with the
|
|
full UUID string after the equals sign. Then in Tools > Zutlio Preferences you
|
|
can set up a contextual menu item and a hotkey for QuickCopy Alt1 (I used
|
|
<kbd>Cmd+Ctrl+t</kbd>).</p>
|
|
<p>At this point, you should be able to select one or more references in Zotero,
|
|
use your hotkey to trigger the QuickCopy Alt1 command and find that your
|
|
clipboard contains the JSON with the metadata of those references.</p>
|
|
<h3 id="go-script">Go script</h3>
|
|
<p>As I mentioned above, I’m only using Go here because I’m learning it and so it
|
|
was a nice opportunity for a practical application. It does also enable you to
|
|
build a self-contained binary that you can put somewhere in your path so that
|
|
you don’t have to maintain a whole ecosystem of packages and their dependencies.
|
|
However, if you are proficient in Ruby or Python (or shell scripting for that
|
|
matter), it should be perfectly possible to use the Go script in my <a href="https://gist.github.com/bsag/638f3380ac7fba12cb7e1abb5e5f272f">gist</a> as
|
|
model to write a version in your language of choice. I set it up so that the
|
|
compiled binary (which I called <code>twimport</code>) takes the pasted JSON data exported
|
|
using the translator as piped data. It parses this to extract the title and
|
|
structure of each of the items (if more than one was selected), and then constructs
|
|
a <code>PUT</code> request to the Tiddlywiki API to create the item.</p>
|
|
<p>The API requires you to specify the title of the tiddler as one of the request
|
|
parameters. I decided to use the citekey field of the reference in Zotero as my
|
|
title, as (the way I have it set up), it includes the first author’s name, the
|
|
first few words of the title, and year, and I can guarantee that it is unique.
|
|
The translator has this citekey as one of its fields, so the script just grabs
|
|
that to construct the API call. It is also worth noting (as this tripped me up
|
|
initially) that the translator stores the JSON items as an array, even if you
|
|
only select one item. However, the Tiddlywiki API expects a single item and
|
|
throws an error if it gets an array. The script must therefore iterate over the
|
|
array and construct an API request for each item. If you are trying to work out
|
|
how to do this in another language, I heartily recommend trying out <a href="https://paw.cloud">Paw</a> (which
|
|
is available on Setapp). I’ve used it a lot recently when trying to use APIs,
|
|
and it is fantastic. You can try out different parameters and see exactly what
|
|
response you get back. I never know exactly how to format headers or bodies, so
|
|
being able to click around and set it up in the Paw UI makes it so much easier.
|
|
It also enables you to export a request once you have sorted out the correct
|
|
setup (using File > Export Request) in a variety of different languages. This
|
|
was how I got the skeleton of the Go version, which was a huge help.</p>
|
|
<p>If you get this stage working, you should be able to pipe your clipboard
|
|
contents (containing the JSON exported by the translator) into this script to
|
|
trigger Tiddlywiki to create a new tiddler. In my case, I can do this with
|
|
<code>pbpaste | twimport</code> (as <code>twimport</code> is in my path).</p>
|
|
<h3 id="gluing-the-parts-together">Gluing the parts together</h3>
|
|
<p>Now that we can export JSON containing the metadata to the clipboard, and pipe
|
|
that clipboard data to our Go script to create a new tiddler, it would be handy
|
|
to be able to trigger this thing with a keyboard shortcut rather than going to
|
|
the command line. In other words, it’s time to get out the gaffer tape again and
|
|
stick a few more components on to our beautiful mess of code!</p>
|
|
<p>As I own it anyway, and use it for everything, my method of choice here was
|
|
<a href="https://www.keyboardmaestro.com/main/">Keyboard Maestro</a>. This is fairly simple, as it just automates the steps we would
|
|
otherwise perform manually.</p>
|
|
<ol>
|
|
<li>
|
|
<p>We start with Zotero frontmost, with one or more references selected.</p>
|
|
</li>
|
|
<li>
|
|
<p>Keyboard Maestro then triggers the keyboard shortcut to QuickCopy the
|
|
translated reference (in my case <kbd>Cmd+Ctrl+t</kbd> as explained above). That puts
|
|
the JSON in the clipboard.</p>
|
|
</li>
|
|
<li>
|
|
<p>An ‘execute shell script’ action then uses the <code>pbpaste | twimport</code> command to
|
|
run the Go script and create the tiddler(s).</p>
|
|
</li>
|
|
<li>
|
|
<p>I use a single site app to view my Tiddlywiki (using <a href="https://www.bzgapps.com/coherence?lightbox=dataItem-k9x4zon11&ref=producthunt">Coherence X</a>, again
|
|
available on Setapp), to make it feel more like a real macOS app. The final
|
|
step in my macro activates this app, then reloads the page. This is necessary
|
|
because externally adding files to the server version of Tiddlywiki does not
|
|
automatically trigger the interface to refresh and register those files. For
|
|
completeness, I also use the macro to trigger a notification with the name of
|
|
the tiddler we’ve added.</p>
|
|
</li>
|
|
</ol>
|
|
<p>Of course, the node <code>tiddlywiki</code> server needs to be running for the API to work.
|
|
Again, you can just start it at the command line, or you can use <a href="http://undefinedvalue.com/setting-personal-tiddlywiki-server-os-x.html">these
|
|
instructions</a> to create a <code>.plist</code> file to go in <code>~/Library/LaunchAgents</code> so that the
|
|
server starts when you log in.</p>
|
|
<p>Within Tiddlywiki, tiddlers tagged with the ‘Source’ tag (like the ones created
|
|
with this workflow) have a template automatically applied which adds an HTML
|
|
table to the top of the tiddler containing selected fields (such as authors,
|
|
year, title and so on). I also copied the ‘Idea Explorer’ table from Soren’s own
|
|
<a href="https://zettelkasten.sorenbjornstad.com">Zettelkasten</a>, which automatically summarises and links to links and backlinks from the
|
|
current tiddler.</p>
|
|
<p>I’m enjoying this setup. It started as an experiment driven by curiosity more
|
|
than anything, but I’m <em>enjoying</em> taking notes and making links between ideas with
|
|
Tiddlywiki, which means that I’m writing notes more often. Tiddlywiki also
|
|
enables a lot of flexibility in the way that tiddlers can be searched, grouped
|
|
and displayed, which means that if I decide in the future that I need to extract
|
|
particular information from my notes, I can do so easily.</p> |