Keeping a lab notebook with org-mode, git, Papers, and Pandoc (Part II)

I promised a while ago that I’d post the code I used to make org-mode a powerful lab notebook, and here it is. In Step 1, I present the emacs lisp for the lab-notebook minor mode. In Step 2, I show you how to leverage Papers to work with Emacs and Pandoc. In Step 3, I show you how to hook Pandoc up with Emacs.

As I mentioned previously, these are all hacked-together solutions. My requirements and setup will likely be different from yours, and I have probably forgotten a few key steps. For instance, while I use a Mac, the only parts that are specific to Macs are the hooks with Papers, a reference library application. Emacs, Pandoc, and bibtex are all fairly cross-platform solutions, I think.

Step 1: lab-notebook mode in emacs

The code in the gist below provides a minor-mode for emacs that enables checkin-on-save and auto-export from Papers (using the Applescript shown in Step 2). Load this in your .emacs file and initiate by M-x lab-notebook-mode, or add -*- eval: (lab-notebook-mode) -*- to the top of your notebook file.

Step 2: Papers auto-export and citekeys

Papers comes with a Citations tool that allows you to insert a citekey in any text field, including Emacs. A citekey is a short, unique reference to a publication or entry in your reference library, used by tools to automatically format your citations upon printing or export. To use, summon Citations using whatever keybinding you chose in Papers’ preferences and add a citekey like you would in a Word document. In our setup, we’ll be using Pandoc, so be sure to change the citekey type to ‘Pandoc’ (Papers -> Preferences -> Citations -> Format: Pandoc Citation).

Pandoc doesn’t talk to Papers, but Papers can export its library in a format that Pandoc will understand, called bibtex. We will want this to happen automatically before Pandoc attempts to render our notebook. The applescript below exports my Papers library as a bibtex file in the location specified by “path_to_notebook_folder” (replace with whatever you want). Save this file in the same location (or wherever) and update the part of the Gist above that starts (shell-command "osascript ... with the right path and name that matches whatever you saved it as.

The benefit of adding this into Emacs is that you can either press a key (by default, F5) to automatically keep your bibtex file up-to-date, or add a hook so that when you render your notebook using Pandoc, it automatically updates your bibtex file first before trying to build your citations.

Step 3: Use ox-pandoc in emacs to export a pretty version of your notebook.

While org-mode does include its own LaTeX and HTML export commands, I found it easier to use Pandoc and the ox-pandoc Emacs package. The benefits of Pandoc is that it exports to a much wider variety of formats and has a simple and easy citations manager, pandoc-citeproc. Citeproc can use the bibtex library we exported in Step 2- all we have to do is add the following somewhere in our notebook org file:
#+PANDOC_OPTIONS: bibliography:library.bib
Make sure there is a library.bib file in the same folder as your notebook file, of course. Now, to export to Latex with formatted references, simply use M-x org-pandoc-export-to-latex-pdf and it will produce a beautiful PDF copy of your notebook. Note: in case it wasn’t clear, you will need to install Pandoc and ox-pandoc since they are not built in. Only recent versions of Pandoc handle org-mode files, so make sure you’re up-to-date. ox-pandoc is available on the Emacs package repo Marmelade.

Example file

An example file. Since Github automatically renders org-mode files when they’re displayed, this is a link to the raw source so you can see the markup used.

Pandoc can export to Github-flavored markdown, so here is the same file rendered:

Further reading


3 thoughts on “Keeping a lab notebook with org-mode, git, Papers, and Pandoc (Part II)

  1. Thanks for sharing this. I like the way you made a minor mode for the save hook. Seems pretty simple.
    However, you might want to take a look at Firestarter[1], which is an Emacs package that: “lets you execute all kinds of things after saving a buffer. You can run shell commands, execute interactive Emacs commands and evaluate arbitrary pieces of Emacs Lisp. Support for shell commands includes asynchronous process control, format chars in command strings and report of process on common process termination states.”


  2. Thanks for sharing this! For a while I was struggling with how to connect my lab notebook in Emacs with my papers in Mendeley. This also would have been super useful when I kept a reading list for qualifying exams in Org mode. I was able to implement your idea using the citation key in Mendeley (instead of Papers) on Linux.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s