Zotero and DEVONThink

I have a bibliographic database in Zotero. The citation information is easy to scrape from web databases such as my own library, Amazon, and the many journal databases that I use. It is convenient to be able to tag and organize my sources and can use Zotero to format footnotes and bibliographies for my dissertation and other papers.

I recently shifted my note taking to a knowledge database called DEVONthink and as I recently discussed here, I created a special template script to add individual note files (like note cards in the old days) for each fragment or note I take on a source and keep them in a folder with, and linked to a main overview note file for the source. Each fragment created duplicates the tags of the main note file.

Automating a Zotero to DEVONthink Workflow with Applescript

I thought it be nice to automate the creation of a folder and a main note file in DEVONthink for each and every source I have in my Zotero database with a script. I also thought it would be nice if the script brought over any and all tags the source had in Zotero, including tags for whatever collections the source was found in. Finally, I wanted the script to only create folders and note files for those sources that I did not already have a folder for in DEVONthink so that I can run the script every once in a while to keep it synced with my Zotero without too much difficulty.

I created such a script this evening and it can be downloaded here:

Zotero to DEVONthink (Version 1.6 2010.10.16)

To use it:

1. Download and unzip the script

2. Open it in the Applescript Editor and edit the two configuration variables (the name of the group you want to put all the source folders and note files in, and the location of your Zotero database)

3. Put a copy of the saved script into your DEVONthink scripts folder and run it every time you want to import Zotero all your entries or, thereafter, check if there are new sources to be added.

4. Please read my notes in the script for more details on how it works and some things to keep in mind.

Note to developers who can do better:

My script accomplishes its task by using sqlite3 shell commands to directly query the Zotero database. While this is read only and seems to work fine, this is not the most graceful way of going about this.

Zotero has an open API and a developer who is less of an amateur than I could probably think of some effective way of using applescript and perhaps a combination of something else like a Firefox plugin to talk to this API directly in Firefox and get information out of the database that way, which is recommended by the Zotero team. Read more here, here, and here.

Other possible future improvements that I think could be made to this script or some combination of it and a Firefox plugin if someone has the time to work on it. I unfortunately don’t have any time to work on any of these. Please let me know if you add such features and I’ll post updates to the script:

1. If Zotero has any attachments, such as PDFs of the sources or snapshots for the webpages, the script could be improved so that these could be imported along with the entry.

2. It would be great if a formatted bibliographic entry for the source was added to the main note file created in DEVONthink. Currently this must be done by hand by dragging and dropping citation into the note file.

3. Any notes already in the Zotero entry for a source should be added to the main note file created in DEVONthink.

4. Ideally, the script would recreate the collections structure found in Zotero within DEVONthink.

5. Ideally, the script would check to see if any tags have been added or deleted from Zotero and not only add tags on the first import of the source.

6. Ideally, the script would keep track of itemID info for each entries and use that to judge whether an entry has already been imported into DEVONthink. That way the user can shorten or edit the folder titles etc. in DEVONthink without the script re-importing that entry because it doesn’t find an exact match by title.

7. Ideally, the script or a script and a plugin would somehow eliminate the need to be run – that somehow every time I added a new source to Zotero, DEVONthink would automatically get updated.

UPDATE 1.1

-I got rid of a “display dialog” command leftover from my last minute debugginge
-I added a check to see if Firefox is running, and gives the user the opportunity to quit Firefox, otherwise the script cannot run since the Zotero database is locked.

UPDATE 1.5

-The script now puts all the sources in a sub group “_All” (option to change the name) and then recreates the collection group hierarchy you had in Zotero. (Note: if you move or change the hierarchy of folders in Zotero and run the script again later, it will not delete the old groups or move them)

-I added a few more configurations, an option to control whether you are warned that Firefox will be quit, an option to add the author name to the name of the group/note file created

-Firefox is automatically restarted at the end of the script

UPDATE 1.6

-The script now supports logging. Every new group and file created will log an entry in a log file. This can be turned off and the location of the log file can be customized in the configuration section of the script.

UPDATE:

There are some problems with the way it handles unusual titles of books and although the script works fine for me, I have seen some comments saying others are having trouble. Please post your comments on the script at the DEVONthink thread for the script where I hope we can get some help from more skilled scripters who might have time to work more on this:

DEVONthink and Zotero – DEVONthink Forum

6 thoughts on “Zotero and DEVONThink”

  1. Thanks! Can’t wait to try this. Integration of Zotero with DEVONThink would be scholarly nirvana!

    I find DT to be simply amazing. It finds so many things, and the previews of all those various file formats is so darned helpful.

    BUT I sure miss having my Zotero data (refs, abstracts, notes) in the DT database.

    My cheat has been to periodically create a Report of all my refs in the Zdb. But I have nearly 2K refs now, so that is a big job and file…

    BTW, I’ve given up on Collections. I find that I wasn’t faithfully updating the allocation of refs to them. And the Zotero Find feature is so good anyway that I really didn’t feel the need.

    Appreciate your effort and generosity.

    Don
    PhD student (proposal stage)
    Human Development

  2. This is a terrific idea — and should, in my opinion, be integrated into DevonThink — but there seems to be a problem in the 1.6 version of the script. I’ve got it configured and running without errors, but it stops after importing the title of the first record. It looks like a colon is causing the problem — the title of the record is “An Introduction to Design and Culture: 1900 to the Present” and I’m getting one group and one record, both titled “An Introduction to Design and Culture”. The log file reads as follows:

    CREATED group: /From Zotero 2/_All/An Introduction to Design and Culture/
    CREATED file: /From Zotero 2/_All/An Introduction to Design and Culture/An Introduction to Design and Culture

    and that’s it.

    Any ideas on where to look for a solution? I’m not an AppleScript expert but my guess is that a delimiter or string type needs to be tweaked.

    Thanks a million! I’m very excited to get this working.

  3. Hi – I’m having the same problem as Jonathan above. Thanks for offering this, will be a great help.

    Jon

  4. Thanks – I have had reports it doesn’t handle a few characters well at all, such as colons. This should be relatively easy to fix in the script but I am a bit short of time at the moment.

  5. Hi, I’m told that the script’s been saved in “read only” mode, so I can’t open it in Editor in order to change the variables. What am I doing wrong?
    Thanks!

  6. Hello everyone.

    Although the script continues to work fine for me, there appears to be a few minor problems with the way it handles some titles. I would love to have someone more skilled in AppleScript give the script a good going over.

    Please post further comments over on the DEVONthink forum posting for this, where I have pasted a full text version of the script and hopefully we can catch the attention of some of the skilled scripters there:

    http://www.devon-technologies.com/scripts/userforum/viewtopic.php?f=20&t=11502

    Lucas: In addition to finding a text version at the above posting, the downloaded script can be opened if you open it from within AppleScript editor or drag/drop it on AppleScript editor, as opposed to double clicking it.

Comments are closed.