“A nearly impenetrable thicket of geekitude…”

Drupal and Nanoc

Posted on December 20, 2017 at 21:01

I started using Drupal to manage most of this site a little under six years ago. That wasn’t a mistake — it solved the problems I wanted to solve at the time — but it hasn’t been an unqualified success either. It’s time to move on to the next thing, which for me looks like Nanoc, a static-site generator written in Ruby.

The Drupal Experiment

It’s probably worth summarising what I learned from the experiment of migrating the majority of this site to Drupal 7 back in 2011–2012.

Drupal’s big attraction at the time was that it allowed me to consolidate a variety of different systems: principally Movable Type for the two blogs Technology Stir Fry and iay@there, and Adobe GoLive for a lot of the more conventional “web site” content.

  • Drupal allowed me to apply a consistent style across my blog and non-blog content.

  • I didn’t like being dependent on dead tools: GoLive had been discontinued in 2009.

  • Changing content layout and structure was very easy with Drupal; it has a very impressive URL management system internally. For example, it allows you to move items around and have references automatically redirected.

  • With a plugin, Drupal supported Markdown syntax for all content; I had already started using Markdown heavily in other work.

The conversion wasn’t without problems, but it went reasonably well. One aspect of the way Drupal works which helped out a lot was that by default (via its .htaccess configuration) it ignores all URLs for which files already exist. This means you can migrate to (or from) Drupal by combining it with static content.

The problems with Drupal, which over time convinced me that I should move away from it for this site, include the following:

  • Drupal is a huge collection of software written in PHP, a language that is not exactly famed for the ease by which it can be used to write secure software. There have been a lot of security problems with Drupal since I adopted it. That’s not necessarily bad if you’re a corporation with dedicated staff doing software maintenance; for a one-person consultancy, it means a lot of patching.

    One security event in particular (SA-CORE-2014-005) was so serious that the follow-up PSA warned deployers as follows:

    You should proceed under the assumption that every Drupal 7 website was compromised unless updated or patched before Oct 15th, 11pm UTC, that is 7 hours after the announcement.

  • I’m running Drupal in a shared hosting environment. It’s a large, complex system written in a scripting language, making multiple requests to a back-end database to service each page. This makes it pretty slow a lot of the time, with Google’s webmaster tools reporting average page retrieval times in the half to one second range. Again, probably not a problem in a corporate environment where you can throw money at the system until you reach your performance goals.

  • You work with Drupal on-line: the advantage of this is that you can easily collaborate with others in your organisation; the disadvantage is that its preview tools are quite primitive and, again, may be very slow in a shared hosting environment. My solution to this has been to compose significant content (like this article) in an external Markdown-capable editor. Today, for example, I’m composing in Atom and previewing in Marked.

    There is of course little or no advantage to being able to collaborate if you’re working on a single-person project like this one.

  • There’s no real way to experiment with things like new layouts and styles without setting up a duplicate environment, including a copy of the database.

So, if I’ve had all these problems using Drupal, what does a modern solution to them look like?

Static-site Generators

I am far from the first or only person to have come to these conclusions, which means that by now there’s a whole category of alternatives: static-site generators.

A static-site generator renders a collection of source files into a collection of files suitable for uploading to a web server. These days, of course, it’s natural to manage those source files in something like a Git repository; most of them are just text, after all.

This rendering process takes place on the site owner’s machines independent of the web server; the software involved does not need to be exposed to the horrors of the public Internet. This reduces the attack surface of such a static web site to the web server software itself. This arrangement is therefore much simpler to secure.

Because all the web server is doing is delivering static resources, page delivery is very fast compared to a dynamic site using something like Drupal, even on relatively under-resourced shared web hosting systems.

Composing and previewing individual documents works pretty much the same way as it does in practice with Drupal: just use Markdown-aware editors and previewers on the local files. Most static-site generators also include a local web server and can dynamically generate a local preview of the whole site while you’re composing.

If you want to experiment with styles or radically change layouts, you can branch your repository and play around with the local live preview, merging back into your master branch only when you’re happy that everything is the way you want it.

Of course, there are downsides to this approach:

  • At least potentially, the whole site needs to be re-rendered every time you change any part of it. If you’re planning on building a really large site, you’d want to benchmark this before committing to a static-site strategy. For my situation, though, it’s not a real concern.

  • If you want to do complicated things with layouts or styles, you may find that you end up needing to write code rather than just drag boxes around as you might with something like Drupal. This isn’t something that is likely to stop me (this site’s subtitle is, after all, “an impenetrable thicket of geekitude…“ for a reason) but your mileage may well vary.

  • A related point is that most static-site generators are written in languages I don’t know well, or at least am far from current with. The last time I used Ruby in anger was in 2006, for example. This isn’t a particularly big issue: learning new languages is something that I have always rather enjoyed.

  • Collaboration is much more difficult; you’d need to treat a site in the same way that you’d treat a multi-developer software project. As mentioned above, this isn’t something I personally need to worry about.

  • Truly dynamic content — specifically allowing comments — isn’t really possible directly. Although you can bolt on things like Disqus to address this, I won’t be going down this route. Although the idea of commenting on blogs still seem to be working for some sites (registration and a lot of work put into moderation both help), for many others it looks like a failed experiment in a world of universal trolls and spammers. In my case, I was forced to give up and disable comments some years ago.

  • If you’re an inveterate content tweaker like me, the time from noticing that you should really change the punctuation on that old article to seeing that change on the live site is a lot higher than with a conventional CMS.

Nanoc

There are more than 200 static-site generators listed at StaticGen at the time of writing in December 2017. Obviously, I didn’t perform an in-depth analysis of each of them before making my selection. I didn’t pick blindly either, though: I have been involved in a few static-site projects recently and had performed smaller-scale surveys as part of those.

The two generators I considered seriously for my own site were Jekyll and Nanoc:

  • I have used Jekyll for a couple of very simple static sites, and it seems to be a pretty good alternative for a site which is basically, or entirely, a blog. It has a fairly good collection of plugins and themes even before getting to the “write some Ruby” level of customisation, so as long as what you are trying to do fits within those boundaries it is very quick to get started with.

  • I have only used Nanoc for one project before this. Nanoc is not so “blog-aware” as something like Jekyll, and it does not have as many plugins or any themes. On the other hand, its pattern-based Rules configuration make it very suitable to the kind of stitched-together Frankenstein’s monster of a web site that this one has become over the years. Source files in different parts of the source tree can for example make use of different processing filters and layout templates.

Nanoc is therefore my pick for this site. I’ll probably still use Jekyll for some simpler projects where the extra flexibility isn’t required.

The Nanoc Experiment

I mentioned earlier that Drupal has one property which allows it to co-exist with static content:

One aspect of the way Drupal works which helped out a lot was that you can configure it (through the .htaccess file) to ignore all URLs for which files exist already. This means you can migrate to (or from) Drupal by combining it with static content.

This means that migrating to Nanoc doesn’t have to be done all at once. It also means that it was relatively simple to test Nanoc before adopting it by replacing just one part of the site with a static equivalent.

The part I chose, as described in this blog entry, was the There technical wiki. Although this was previously managed by an instance of the Usemod wiki software, it hasn’t seen a content update in quite a few years so was essentially static already. It also had the advantage that the wiki had its own CSS stylesheet, so I could just reuse that.

Extracting the content from the Usemod database and converting it to Markdown turned out to be the majority of the work involved in this: Usemod’s markup has some pretty interesting differences in the ways that you represent things and link pages together. I started with someone else’s Usemod to Jekyll script, tweaked that until the results looked decent, then worked through the pages manually.

The way Usemod handles naming and accessing pages, and redirections, is quite complicated and explicitly dynamic through the use of query parameters. I’ve handled that through the Apache server’s mod_rewrite module and an .htaccess file containing things like this:

#
# Old URLs looked like this:
#    /there/wiki/wiki.pl?PageName
# Now that looks like:
#    /there/wiki/PageName/
#
RewriteCond %{QUERY_STRING} ^(.+)$
RewriteRule "^wiki\.pl$" "/there/wiki/%1/?" [Last,Redirect=301]

Redirect 301 /there/wiki/Animob/ /there/wiki/animob/
Redirect 301 /there/wiki/Animobs/ /there/wiki/animob/

This is not what I’d like a statically generated site to look like; for example, it relies on an Apache web server and mod_rewrite, which means that it doesn’t work in my development and preview environments as they are fronted by something else. As with so many other things, though, it seems like a reasonable pragmatic compromise to allow people who have bookmarked URLs on my site to have those bookmarks continue working. New visitors to the site don’t need to know what the URL system used to be.

What’s Next?

I’m not quite ready to move any of the substantial content currently managed by Drupal. Before I do that, I need to develop CSS and templates to fit in with the Drupal part of the site. I’ll apply that to the There wiki first before moving on to any real Drupal content.

When I do start on the content currently held in Drupal, the order will probably be something like this:

  • Convert a few old blog entries (probably starting with this one) so that I can verify that overlaying content works in a more general context than with the isolated wiki pages.

  • Convert some of the site’s non-blog static content that is merely managed by Drupal at present, such as the about page and the colophon.

  • Migrate the iay@there blog, which isn’t going to see any more updates which might complicate the process.

  • Convert the main blog.

It’s not obvious to me whether this process will include things like RSS feeds or not. I’m not sure there are enough subscribers to that at present to make it worthwhile.

Tags: