Sometimes I play around with Tails and on rare occasions I also build a Tails image myself. One thing that makes the build of Tails a bit tedious is that it a also builds the Tails Website, which contains the whole documentation (which is really cool, because that way users have the most up to date documentation on their desktop!). The problem is, that the website takes a looooong time to build- on my Laptop (i7-5600U) it takes around 11 minutes.

I was curious if it was possible to convert the whole website, which is based on ikiwiki, to the hugo static site generator which is known to be pretty fast ("with its amazing speed and flexibility, Hugo makes building websites fun again" as the hugo website puts it ;)). I did some research if there was some tooling to do so- the Hugo website lists some migration tools but nothing for ikiwiki, but I stumbled upon anarcat’s conversion notes which has a lot of information and also links to the write up jak did on his conversion. Anarcat also published a python script to convert ikiwiki to hugo which I tried, but there were some important parts missing. As it happened I also had to prepare for an exam at that time and I am a bit of a procrastinator, so I started adapting the script for the Tails wiki and then rewrote some parts and now its just another ikiwiki2hugo script:

What does it do?

For every ikiwiki *.mdwn or *.html file, it creates a Markdown file with hugo frontmatter. The information for the hugo frontmatter comes from parsing the [[!meta directives in the files.

If there is a *.po file with the same name, the script creates a *.XX.md file with the strings translated.

Then the content is converted:

The content of the Markdown files is a copy of the content of the ikiwiki files with the ikiwiki directives replaced by hugo shortcodes. There are a lot of directives out there and I only implemented replacements for some of them. My test ikiwiki instance was the the Tails ikiwiki source, so I mostly implemented directives used there. Merge requests or patches for additional directives are welcome- to add a replacement for a directive, look in the directives folder, all the replacement python modules inherit from the Directive class. The shortcode files are assets/shortcodes folder.

What does it not do?

Probably the more important question ;)

There are a lot of directives whose replacements are not implemented. Most important probably the [[!inline directive with multiple (and negative) arguments and the [[!map directive.

Also, Hugo is more strict regarding Markdown syntax. It does not convert markdown syntax that is embedded in <div>...</div> and other block-level elements.

The result

Disclaimer: This comparison only looks at the build time- there is a lot of functionality of ikiwiki that hugo does not provide, for example the possibility to edit websites through the browser, which is used for the Tails blueprints. Another feature are the traillink directives which i simple replaced by normal markdown links.

Anarcat also provides a nice oneliner to check which directives are used in the ikiwiki repository:

grep -h -r '\[\[!' * | sed 's/\[\[!/\n[[!/g' | grep '\[\[!' | sed 's/ .*//' | sort | uniq -c | sort -n

Before running the script (I’ve removed listings of directives that occure less than 100 times):

    ...
    ...
    122 [[!wikipedia\n"
    124 [[!debsa2012
    130 [[!debsa2018
    134 [[!tails_roadmap
    152 [[!debsa2014
    188 [[!debsa2015
    204 [[!debsa2017
    219 [[!tails_gitweb_branch
    222 [[!debsa2016
    247 [[!tails_website
    283 [[!mfsa
    604 [[!cve
    655 [[!toggleable
    696 [[!tails_roadmap]]
    775 [[!"
    806 [[!tails_gitweb
   1143 [[!toggle
   1203 [[!wikipedia
   1704 [[!traillink
   2411 [[!pagetemplate
   2434 [[!toc
   2572 [[!tag
   5627 [[!inline
   5747 [[!tails_ticket
   8673 [[!img
  13577 [[!meta

After the conversion :

      1 [[!immagine
      1 [[!parent]]`.
      1 [[!tails_gitweb
      1 [[!tails_ticket
     30 [[!map
     66 [[!inline

Hugo takes nearly two seconds to build the website:

                   | DE  |  EN  | ES  | FA  | FR  | IT  | PT
+------------------+-----+------+-----+-----+-----+-----+-----+
  Pages            | 286 | 1023 | 286 | 286 | 286 | 286 | 286
  Paginator pages  |   0 |    0 |   0 |   0 |   0 |   0 |   0
  Non-page files   |   9 |  400 |   9 |   9 |  14 |   9 |   9
  Static files     |   0 |    0 |   0 |   0 |   0 |   0 |   0
  Processed images |   0 |    0 |   0 |   0 |   0 |   0 |   0
  Aliases          |   1 |    0 |   0 |   0 |   0 |   0 |   0
  Sitemaps         |   2 |    1 |   1 |   1 |   1 |   1 |   1
  Cleaned          |   0 |    0 |   0 |   0 |   0 |   0 |   0

Total in 1711 ms

Finally, a screeshot of the Tails website built with hugo- it looks pretty broken. Most of the syntax that does not work is because its embedded in HTML tags. Apparently hugo thinks about switching to another markdown renderer, so that could make a conversion from systems with a lot of mixed HTML/Markdown code easier.