Sometimes I play around with Tails and on rare occasions I also build a Tails image myself. One thing that makes the build of Tails a bit tedious is that it a also builds the Tails Website, which contains the whole documentation (which is really cool, because that way users have the most up to date documentation on their desktop!). The problem is, that the website takes a looooong time to build- on my Laptop (i7-5600U) it takes around 11 minutes.
I was curious if it was possible to convert the whole website, which is based on ikiwiki, to the hugo static site generator which is known to be pretty fast (”with its amazing speed and flexibility, Hugo makes building websites fun again” as the hugo website puts it ;)). I did some research if there was some tooling to do so- the Hugo website lists some migration tools but nothing for ikiwiki, but I stumbled upon anarcat’s conversion notes which has a lot of information and also links to the write up jak did on his conversion. Anarcat also published a python script to convert ikiwiki to hugo which I tried, but there were some important parts missing. As it happened I also had to prepare for an exam at that time and I am a bit of a procrastinator, so I started adapting the script for the Tails wiki and then rewrote some parts and now its just another ikiwiki2hugo script:
What does it do?
For every ikiwiki
*.html file, it creates a Markdown file with
hugo frontmatter. The
information for the hugo frontmatter comes from parsing the
in the files.
If there is a *.po file with the same name, the script creates a
with the strings translated.
Then the content is converted:
The content of the Markdown files is a copy of the content of the ikiwiki files
with the ikiwiki directives replaced by hugo
shortcodes. There are a lot
of directives out there and I only
implemented replacements for some of them. My test ikiwiki instance was the
the Tails ikiwiki source, so I mostly implemented
directives used there. Merge requests or patches for additional directives are
welcome- to add a replacement for a directive, look in the directives
all the replacement python modules inherit from the
Directive class. The
shortcode files are
What does it not do?
Probably the more important question ;)
There are a lot of directives whose replacements are not implemented. Most
important probably the
[[!inline directive with multiple (and negative)
arguments and the
Also, Hugo is more strict regarding Markdown syntax. It does not convert
markdown syntax that is embedded in
<div>...</div> and other block-level
Disclaimer: This comparison only looks at the build time- there is a lot of functionality of ikiwiki that hugo does not provide, for example the possibility to edit websites through the browser, which is used for the Tails blueprints. Another feature are the traillink directives which i simple replaced by normal markdown links.
Anarcat also provides a nice oneliner to check which directives are used in the ikiwiki repository:
grep -h -r '\[\[!' * | sed 's/\[\[!/\n[[!/g' | grep '\[\[!' | sed 's/ .*//' | sort | uniq -c | sort -n
Before running the script (I’ve removed listings of directives that occure less than 100 times):
... ... 122 [[!wikipedia\n" 124 [[!debsa2012 130 [[!debsa2018 134 [[!tails_roadmap 152 [[!debsa2014 188 [[!debsa2015 204 [[!debsa2017 219 [[!tails_gitweb_branch 222 [[!debsa2016 247 [[!tails_website 283 [[!mfsa 604 [[!cve 655 [[!toggleable 696 [[!tails_roadmap]] 775 [[!" 806 [[!tails_gitweb 1143 [[!toggle 1203 [[!wikipedia 1704 [[!traillink 2411 [[!pagetemplate 2434 [[!toc 2572 [[!tag 5627 [[!inline 5747 [[!tails_ticket 8673 [[!img 13577 [[!meta
After the conversion :
1 [[!immagine 1 [[!parent]]`. 1 [[!tails_gitweb 1 [[!tails_ticket 30 [[!map 66 [[!inline
immagineis a typo in a translation; I also found a couple of other typos which are fixed now; the
tails_ticketdirective is a code-quote from the release process document and the
tails_gitwebis the one occurence of this directive which lists the description before the path ;))
Hugo takes nearly two seconds to build the website:
| DE | EN | ES | FA | FR | IT | PT +------------------+-----+------+-----+-----+-----+-----+-----+ Pages | 286 | 1023 | 286 | 286 | 286 | 286 | 286 Paginator pages | 0 | 0 | 0 | 0 | 0 | 0 | 0 Non-page files | 9 | 400 | 9 | 9 | 14 | 9 | 9 Static files | 0 | 0 | 0 | 0 | 0 | 0 | 0 Processed images | 0 | 0 | 0 | 0 | 0 | 0 | 0 Aliases | 1 | 0 | 0 | 0 | 0 | 0 | 0 Sitemaps | 2 | 1 | 1 | 1 | 1 | 1 | 1 Cleaned | 0 | 0 | 0 | 0 | 0 | 0 | 0 Total in 1711 ms
Finally, a screeshot of the Tails website built with hugo- it looks pretty broken. Most of the syntax that does not work is because its embedded in HTML tags. Apparently hugo thinks about switching to another markdown renderer, so that could make a conversion from systems with a lot of mixed HTML/Markdown code easier.