Home | Software | Rules and About | Atom feed
Admin | Edit
published: Sunday 5 May 2019
modified: Monday 6 May 2019
author: Hales
markup: html

Minisleep

Meet David, the sleepy WYSIWYG magician:

Minisleep is a tiny wiki engine designed to be easy to use and safe to leave web-facing.  It was born out of frustration from setting up and maintaining many other wikis, and the desire to make something that's portable between different hosts.

Notable features:

Lightweight: Zero dependencies other than bash and a CGI-compatible webserver (lighttpd, apache, etc)
WYSIWYG html editor, scripted pages and any markup language/program you want
  • Drag-and-drop image support using the in-built HTML WYSIWYG editor
  • Adding support for any markup language conversion tool (eg for mediawiki, bbcode, reStructuredText, textile, markdown) takes one line of code.
  • Optional feature: a page can be an executable script, where any text printed becomes the page.  Useful for generating indexes.

Statically compiles pages: fast & resilient to failure
  • All scripts can be disabled or fail and the site will still stay online.  You can repair problems in your own time.
  • Faster page delivery than any traditional on-the-fly wiki software.
Dumb and simple security
  • HTTP authentication, handled by your webserver instead of this wiki's scripts.
  • This wiki has no exposed attack surface if you keep these credentials secure & HTTP_AUTH on.
No database: pages stored as files and folders
  • Dead simple to administrate and backup.
  • Very easy to migrate page content to and from other wikis and systems, so you don't feel trapped.
Easy to theme
  • Comes with a very short CSS file, rather than one filled with magic (David keeps it to himself).
  • One short script acts as the page template, intended to be edited & kept across updates.

What For?

Recommended:
  • Small project wikis
  • Documentation at work
  • Personal websites & projects
Not recommended:
  • Anything that needs gradated access control (eg preventing certain authorised users from editing certain pages)
  • Any situation where authorised users may try to attack the site (eg environments where dozens+ of people get login credentials).  See the next section.
  • CMS duties where you need to regularly store non-image files (eg PDFs, treating it like cloud file storage)

Security

Minisleep is divided into two parts:
  1. Normal pages: what users see.  Static .html files in folders making up the public side of the wiki.
  2. The CGI editor script: one special URL used to edit pages.
This CGI script is designed for use with HTTP authentication:
HTTP auth is implemented in your HTTP server rather than in Minisleep itself.  If a person does not have a valid username+password then their requests will never reach the minisleep scripts.  This arguably exposes less attack surface than traditional custom login forms + cookie mechanisms.  HTTP auth features of popular webservers (such as apache and lighttpd) are generally well tested.

To keep a web-facing minisleep install safe you need to:
  1. Keep all login credentials safe.
  2. Only give login credentials to people you trust.
  3. Use HTTPS to prevent credential theft.
  4. Keep HTTP authentication enabled in your webserver config.
Once a user has working credentials they can perform attacks.  They can edit pages to add javascript coin miners and other web-malware (a problem shared by most wikis).  They can also look for unpatched bugs in minisleep.cgi to try and execute arbitrary code on the server.

Why was Minisleep made?

Many years ago I wanted to start my own personal website and I thought a wiki backend would be neat.  I had spent a lot of time writing for a game project's Mediawiki site and I had grown to love the wiki-style markup & features.

My adventures setting up my own wiki didn't go well:
  • Mediawiki is a behemoth, with lots to do and go wrong during the setup process.  Great for big projects, not so much for one person's personal site.
  • Tiddlywiki has some cool concepts, but lots of javascript.
  • ikiwiki looked absolutely perfect, but when I tried to set it up on my cheap shared host I had to fetch hundreds of megabytes of perl dependencies.  It took me a few attempts to get it right and it broke for me when the host updated it systems.
I also found that every wiki backend I tried had annoying usability problems. Inability to have multiple lines of text in a table cell using the supported markup, complicated image upload procedures and extremely complex page templates were common issues.

Here's my stab at insulting many of the other wiki engines:

(Counter-depictions of Minisleep are accepted, and if they're particularly nice I'll feature them here)

In response I started writing my own backend called Darksleep that I have since used on my own personal site for several years now.  Originally I had to SSH in to make changes, but eventually I added an online interface for page creation and editing, slowly morphing it into something more like a wiki.

Minisleep is a stripped down version of Darksleep, notably missing support for comments and submission throttling.  I made it because I needed a simple wiki backend to throw at some problems at work; and I wanted to make the code as close to perfect as possible to prevent reliability & security problems in the future.

Why shell scripts?

Shell scripts are one of the lowest common denominators across all webhosts.  You're almost guaranteed to have bash already.

My experiences wrangling dependencies for other website backends drove me mad.  A good tool does a lot with a little, not the other way around.

I've written Minisleep in as portable of a manner as I can, so that moving your site from one webhost to another can be as simple as possible.  You don't have to track down a pile of PHP, Python or Perl (PPP) dependencies each time, apart from fighting with your HTTP server's config it should work out of the box.

Why HTTP_AUTH?

It's damned simple compared to cookie-based auth methods and it dramatically reduces the attack surface for a site.  See the section on 'Security' above.

Isn't CGI slow?

  1. It's only used in Minisleep when you edit pages, not for viewing them.
  2. No.  Webapp latency is a complex topic, server interfaces are only one part of the puzzle.
In pratice: I think Minisleep is the fastest site engine I have ever used.  Slowdowns tend to be dominated by uncached disk reads, so on shared hosts the first edit will take a while to load but after that things are snappy.


Downloads & changelogs

License: GPLv3 or later
Contact: minisleep AT halestrom DOT net
Copyright: William Hales 2019

  • Fixed typos in the documentation
  • Fixed some variables unintentionally being substituted in the documentation
  • Initial release

Quickstart in 5 minutes

Minisleep comes with example server configs you can run in-place to try it out.

Option 1: lighttpd

Lighttpd's configuration files tend to be simpler than that of Apache.

1. Install lighttpd on your computer. Eg:
sudo xbps-install lighttpd     # Void
sudo apt-get install lighttpd # Devuan, Debian, Ubuntu, Mint, etc
sudo yum install lighttpd # Fedora
2. Enter the folder 'minisleep/docs/lighttpd'

3. Try to run lighttpd with the provided config:
$ lighttpd -f lighttpd.conf -D
You may need to provide the full path of lighttpd, depending on your distro:
$ /usr/sbin/lighttpd -f lighttpd.conf -D
4. Point your web browser to http://localhost:8080/minisleep/

If you want to edit any pages: the username is 'david' and the password is 'magic'.

If you plan to use Lighttpd yourself then pay attention to:
  • Enabling server.follow-symlink
  • Inserting mod_auth and mod_cgi in the right order to avoid module-loading problems.
  • Configuring page expiry, so that browsers don't keep old copies of pages cached.

Option 2: Yaws

"Yet Another Webserver" also has a nice config file format.

1. Install yaws

2. Enter the folder 'minisleep/docs/yaws'

3. Run yaws with the provided config:
$ yaws --conf yaws.conf
4. Point your web browser to http://localhost:8080/minisleep/

If you want to edit any pages: the username is 'david' and the password is 'magic'.

Points of note:
  • Yaws aggressively caches pages by default.  You may have to wait up to 30 seconds before refreshing will show changed page contents.

Option 3: Hiawatha

Hiawatha is another easy to configure webserver with some really nice features and a long history.  It's not in the Debian repos, but many other distros package it. 

Unfortunately Hiawatha's future is unclear as of 2019 with the lead author locking the forums and wanting to scale down the project.

1. Install hiawatha

2. Enter the folder 'minisleep/docs/hiawatha/'

3. Get your current path using the 'pwd' command:
$ pwd
/home/valentine/library/code/minisleep/docs/hiawatha
4. Edit hiawatha.conf to reflect this path:
set START_POINT=/home/valentine/library/code/minisleep/docs/hiawatha
5. Run hiawatha with the provided config:
$ hiawatha -c . -d
6. Point your web browser to http://localhost:8080/minisleep/

If you want to edit any pages: the username is 'david' and the password is 'magic'.

If you want to use Hiawatha yourself then pay attention to:
  • MaxRequestSize (for uploading page edits with lots of big images)
  • Enabling FollowSymlinks

Installation Procedure

First you need to get a HTTP webserver that supports CGI.   If you are on a shared host then one will probably have already been setup for you, otherwise I recommend you install apache or lighttpd (two of the most popular options).  You should also read the section "Tip: Testing CGI" further down in this document.

Next you need to choose two URLs for minisleep to use.  One URL for all of the normal static pages to be under and one special URL for the editor's CGI script.  Valid choices include:
http://example.com/minisleep/
http://example.com/minisleep.cgi

http://example.com/bobs_barbarians/
http://example.com/cgi-bin/bobs_barbarians.cgi

...etc...
Note: Many webservers only allow you to enable HTTP auth for folders, not files.  You will probably want to put the CGI file into its own folder (eg cgi-bin/) to work around this.
Download and extract your copy of minisleep somewhere safe.  Do not extract it into anywhere that your HTTP server will serve (as you would with many php websites).  Instead keep it somewhere such as your home directory where other people cannot get access to any secrets in it.

Edit your minisleep 'config' to reflect your chosen URLs:
export URLPUBLIC='/bobs_barbarians'
export URLCGI='/cgi-bin/bobs_barbarians.cgi'
You will now want to update minisleep's pages to reflect these changed URLs, otherwise the links on the pages will be broken:
$ source config
$ scripts/rebuild_all_pages.sh
Now add two symbolic links between your minisleep setup and your web server's WWW directory.  Examples include:
# On a shared host
ln -s ~/minisleep/public ~/public_html/bobs_barbarians
ln -s ~/minisleep/scripts/minisleep.cgi ~/public_html/cgi-bin/bobs_barbarians.cgi

# On my own server
ln -s ~/minisleep/public /var/www/html/bobs_barbarians
ln -s ~/minisleep/scripts/minisleep.cgi /var/www/html/cgi-bin/bobs_barbarians.cgi
Configure your webserver to allow following symlinks too, some disable this by default.
Note: Some webservers, such as Hiawatha and Yaws, will make the minisleep.cgi script hang and time-out when you try to use it.

To work around this problem: use scripts/minisleep_lessgreedy.cgi instead of scripts/minisleep.cgi

This issue occurs due to the differences in how each HTTP server provides stdin for CGI scripts.  The lessgreedy script is a thin wrapper to fix the problem, read its code for more information.
At this point your copy of minisleep should be working.

Now we need to put two security features in place:
  1. Enable HTTP auth on the page editing minisleep.cgi script (so that people need a username+password to edit pages).
  2. Setup TLS (https) so that you can access and edit your website securely.
Lets Encrypt is a popular free service for obtaining HTTPS certificates, and many shared hosting providers automatically set you up with a free certificate anyway.

See the next section 'HTTP Authentication'.

HTTP Authentication

Examples of how to enable HTTP auth for some servers are in the docs/ directory.  Refer to your HTTP server's own documentation for more information.

The most common way of managing HTTP auth credentials is to use the 'htpasswd' utility.  This tool "should" come with your HTTP server, but some distros only bundle a copy with apache.  On Debian based distros it's separated into the apache2-utils package.

Use it like so:
$ htpasswd -c myauthfile.htpasswd bobuser
$ htpasswd myauthfile.htpasswd another
$ htpasswd myauthfile.htpasswd thirduser
Htpasswd supports some better hash types than it's default of apr1 (a variant of MD5), but be warned that some HTTP servers do not support them.  In practice many servers tend to simply refuse your credentials if they don't understand the hashes, rather than being polite and telling you they don't know what to do.  To start with I recommend sticking to the defaults.

Hiawatha comes with its own version of htpasswd called wigwam.

If you have troubles getting a copy of htpasswd then a shell-script imitation is provided in the docs/ directory.  It requires an openssl variant to be installed (generally true for any Linux server these days), so it breaks the spirit of this project slightly.

If all else fails: many HTTP servers also support 'plaintext' passwd files like this:
bob:bobs password in the clear
mary:turduckinator 3000
admin:password
If you are on a shared host then this may be unwise, as there's a higher chance of someone finding a way to read your files and find your passwords.

Things you do not need to do

1. Tell minisleep where it's installed.

Minisleep's scripts will automatically detect where they are installed, even if they are moved, provided the symlinks to them are kept correct.

2. Setup an SQL database.

Minisleep keeps pages as files and folders.  There is little to minisleep that isn't hierarchical, so a database is not needed.

Tip: Testing CGI

Whilst many HTTP servers support CGI, they don't have it enabled by default and enabling can be fiddly.  If you want to use Minisleep then it's worth trying to get CGI working first.

Create a text file with the following contents:
#!/bin/sh

printf 'status: 200\n'
printf 'content-type: text/html\n'
printf '\n'
printf '<b> Moo said the cow </b>'
printf '<p> If you can read this then CGI is working. </p>'
Depending on your host & setup you'll need to work out where to save it and under what name.  Examples include:
~/public_html/cgi_bin/mytest.cgi  # Common path on shared hosts
/var/www/html/mytest.cgi  # Your own HTTP server, you will probably have to enable CGI for this particular path/URL
Make the script executable:
$ chmod a+x /var/www/html/mytest.cgi
Now try and browse to the relevant URL in your web browser.  If everything is working then you will see:


If instead you see the sourcecode to your script, are prompted to download the script or get an error: CGI is not setup correctly yet.  Work harder, read more, try a different webserver if you can :) 

CGI is one of the simplest ways of making websites interactive, so it's amazing that the big HTTP servers make it so easy to screw up the config of.  Don't worry, you're not alone.

Recommendation: Use https

(If you are running minisleep on your home LAN or in another controlled network then you can safely ignore this section).

Every time you login to a website you will want to make sure your connection is encrypted and secured.  If it's not then people can steal your username+password and do all sorts of naughty things to your website.  This isn't a problem unique to minisleep, which is why the vast majority of websites now support HTTPS.

Every single HTTP server and environment has a different way of setting up TLS/SSL/HTTPs.  You also need to create or get a valid certificate -- as of the time of writing lets encrypt is a very popular free service. 

If you are on a shared host then they may be able to do this for you (and some do it automatically for free without asking).

Modifying buildpage.sh

Minisleep is split into two main scripts:
  • scripts/buildpage.sh
  • scripts/minisleep.cgi
The buildpage.sh file is intended for users to edit and keep their edits across updates of the wiki.  The minisleep.cgi file on the other hand is intended to remain unedited so that it can be easily replaced with newer versions when updating.

buildpage.sh is not very long.  You can skip all of the initial setup stuff and go right to the page rendering bits.

Adding support for your favourite markup language converter

Anything that can input a text file and output HTML will work.

In scripts/buildpage.sh:
# Any script/program/method that you want can be used to markup your pages into
# HTML. Minisleep comes with several examples included below, however they will
# probably need adjusting to meet you needs.
#
# Tips:
# - There are many different 'markdown' converters out there. If you use
# mardown then make sure to adjust the command line options below to match
# your variant.
# - 'pandoc' supports pretty much every format under the sun and is really
# convenient, but it's often in the form of a single >100MiB executable.
# - Stick to HTML if you're unsure, it requires no setup of external programs.
#
# 'script' is disabled by default, because it is suspected that many people do
# not like interfaces for executing arbitrary code on their server to exist.
case "$markup"
in
html) cp temp_pre temp_post ;;
html_bug) cp temp_pre temp_post ;;
markdown) markdown temp_pre > temp_post ;;
textile) pandoc -f textile -t html temp_pre > temp_post ;;
mediawiki) pandoc -f mediawiki -t html temp_pre > temp_post ;;
plaintext)
echo '<pre>' > temp_post
cat temp_pre | sed 's|<|\&lt;|g ; s|>|\&gt;|g ; s|'\''|\&apos\;|g ; s|"|\&quot\;|g' >> temp_post
echo '</pre>' >> temp_post
;;
#script)
# chmod u+x temp_pre
# . temp_pre > temp_post
# ;;
*) echo "Page generation error (buildpage.sh): unknown markup type '$markup'."
esac
Let's say we want to add support for reStructuredText using a program called 'rst2html' from the Debian package 'docutils-common'.  After installing this tool we can simply add the following line to the mix:
restructuredtext)  rst2html temp_pre > temp_post ;;
Done.  Try it out by specifying 'restructuredtext' as the markup for a page when editing it.

Customise appearance (aka templating)

Minisleep does not use a seperate template file, instead code and template are mixed together in buildpage.sh:

# ------------------------------------------------------------------------------
# -- Final Content Render
# ------------------------------------------------------------------------------

exec 1>index.html.temp
# Header
echo "
<!DOCTYPE html>
<html>
<head>
<meta http-equiv='Content-Type' content='text/html;charset=UTF-8' />
<title> $title </title>
<link rel='stylesheet' href='${URL_CSS}' type='text/css' />
<link rel='icon' href='${URL_FAVICON}' />
<meta name='expires' content='0' />
<meta http-equiv='pragma' content='no-cache' />
<meta name='viewport' content='width=device-width, initial-scale=1.0'/>
</head>
<body>

<header>
<div class='left'>
<a HREF='${/minisleep}/'>Home</a> |
<a HREF='http://www.autofish.net/'>Somewhere Else</a> |
<a HREF='https://libraryofbabel.info/'>Deeper</a>
</div>
<div class='right'>
<a HREF='ds_revisions/'>Revisions</a> |
<a HREF='${/cgi/minisleep.cgi}?action=getcontrols&path=${PAGEPATH}'>Edit</a>
</div>
</header>

<main>
<h1> $title </h1>"

cat temp_post

echo "</main></body></html>"

# Shift the now completed page into production
# The 'mv' step is added for atomicity
mv index.html.temp index.html
rm temp_pre temp_post
rm ds_lockfile
That's it.  Nothing more.  Compare that to some default templates provided by other wikis :D

You will note single quotes ( ' ) are used instead of double quotes ( " ) in the HTML.  This is 100% valid HTML and makes it easier to avoid quoting problems in the script, otherwise you have to write with slashes ( \" ) everywhere.

By default a CSS file is kept in public/misc/style.css:

/* -----------------------------------------------------------------------------
* Default HTML constructs
* ---------------------------------------------------------------------------*/
body { margin: 0; font-family: Sans, Sans-Serif; }
h1, h2, h3, h4 { clear: left; }
pre
{
white-space: pre-wrap;
margin-left: 2rem;
}
img
{
height: auto;
margin: 0;
max-width: 100%;
padding: 0;
}

/* -----------------------------------------------------------------------------
* Main page components
* ---------------------------------------------------------------------------*/
main { margin: 1rem; }

header
{
background-color: #8A0000;
color: #AAAAAA;
padding: 0.3rem 0.5rem;
margin: 0;
overflow: hidden;
}
header a { color: white; }
header a:visited { color: white; }


/* -----------------------------------------------------------------------------
* Misc
* ---------------------------------------------------------------------------*/
.left { float: left; }
.right { float: right; }

#content h1:first-of-type {
clear: none;
margin-top: 0;
}

This CSS is intentionally short and bereft of magic. You can either work from it or wipe it and start from scratch, nothing will break.  The editor interface provides its own CSS, so you don't have to worry about harming it.


Bugs & Plans

WYSIWYG editor:
  • Forms get destroyed when the editor is enabled.
  • Dragging and dropping some types of files onto the editor (such as videos) embeds them in a broken manner.  
  • No table support.  This used to exist in Firefox and was really nice, but it is now gone.
  • Clicking on a formatting button removes focus from the typing area.
  • Needs key shortcuts.
  • Needs more features (text colour, boxes, floats, tables, etc)
  • Needs button for 'remove formatting'
General:
  • Documentation: info on cache control & page expiry.
  • Feature: Page deletion (move deleted folders to a safe place).
  • Username detection will likely fail if you use a HTTP_AUTH type other than basic (eg digest)
  • Feature: Automatic table of contents generation.  ie a simple script pass that looks at <h2>, <h3> etc tags and prepends some extra HTML.
  • More elegant solution to the read(stdin) hang problem of some HTTP servers (see minisleep/scripts/minisleep_lessgreedy.cgi).
  • Errant newlines are added to the end of a page every time it is edited.
Debatable:
  • Add our own support for HTTP auth & reading htpasswd files, so we don't need to rely on the webserver providing it? 
  • HTTP keywords (POST, GET) are probably not used correctly, PUT and some others might be worth considering.
Limitations (thing that probably won't be changed or fixed):
  • Gradated access control (different user roles)
  • Namespaces (if really needed: run another copy of minisleep?)
  • Whenever a HTML page with inline images (dragged and dropped with the WYSIWYG editor) is edited the whole page, including all images, gets re-uploaded.  This can be slow if the images are large.

Contact

Please send all of your comments, suggestions, workarounds, stories, bugreports, code and complaints to:  minisleep AT halestrom DOT net