The Ukiyoe CMS


Tutorial

Before starting, keep in mind that ukiyoe has been mostly tested under Linux, runs on MacOS X, and has been tried under Windows (running with Apache remains untested in the latter two).

Quick start

To install, download and unzip ukiyoe+maiko.zip (this creates two new directories ukiyoe/ and maiko/). From a terminal, change into the new directory and run ./ukiyoe.py (python ukiyoe.py under Windows). You can then click on http://localhost:8080/ to access the CMS (note that, aside from RSS, the localhost links in the guide will only work when viewing this tutorial on your computer).

To create a new item, make a directory under ukiyoe/page/, say test/, and then start a text editor to copy into it the following:

# Lines starting with "#" are comments. The "#" must be the first character.
release: 23:00/16/dec/2013
title: A test
<-- uki -->

*Links to files in the item can be specified by prepending {/}*

[Link]({/}file.pdf)

(the link **won't** work, of course, but if clicked a custom 404 file will be shown).

Save this file (the article) as basic.uki in ukiyoe/page/test/ (any other files in that directory are the article's auxiliary files). Click on http://localhost:8080/page/test/ to see the result. Note that {/} has been replaced by /page/test/.

Regarding Windows in particular, it's important that the terminal where ukiyoe runs supports UTF-8 encoding (otherwise it may fail in DEBUG mode). Instructions can be found here, but essentially it involves choosing the "Lucida Console" font for the DOS terminal and then typing chcp 65001 at the prompt before running python ukiyoe.py (characters will most likely be missing, but it should run). Recall that Windows has a tendency to append a file extension which isn't shown in the file manager, so make sure the files are named properly e.g. file.uki and not file.uki.txt. Futhermore, the proper temporary directories must be specified (see the included uconfig.py file).

Ukiyoe Markdown files

The lines below the <-- uki --> separator are the contents of the article. They can be written in Markdown and, if need be, mixed with HTML (or, in the absence of the Markdown module, pure HTML). Above the separator are key: value definitions of metadata. The metadata serves to identify certain aspects of the article, such as its title. It's important to note that only one line per key: value pair is allowed (but there is no limit to its length).

There can be any number of metadata keys, but ukiyoe recognizes the following for internal use:

release specifies the time and date on which an article (and its auxiliary files) is to be served to the web. The format (configurable) is %H:%M/%d/%b/%Y e.g. 17:05/09/oct/2014. All articles must have a specified release, otherwise they will never be browsable (it is also used to create blogs). In combination with utc_offset, which is the difference, in hours, between the server time and the author's local time, a release time can be specified regardless of location (by default the local time is the time on the server). Unless needed one can simply forget about utc_offset and just think of release as the article's creation date.

rescind is optional, but similar to release. It specifies the time and date on which an article (and its auxiliary files) will no longer be served to the web.

Note that text editors sometimes add a byte order mark (BOM) at the beginning of a file, and if it precedes a key it will effectively rename it (in particular, if the release key is so modified the article will not be served). To avoid this either make the first line of the file a comment, and/or delete the BOM, and/or use an altogether different text editor (Notepad and some other Windows and Mac editors are guilty of this). Furthermore, the last line must be properly terminated, otherwise parsing of the file might fail (pathological files with mixed line endings will most likely be problematic).

keywords is a list of space-separated words which are relevant to the content of the article. If the site gets big these lists of words will be invaluable in speeding up searches and improving the results. The title of the article is also included in the search.

If disable_markdown is any of y/yes/true/1 (regardless of case) then the page will not be run throught the Markdown engine. Note that this can only disable Markdown processing, not turn it on.

allowed_users and allowed_IPs are space-separated lists of users and IPs, respectively. If allowed_users is present then authentication via HTTPS is required using a .htpasswd file, making it possible to control which of those users can actually access a given item. Similarly, access can also be limited on a per-IP basis (regardless of whether authentication is enabled or not). Note that IPv4 network classes are allowed e.g. 127.0.0. (IPv6 is untested). If neither is present no ACLs are enforced. RSS and search operations are not subject to ACLs.

The rss_title and rss_description are self-evident. Internally these are used for the RSS feed, which can be found at http://localhost:8080/rss. As with all metadata, these variables can be accessed by the templates as explained below.

Directory structure

Under the ukiyoe/ directory are three subdirectories (all configurable), two of which are called areas: public/ and private/, and static/. The first two can in turn contain sub-directories (called items) which store the CMS data:

ukiyoe/
     |-ukiyoe.py
     |-uconfig.py (optional)
     |-bottle.py
     |-ukiyoe.wsgi
     |-yokan.zip
     |--log (optional)
     |    |-lock (transient)
     |    +-stats.log
     |--public/
     |       |--item1/ (one .uki article + auxiliary files e.g. .pdf, .png...)
     |       |--item2/ (one .uki article + auxiliary files e.g. .pdf, .png...)
     |       +...
     |--private/
     |        |-.htpasswd
     |        |--item1/ (one .uki article + auxiliary files e.g. .pdf, .png...)
     |        |--item2/ (one .uki article + auxiliary files e.g. .pdf, .png...)
     |        +...
     + --static/ (templates (.tpl, .meta), CSS (.css) and other static files)

The path to the item on the file system is called the file system entry.

Templates

Ukiyoe uses the SimpleTemplate Engine provided by Bottle, please refer to its manual for details. Below is an overview of its functionality.

Templates and CSS files are stored under ukiyoe/static/. An article is associated to its template by its name e.g. basic.uki will use template basic.tpl and will have defined config['global_css'] and layout['local_css'], which contain basic.css — the local CSS file — and global.css — the global CSS file (although of course, any CSS file can be imported explicitly). The basic.tpl template is as follows:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    % if 'title' in metadata:
    %   title = metadata['title']
    % end
    <title>{{get('title', '')}}</title>
    <style type="text/css">
      @import url(/{{config['static_dir']}}/{{config['global_css']}});
      @import url(/{{config['static_dir']}}/{{layout['local_css']}});
    </style>
  </head>
  <body>
    <article>
      {{!content}}
    </article>
  </body>
</html>

The content of the article is passed on to the template and invoked as {{!content}} (the ! prevents HTML codes from being escaped, and since content is trusted and may contain markup it's usually necessary). The static directory, local and global CSS files are specified as entries of the config and layout dictionaries. Python code can be prefixed by % (or within a <% %> code block), in this case setting title if it's present in the metadata. The construct {{get('title', '')}} has the following meaning: if title is defined, show it, otherwise leave it blank. In addition to the parsed content and the metadata dictionary, the templates are provided with the following dictionaries:

Release information is provided by the release tuple (released, utc_release) (e.g. (True, 1387439310.0)).

A file in the static directory named after the template type but with a .meta extension (as opposed to .tpl) can contain, if present, the following metadata tags:

where [template] is the name of any template type.

If the template metadata contains the marker <!-- index:full --> an index dictionary will be made available to the template and populated as follows:

Where area can be 'public' or 'private', spanning the sub-directories corresponding to CONFIG['public_area'] and CONFIG['private_area'], respectively (note that the 'private' key is only present if CONFIG['index_private'] is True). article is the full path to the .uki articles on the file system (i.e. the file system entry + .uki file). If, in addition to <!-- index:full -->, the template metadata contains the marker <!-- index:parse --> the content in index[area][article]['content'] will be parsed (CONFIG['uri_tag'/'uki_dict'] substitutions and Markdown processing).

Using the <!-- index:uri --> marker instead will only return index[area][article]['uri']. Setting a filter via <!-- filter:[template] --> will only index items with the template specified in the tag. This functionality can be used to create site listings and blogs, as discussed below.

If no marker is present index is an empty dictionary.

Statistics

If site statistics are enabled then views per article and access times (in server time epoch) can be found in the stats dictionary, e.g.:

{'site': {'views': 34, 'access': 1387999915.2},
 '/ukiyoe/page/hello': {'views': 9, 'access': 1387989465.2}, ...}

As a side-note, the above can be readily converted into a ranking list within the template by doing:

ranking = sorted(stats.items(), key=lambda x: x[1]['views'], reverse=True)

which results in:

[('site', {'views': 34, 'access': 1387999915.2}),
 ('/ukiyoe/page/test', {'views': 18, 'access': 1387999910.7}), ...]

Note that articles with zero views are not listed, and that 'site' represents the total view count. Articles in the private area are also included, so care should be taken when listing site statistics. If site statistics are unavailable the stats dictionary will be present but empty. Note that high loads have been known to corrupt the statistics log file. If this occurs the record will no longer be updated but otherwise operations should continue unimpeded (the stats.log file can be removed and the server restarted to resume log updates).

Creating a blog

Make a directory ukiyoe/page/hello-world/ and save the following as post.uki inside of it:

release: 22:00/1/feb/2014
title: Welcome to my first post!
author: Jane Blogger
description: A truly momentous event
<-- uki -->
Here is a test entry for my new blog.

Now create the directory ukiyoe/page/blog/ and save the following into blog.uki:

release: 22:00/1/feb/2014
title: My beautiful blog
<-- uki -->

The individual post entry can be seen at: http://localhost:8080/page/hello-world/

The blog is available at: http://localhost:8080/page/blog/

And the new blog entry will show up on RSS at: http://localhost:8080/rss

Both post.tpl and blog.tpl templates (with their associated metadata) are packaged with ukiyoe in the static/ directory, but are fairly trivial and really only serve as examples for this tutorial. The relevant code for post.tpl is:

...
  <header>
    % import time
    % utc_offset = -time.timezone
    % if 'utc_offset' in metadata:
    %   utc_offset = float(metadata['utc_offset'])*3600
    % end
    <!-- "release" is the two-element tuple: (True, utc_release) -->
    % local_time = time.localtime(release[1] + utc_offset)
    % date = time.strftime('%b %d, %Y at %I:%M%p', local_time)
    % if 'author' in metadata:
    %   author = metadata['author']
    % end
    <h1>{{get('title', 'No Subject')}}</h1>
    <p class="byline">posted by {{get('author', 'Unknown')}} on {{date}}</p>
  </header>
  {{!content}}
...

Note that python modules can be imported into a template, and that the UTC offset will be calculated in the template (server local time) unless specified in the metadata. Finally, time.strftime() is used to format the date (in this case the release date) in a more friendly way.

The relevant section of the blog template is:

<!DOCTYPE html>
<html lang="en">
...
      <header>
        <h1>Post listing</h1>
      </header>
      <br class="clearfloat" />
      % posts = []
      % for i in index['public']:
      %   entry = index['public'][i]
      %   if entry['release'][0]:
      %     uri = entry['uri']
      %     metadata = entry['metadata']
      %     title = uri
      %     if 'title' in metadata:
      %       title = metadata['title']
      %     end
      %     views = 0
      %     if uri in stats:
      %       views = stats[uri]['views']
      %     end
      %     posts.append([uri, title, entry['release'][1], views])
      %   end
      % end
      % # Sort posts according to their release times:
      % for post in sorted(posts, key=lambda x: x[2], reverse=True):
          <p class="blogpost"><a href="{{post[0]}}">{{post[1]}}</a> (views: {{post[3]}})</p>
      % end
...

While the blog template metadata (in blog.meta) reads:

<!-- index:full -->
<!-- filter:post -->

In this case indexing is requested by inserting the <!-- index:full --> tag, which generates an index dictionary containing information about every post (and, due to the filter, only posts) on the site. First, public articles are chosen (for i in index['public']) and from those only released (if entry['release'][0]) posts are processed to generate a list of lists as so:

[['/ukiyoe/page/hello-world', 'Welcome to my blog!', 1387440000.0, 4]... ]

Finally, posts are shown in descending order (oldest last) by sorting the list according to the time entry in each sub-list:

sorted(posts, key=lambda x: x[2], reverse=True)

A fast blog

It would be convenient to navigate between posts, and this can be readily accomplished by post.tpl in a way similar to how it's done in blog.tpl, namely, creating a full index and sorting chronologically. However, this would require indexing and parsing the metadata from all entries every time time a post is read (which, admittedly, should not be an issue unless there are hundreds of posts). Being methodical early on regarding site layout can provide simple way to make site navigation more efficient as it grows (systematic use of keywords is another way to improve performance).

Consider, for example, the following item naming scheme:

cd ukiyoe/page/
mv hello-world post000-hello-world
mkdir post001-hello-world post002-hello-world
sed 's/Welcome/Enjoy/' post000-hello-world/post.uki | \
sed 's/first/second/' > post001-hello-world/post.uki
sed 's/Welcome to/Behold/' post000-hello-world/post.uki | \
sed 's/first/third/' > post002-hello-world/post.uki

(this example is artificial in that the release dates are all identical, but this does not change the point of the discussion that follows). Note that if you are keeping statistics and wish to preserve the views of the original blog post you must edit stats.log to reflect the change, i.e. after changing directory into ukiyoe/log/ do:

sed -i 's+/page/hello-world+/page/post000-hello-world/+' stats.log

Re-visiting the post entry now shows: http://localhost:8080/page/post000-hello-world/

The code which provides post navigation is:

<!DOCTYPE html>
<html lang="en">
...
        % posts = []
        % for i in index['public']:
        %   posts.append(index['public'][i]['uri'])
        % end
        % if posts and url['uri'] in posts:
        %   posts.sort()
        %   current = posts.index(url['uri'])
        %   previous = current-1
        %   next = current+1
        %   if previous >= 0:
            <a href="{{posts[previous]}}">Previous post</a>&nbsp;
        %   end
        %   if next < len(posts):
            &nbsp;<a href="{{posts[next]}}">Next post</a>
        %   end
        % end
...

with metadata:

<!-- index:uri -->
<!-- filter:post -->

In this case only a URI index is created, and posts are ordered according to the name of the item (post###-...). This only works, of course, if the same naming convention is maintained throughout. Otherwise, a chronological sorting over a full index is required.

Another potentially useful templating example can be found in gallery.tpl, which can automatically generate an image gallery. The gallery is available at the ukiyoe-gallery-example page, although it has no images at the moment (please see the file gallery.uki for usage instructions). For using MathJax in ukiyoe please refer to the ukiyoe-mathjax guide.

Previous: Introduction

Next: Advanced