@namgoe/gcmsgen v0.0.11

gcmsgen

A GCMS (Göttinger Content Management System) lookalike based on the Metalsmith static site generator.

Disclaimer

Not everything in this README is working as advertised. In particular, bibliography management is still a bit wonky.

Quick start

1. Install Node.js 8 or greater, either via the linked website, or via your distribution's package manager (Note: in our system node.js v8 is installed already).

   Note: gcmsgen has only been tested on Linux. MacOS should work, and Windows might.

2. Install gcmsgen. The quickest way is to use

   npm install -g @namgoe/gcmsgen

   Make sure the gcmsgen script is in your PATH. The install location depends on how you installed Node.js (Note: here -g means global, so for this you need root-permissions).

   There are other installation options which have some advantages over the global method, which are described in the Installation section below.

3. Create a directory for your website and a subdirectory named src and data. Add some content to the src directory:

   src/index.de.md

   ---
   title: Willkommen
   ---
   
   Sehr informativer Inhalt hier.

   src/index.en.md

   ---
   title: Welcome
   ---
   
   Very informative content here.

4. Add optional additional files to the directory data. They are not processed at all (important for html-files)

5. From the base directory, generate the website by running

   gcmsgen

   The output will be written to the build directory. The site will be static, i.e. consist mainly of a set of HTML files which can be directly opened in a browser or pushed to the document root of a web server.

   Note: You will however not be able to view the generated website without a network connection, as some resources (jQuery, Bootstrap, ...) are fetched from external servers. We will (probably) add an option for fully offline sites later.

6. For development, you can also run

   gcmsgen --serve

   This will start a local web server on http://localhost:8080. It will also monitor files for changes, rebuild the website if needed, and tell the browser to reload.

   Note: The HTML generated by gcmsgen --serve contains additional javascript for the livereload functionality. Make sure to always run gcmsgen without --serve once before pushing to a web server to get rid of the script. It may also be useful to completely remove the build directory before the final gcmsgen, just to be sure.

Installation

As alternatives to the global installation described above, the following methods are available.

Locally in the site source directory

This method is most useful when putting your site source under version control (which generally is a good idea anyway) and cooperatively working with others. From the repository's root directory, run

npm init

once. The command will generate a package.json file, asking some questions in the process. The answers do not really matter as long as you do not intend to publish the site source itself via npm, so you can just press Return a few times. After that, run

npm install --save @namgoe/gcmsgen

which will install gcmsgen and all dependecies into the node_modules subdirectory of your repository and record the installed version in package.json. This file should be checked into version control, but the node_modules directory should not be. So if you are using Git, your .gitignore should at least contain

build/
node_modules/

The gcmsgen script is installed as node_modules/.bin/gcmsgen, which you may want to symlink somewhere more accessible.

The advantage of this method is that anyone else checking out the repository can simply run

npm install

to install the same gcmsgen version you are using into their local node_modules.

From Git

You can install the most recent version by cloning the repository from GitLab:

git clone https://gitlab.gwdg.de/nam/gcmsgen.git

From inside the cloned directly, run

npm install

to download all dependencies. The bin/gcmsgen script can then be executed directly.

This method is particularly useful if you want to modify gcmsgen itself. As you do not have push access to the main Git repository, it may furthermore be useful to work on your own fork of gcmsgen to which you have write access. You can create a fork by clicking the Fork button on gcmsgen's GitLab page.

If you intend to modify gcmsgen, it would be great if you told us about it or even contributed your changes back. When working from your own fork, the simplest way is to create a merge request.

Structure of the src directory

The structure of src is mostly mirrored in build. Markdown files (.md) will be processed to .html, dotfiles removed, and some other file types will be processed and removed as well (see below for details).

The biggest difference is that all language-specific files will be moved to separate directory hierarchies. Given an src directory like

index.de.md
index.en.md
foobar.de.md
subpage/
  index.de.md
  index.en.md
img/
  logo.png

the build directory will contain

de/
  index.html
  foobar.html
  subpage/
    index.html
en/
  index.html
  subpage/
    index.html
img/
  logo.png
static/
index.html

The static subdirectory contains stylesheets and scripts mostly copied from GCMS. The top-level index.html is (almost) identical to de/index.html and serves as the default landing page.

Top navigation bar

The subdirectories of the current language's top-level directory, i.e. the subpage directories in the example above, are listed in the page header instead of the sidebar. For every subdirectory there will be a top navigation link with the name of the title of the index-file, which is also the main file of that navigation entry.

Navigation sidebar

All pages by default have a navigation sidebar on the right hand side consisting of up to three boxes containing

• links to all pages in the same directory,
• links to the index.html files of all subdirectories, and
• optional supplementary links that can be configured in the front matter (see below).

Links

In-site links in Markdown or HTML should usually be written relative to the current directory so as not to break when moving files to the language-specific subdirectories. If you use absolute links, make sure to take the target's final location into account.

As a final processing step, after the directory structure is fixed, all absolute links are automatically rewritten to relative links. This makes it possible to serve the generated site from a subdirectory of the web server or your local drive.

Metadata

Files can be preceded by a front matter, which is a block of YAML data delimited by lines containing only ---.

---
title: Some title
key1: value1
key2:
  subkey: value
somelist:
  - a
  - b
---

(...)

YAML is a simple language for structured data, consisting essentially of lists, key-value maps and some elementary data types like strings and numbers. To quickly learn about the basics, Learn YAML in Y minutes is a good resource.

The front matter can be an arbitrary map. Some keys have special meaning, but all keys defined here are available when processing the input files via the template engine (see Templates below).

Remark: File Encoding

In order not to get confused by arbitrary binary data, the library gcmsgen is based on will only try parsing the front matter in files encoded as UTF-8 (which includes all plain ASCII files). The file encoding depends on the text editor used to create the file, and on the underlying operating system. While UTF-8 is often the default, other encodings like ISO-8859-1 (also Latin-1) are still used. The file encoding can be determined by

file -i index.md

(file -I on MacOS), which outputs something like

index.md: text/plain; charset=iso-8859-1

If the charset is not UTF-8 or ASCII, the iconv tool can be used to convert the file:

iconv -f iso-8859-1 -t utf-8 index.md > tmp
mv tmp index.md

Special properties

The following file properties have special meaning. Some of them can be set to control how the file contents will be processed, others are automatically generated to be used by templates if needed.

• layout

  The content generated from each file can optionally be wrapped in a layout. Currently, the only available layout is default, which is enabled for all Markdown and HTML files and produces the GCMS-like look. If unset, the generated content will be used as-is.

  The default layout uses some additional properties.

  • title

    Sets the page title.

  • navtitle

    An alternative title to be used in the navigation sidebar. If unset, title will be used.

  • order

    Determines where in sidebar the page will be listed. Pages will be sorted by the value of this parameter, and pages with identical order will be sorted by path and filename.

  • sidebar

    Whether to show the sidebar at all. Defaults to true.

  • links

    A list of supplementary links to be displayed in the sidebar. This should be a YAML list of strings, which will be inserted into the sidebar verbatim as <li> elements. They should be valid HTML, most likely <a> tags.

  • stylesheets

    A YAML list of additional CSS stylesheets to be loaded in the generated page header.

  • mathjax

    Setting this to true adds a <script> header to fetch MathJax from external servers. This allows you to write LaTeX math in .html files, or, in combination with the pre-installed MathJax Markdown plugin, in .md files.

    The MathJax scripts are somewhat heavy, so they should only be activated on pages that actually need them.

  • livereload

    Add a script supporting automatic page reloading when using gcmsgen --serve. Defaults to true, and only has an effect if the --serve option is actually given. There should be little reason to set this to false.

• template

  Setting this to false causes the template engine to ignore this file. Defaults to true for Markdown and HTML, false otherwise.

• partial

  Declares the file to be a partial template (see below). The value should be the intended partial name, or true, in which case the partial name is derived from the filename up to the first dot. The file itself will not appear in the generated output. All other properties of the file are ignored. Defaults to false.

• bibname

  Declares the file to be a BibTeX collection. The value is processed just as the partial parameter above. See below for more details on the BibTeX plugin.

  The plugin will remove the file from the generated output. If you want to also make a BibTeX file available for download, you will have to include it a second time.

• data

  Used by the data loader plugin. Can be set to the (relative) path of a supplementary YAML or JSON file, a list of such paths, or a map with such paths as values. The files will be loaded and inserted in place of the respective paths.

  This feature has some conceptual overlap with the defaults.yaml mechanism described below. Its main usecase is to have an external script generate some additional data which can be processed by the template.

• contents, stats

  Auto-generated, mostly for internal use; these properties should not be assigned otherwise.

Multiple languages

There are some additional properties that govern the multiple language support. Their behaviour is slightly more complex. For the most part, using files ending in .de.md and .en.md (or ...html) should just work.

• lang

  This is the language of the file and determines where in the directory hierarchy the generated file ends up. It defaults to en for all .en.md and .en.html files, and to de for all other .md and .html files (its probably a good idea to use .de.md nevertheless). All other files do not have lang set by default and will not be moved to the language-specific directories.

  lang can also be set to a list of languages. In this case, multiple copies of the file will be generated, one for each entry. When processing the individual templates, lang will be set to the respective language.

  Finally, lang can be map from languages to addtional language-specific metadata. Processing will be performed as for the list case above, but with the respective metadata available, possibly overriding existing values. Example:

  ---
  lang:
    de:
      title: Deutscher Seitentitel
    en:
      title: English page title
  ---
  
  ...

• basename

  The file name of the generated file in the language-specific directory. Defaults to the file name itself with .${lang}. replaced by . if lang is a string, and to the plain file name otherwise. Files are regarded as translations of each other if they have the same basename and path inside the language-specific directory, and corresponding links below the header will be generated.

• langinfo

  Auto-generated property that contains supplementary language information from the site's meta.yaml (see below).

• translations

  Auto-generated map from languages to translations of the page. This is mainly used to generate the "read in other language" links below the page header.

defaults.yaml

In addition to specifying metadata in the front matter, the source will be searched for files named defaults.yaml. These should contain YAML maps, the keys of which are taken as shell-style glob patterns, and the values are again maps which are applied to all files matching the pattern relative to the subdirectory the defaults.yaml is located in. For example,

'*.md':
  key1: val1

'**/*.txt':
  key2: val2

will set key1 to val1 in all .md files, and key2 to val2 in all .txt files in all subdirectories (**/ matches arbitrarily many subdirectories).

If a key is specified both in the front matter and a defaults.yaml, the former takes precedence. On conflicts between multiple defaults.yaml files, the one further down the directory hierarchy wins.

Standard settings

The default properties, except for the ones auto-generated during processing, are set from gcmsgen's own defaults.yaml. Here are its contents at the time of writing this README:

'**/*.de.{md,htm,html}':
  lang: de

'**/*.en.{md,htm,html}':
  lang: en

---

'**':
  livereload: true
  sidebar: true

'**/*.{md,htm,html}':
  layout: default
  template: true
  lang: de

The --- ensures that the lower part is applied after the upper part, so the upper part takes precedence. Otherwise, the order of applying the lang properties would be undefined.

meta.yaml

Finally, some configuration options are site-wide rather than page-specific. The defaults of these are set in gcmsgen's meta.yaml, which can be overridden by adding a meta.yaml to your src directory. The defaults are:

langs:
  de:
    name: Deutsch
    sitename: Institut für Numerische und Angewandte Mathematik
    menu: Menü
    navigation: Navigation
    subpages: Unterseiten
    links: Links
    footer: footer-de
  en:
    name: English
    sitename: Institute for Numerical and Applied Mathematics
    menu: Menu
    navigation: Navigation
    subpages: Subpages
    links: Links
    footer: footer-en

indexlang: de

All of these are available in templates as well, but get overriden by page-specific settings. The applicable branch of the the langs property is availabe as langinfo in templates, and allows you e.g. to translate short strings via template variables. indexlang determines the language of the top-level index.html.

If you want to support another language, adding it is only a matter of extending langs and the various glob patterns in defaults.yaml and implementing the partial template referenced by the footer key. de and en are not hard-coded otherwise.

Finally, some plugins can be configured in meta.yaml. Currently, this only applies to the BibTeX plugin.

Markdown

gcmsgen uses the markdown-it Markdown parser, which is CommonMark compliant and extensible by plugins. If you never used Markdown, the plain text of this README or markdown-it's homepage should give you an idea.

Markdown is simple, but can be a bit limited at times. If needed, you can also include most HTML constructs directly in .md files.

gcmsgen currently includes the following markdown-it plugins.

MathJax

This was already mentioned for the mathjax file property above. It allows you to use a subset of LaTeX math via MathJax.

Decorate

As an alternative to inline HTML, this plugin allows you to add HTML attributes to Markdown elements, which can be useful in combination with custom CSS, Bootstrap or included images.

BibTeX

gcmsgen includes a BibTeX plugin, which adds some template helpers to include bibliography listings in a customizable format on a page, and to cite references inline. Simply add your BiBTeX-files in the directory where you want to use them. More Configuration can be done via the bibtex key in meta.yaml, the value of which is directly passed to the plugin. Collections, i.e. bibtex source files, can either be configured there, or via the bibname file property described above.

Inside the file you then can access the full bibliography with the template expansion:

{{{bibliography '<bibfile>' style=<style> lang='en-US'}}}

The citation-style can be either given here, or via the file property style. If you want to access parts of the BiBTeX-file you can use the following block expansion in Handlebars:

{{#bibliography '<bibfile>' style=<style> lang='en-US' }}
text {{cite '<citekey1>' '<citekey2>'}} some more text
{{/bibliography}}

There are many possible citation-styles, there are some examples in csl/styles/. One possible is ieee-with-url, which sorts after date.

You can view, test, edit and download styles from citationstyles.

To filter the entries in the BiBTex-file you can use include

{{{bibliography '<bibfile>' style=style lang='en-US' include=(list '<bibkey1>' '<bibkey2>')}}}

Because csl sadly is not distinguishing between master and PhD theses, we need to split that up in two separate BiBTex files which then can be loaded under a heading like this

## PhD theses
{{{bibliography 'phd' style='<style>' lang='en-US'}}}

## Master theses
{{{bibliography 'master' style='<style>' lang='en-US'}}}

Bootstrap

GCMS' responsive page layout is based on the Bootstrap framework, and gcmsgen inherits this dependency.

Presence of the framework should be mostly transparent, as it is handled by the default layout, but you may use it for your own purposes, for example to create a responsive multi-column layout. The default layout wraps the page content in a .container-fluid, so all you need to do is to add a row and two columns:

---
title: Column example
---

<div class="row">
<div class="col-sm-6">

## First column

</div>
<div class="col-sm-6">

## Second column

</div>
</div>

Note: The default layout adds the page title as an <h1>, so in-page headings should mostly be <h2> (i.e. ## in Markdown) or higher.

Templates

gcmsgen uses the Handlebars template engine on all files having the template property set. Basics usage of handlebars is rather simple. Variable expansion happens between {{ and }}, so {{title}} expands to the page title set in the front matter. For nested data structures, {{key.subkey}} expands to the value of subkey in the map referred to by key, and list elements can be accessed like {{list.[2]}} (0-based).

Conditionals are written like

{{#if someproperty}}It's set.{{else}}It's unset.{{/if}}
{{#unless someproperty}}It's set.{{else}}It's unset.{{/unless}}

where in both cases the {{else}} branch is optional. Iteration over lists and maps can be done by

{{#each somelist}}{{somefield}}{{/each}}

which expands to the the somefield properties of all elements in turn.

For debugging, there is a log helper:

{{log something}}

outputs the value of something to the console on expansion, and expands to nothing.

For full documentation, refer to the website linked above.

Partials

A partial is a template that can be included in other templates and is expanded in the context of the containing template (by default). Partials in gcmsgen are registered using the partial property, and are included by

{{> somepartial}}

Partials can also receive other contexts, additional variables or entire template blocks as parameters, which allows for some pretty flexible trickery.

Helpers

Handlebars by itself is very minimal and delegates most tasks to helpers, which can be arbitrary Javascript functions. Registering helpers is (somewhat intentionally) not possible without modifying gcmsgen. If you need some helper functions, please tell us.

Currently, our custom helpers only include elementary logic operations:

• or

  Takes an arbitraty number of arguments and evaluates to first non-"false" one, otherwise to undefined (which expands to nothing in templates). "false" has a somewhat broad definition; it's the same one Handlebars' own if helper uses.

  {{or false someproperty}}

  expands to someproperty if it is non-"false". or can also be used in conditionals:

  {{if (or a b)}} ... {{/if}}

  and as a block helper, i.e.

  {{#or a b}} ... {{/or}}

  In the latter case, it evaluates the block in the context of the first non-"false" argument.

• and

  Evaluates to the last argument if all arguments are non-"false", and undefined otherwise. The rest of the behaviour is the same as for or.

• eq and eqw

  These correspond to Javascripts' === and ==, respectively. If unsure about the difference, eq is the one you want.

  {{if (eq key1 key2)}} ... {{/if}}

  does the obvious thing.

• not

  {{#if (not something)}}...{{/if}} does what you think it does.

Updating

Methods for updating gcmsgen depend on how it was installed.

Global install

Running

npm update -g

should suffice.

Local install

Running

npm update

from inside your repository performs updates, but only for minor version changes which are compatible with the major version your package.json specifies. This is intended to make sure that updates give you bug fixes, but do not break your site.

For major version updates, there is an external tool npm-check-updates, which you can install by

npm install -g npm-check-updates

and run by issuing

ncu

from inside the repository. To actually install the updates and update package.json, use

ncu -u

Afterwards, you should check that nothing is broken and then commit the changes to package.json.

Git install

Occasionally running

npm update

is a good thing also in Git checkouts, as it gives you minor version (mostly bugfix) updates of gcmstatic's dependencies. Apart from that, the ususal means of updating a Git repository should generally work. If you are working from your own fork, you will need to add the main repository as a remote:

git remote add upstream https://gitlab.gwdg.de/nam/gcmsgen.git

To merge upstream changes, you can then use

git fetch upstream
git merge upstream/master

After resolving possible merge conflicts, push the changes to your fork on GitLab with

git push

In case dependencies have changes during the update, you should run

npm install

to get new ones, and

npm prune

to get rid of obsolete ones.

Bugs

Found a bug? Need some additional functionality, template helpers, markdown plugins or custom javascript? Tell us or create an issue on GitLab.