IoT Shaman

Announcing Shaman Website Compiler v4

by Kyle Brown, August 15th 2020

We at IoT Shaman are excited to announce version 4 of Shaman Website Compiler. If you are not familiar, Shaman Website Compiler is a utility that compiles website files using compression, minification, bundles, etc. It also encourages developers to pre-render their public-facing templates, improving response time and reducing server load. Below is a list of the most important features included in version 4 of Shaman Website Compiler:

  • Development of a CLI (shaman-factory) to scaffold websites from pre-defined templates.
  • Drastic improvements in compilation time
  • Removing the need for javascript configuration, opting for JSON configuration instead.
  • Abstracting to process of quering the database, allowing developers to configure queries the file models associated with each web page.
  • Removing the need to code dynamic file generation. Now you can use file model configuration to declare that a template is dynamic, compiling the template once for each query result.
  • Each file compiles independently, meaning when you are working in development mode and change a file, only that file (and any dependencies) gets re-compiled.

Shaman Factory CLI

The primary inspiration for the development of version 4 was to make it extremely easy to get started designing high-quality websites by reducing the time it takes to scaffold applications. To really streamline this process we developed a new CLI tool, called shaman-factory. This CLI has a command called 'create' which takes 2 parameters, "name" (name of the website) and "template" (pre-defined application scaffolding), and will create the entire scaffolding required to build a website using the Shaman Website Compiler. At the time of writing this blog, there are currently 4 pre-built templates, ranging from a simple implementation of a static website, to full-featured websites that include user authentication, form / blog management, user management, and more.

Compilation Performance Improvements

Version 4 of the compiler included a complete re-architecture of how files were compiled. Previous versions of the compiler processed the rendering, bundling, compression, minification, etc. synchronously, which caused performance bottlenecks. In the newest version (v4), each file is processed as part of an asynchronous event stream; when a website file is located, a 'file-found' event is created, starting a chain of events that result in a fully compiled file. In the event a developer configures an HTML file to be dynamic, multiple files may be generated.

For example, the below command will create a website that contains a splash page with all blogs listed, an admin panel complete with user authentication, user management, and blog management:

factory create my-website blog

JSON Website Configuration

In previous versions of the compiler, a javascript file was required for configuration. This was a result of not having an abstraction layer for database communication; since queries needed to be ran manually in custom code, this had to be done before configuring the compiler. With version 4, we have created the database abstraction layer, allowing every configuration element to be defined in a static JSON file. This is the key piece that allows the shaman factory CLI to be able to generate websites with a simple command.

Below is a sample configuration that contains a database abstraction adapter, a folder to output files, and declares this website should be built in production mode:

{
  "production": false,
  "output": "./wwwroot",
  "adapter": {
    "name": "JsonRepoAdapter",
    "configuration": {
      "dataPath": "./data/db.json",
      "models": ["blogs"]
    }
  }
}

Database Query Abstraction

Another key feature of version 4 is the ability to define queries using a standardized JSON syntax. Without this feature, queries would need to be defined in custom code, making it very difficult to have a simple JSON configuration for the entire website. A database abstraction layer also allows developers to keep their views and their models separate; view models can declaratively define queries using a simple JSON syntax, and views can simply reference the result sets by name.

Below is an example query, which will return the top 10 most recent published blogs, and sort them in descending order (newest first):

{
    "shaman": {
      "query": [{
        "name": "recentBlogs",
        "path": "blogs",
        "args": ["published", true],
        "limit": 10,
        "sort": { "key": "createdDate", "descending": true }
      }]
    }
}

Declarative Dynamic File Generation

Previous versions of the compiler required you to run database queries before creating the compiler configuration object, then iterating over the result sets to define your dynamic files. This was the primary reason previous versions required javascript configuration. Since version 4 contained the ability to declaratively define database queries in JSON format, we could now use HTML file models to declaratively define a template as a dynamic file.

The below file model contains all the configuration necessary to define a file as dynamic. When this file gets compiled, every object in the query's result set will get rendered against the corresponding HTML template, and a route will be generated.

{
  "shaman": {
    "dynamic": {
      "path": "blogs/",
      "name": "filename"
    },
    "query": [{
      "path": "blogs",
      "dynamic": true,
      "args": ["*"]
    }]
  }
}

Learn More About Shaman Website Compiler

If you are interested in the project and want to learn more, are are looking to contribute, click here to navigate to the github project page.