Embedded Template in Go

Getting Start

Part of developing a web application usually revolves around working with HTML as user interface. To ease development process each web framework have some sort of template engine which allow developer to write server side logic to manipulate HTML output for example Ruby on Rails used embedded ruby as it's template engine. The same goes with Go ecosystem too, but one thing in common is that they required the template file to be presence on the web server after deploy. We can take this a step further in Go by compile the template directly into Go binary and thus eliminate the need to copy the template file to production web server. In this artcle I will show the step on how to do that.

Embedded Template Generator

For embedded template to work we need to convert the HTML template into a binary data and load it into memory and to do that without copy the template file themselves is to generate a Go code during compile time. We can achieve that by write a Go generator like below.

// generator.go
var conv = map[string]interface{}{"conv": fmtByteSlice}
var tmpl = template.Must(template.New("").Funcs(conv).Parse(`package box

// Code generated by go generate; DO NOT EDIT.

func init() {
	{{- range $name, $file := . }}
		box.Add("{{ $name }}", []byte{ {{ conv $file }} })
	{{- end }}
}`),
)

func fmtByteSlice(s []byte) string {
	builder := strings.Builder{}

	for _, v := range s {
		builder.WriteString(fmt.Sprintf("%d,", int(v)))
	}

	return builder.String()
}

func main() {
    // more code
}

The code above is the template code for generator that will be generated Go code which loop through each template file, convert them to byte form and store it into a variable box, in package box which we will talk about in the later section.

Next lets take a look at the piece of code that read template files.

// generator.go
package main

import (
	"bytes"
	"fmt"
	"go/format"
	"io/ioutil"
	"log"
	"os"
	"path/filepath"
	"strings"
	"text/template"
)

const (
	blobFileName = "blob.go"
	embedFolder  = "../../static"
)

func main() {
    if _, err := os.Stat(embedFolder); os.IsNotExist(err) {
		log.Fatal("Configs directory does not exists!")
	}

	configs := make(map[string][]byte)

	err := filepath.Walk(embedFolder, func(path string, info os.FileInfo, err error) error {
		relativePath := filepath.ToSlash(strings.TrimPrefix(path, embedFolder))

		if info.IsDir() {
			log.Println(path, "is a directory, skipping...")
			return nil
		}

		log.Println(path, "is a file, packing in...")

		b, err := ioutil.ReadFile(path)
		if err != nil {
			log.Printf("Error reading %s: %s", path, err)
			return err
		}

		configs[relativePath] = b
		return nil
	})
	if err != nil {
		log.Fatal("Error waling through embed directory:", err)
	}

	f, err := os.Create(blobFileName)
	if err != nil {
		log.Fatal("Error creating blob file:", err)
	}
	defer f.Close()

	builder := &bytes.Buffer{}
	if err := tmpl.Execute(builder, configs); err != nil {
		log.Fatal("Error executing template", err)
	}

	data, err := format.Source(builder.Bytes())
	if err != nil {
		log.Fatal("Error formatting generated code", err)
	}

	if err = ioutil.WriteFile(blobFileName, data, os.ModePerm); err != nil {
		log.Fatal("Error writing blob file", err)
	}
}

This code is simple, it basically read content of each file in embedFolder store them into a map with corresponding file path, which in turn was used in a generator template execution tmpl.Execute. format.Source is only to make the output code formated nicely. Lastly write the output from template execution to blobFileName file which result in Go code being generated.

The Box Package

In order to use the compiled templates we need a way to access it in our program and to do that we need to create a package (probably internal package). We called it box as you can see from the previous generator template. Below is the code to access the binary template.

//go:generate go run generator.go

package box

type embedBox struct {
	storage map[string][]byte
}

func newEmbedBox() *embedBox {
	return &embedBox{storage: make(map[string][]byte)}
}

func (e *embedBox) Add(file string, content []byte) {
	e.storage[file] = content
}

func (e *embedBox) Get(file string) []byte {
	if c, ok := e.storage[file]; ok {
		return c
	}
	return nil
}

var box = newEmbedBox()

func Add(file string, content []byte) {
	box.Add(file, content)
}

func Get(file string) []byte {
	return box.Get(file)
}

box package exposes two public functions, Add which we used in generator code to store the content of template file and Get which is intended to be used in our web application code like below.

func someHandler(w http.ResponseWriter, r *http.Request) {
    // some logic
    w.Write(box.Get("/path"))
}

Final Touch

To generate the code simple run go generate ./... in the directory that contain generator code and Go will scan through the files that contain go:generate tag as you can see from the above code.