Continuing in what has become a bit of a mini-series on ClojureScript with Electron, it’s time to review what we have and clean-up the prototype, flushing out the missing scenarios.

For the previous two articles, see:

• ClojureScript + Electron
• ClojureScript + Electron + Packaging

Where We Are At

When I’m working on a project, especially in the exploration stages, I like to iteratively approach the problem and narrow down on the “sweet spot”. I’ve tried to do that with this mini-series exploration as well.

So, that begs the question: “what is the sweet spot?”

Honestly, I don’t think there is one. Rather, it’s the place where the trade-offs that you made to get you there are comfortable for you. At this point in the process, these are my dependencies:

1. Web-technology based app. This is a big one that I’m not completely sold on, but some of the positives include:

• Great cross-platform support out-of-the-box.
• Lots of great community support in various technical spaces.
• Relatively future-proof.
• Easy to hook up external scripting of my app through JavaScript or one of the many dialects. This is huge for my actual project I’m working on.
• Reasonable performance ceiling for my needs.

2. ClojureScript – it compiles into JavaScript, and I can use Clojure on the backend sharing a lot of code.
3. Electron – this is the native app shell I’ll be using. It’s the major player in this space and has great backing.

Those are the big foundational dependencies. Switching out any of those essentially requires a complete re-write. Those are the ones to nail down first.

Next are the more transient requirements. These can be swapped out, though they may cause some pain.

1. Leiningen – This is the build tool for Clojure and ClojureScript, by extension. There are some others, but I see no reason to buck the norm here.
2. npm – This is the packaging tool for JavaScript. Now, there is actually a lot I do not like about npm. However… I can mitigate a lot of that, thus I see no particularly strong reason to ditch it.
3. Grunt – This is a very popular JavaScript task runner. But more importantly than that, it literally took me about 5 minutes to write my first set of scripts for it with this project. So ease of use was a huge win for me here.
4. Shell script – I have a single shell script. This means that my project will currently only build on macOS (technically, any Bash shell should work). It’s a very simple script though, so porting would take a manner of minutes.

These are not foundational even though they do encompass the very essence of being able to built out our project. They are not foundational though because each and everyone of them could be swapped out and the project could still be built. The project size will quickly dwarf the size of these components, especially if we use them wisely.

This bring us to ask the question: how are we using them? Not well. Another way to state it is: not in a very maintainable way. I’m very much of the philosophy that you don’t actually know the right answer right off the bat. If you do, it’s only because you’ve had the experience or training from someone else with that experience. Me, I’m new to this setup. What I have in the previous two articles are simply a way for me to explore the “how” of where I want to get. I know where I want to get, and now I’ve seen a path to get there. Now it’s time to clean up that debt so others can travel that same path in a more efficient and maintainable way.

Alias Cleanup

Currently the project has a bunch of aliases in the project. The problem is simple:

1. They duplicate existing functionality within Leiningen.
2. They duplicate existing build commands.
3. They do not scale well for both “development” and “production” uses.

Again, just to re-iterate: it’s not necessarily bad if this is what you have. I got here organically trying to figure out my needs. I see my needs now growing, so instead of accruing technical debt, I’m going to essentially refactor my build file.

electron-clean

The first to do is the electron-clean. We can already use the lein clean task to do this for us. The only thing we need to do is tell it which folders to delete in addition to its normal set.

 :clean-targets [“.out” “.tmp” “.dist”]

That’s it. Now those folders will be deleted every time lein clean is done. The biggest benefit here is that people already familiar with Leiningen will immediately know how to clean up and get back to a clean state.

electron-main and electron-ui

Now, there might be a time where I only want to build one portion of my project… however, I can actually already do that by using the cljsbuild task anyway. So really, these are basically duplicating those commands.

The other problem: I want a dev and a prod version. I really do not want to be in the business of dealing with multiple versions of each of these. Instead, I’d rather just have: electron-dev and electron-prod. Those two aliases will build my project using my more granular build configurations: main-dev, ui-dev, main-prod, and ui-prod.

They will look something like this:

  “electron-dev” [“do”
                  [“cljsbuild” “once” “main-dev”]
                  [“cljsbuild” “once” “ui-dev”]]
                  ;; other build steps…
  “electron-prod” [“do”
                   [“cljsbuild” “once” “main-prod”]
                   [“cljsbuild” “once” “ui-prod”]]
                   ;; other build steps…

Of course, there are some more steps in there, but we’ll talk about those next.

Isolating Dependencies

If you remember from before, we have this strange interaction between Leiningen, npm, and Grunt each needing to know a little bit of information, and in some cases, we are even duplicating that information. There’s few things I hate more in a project that needed to update a project in multiple places to keep things in sync.

package.json

First up: package.json

Now, I said that npm is a dependency. However, the “one source of truth” for my project configuration is my project.clj file. That is the file that contains all of the information that should funnel down to the other tools. So instead of having a package.json that is checked in to our repository, let’s generate it!

Something to note up-front: I have a trade-off here. I went with ease of implementation on the script vs. full automation from within the project.clj file. I think this is more readable and slightly easier to maintain as well.

Here’s the script:

#!/bin/bash

CURRENT_DIR=“$( cd “$( dirname “${BASH_SOURCE[0]}” )” && pwd )”
PROJECT_ROOT=$CURRENT_DIR/..

PACKAGE_FILE=./package.json
PRODUCT_NAME=$1
PRODUCT_DESCRIPTION=$2
PRODUCT_VERSION=$3
PRODUCT_URL=$4
ELECTRON_VERSION=$5
ELECTRON_PACKAGER_VERSION=$6

read -d ‘’ FILE_CONTENTS << EOF {
   “name”: “$PRODUCT_NAME”,
   “version”: “$PRODUCT_VERSION”,
   “description”: “$PRODUCT_DESCRIPTION”,
   “devDependencies”: {
     “grunt”: “^1.0.0”,
     “grunt-contrib-symlink”: “^1.0.0”,
     “electron”: “$ELECTRON_VERSION”,
     “electron-packager”: “$ELECTRON_PACKAGER_VERSION”
   },
   “license”: “MIT”,
   “repository”: “$PRODUCT_URL”
 }
 EOF

 echo $FILE_CONTENTS > $PACKAGE_FILE

Basically, it just creates the package.json from a template specified in the file. The “ease of development” part is not having the devDepedencies passed in as a set of parameters generating the file. I don’t actually see this file changing very often, so I’m OK with this trade-off.

To use this, we’ll add a new alias: electron-init. So instead of cloning the project and running npm install, we’ll instead run lein electron-init. The great thing about this, in my opinion of course, is that we’ve essentially made npm an implementation detail. The public interface for working with our project is always a single tool: lein.

That’s a big win.

So here’s the alias:

  “electron-init” [“do”
                   [“shell” “scripts/setup.sh”
                    :project/name :project/description
                    :project/version :project/url
                    :project/electron-version
                    :project/electron-packager-version]
                   [“shell” “npm” “install”]]

As you can see, it takes in a few configuration bits from our project file and passes it along to the script. After that, it runs npm install to get our project up and running.

The configuration for the electron bits looks like this:

 :electron-version “1.5.0”
 :electron-packager-version “^8.5.1”

Now, I would have loved to have create a map like:

 :electron {:version “1.5.0” :packager-version “^8.5.1” }

However, I could not figure out how to make Leiningen happy to parse those values and pass them along to my script easily. Alas… a trade-off.

Symlinks

Now, previously we already handled the creation of the symlink. It looked like this:

[“shell” “grunt” “symlink”]

Easy, right? Well… yes. But it also hard-coded some information in the script. So again, if we wanted to make any changes to the output location of this, we’d need to go to another file and update that. This is something we should clean up, especially given that I will be calling this twice now: once for dev and once for prod.

Instead, we’ll have something like this:

[“shell” “grunt” “symlink”
 “—source=ui/public” “—target=.out/dev/app/public”]

Now, it contains all that we need. A simple update to the Gruntfile gives us this:

“symlink”: {
  options: {
    overwrite: true
  },
  explicit: {
    src: grunt.option(“source”),
    dest: grunt.option(“target”)
  }

Now Grunt is parsing out the command line parameters —source and —target.

Wrangling ClojureScript, Google Closure Compiler, and Electron

The next bit is more involved. This is actually something I found extremely frustrating. One thing I really dislike about tools is when they do “magic”. Especially more so when the output of tools gives you different ways to handle that output. That’s what happens here…

Here’s what the full cljsbuild description looks like now:

 :cljsbuild
 {:builds
  {;; The developement profiles contain no optimizations.
   ;; NOTE!! When optimizations are set to :none, then the :output-dir *must*
   ;; also be point to the `.out` location as these files will be referrenced
   ;; by the output file.
   :main-dev
   {:source-paths [“app/src”]
    :compiler {:output-to “.out/dev/app/goog/electron-deps.js”
               :output-dir “.out/dev/app”
               :target :nodejs
               :optimizations :none}}
   :ui-dev
   {:source-paths [“ui/src”]
    :compiler {:output-to “.out/dev/app/ui.js”
               :output-dir “.out/dev/app/lib-ui”
               :optimizations :none}}

   ;; The production profiles contain full optimizations.
   ;; NOTE!! When optimizations are set to something other than :none, then it
   ;; is safe to output the build collateral outside of the target location.
   :main-prod
   {:source-paths [“app/src”]
    :compiler {:output-to “.out/prod/app/electron/host.js”
               :output-dir “.tmp/prod/app”
               :target :nodejs
               ;; :simple is preferred so any externs do not need to be created.
               ;; Also, there is no real performance difference here.
               :optimizations :simple}}
   :ui-prod
   {:source-paths [“ui/src”]
    :compiler {:output-to “.out/prod/app/ui.js”
               :output-dir “.tmp/prod/ui”
               :optimizations :simple}}}

As you can see, there are basically the four different build configurations:

1. 1. main-dev – builds the development version of our Electron hosting bits.
2. 2. ui-dev – builds the development version of our app.
3. 3. main-prod – builds the production version of our Electron hosting bits.
4. 4. ui-prod – builds the production version of our app.

Build Output Location Matters

One of the first things to note is that the output-dir is different for each of the profile types. The reason behind this is when optimizations is set to :none, none of the JavaScript files are combined together. This means that we are required do deal with loading in all of the right files to start.

That’s dumb. I would have hoped that the required file would be created for us to load up all of the requirements, but alas, it’s not (or I simply couldn’t find it). Not the end of the world though, we can fix this!

The first fix is to put all of the “temporary” build output in the app folder. Just to reiterate, this is required because our electron-host.js file is going to need to reference these files individually so we need them local to the app assets.

The second fix is to actually put the electron-host.js in the app/goog folder. This is required as this is one of those “magic” things that you are just supposed to know. Hopefully you read all of the document before you started and didn’t try to just piece together and spend a bunch of time trying to figure it out…

In theory, you can specify a :main to handle a lot of this boiler-plate for you. However, I found that setting causes just as many other magic settings to tweak and was simply not helpful. Maybe I was just missing something though.

Electron’s package.json

Electron requires a package.json file to describe how the app bundle works. Now, we did generate this before. However, this time around, we are going to tweak it just a bit.

grunt.registerTask("generate-manifest", "Generate the Electron manifest.", function() {
    grunt.config.requires("generate-manifest.name");
    grunt.config.requires("generate-manifest.version");
    grunt.config.requires("generate-manifest.manifestDir");

    var config = grunt.config("generate-manifest");
    var json = {
        name: config.name,
        version: config.version,
        main: "./main.js"
    };

    var manifestFile = config.manifestDir + "/package.json";
    grunt.file.write(manifestFile, JSON.stringify(json, null, 2));
}); 

This time, we are going to hard-code the main values to ./main.js. We are going to do this so that both our dev and prod versions have the same basic structure.

Remember how I mentioned above that we need to load different content? Well, this is one of the ways that we are going to do it. Instead of having a single main.js file, we’ll have two versions that we can maintain. I put them under the app/hoist source folder.

main-dev.js

require('./goog/bootstrap/nodejs');
require('./goog/base');
require('./goog/electron-deps.js');
require('./blog_post/electron.js');
blog_post.electron._main();

main-prod.js

require('./electron/host.js');

Now, we can just make a copy-file Grunt task and add it to our aliases:

"electron-dev" ["do"
                ...
                ["shell" "grunt" "copy-file" "--source=./app/hoist/main-dev.js" "--target=.out/dev/app/main.js"]
                ["shell" "grunt" "copy-file" "--source=./ui/hoist/index-dev.html" "--target=.out/dev/app/index.html"]
                ...

"electron-prod" ["do"
                 ...
                 ["shell" "grunt" "copy-file" "--source=./app/hoist/main-prod.js" "--target=.out/prod/app/main.js"]
                 ["shell" "grunt" "copy-file" "--source=./ui/hoist/index-prod.html" "--target=.out/prod/app/index.html"]
                 ...

Now, when we build each build flavor, we’ll copy over the correct version of our main.js.

Of course, the UI section is going to need a similar treatment as it also deals with the same basic problem from its compiled code. So instead of trying to generalize an index.html file, we’ll create two versions and copy over the right one.

index-dev.html

index-prod.html

Now, we do have a little bit of duplication here. There are some other solutions we could do here, such as:

1. Generate the HTML content and run-time in our main.js file.
2. Create another single JavaScript file to load that handles the right includes.
3. Create an empty stub for lib-ui/goog/base.js for the prod flavor.
4. Generate the HTML content from a Gruntfile task.

Now, for my app, the long-term plan is to actually use https://github.com/weavejester/hiccup for all of my HTML. So I’m likely to just have a fairly empty HTML file like so:

And at this point, it seems just as easy to maintain these two HTML files vs. maintaining one of the proposed solutions above. Your mileage may vary, of course.

Wrapping Up

So I think that is about it. With these changes I’m now in a pretty good place with the setup of my project. I’m able to:

• Build different build flavors
• Package up a distributable bundle
• Use a single build tool interface (lein)

Here’s the link to the full source for this version: GitHub – owensd/electron-blog-post-sample at optimization-settings.