Scientific workflows are readily and most easily expressed as shell scripts. However, shell scripts are notorious for being "duct tape", and are hard to adapt to different hardware and software environments. Workflow languages such as the Common Workflow Language (CWL) were developed to address this pain point. However, CWL does come at the cost of increased verbosity and reduced readability. Even seasoned scientists shy away from writing CWL, and fall back to the familiarity of shell scripts.
This need not be. There is no reason to inflict the behind-the-scenes complexity of workflow languages on the user. It should be possible to automatically translate (in other words, compile) a simple shell-like script into a workflow language specification. In effect, we create a new domain specific language that is almost as easy to write as a shell script, but compiles to CWL. No other language is better suited to this task of creating domain specific languages than lisp. The lisps with their homoiconicity and unique macro system empower the programmer to use ordinary code to hack the compiler itself.
In this talk, I will present the Concise Common Workflow Language (ccwl). The Concise Common Workflow Language (ccwl) is a concise syntax to express CWL workflows. It is a compiler that compiles a simple lispy shell-like workflow specification into a CWL workflow. It is implemented in the Scheme programming language, a minimalist dialect of lisp. I will discuss the considerations that went into the design of ccwl, and demonstrate its expressive power with illustrative examples.
