Written on 2025-08-21 by Daniel Kochmański

Scripting a website with Common Lisp
JS-FFI – low level interface
LIME/SLUG – interacting from Emacs
Injecting CL runtime in arbitrary websites
Current Caveats
Funding

Web Embeddable Common Lisp is a project that brings Common Lisp and the Web Browser environments together. In this post I'll outline the current progress of the project and provide some technical details, including current caveats and future plans.

It is important to note that this is not a release and none of the described APIs and functionalities is considered to be stable. Things are still changing and I'm not accepting bug reports for the time being.

The source code of the project is available: https://fossil.turtleware.eu/wecl/.

Scripting a website with Common Lisp

The easiest way to use Common Lisp on a website is to include WECL and insert script tags with a type "text/common-lisp". When the attribute src is present, then first the runtime loads the script from that url, and then it executes the node body. For example create and run this HTML document from localhost:

<!doctype html>
<html>
  <head>
    <title>Web Embeddable Common Lisp</title>
    <link rel="stylesheet" href="https://turtleware.eu/static/misc/wecl-20250821/easy.css" />
    <script type="text/javascript" src="https://turtleware.eu/static/misc/wecl-20250821/boot.js"></script>
    <script type="text/javascript" src="https://turtleware.eu/static/misc/wecl-20250821/wecl.js"></script>
  </head>
  <body>
    <script type="text/common-lisp" src="https://turtleware.eu/static/misc/wecl-20250821/easy.lisp" id='easy-script'>
(defvar *div* (make-element "div" :id "my-ticker"))
(append-child [body] *div*)

(dotimes (v 4)
  (push-counter v))

(loop for tic from 6 above 0
      do (replace-children *div* (make-paragraph "~a" tic))
         (js-sleep 1000)
      finally (replace-children *div* (make-paragraph "BOOM!")))

(show-script-text "easy-script")
    </script>
  </body>
</html>

We may use Common Lisp that can call to JavaScript, and register callbacks to be called on specified events. The source code of the script can be found here:

Because the runtime is included as a script, the browser will usually cache the ~10MB WebAssembly module.

JS-FFI – low level interface

The initial foreign function interface has numerous macros defining wrappers that may be used from Common Lisp or passed to JavaScript.

Summary of currently available operators:

define-js-variable: an inlined expression, like document
define-js-object: an object referenced from the object store
define-js-function: a function
define-js-method: a method of the argument, like document.foobar()
define-js-getter: a slot reader of the argument
define-js-setter: a slot writer of the first argument
define-js-accessor: combines define-js-getter and define-js-setter
define-js-script: template for JavaScript expressions
define-js-callback: Common Lisp function reference callable from JavaScript
lambda-js-callback: anonymous Common Lisp function reference (for closures)

Summary of argument types:

type name	lisp side	js side
:object	Common Lisp object	Common Lisp object reference
:js-ref	JavaScript object reference	JavaScript object
:fixnum	fixnum (coercible)	fixnum (coercible)
:symbol	symbol	symbol (name inlined)
:string	string (coercible)	string (coercible)
:null	nil	null

All operators, except for LAMBDA-JS-CALLBACK have a similar lambda list:

(DEFINE-JS NAME-AND-OPTIONS [ARGUMENTS [,@BODY]])

The first argument is a list (name &key js-expr type) that is common to all defining operators:

name: Common Lisp symbol denoting the object
js-expr: a string denoting the JavaScript expression, i.e "innerText"
type: a type of the object returned by executing the expression

For example:

(define-js-variable ([document] :js-expr "document" :type :symbol))
;; document
(define-js-object ([body] :js-expr "document.body" :type :js-ref))
;; wecl_ensure_object(document.body) /* -> id   */
;; wecl_search_object(id)            /* -> node */

The difference between a variable and an object in JS-FFI is that variable expression is executed each time when the object is used (the expression is inlined), while the object expression is executed only once and the result is stored in the object store.

The second argument is a list of pairs (name type). Names will be used in the lambda list of the operator callable from Common Lisp, while types will be used to coerce arguments to the type expected by JavaScript.

(define-js-function (parse-float :js-expr "parseFloat" :type :js-ref)
    ((value :string)))
;; parseFloat(value)

(define-js-method (add-event-listener :js-expr "addEventListener" :type :null)
    ((self :js-ref)
     (name :string)
     (fun :js-ref)))
;; self.addEventListener(name, fun)

(define-js-getter (get-inner-text :js-expr "innerText" :type :string)
    ((self :js-ref)))
;; self.innerText

(define-js-setter (set-inner-text :js-expr "innerText" :type :string)
    ((self :js-ref)
     (new :string)))
;; self.innerText = new

(define-js-accessor (inner-text :js-expr "innerText" :type :string)
    ((self :js-ref)
     (new :string)))
;; self.innerText
;; self.innerText = new

(define-js-script (document :js-expr "~a.forEach(~a)" :type :js-ref)
    ((nodes :js-ref)
     (callb :object)))
;; nodes.forEach(callb)

The third argument is specific to callbacks, where we define Common Lisp body of the callback. Argument types are used to coerce values from JavaScript to Common Lisp.

(define-js-callback (print-node :type :object)
    ((elt :js-ref)
     (nth :fixnum)
     (seq :js-ref))
  (format t "Node ~2d: ~a~%" nth elt))

(let ((start 0))
  (add-event-listener *my-elt* "click"
                      (lambda-js-callback :null ((event :js-ref)) ;closure!
                        (incf start)
                        (setf (inner-text *my-elt*)
                              (format nil "Hello World! ~a" start)))

Note that callbacks are a bit different, because define-js-callback does not accept js-expr option and lambda-js-callback has unique lambda list. It is important for callbacks to have an exact arity as they are called with, because JS-FFI does not implement variable number of arguments yet.

Callbacks can be referred by name with an operator (js-callback name).

LIME/SLUG – interacting from Emacs

While working on FFI I've decided to write an adapter for SLIME/SWANK that will allow interacting with WECL from Emacs. The principle is simple: we connect with a websocket to Emacs that is listening on the specified port (i.e on localhost). This adapter uses the library emacs-websocket written by Andrew Hyatt.

It allows for compiling individual forms with C-c C-c, but file compilation does not work (because files reside on a different "host"). REPL interaction works as expected, as well as SLDB. The connection may occasionally be unstable, and until Common Lisp call returns, the whole page is blocked. Notably waiting for new requests is not a blocking operation from the JavaScript perspective, because it is an asynchronous operation.

You may find my changes to SLIME here: https://github.com/dkochmanski/slime/, and it is proposed upstream here: https://github.com/slime/slime/pull/879. Before these changes are merged, we'll patch SLIME:

;;; Patches for SLIME 2.31 (to be removed after the patch is merged).
;;; It is assumed that SLIME is already loaded into Emacs.
(defun slime-net-send (sexp proc)
  "Send a SEXP to Lisp over the socket PROC.
This is the lowest level of communication. The sexp will be READ and
EVAL'd by Lisp."
  (let* ((payload (encode-coding-string
                   (concat (slime-prin1-to-string sexp) "\n")
                   'utf-8-unix))
         (string (concat (slime-net-encode-length (length payload))
                         payload))
         (websocket (process-get proc :websocket)))
    (slime-log-event sexp)
    (if websocket
        (websocket-send-text websocket string)
      (process-send-string proc string))))

(defun slime-use-sigint-for-interrupt (&optional connection)
  (let ((c (or connection (slime-connection))))
    (cl-ecase (slime-communication-style c)
      ((:fd-handler nil) t)
      ((:spawn :sigio :async) nil))))

Now we can load the LIME adapter opens a websocket server. The source code may be downloaded from https://turtleware.eu/static/misc/wecl-20250821/lime.el:

;;; lime.el --- Lisp Interaction Mode for Emacs -*-lexical-binding:t-*-
;;; 
;;; This program extends SLIME with an ability to listen for lisp connections.
;;; The flow is reversed - normally SLIME is a client and SWANK is a server.

(require 'websocket)

(defvar *lime-server* nil
  "The LIME server.")

(cl-defun lime-zipit (obj &optional (start 0) (end 72))
  (let* ((msg (if (stringp obj)
                  obj
                (slime-prin1-to-string obj)))
         (len (length msg)))
    (substring msg (min start len) (min end len))))

(cl-defun lime-message (&rest args)
  (with-current-buffer (process-buffer *lime-server*)
    (goto-char (point-max))
    (dolist (arg args)
      (insert (lime-zipit arg)))
    (insert "\n")
    (goto-char (point-max))))

(cl-defun lime-client-process (client)
  (websocket-conn client))

(cl-defun lime-process-client (process)
  (process-get process :websocket))

;;; c.f slime-net-connect
(cl-defun lime-add-client (client)
  (lime-message "LIME connecting a new client")
  (let* ((process (websocket-conn client))
         (buffer (generate-new-buffer "*lime-connection*")))
    (set-process-buffer process buffer)
    (push process slime-net-processes)
    (slime-setup-connection process)
    client))

;;; When SLIME kills the process, then it invokes LIME-DISCONNECT hook.
;;; When SWANK kills the process, then it invokes LIME-DEL-CLIENT hook.
(cl-defun lime-del-client (client)
  (when-let ((process (lime-client-process client)))
    (lime-message "LIME client disconnected")
    (slime-net-sentinel process "closed by peer")))

(cl-defun lime-disconnect (process)
  (when-let ((client (lime-process-client process)))
    (lime-message "LIME disconnecting client")
    (websocket-close client)))

(cl-defun lime-on-error (client fun error)
  (ignore client fun)
  (lime-message "LIME error: " (slime-prin1-to-string error)))

;;; Client sends the result over a websocket. Handling responses is implemented
;;; by SLIME-NET-FILTER. As we can see, the flow is reversed in our case.
(cl-defun lime-handle-message (client frame)
  (let ((process (lime-client-process client))
        (data (websocket-frame-text frame)))
    (lime-message "LIME-RECV: " data)
    (slime-net-filter process data)))

(cl-defun lime-net-listen (host port &rest parameters)
  (when *lime-server*
    (error "LIME server has already started"))
  (setq *lime-server*
        (apply 'websocket-server port
               :host host
               :on-open    (function lime-add-client)
               :on-close   (function lime-del-client)
               :on-error   (function lime-on-error)
               :on-message (function lime-handle-message)
               parameters))
  (unless (memq 'lime-disconnect slime-net-process-close-hooks)
    (push 'lime-disconnect slime-net-process-close-hooks))
  (let ((buf (get-buffer-create "*lime-server*")))
    (set-process-buffer *lime-server* buf)
    (lime-message "Welcome " *lime-server* "!")
    t))

(cl-defun lime-stop ()
  (when *lime-server*
   (websocket-server-close *lime-server*)
   (setq *lime-server* nil)))

After loading this file into Emacs invoke (lime-net-listen "localhost" 8889). Now our Emacs listens for new connections from SLUG (the lisp-side part adapting SWANK, already bundled with WECL). There are two SLUG backends in a repository:

WANK: for web browser environment
FRIG: for Common Lisp runtime (uses websocket-driver-client)

Now you can open a page listed here and connect to SLIME:

<!doctype html>
<html>
  <head>
    <title>Web Embeddable Common Lisp</title>
    <link rel="stylesheet" href="easy.css" />
    <script type="text/javascript" src="https://turtleware.eu/static/misc/wecl-20250821/boot.js"></script>
    <script type="text/javascript" src="https://turtleware.eu/static/misc/wecl-20250821/wecl.js"></script>
    <script type="text/common-lisp" src="https://turtleware.eu/static/misc/wecl-20250821/slug.lisp"></script>
    <script type="text/common-lisp" src="https://turtleware.eu/static/misc/wecl-20250821/wank.lisp"></script>
    <script type="text/common-lisp" src="https://turtleware.eu/static/misc/wecl-20250821/easy.lisp">
      (defvar *connect-button* (make-element "button" :text "Connect"))
      (define-js-callback (connect-to-slug :type :null) ((event :js-ref))
        (wank-connect "localhost" 8889)
        (setf (inner-text *connect-button*) "Crash!"))
      (add-event-listener *connect-button* "click" (js-callback connect-to-slug))
      (append-child [body] *connect-button*)
    </script>
  </head>
  <body>
  </body>
</html>

https://turtleware.eu/static/misc/wecl-20250821/slug.html.

This example shows an important limitation – Emscripten does not allow for multiple asynchronous contexts in the same thread. That means that if Lisp call doesn't return (i.e because it waits for input in a loop), then we can't execute other Common Lisp statements from elsewhere because the application will crash.

Injecting CL runtime in arbitrary websites

Here's another example. It is more a cool gimmick than anything else, but let's try it. Open a console on this very website (on firefox C-S-i) and execute:

function inject_js(url) {
    var head = document.getElementsByTagName('head')[0];
    var script = document.createElement('script');
    head.appendChild(script);
    script.type = 'text/javascript';
    return new Promise((resolve) => {
        script.onload = resolve;
        script.src = url;
    });
}

function inject_cl() {
    wecl_eval('(wecl/impl::js-load-slug "https://turtleware.eu/static/misc/wecl-20250821")');
}

inject_js('https://turtleware.eu/static/misc/wecl-20250821/boot.js')
    .then(() => {
        wecl_init_hooks.push(inject_cl);
        inject_js('https://turtleware.eu/static/misc/wecl-20250821/wecl.js');
    });

With this, assuming that you've kept your LIME server open, you'll have a REPL onto uncooperative website. Now we can fool around with queries and changes:

(define-js-accessor (title :js-expr "title" :type :string)
  ((self :js-ref)
   (title :string)))

(define-js-accessor (background :js-expr "body.style.backgroundColor" :type :string)
  ((self :js-ref)
   (background :string)))

(setf (title [document]) "Write in Lisp!")
(setf (background [document]) "#aaffaa")

Current Caveats

The first thing to address is the lack of threading primitives. Native threads can be implemented with web workers, but then our GC wouldn't know how to stop the world to clean up. Another option is to use cooperative threads, but that also won't work, because Emscripten doesn't support independent asynchronous contexts, nor ECL is ready for that yet.

I plan to address both issues simultaneously in the second stage of the project when I port the runtime to WASI. We'll be able to use browser's GC, so running in multiple web workers should not be a problem anymore. Unwinding and rewinding the stack will require tinkering with ASYNCIFY and I have somewhat working green threads implementation in place, so I will finish it and upstream in ECL.

Currently I'm focusing mostly on having things working, so JS and CL interop is brittle and often relies on evaluating expressions, trampolining and coercing. That impacts the performance in a significant way. Moreover all loaded scripts are compiled with a one-pass compiler, so the result bytecode is not optimized.

There is no support for loading cross-compiled files onto the runtime, not to mention that it is not possible to precompile systems with ASDF definitions.

JS-FFI requires more work to allow for defining functions with variable number of arguments and with optional arguments. There is no dynamic coercion of JavaScript exceptions to Common Lisp conditions, but it is planned.

Funding

This project is funded through NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet program. Learn more at the NLnet project page.