Recent Content
My everlasting Common Lisp TODO list
posted on 2019-01-05 17:30 by Daniel "jackdaniel" Kochmański
We have minds capable of dreaming up almost infinitely ambitious plans and only time to realize a pathetic fraction of them. If God exists this is a proof of his cruelty.
This quote is a paraphrase of something I've read in the past. I couldn't find where it's from. If you do know where it comes from – please contact me!
I've hinted a few times that I have "lengthy" list of things to do. New year is a good opportunity to publish a blog post about it. I'm going to skip some entries which seem to be too far fetched (some could have slipped in anyway) and some ideas I don't want to share yet.
Please note, that none of these entries nor estimates are declarations nor project road maps – this is my personal to do list which may change at any time. Most notably I am aware that these estimates are too ambitious and it is unlikely that all will be met.
ECL improvements
In its early days ECL had both. They were removed in favor of native threads. I think that both are very valuable constructs which may function independently or even work together (i.e native thread have a pool of specialized green threads sharing data local to them). I want to add locatives too since I'm at adding new built-in classes.
ETA: first quarter of 2019 (before 16.2.0 release).
There might be better interfaces for the same goal, but there are already libraries which benefit from APIs defined in CLtL2 which didn't get through to the ANSI standard. They mostly revolve around environment access and better control over compiler workings (querying declarations, writing a code walker without gross hacks etc).
ETA: first quarter of 2019 (before 16.2.0 release).
ECL has two major performance bottlenecks. One is compilation time (that is actually GCC's fault), second is generic function dispatch. In a world where many libraries embrace the CLOS programming paradigm it is very important area of improvement.
Professor Robert Strandh paved the way by inventing a method to greatly improve generic function dispatch speed. The team of Clasp developers implemented it, proving that the idea is a practical one for ECL (Clasp was forked from ECL and they still share big chunks of architecture – it is not rare that we share bug reports and fixes across our code bases). We want to embrace this dispatch strategy.
ETA: second quarter of 2019 (after 16.2.0 release).
I think about adding optional modules for SSL and file SQL database. Both libraries may be statically compiled what makes them good candidates which could work even for ECL builds without FASL loading support.
ETA: third quarter of 2019.
- Compiler modernization
Admittedly I already had three tries at this task (and each ended with a failure - changes were too radical to propose in ECL). I believe that four makes a charm. Currently ECL has two passes (which are tightly coupled) - the first one for Common Lisp compilation to IR and the second one for transpiling to C/C++ code and compiling it with GCC.
The idea is to decouple these passes and have: a frontend pass, numerous optimization passes (for sake of maintainability) and the code generation pass which could have numerous backends (C, C++, bytecodes, LLVM etc).
ETA: third quarter of 2019.
CDR has many valuable proposals I want
to implement (some proposals are already implemented in ECL). Functions
compiled-file-p and abi-version are a very useful addition from the
implementation and build system point of view. Currently ECL will "eat" any
FASL it can (if it meets certain criteria, most notably it will try to load
FASL files compiled with incompatible ECL build). ABI validation should depend
on a symbol table entries hash, cpu architecture and types used by the compiler.
ETA: (this task is a sub-task of compiler modernization).
- Replacing ecl_min with EuLisp Level 0 implementation
ecl_min is an small lisp interpreter used to bootstrap the implementation (it is a binary written in C). Replacing this custom lisp with a lisp which has the standard draft would be a big step forward.
I expect some blockers along the way – most notably EuLisp has one namespace for functions and variables. Overcoming that will be a step towards language agnostic runtime.
ETA: fourth quarter of 2019.
McCLIM improvements
- Thread safety and refactored event processing loop
standard-extended-input-stream has a quirk - event queue is mixed with the input buffer. That leads to inconsistent event processing between input streams and all other panes. According to my system and specification analysis this may be fixed. This task requires some refactor and careful documentation.
Thread safety is about using CLIM streams from other threads to draw on a canvas being part of the CLIM frame from inside the external REPL. This ostensibly works, but it is not thread safe - output records may get corrupted during concurrent access.
ETA: first quarter of 2019.
- Better use of XRender extension
We already use XRedner extensions for drawing fonts and rectangles with a solid background. We want to switch clx backend to use it to its fullest. Most notably to have semi-transparency and accelerated transformations. Some proof of concept code is stashed in my source tree.
ETA: second quarter of 2019.
- Extensive tests and documentation improvements
I've mentioned it in in the last progress report. We want to spend the whole mid-release period on testing, bug fixes and documentation improvements. Most notably we want to write documentation for writing backends. This is a frequent request from users.
ETA: third quarter of 2019
This task involves writing a console-based backend (which units are very big compared to a pixel and they are not squares). That will help me to identify and fix invalid assumptions in McCLIM codebase. The idea is to have untyped coordinates being density independent pixels which have approximately the same size on any type of a screen. A natural consequence of that will be writing examples of specialized sheets with different default units.
ETA: fourth quarter of 2019
Other tasks
- Finish writing a CLIM frontend to ansi-test (tested lisps are run as external processes).
- Create a test suite for Common Lisp pathnames which goes beyond the standard.
- Contribute UNIX domain sockets portability layer to
usocket. - Explore the idea of making CLX understand (and compile) XCB xml protocol definitions.
- Write a blog post about debugging and profiling ECL applications.
- Resume a project with custom gadgets for McCLIM.
- Do more research and write POC code for having animations in McCLIM.
- Resume a project for writing Matroska build system with ASDF compatibility layer.
- Use cl-test-grid to identify most common library dependencies in Quicklisp which doesn't support ECL and contribute such support.
Conclusions
This list could be much longer, but even suggesting more entries as something scheduled for 2019 would be ridiculous – I'll stop here. A day has only 24h and I need to balance time between family, friends, duties, commercial work, free software, communities I'm part of etc. I find each and every task in it worthwhile so I will pursue them whenever I can.
McCLIM demo – Gadgets
posted on 2018-04-28 by Daniel "jackdaniel" Kochmański
My second video where I explain CLIM gadget abstraction. This is a live coding session which takes around 1h30m. Enjoy!
If you like it – let me know. If you hate it – let me know. :-)
cl-charms crash course
posted on 2018-02-04 17:30 by Daniel "jackdaniel" Kochmański
Writing a new CLIM backend
This work is meant as a showcase how to write a new McCLIM backend. To make it
more interesting to me I'm writing it using cl-charms library which is a
Common Lisp library for ncurses - console manipulation library for UNIX
systems. During development I'm planning to make notes about necessary steps. If
possible I'll also write a test suite for backends which will test the
functionality from most basic parts (like creating windows) to more
sophisticated ones (transformations and drawing). That should simplify
verifying, if a new backend works fine and to what degree it is complete. We
start with a crash course for cl-charms library.
Pointers
- charming clim is a main project repository
- cl-charms is a library upstream repository.
cl-charms crash course
Ensure you have ncurses development package installed on your system. Start
the real terminal (Emacs doesn't start *inferior-lisp* in something ncurses
can work with) and launch your implementation. I use CCL because usually
software is developed with SBCL and I want to catch as many problems with
cl-charms as possible. After that start swank server and connect from your
Emacs session.
~ ccl
? (defvar *console-io* *terminal-io*) ; we will need it later
*CONSOLE-IO*
? (ql:quickload 'swank :silent t)
(SWANK)
? (swank:create-server :port 4005 :dont-close t)
;; Swank started at port: 4005.
4005
? (loop (sleep 1))We loop over sleep because we don't want console prompt to read our first line for the console. If you don't do that you may have somewhat confusing behavior.
In Emacs: M-x slime-connect *Host:* localhost *Port:* 4005. Now we are working
in *slime-repl ccl* buffer in Emacs and we have ncurses output in the terminal
we have launched server from. Try some demos bundled with the library:
CL-USER> (ql:quickload '(cl-charms bordeaux-threads alexandria))
(CL-CHARMS BORDEAUX-THREADS ALEXANDRIA)
CL-USER> (ql:quickload '(cl-charms-paint cl-charms-timer) :silent t)
(CL-CHARMS-PAINT CL-CHARMS-TIMER)
CL-USER> (charms-timer:main) ; quit with Q, start/stop/reset with [SPACE]
CL-USER> (charms-paint:main) ; quit with Q, move with WSAD and paint with [SPACE]Now we will go through various charms (and ncurses) capabilities. Our final
goal is to have a window with four buttons and text input box. Navigation should
be possible with [TAB] / [SHIFT]+[TAB] and by selecting gadgets with a mouse
pointer. Behold, time for the first application.
First application
Lets dissect this simple program which prints "Hello world!" on the screen:
(defun hello-world ()
(charms:with-curses ()
(charms:disable-echoing)
(charms:enable-raw-input)
(loop named hello-world
with window = (charms:make-window 50 15 10 10)
do (progn
(charms:clear-window window)
(charms:write-string-at-point window "Hello world!" 0 0)
(charms:refresh-window window)
;; Process input
(when (eql (charms:get-char window) #\q)
(return-from hello-world))
(sleep 0.1)))))Program must be wrapped in charms:with-curses macro which ensures proper
initialization and finalization of the program. In this operator context
charms functions which configure the library are available. We use
charms:disable-echoing to prevent unnecessary obfuscation of the window (we
interpret characters ourself) and (charms:enable-raw-input) to turn off line
buffering. charms:*standard-window* is a window covering whole terminal
screen.
We create a Window for output (its size is 50x15 and offset is 10x10) and then
in a loop we print "Hello world!" (at the top-left corner of it) until user
press the character q.
Extending cl-charms API
All functions used until now come from higher-level interface. charms has also
a low-level interface which maps to libncurses via CFFI. This interface is
defined in the package named charms/ll. I highly recommend skimming through
http://www.tldp.org/HOWTO/NCURSES-Programming-HOWTO which is a great overview of
ncurses functionality.
We want borders around the window. CFFI interface is a bit ugly (i.e we would
have to extract a window pointer to call wborder on it). We are going to
abstract this with a function which plays nice with the lispy abstraction.
(defun draw-window-border (window
&optional
(ls #\|) (rs #\|) (ts #\-) (bs #\-)
(tl #\+) (tr #\+) (bl #\+) (br #\+))
(apply #'charms/ll:wborder (charms::window-pointer window)
(mapcar #'char-code (list ls rs ts bs tl tr bl br))))
(defun draw-window-box (window &optional (verch #\|) (horch #\-))
(charms/ll:box (charms::window-pointer window) (char-code verch) (char-code horch)))Now we can freely use draw-window-border. Put (draw-window-box window) after
(charms:clear-window window) in hello-world program and see the result. It
is ugly, but what did you expect from a window rendered in the terminal?
It is worth mentioning that border is drawn inside the window, so when we start writing string at point [0,0] - it overlaps with the border. If we want to paint content inside the border we should start at least at [1,1] and stop at [48,13].
Somewhat more appealing result may be achieved by having distinct window background instead of drawing a border with characters. To do that we need to dive into the low-level interface once more. We define colors API.
(defun start-color ()
(when (eql (charms/ll:has-colors) charms/ll:FALSE)
(error "Your terminal does not support color."))
(let ((ret-code (charms/ll:start-color)))
(if (= ret-code 0)
T
(error "start-color error ~s." ret-code))))
(eval-when (:load-toplevel :compile-toplevel :execute)
(defconstant +black+ charms/ll:COLOR_BLACK)
(defconstant +red+ charms/ll:COLOR_RED)
(defconstant +green+ charms/ll:COLOR_GREEN)
(defconstant +yellow+ charms/ll:COLOR_YELLOW)
(defconstant +blue+ charms/ll:COLOR_BLUE)
(defconstant +magenta+ charms/ll:COLOR_MAGENTA)
(defconstant +cyan+ charms/ll:COLOR_CYAN)
(defconstant +white+ charms/ll:COLOR_WHITE))
(defmacro define-color-pair ((name pair) foreground background)
`(progn
(start-color)
(defparameter ,name (progn (charms/ll:init-pair ,pair ,foreground ,background)
(charms/ll:color-pair ,pair)))))
(define-color-pair (+white/blue+ 1) +white+ +blue+)
(define-color-pair (+black/red+ 2) +black+ +red+)
(defun draw-window-background (window color-pair)
(charms/ll:wbkgd (charms::window-pointer window) color-pair))
(defmacro with-colors ((window color-pair) &body body)
(let ((winptr (gensym)))
(alexandria:once-only (color-pair)
`(let ((,winptr (charms::window-pointer ,window)))
(charms/ll:wattron ,winptr ,color-pair)
,@body
(charms/ll:wattroff ,winptr ,color-pair)))))start-color must be called when we configure the library. We map charm/ll
constants to lisp constants and create define-color-pair macro. This
abstraction could be improved so we are not forced to supply pair numbers and
providing proper association between names and integers. We skip that step for
brevity. Define two color pairs, function for filling a window background and
macro with-colors for drawing with a specified palette. Finally lets use the
new abstraction in pretty-hello-world function:
(defun pretty-hello-world ()
(charms:with-curses ()
(charms:disable-echoing)
(charms:enable-raw-input)
(start-color)
(loop named hello-world
with window = (charms:make-window 50 15 10 10)
do (progn
(charms:clear-window window)
(draw-window-background window +white/blue+)
(with-colors (window +white/blue+)
(charms:write-string-at-point window "Hello world!" 0 0))
(with-colors (window +black/red+)
(charms:write-string-at-point window "Hello world!" 0 1))
(charms:refresh-window window)
;; Process input
(when (eql (charms:get-char window :ignore-error t) #\q)
(return-from hello-world))
(sleep 0.1)))))Result looks, as promised in the function name, very pretty ;-)
Asynchronous input
Printing Hello world! doesn't satisfy our needs, we want to interact with a
brilliant software we've just made while its running. Even more, we want to do
it without blocking computations going on in the system (which are truly
amazing, believe me). First lets visualise these computations to know that they
are really happening. Modify the program loop to draw Hello World! in
different color on each iteration.
(defun amazing-hello-world ()
(charms:with-curses ()
(charms:disable-echoing)
(charms:enable-raw-input)
(start-color)
(loop named hello-world
with window = (charms:make-window 50 15 10 10)
for flip-flop = (not flip-flop)
do (progn
(charms:clear-window window)
(draw-window-background window +white/blue+)
(with-colors (window (if flip-flop
+white/blue+
+black/red+))
(charms:write-string-at-point window "Hello world!" 0 0))
(charms:refresh-window window)
;; Process input
(when (eql (charms:get-char window :ignore-error t) #\q)
(return-from hello-world))
(sleep 1)))))Something is not right. When we run amazing-hello-world to see it flipping –
it doesn't. Our program is flawed. It waits for each character to verify that
the user hasn't requested application exit. You press any key (i.e space) to
proceed to the next iteration. Now we must think of how to obtain input from
user without halting the application.
To do that we can enable non blocking mode for our window.
with window = (let ((win (charms:make-window 50 15 10 10)))
(charms:enable-non-blocking-mode win)
win)
This solution is not complete unfortunately. It works reasonably well, but we
have to wait a second (because "computation" is performed every second, we call
sleep after each get-char) before the character is handled. It gets even worse
if we notice, that pressing five times character b and then q will delay
processing by six seconds (characters are processed one after another in
different iterations with one second sleep between them). We need something
better.
I hear your internal scream: use threads! It is important to keep in mind, that
if you can get without threads you probably should (same applies for cache and
many other clever techniques which introduce even cleverer bugs). Keep also in
mind that ncurses is not thread-safe. We are going to listen for events from
all inputs like select does and generate "recompute" event each second. On
implementation which support timers we could use them but we'll use... a thread
to generate "ticks". Note that we use a thread as an asynchronous input rather
than asynchronous charms access.
;;; asynchronous input hack (should be a mailbox!)
(defparameter *recompute-flag* nil "ugly and unsafe hack for communication")
(defvar *recompute-thread* nil)
(defun start-recompute-thread ()
(when *recompute-thread*
(bt:destroy-thread *recompute-thread*))
(setf *recompute-thread*
(bt:make-thread
#'(lambda ()
(loop
(sleep 1)
(setf *recompute-flag* t))))))
(defun stop-recompute-thread ()
(when *recompute-thread*
(bt:destroy-thread *recompute-thread*)
(setf *recompute-thread* nil)))In this snippet we create an interface to start a thread which sets a global
flag. General solution should be a mailbox (or a thread-safe stream) where
asynchronous thread writes and event loop reads from. We will settle with this
hack though (it is a crash course not a book after all). Start recompute thread
in the background before you start new application. Note, that this code is not
thread-safe, we concurrently read and write to a global variable. We are also
very drastic with bt:destroy-thread, something not recommended in any code
which is not a demonstration like this one.
Time to refactor input and output functions: display-amazing-hello-world and
get-amazing-hello-world-input.
(defun display-amazing-hello-world (window flip-flop)
(charms:clear-window window)
(draw-window-background window +white/blue+)
(with-colors (window (if flip-flop
+white/blue+
+black/red+))
(charms:write-string-at-point window "Hello world!" 0 0))
(charms:refresh-window window))
(defun get-amazing-hello-world-input (window)
(when *recompute-flag*
(setf *recompute-flag* nil)
(return-from get-amazing-hello-world-input :compute))
(charms:get-char window :ignore-error t))And finally improved application which takes asynchronous input without blocking.
(defun improved-amazing-hello-world ()
(charms:with-curses ()
(charms:disable-echoing)
(charms:enable-raw-input)
(start-color)
(let ((window (charms:make-window 50 15 10 10))
(flip-flop nil))
(charms:enable-non-blocking-mode window)
(display-amazing-hello-world window flip-flop)
(loop named hello-world
do (case (get-amazing-hello-world-input window)
((#\q #\Q) (return-from hello-world))
(:compute (setf flip-flop (not flip-flop))
(display-amazing-hello-world window flip-flop))
;; don't be a pig to a processor
(otherwise (sleep 1/60)))))))When you are done with demo you may call stop-recompute-thread to spare your
image unnecessary flipping a global variable.
Gadgets and input handling
So we have created an amazing piece of software which does the computation and reacts (instantaneously!) to our input. Greed is an amazing phenomena – we want more... We want interactive application – buttons and input box (allowing us to influence the amazing computation at run time).
First we define abstract class gadget.
(defparameter *active-gadget* nil)
;;; gadget should be type of `window' or `panel' – we are simplistic
(defclass gadget ()
((position :initarg :position :accessor gadget-position)))
(defgeneric display-gadget (window gadget &key &allow-other-keys)
(:method ((window charms:window) (gadget gadget) &key)
(declare (ignore window gadget))))
(defgeneric handle-input (gadget input &key &allow-other-keys)
(:method (gadget input &key)
(declare (ignore gadget input))))In our model each gadget has at least position, display function and method for
handling input. Both methods are gadget-specific with defaults doing nothing. We
define also a parameter *active-gadget* which holds gadget receiving input.
handle-input returns T only if this input causes, that gadget has to be
redisplayed. Otherwise it should return NIL (this is a small optimization
which we will use later in main application loop).
Lets define something what we will use in our application.
(define-color-pair (+black/white+ 3) +black+ +white+) ; color for text-input (inactive)
(define-color-pair (+black/cyan+ 4) +black+ +cyan+) ; color for text-input (active)
(defparameter *computation-name* "Hello world!")
(defclass text-input-gadget (gadget)
((buffer :initarg :buffer :accessor gadget-buffer)
(width :initarg :width :reader gadget-width)))
(defun make-text-input-gadget (width x y)
(make-instance 'text-input-gadget
:width width
:position (cons x y)
:buffer (make-array width
:element-type 'character
:initial-element #\space
:fill-pointer 0)))
(defmethod display-gadget ((window charms:window) (gadget text-input-gadget) &key)
(with-colors (window (if (eql gadget *active-gadget*)
+black/cyan+
+black/white+))
(let ((background (make-string (gadget-width gadget) :initial-element #\space)))
(destructuring-bind (x . y) (gadget-position gadget)
(charms:write-string-at-point window background x y)
(charms:write-string-at-point window (gadget-buffer gadget) x y)))))
(defmethod handle-input ((gadget text-input-gadget) input &key)
(let ((buffer (gadget-buffer gadget)))
(case input
((#\Backspace #\Rubout)
(unless (zerop (fill-pointer buffer))
(vector-pop buffer)))
((#\Return #\Newline)
(unless (zerop (fill-pointer buffer))
(setf *computation-name* (copy-seq buffer)
(fill-pointer buffer) 0)))
(#\ESC
(setf (fill-pointer buffer) 0))
(otherwise
(when (ignore-errors (graphic-char-p input))
(vector-push input buffer))))))First gadget we define is text-input-gadget. What we need as its internal
state is a buffer which holds text which is typed in the box. We care also
about the string maximal width.
Moreover define colors for it to use in display-gadget (we depend on global
parameter *active-gadget* what is a very poor taste). In display function we
create a "background" (that wouldn't be necessary if it were a panel,
abstraction defined in a library accompanying ncurses) and then at the gadget
position we draw background and buffer contents (text which was already typed
in text-input-gadget).
Second function handle-input interprets characters it receives and acts
accordingly. If it is backspace (or rubout as on my keyboard with this
terminal settings), we remove one element. If it is return (or newline) we
change the computation name and empty input. escape clears the box and
finally, if it is a character which we can print, we add it to the text-input
(vector-push won't extend vector length so extra characters are ignored).
(defparameter *gadgets*
(list (make-text-input-gadget 26 2 13)))
(defun display-greedy-hello-world (window flip-flop)
(charms:clear-window window)
(draw-window-background window +white/blue+)
(with-colors (window (if flip-flop
+white/blue+
+black/red+))
(charms:write-string-at-point window *computation-name* 2 1))
(dolist (g *gadgets*)
(if (eql g *active-gadget*)
(display-gadget window g)
(charms:with-restored-cursor window
(display-gadget window g))))
(charms:refresh-window window))We maintain a list of gadgets which are displayed one-by-one in the window
(after signalling, that computation is being performed). Previously we had hard
coded "Hello world!" name but now we depend on variable *computation-name*
which may be modified from the input box.
Each gadget is displayed but only *active-gadget* is allowed to modify cursor
position. Other rendering is wrapped in charms:with-restored-cursor macro
which does the thing name suggests. That means, that cursor will be position
whenever *active-gadget* puts it (or if it doesn't modify its position –
cursor will be at the end of the computation string).
(defun get-greedy-hello-world-input (window)
(when *recompute-flag*
(setf *recompute-flag* nil)
(return-from get-greedy-hello-world-input :compute))
(charms:get-char window :ignore-error t))
(defun greedy-hello-world ()
(charms:with-curses ()
(charms:disable-echoing)
(charms:enable-raw-input)
(start-color)
(let ((window (charms:make-window 50 15 10 10))
(flip-flop nil))
(charms:enable-non-blocking-mode window)
(display-greedy-hello-world window flip-flop)
(catch :exit
(loop
do (let ((input (get-greedy-hello-world-input window)))
(case input
(#\Dc1 ;; this is C-q
(throw :exit :c-q))
(#\Dc2 ;; this is C-r
(charms:clear-window charms:*standard-window*)
(charms:refresh-window charms:*standard-window*)
(display-greedy-hello-world window flip-flop))
(#\tab
(alexandria:if-let ((remaining (cdr (member *active-gadget* *gadgets*))))
(setf *active-gadget* (car remaining))
(setf *active-gadget* (car *gadgets*)))
(display-greedy-hello-world window flip-flop))
(:compute
(setf flip-flop (not flip-flop))
(display-greedy-hello-world window flip-flop))
(otherwise
(if (handle-input *active-gadget* input)
;; redisplay only if handle-input returns non-NIL
(display-greedy-hello-world window flip-flop)
;; don't be a pig to a processor
(sleep 1/60))))))))))get-greedy-hello-world-input is fairly the same as
get-amazing-hello-world-input for now. greedy-hello-world on the other hand
handles input differently. For Quit we reserve C-q instead of q because we
want to be able to type this character in the input box. We also add C-r
sequence to refresh whole screen (just in case if we resize the terminal and
some glitches remain). :compute is handledthe same way as it was
previously. Finally if input is something else we feed it to the
*active-gadget*. If handle-input returns something else than NIL we
redisplay the application.
Note that if it is not redisplayed (i.e input is not handled because
*active-gadget* was NIL or it didn't handle the input) we are a good citizen
and instead of hogging the processor we wait 1/60 of a second. On the other hand
if events come one by one we are legitimately busy so we skip this rest time and
continue the event loop.
We don't have means to change which gadget is active yet. For that we have to
add a new key which will be handled in the main event loop of the application
#\tab. At least we wrap whole loop body in (catch :exit) to allow dynamic
exit (instead of lexical one with block/return-from pair).
Start the application and make text-input-gadget active by pressing
[tab]. Notice that its background changes. Now we have working input box where
we can type new string and when we confirm it with [enter] string at the top
will change.
We want more gadgets. For now we will settle with buttons.
(defclass button-gadget (gadget)
((label :initarg :label :reader gadget-label)
(action :initarg :action :reader gadget-action)))
(defun make-button-gadget (text callback x y)
(make-instance 'button-gadget :label text :action callback :position (cons x y)))
(defmethod display-gadget ((window charms:window) (gadget button-gadget) &key)
(with-colors (window (if (eql gadget *active-gadget*)
+red/black+
+yellow/black+))
(destructuring-bind (x . y) (gadget-position gadget)
(charms:write-string-at-point window (gadget-label gadget) x y))))
(defmethod handle-input ((gadget button-gadget) input &key)
(when (member input '(#\return #\newline))
(funcall (gadget-action gadget))))Each button is a gadget which (in addition to inherited position) has a
label and the associated action. Active button label is drawn with yellow
and red ink depending on its state. Handling input reacts only to pressing
[enter]. When button action is activated gadget's action is funcall-ed
(without arguments).
Modify *gadgets* parameter to have four buttons of ours and run the
application (try pressing [tab] a few times to see that active widget is
changing).
(defparameter *gadgets*
(list (make-text-input-gadget 26 2 13)
(make-button-gadget " Toggle " 'toggle-recompute-thread 30 11)
(make-button-gadget " Exit " 'exit-application 40 11)
(make-button-gadget " Accept " 'accept-input-box 30 13)
(make-button-gadget " Cancel " 'cancel-input-box 40 13)))
(defun toggle-recompute-thread ()
(if *recompute-thread*
(stop-recompute-thread)
(start-recompute-thread)))
(defun accept-input-box ()
(handle-input (car *gadgets*) #\return))
(defun cancel-input-box ()
(handle-input (car *gadgets*) #\esc))
(defun exit-application ()
(throw :exit :exit-button))We have created four buttons – Toggle starts/stops the "computation" thread
which feeds us with input, Exit quits the application (that's why we have
changed block to catch in greedy-hello-world main loop) and Accept /
Cancel pass return and escape input to the text-input-gadget. Once again
we prove that we lack a good taste because we aim at first element of
*gadgets* parameter assuming, that first element is text input field.
The result looks a little like a user interface, doesn't it? Try activating
various buttons and check out if text input works as desired. When you select
Toggle and press [enter] computation label at the top will start / stop
blinking in one second intervals.
Mouse integration
Our software has reached the phase where it is production ready. Investors crawl at our doorbell because they feel it is something what will boost their income. We are sophisticated and we have proposed a neat idea which will have a tremendous impact on the market. We have even created buttons and text-input gadgets which are the key to success. Something is still missing though. After a brainstorm meeting with the most brilliant minds of our decade we've came to a conclusion – what we miss is the mouse integration. That's crazy, I know. Mouse on a terminal? It is a heresy! Yet we believe that we have to take the risk if we want to outpace the competition. Roll up your sleeves – we are going to change the face of the modern UI design! ;-)
First we will start with abstracting things to make it easy to distribute events among gadgets. To do that we need to know where pointer event has happened. Lets assume that mouse event is composed of three parameters: button, x coordinate and y coordinate. Each gadget occupies some space on the screen (lets call it a region) which may be characterized by its bounding rectangle with [min-x, min-y] and [max-x, max-y] points.
(defgeneric bounding-rectangle (gadget)
(:method ((gadget text-input-gadget))
(destructuring-bind (x . y) (gadget-position gadget)
(values x
y
(+ x -1 (gadget-width gadget))
y)))
(:method ((gadget button-gadget))
(destructuring-bind (x . y) (gadget-position gadget)
(values x
y
(+ x -1 (length (gadget-label gadget)))
y))))We could refactor button-gadget to have gadget-width method (that would simplify the implementation), but lets use what we have now. Having such representation of bounding rectangle we can now easily determine if cursor event occurred "over" the gadget or somewhere else.
(defun region-contains-position-p (gadget x y)
(multiple-value-bind (x-min y-min x-max y-max)
(bounding-rectangle gadget)
(and (<= x-min x x-max)
(<= y-min y y-max))))Now distributing mouse event is as easy as iterating over all gadgets and
verifying if event applies to the gadget. If it does we make such gadget
active. Moreover if left mouse button is clicked we simulate [return] key to
be handled by the previously defined handle-input method.
(defun distribute-mouse-event (bstate x y)
(dolist (g *gadgets*)
(when (region-contains-position-p g x y)
(setf *active-gadget* g)
(when (eql bstate charms/ll:button1_clicked)
(handle-input g #\return))
(return))))Notice that we have used low-level charms interface (comparing bstate with
charms/ll:button1_clicked). Other events are also defined in the charms/ll
package so you may expand mouse handling interface.
Time to define display function for our enterprise application. ncurses cursor
should to follow mouse movement, so displaying gadgets can't affect cursor
position. To achieve that we wrap whole display function in
charms:with-restored-cursor macro.
(defun display-enterprise-hello-world (window flip-flop)
(charms:with-restored-cursor window
(charms:clear-window window)
(draw-window-background window +white/blue+)
(if flip-flop
(with-colors (window +white/blue+)
(charms:write-string-at-point window *computation-name* 2 1))
(with-colors (window +black/red+)
(charms:write-string-at-point window *computation-name* 2 1)))
(dolist (g *gadgets*) (display-gadget window g))
(charms:refresh-window window)))Usually terminal emulators doesn't report mouse movement (only clicks). To
enable such reports print the following in the terminal output (note, that slime
doesn't bind *terminal-io* to the right thing, so we use *console-io* which
we have dfined at the beginning):
(format *console-io* "~c[?1003h" #\esc)
This escape sequence sets xterm mode for reporting any mouse event including mouse movement (see http://invisible-island.net/xterm/ctlseqs/ctlseqs.html). This escape sequence is usually honored by other terminal emulators (not only xterm). Without it we wouldn't be able to track mouse movement (only pointer press, release, scroll etc). This is important to us because we want to activate gadgets when mouse moves over them.
To start mouse and configure its events lets create initialization function. We configure terminal to report all mouse events including its position. After that we tell the terminal emulator to report mouse position events.
(defun start-mouse ()
(charms/ll:mousemask
(logior charms/ll:all_mouse_events charms/ll:report_mouse_position))
(format *console-io* "~c[?1003h~%" #\esc))We need to handle new event type :key-mouse (already handled types are C-q,
C-r, TAB, :compute and "other" characterrs). Since event handling gets
more complicated we factor it into a separate function
enterprise-process-event. Note the handler-case – when mouse event queue is
empty (for instance because the mouse event is masked), then function will
return error. We also take into account that mouse events are reported starting
at absolute terminal coordinates while our window starts at point
[10,10]. Additionally we implement shift+tab sequence which on my system is
reported as #\Latin_Small_Letter_S_With_Caron.
(defun get-enterprise-hello-world-input (window)
(when *recompute-flag*
(setf *recompute-flag* nil)
(return-from get-enterprise-hello-world-input :compute))
(let ((c (charms/ll:wgetch (charms::window-pointer window))))
(when (not (eql c charms/ll:ERR))
(alexandria:switch (c)
(charms/ll:KEY_BACKSPACE #\Backspace)
(charms/ll:KEY_MOUSE :KEY-MOUSE)
(otherwise (charms::c-char-to-character c))))))
(defun enterprise-process-event (window flip-flop)
(loop
(let ((input (get-enterprise-hello-world-input window)))
(case input
(#\Dc1 ;; this is C-q
(throw :exit :c-q))
(#\Dc2 ;; this is C-r
(display-enterprise-hello-world window flip-flop))
(#\tab
(alexandria:if-let ((remaining (cdr (member *active-gadget* *gadgets*))))
(setf *active-gadget* (car remaining))
(setf *active-gadget* (car *gadgets*)))
(display-enterprise-hello-world window flip-flop))
(#\Latin_Small_Letter_S_With_Caron ;; this is S-[tab]
(if (eql *active-gadget* (car *gadgets*))
(setf *active-gadget* (alexandria:lastcar *gadgets*))
(do ((g *gadgets* (cdr g)))
((eql *active-gadget* (cadr g))
(setf *active-gadget* (car g)))))
(display-enterprise-hello-world window flip-flop))
(:key-mouse
(handler-case (multiple-value-bind (bstate x y z id)
(charms/ll:getmouse)
(declare (ignore z id))
;; window starts at 10,10
(decf x 10)
(decf y 10)
(charms:move-cursor window x y)
(distribute-mouse-event bstate x y)
(display-enterprise-hello-world window flip-flop))
(error () nil)))
(:compute
(setf flip-flop (not flip-flop))
(display-enterprise-hello-world window flip-flop))
(otherwise
(if (handle-input *active-gadget* input)
;; redisplay only if handle-input returns non-NIL
(display-enterprise-hello-world window flip-flop)
;; don't be a pig to a processor
(sleep 1/60)))))))To make terminal report mouse events in the intelligible way we need to call
function charms:enable-extra-keys (thanks to that we don't deal with raw
character sequences). We also call start-mouse.
(defun enterprise-hello-world ()
(charms:with-curses ()
(charms:disable-echoing)
(charms:enable-raw-input)
(start-color)
(let ((window (charms:make-window 50 15 10 10))
(flip-flop nil))
;; full enterprise ay?
(charms:enable-non-blocking-mode window)
(charms:enable-extra-keys window)
(start-mouse)
(display-enterprise-hello-world window flip-flop)
(catch :exit
(loop (funcall 'enterprise-process-event window flip-flop))))))Our final result is splendid! We got rich :-). Source of the following asciinema video is located here.
To finish our lisp session type in the slime REPL (quit).
Conclusions
In this crash course we have explored some parts of the cl-charms interface
(windows, colors, mouse integration...) and defined an ad-hoc toolkit for the
user interface. There is much more to learn and the library may be expanded
(especially with regard to high level interface, but also to include supplement
ncurses libraries like panel, menu and forms). Ability to run charms
application in a new terminal started with run-program would be beneficial
too.
Some programmers may have noticed that we have defined an ad-hoc, informally-specified, bug-ridden, slow implementation of 1/10th of CLIM. No wonder, it defines a very good abstraction for defining UI in a consistent manner and it is a standard (with full specification).
Instead of reinventing the wheel we want to plug into CLIM abstractions and use
cl-charms as CLIM backend. This option will be explored in a near-term future
– it will help to document a process of writing new backends. Of course such
backend will be limited in many ways – that will be an opportunity to explore
bugs which got unnoticed when the smallest distance is a pixel (in contrast to a
reasonably big terminal character size). Stay tuned :-)
McCLIM demo – St Nicolas Day present
posted on 2017-12-06 by Daniel "jackdaniel" Kochmański
One of the projects I work on is McCLIM which is a GUI toolkit tailored for Common Lisp. For a few weeks now I was thinking about recording a demo session which shows some random usage of this software. If you are interested to take time and watch it, it takes around 30 minutes.
This is my first tutorial video recorded in the home, so I would appreciate any feedback with remarks what I did good and what I did wrong. Thank you and enjoy the video!
Tutorial: Working with FiveAM
posted on 2017-09-05 by Tomek "uint" Kurcz
What is FiveAM?
FiveAM is a simple-yet-mature test framework. It makes test suites for your project easy to implement, maintain, organize and run.
Motivation
While it can't be said that there are no learning materials provided for FiveAM, it feels like they are lacking in both clarity and detail. Beginners are in need of gentle, friendly guidance. Experienced Lisp hackers are able to make do without it, but even they probably spend a little extra time tinkering, experimenting and skimming source code to "get" the framework. This shouldn't be necessary.
This tutorial assumes familiarity with Common Lisp and a basic understanding of ASDF system definitions.
Our building blocks
We will start with a bit of theorizing. Be not afraid, however - there won't be too much of it.
The essential terms you will need to be familiar with are:
- Checks
- Tests
- Test suites
Checks
A check is, essentially, a single assertion - a line of code that makes sure something that should be true is indeed true. FiveAM tries to make assertions as simple as possible. The form of a basic check definition looks like this:
(is test &rest reason-args)In this case,test is the assertion we want to make. A function (or special operator)
application with any number of arguments can be used as the assertion. If it returns
a true value, the assertion succeeds; if it returns NIL, it fails.
If the test parameter matches any of the 4 "templates" below, FiveAM will
try to reason a little about what is what and attempt to print
the explanations of failures in a more readable way. Arguably.
(predicate value)
(predicate expected value)
(not (predicate value))
(not (predicate expected value))The logic FiveAM follows when reasoning is thus:
The first expression checks whether value satisfies predicate.
In the second one, the predicate is usually some form of equality test.
The assertion makes sure the value we got (by calling some function we're
testing) matches the expected value according to the predicate.
The last two tests are the same things, only negated.
In practice, these declarations look like this:
(is (listp (list 1 2))) ; is (list 1 2) a list?
(is (= 5 (+ 2 3))) ; is (+ 2 3) equal 5?Simple, right? If we were implementing standard Lisp functions, we could
use the above to test whether list generates a list as it should, and whether
+ sums properly. Or, well, at least we'd ascertain that for the above cases.
And if we wanted to negate:
(is (not (listp (list 1 2)))) ; is (list 1 2) not a list?
(is (not (= 5 (+ 2 3)))) ; is (+ 2 3) not equal 5?As you may have noticed, we haven't used the optional reason-args
argument. It's used to specify what's printed as the reason
for a failed check. Sometimes FiveAM's reasoning just isn't good
enough. We will get back to it when we start hacking away.
Tests
We know how to write checks, but there's not much we can actually do
with just this knowledge. The is syntax is only available in
the context of a test definition.
A test, as defined by FiveAM, is simply a collection of checks. Each such collection has a name so that we can easily run it later. Defining one is easy:
(test test-+
"Test the + function" ;optional description
(is (= 0 (+ 0 0)))
(is (= 4 (+ 2 2)))
(is (= 1/2 (+ 1/4 1/4))))We're sticking to the basics for now, but you should know there are some additional keyword parameters you can pass in order to declare dependencies, explicitly specify the parent suite, specify the fixture, change the time of compilation and/or collect profiling information.
A fixture is something that ensures a test is run in a specific context. Sometimes it's necessary to reproduce results consistently. For example, if you had a pathfinding algorithm, you'd probably have to load some sort of a map before you could test it. Apparently, using FiveAM's fixture functionality isn't recommended by the current maintainer. Perhaps it's best to just set up macros for those.
As for profiling information, this functionality doesn't seem to actually be implemented yet. Instead, Metering is a good option if needed.
You'll most likely end up defining a single test for a single function, but nothing stops you from slicing the pie up differently. Maybe a particularly complex function requires a lot of checks that are best divided into categories? Maybe a set of simple, related functions can be covered by a single test for simplicity? Your common sense is the best advisor here.
Suites
The final piece of the puzzle. These are not obligatory, but very useful. Suites are containers for tests, good if you need more hierarchy - which, honestly, you will. Speaking of hierarchy: suites can parent other suites, so you can have plenty of that.
The way suites are defined and used is roughly analogous to packages.
(def-suite tutorial-suite
:description "A poor man's suite"
:in some-parent-suite)
(in-suite tutorial-suite)The first form defines a test suite called tutorial-suite.
The in keyword is used to set the parent suite.
Just like in-package sets the *package* special variable, in-suite
sets the *suite* one. Test definitions pick up on it when provided.
Thanks to that, any test definitions after (in-suite tutorial-suite)
will be included in tutorial-suite. Other suite definitions, however,
won't be automagically contained in the suite pointed to by *suite*.
For that reason, you always need to explicitly set the in keyword when
defining a child suite.
And that's actually all there is to suites.
The story so far
Time for a quick summary - from the top, our tests are organized like this:
- (optional) Top-level test suites defined with
(def-suite) - (optional) Child test suites defined with
(def-suite)with:inset - Tests defined with the
(test)macro - Checks (assertions) defined with
(is)expressions within a(test)form
A practical example
Now that all that is clear, let's try doing something with it. Imagine you are building an RPG game according to some existing pen-and-paper system. One day, it will surely rival the likes of AAA+ titles out there.
...for now, though, you only have the character generation facility down. Oh well, got to start somewhere. According to the specification of the system you're using, the stats of a character are generated randomly, but prior to the generation, a player can choose two stats they wish to "favor". Unfavored stats are decided on with a roll of two 8-sided dice, while favored ones - a roll of three 8-sided dice. You've defined a little utility function for rolling an arbitrary number of dice with an arbitrary number of sides.
You've written this basic functionality, wrapped it up in a package, defined an ASDF system, checked that everything compiles without warnings... So far, so good. But now you want to go the extra mile to make sure this is going to be a well-built piece of software. You want to integrate tests.
If you'd like to follow all the outlined steps and integrate FiveAM with me, just clone the master branch of the quasirpg repository.
git clone https://github.com/uint/quasirpg.gitIdeally, if you have quicklisp, do that in ~/quicklisp/local-projects/.
Otherwise, clone the repository to either ~/common-lisp/ or
~/.local/share/common-lisp/source/.
If you wish, you can also look through the commit history of the test branch to see exactly how I've done all the work detailed in the following sections. It might come in useful if you get stuck.
If you want to see the code in action, try these:
CL-USER> (ql:quickload 'quasirpg)
CL-USER> (in-package #:quasirpg)
QUASIRPG> (roll-dice 3 6) ; throw three 6-sided dice
QUASIRPG> (make-character)
QUASIRPG> (make-character "Bob" '("str" "int"))In case you don't have quicklisp, you can use this to load the system:
CL-USER> (asdf:load-system 'quasirpg)Keep in mind that without quicklisp, you will also have to download FiveAM by hand. In the same directory you cloned quasirpg to, try:
git clone https://github.com/sionescu/fiveam.gitGroundwork
Tests shouldn't be a part of your software's main system. Why would they
be? People who simply want to download your application and use it don't
need them. Neither do they need to pull FiveAM as a dependency. So let's
define a new system for tests. We could create a separate .asd file,
but I like to have just one .asd file around. In this case, any
additional systems defined after the main quasirpg one should be named
quasirpg/some-name. So we append this to our quasirpg.asd:
(asdf:defsystem #:quasirpg/tests
:depends-on (:quasirpg :fiveam)
:components ((:module "tests"
:serial t
:components ((:file "package")
(:file "main")))))We also create the new files tests/package.lisp and tests/main.lisp
to make true to the above declaration. As you might guess, we're planning
to define a separate package for tests. This isn't as important as
separate systems, but it's always good to keep namespaces separate.
Nice and tidy.
;;;; tests/package.lisp
(defpackage #:quasirpg-tests
(:use #:cl #:fiveam)
(:export #:run!
#:all-tests))And finally the star of the show:
;;;; tests/main.lisp
(in-package #:quasirpg-tests)
(def-suite all-tests
:description "The master suite of all quasiRPG tests.")
(in-suite all-tests)
(defun test-quasi ()
(run! 'all-tests))
(test dummy-tests
"Just a placeholder."
(is (listp (list 1 2)))
(is (= 5 (+ 2 3))))Defining a simple, argument-less test runner for the whole system
(test-quasi here) isn't strictly necessary, but it's going to
spare us some potential headaches with ASDF.
We define a meaningless test just so we can check whether the whole setup works. If you've done everything correctly, you should be able to load the test system in your REPL
CL-USER> (ql:quickload 'quasirpg/tests)and run the test runner
CL-USER> (quasirpg-tests:test-quasi)
Running test suite ALL-TESTS
Running test DUMMY-TESTS ..
Did 2 checks.
Pass: 2 (100%)
Skip: 0 ( 0%)
Fail: 0 ( 0%)
T
NILSo far, so good!
ASDF integration
Integrating the tests with ASDF is a good idea. That way we get hooked up
to the standard, abstracted way of triggering system tests. First, we
add this somewhere to our quasirpg/tests system definition.
:perform (test-op (o s)
(uiop:symbol-call :fiveam :run! 'quasirpg-tests:all-tests))From now on, we can run all-tests with:
CL-USER> (asdf:test-system 'quasirpg/tests)Next, we tell ASDF that when someone wants to test quasirpg, they really
want to run the quasirpg/tests test-op. Somewhere in the quasirpg
system definition:
:in-order-to ((test-op (test-op "quasirpg/tests")))Now all we need to do to test our game is:
CL-USER> (asdf:test-system 'quasirpg)Adding real tests
Most of the character generation system's math is within the dice-rolling function - it's probably a good idea to tackle that one. The only problem is it's not a very predictable one. We can, however, still do some useful things.
(defun test-a-lot-of-dice ()
(every #'identity (loop for i from 1 to 100
collecting (let ((result (quasirpg::roll-dice 2 10)))
(and (>= result 2)
(<= result 20))))))
(test dice-tests
:description "Test the `roll-dice` function."
(is (= 1 (quasirpg::roll-dice 1 1)))
(is (= 3 (quasirpg::roll-dice 3 1)))
(is-true (test-a-lot-of-dice)))The first two checks simply provide arguments for which the function should always spew out the same values - we're throwing one-sided dice. Just... try not to think too hard about it.
The function test-a-lot-of-dice returns true only if every one
of 100 throws of two 10-sided dice is within the
expected bounds, that is 2-20. All we have to do is check whether that
function returns true. We can just write (is (test-a-lot-of-dice)),
but I recommend using is-true instead, since the way it prints
failures is more readable in cases like this.
In all honesty, test-a-lot-of-dice could be improved in terms of
optimization (for example by making it a macro that wraps the 100 checks
in an and) or functionality (the parameters passed to roll-dice could be
random). But this version is simple and sufficient for this tutorial.
Now let's see this thing in action.
Running test suite ALL-TESTS
Running test DICE-TESTS fff
Did 3 checks.
Pass: 0 ( 0%)
Skip: 0 ( 0%)
Fail: 3 (100%)And there we go. We've just detected a bug that would never be caught by the compiler. A look at the first fail gives us a hint:
(QUASIRPG::ROLL-DICE 1 1)
evaluated to
0
which is not
=
to
1A look at the function in question should be enough to see the problem.
(let ((result (loop for i from 1 to n summing (random sides))))What (random sides) does is generate a number from 0 to (sides - 1).
That's not what we want.
(let ((result (loop for i from 1 to n summing (1+ (random sides)))))And now we re-run the tests:
Running test suite ALL-TESTS
Running test DICE-TESTS ...
Did 3 checks.
Pass: 3 (100%)
Skip: 0 ( 0%)
Fail: 0 ( 0%)The true power of tests, however, is that if we now ever decide to modify our dice-throwing facility, any bugs we introduce by accident will most likely be caught by the tests already in place. And so we'll avoid nasty, hard-to-debug consequences further down the line. All that without having to test things by hand each time we make changes.
Handling invalid parameters
What happens when someone passes a non-positive integer to roll-dice?
Or a fractional one? We should probably control that behavior. And
we should probably test to make sure when the unexpected happens,
it's handled as expected.
Let's say our specification tells us that when any of the arguments is
fractional, it should just be rounded down. So we append two
additional tests to dice-tests:
(is (= 3 (quasirpg::roll-dice 3.8 1)))
(is (= 3 (quasirpg::roll-dice 3 1.9)))The first one actually passes. It just so happens that loop is
responsible for looping N times over the random number generation
for each die. loop rounds down a fractional number if it's passed
one.
The second test requires our attention. It fails. The problem is that
random is passed a fractional argument, and it thinks it's meant
to give a fractional number in response. Simple fix and we're back
on track:
(let ((result (loop for i from 1 to n summing (1+ (floor (random sides))))))Now for something more interesting. Let's say our specification tells us
that if any argument is not a positive number, we should get
a SIMPLE-TYPE-ERROR. It's time to introduce yet another kind of check.
(signals condition &body body)Not a lot to explain here. BODY is expected to cause CONDITION to be signaled. Our check only succeeds if it does. We can use this:
(signals simple-type-error (quasirpg::roll-dice 3 -1))
(signals simple-type-error (quasirpg::roll-dice 3 0))
(signals simple-type-error (quasirpg::roll-dice -1 2))
(signals simple-type-error (quasirpg::roll-dice 0 2))
(signals simple-type-error (quasirpg::roll-dice -1 1))Again, some of the work is already done for us. random will
signal a SIMPLE-TYPE-ERROR in response to a non-positive arg.
What's left to do is to handle the number of throws, so we add
the appropriate code to the beginning of roll-dice:
(if (< n 1)
(error 'simple-type-error
:expected-type '(integer 1)
:datum n
:format-control "~@<Attempted to throw dice ~a times.~:>"
:format-arguments (list n)))And voila. Once more, all checks pass.
Random number generators
So far, we've used specific numbers. We can do better, though. We can run
a large amount of checks based on random data. This is where the
fiveam:for-all check comes in that runs tests 100 times, randomizing
specified variables each time.
(for-all bindings &body body)bindings is a list of forms of this type:
(variable generator)generator is a function (or function-bound symbol) that returns random data.
variable is the variable binding that stores the results from generator.
body can contain other kinds of checks.
For example, let's try replacing (is-true (test-a-lot-of-dice)) with
something more comprehensive.
(for-all ((n (gen-integer :min 1 :max 10))
(sides (gen-integer :min 1 :max 10)))
"Test whether calls with random positive integers give results within expected bounds."
(let ((min n)
(max (* n sides))
(result (quasirpg::roll-dice n sides)))
(is (<= min result))
(is (>= max result))))(gen-integer :min 1 :max 10) is a function provided by FiveAM that
returns a random integer generator with the specified bounds. We keep
the numbers small here so that the tests don't take forever trying
to throw a lot of dice, and so that there's a reasonable chance of edge
cases getting tested.
We can also replace the rounding checks. Since FiveAM doesn't provide a suitable generator, we have to write our own. It's not difficult, though, thanks to CL's ease of creating higher-order functions:
(defun gen-long-float (&key (max (1+ most-positive-long-float))
(min (1- most-negative-long-float)))
(lambda () (+ min (random (1+ (- max min))))))With that definition in place, we can write the new checks:
(for-all ((valid-float (gen-long-float :min 1 :max 100)))
"Test whether floats are rounded down."
(is (= (floor valid-float) (quasirpg::roll-dice valid-float 1)))
(is (>= (floor valid-float) (quasirpg::roll-dice 1 valid-float))))Finally, we can replace our condition checking too:
(for-all ((invalid-int (gen-integer :max 0))
(invalid-int2 (gen-integer :max 0))
(valid-int (gen-integer :min 1)))
"Test whether non-positive numbers signal SIMPLE-TYPE-ERROR."
(signals simple-type-error (quasirpg::roll-dice valid-int invalid-int))
(signals simple-type-error (quasirpg::roll-dice invalid-int valid-int))
(signals simple-type-error (quasirpg::roll-dice invalid-int invalid-int2))))If you run these tests, you'll notice only a few checks in the results.
That's because FiveAM treats each for-all declaration as a single check,
regardless of the contents or the hundreds of tests that actually get run.
REASON-ARGS
When the tests we've written failed, the output we got was mostly descriptive enough. That's not always the case. It's hard to expect the testing framework to know what sort of information is meaningful to us, or what the concept behind the functions we write is.
So let's say when we make-character, we want the name to be
automatically capitalized. We care about punctuation and won't
allow our players to get sloppy with it. Pshaw.
We add a new test:
(test make-character-tests
:description "Test the `make-character` function."
(let ((name (quasirpg::name (quasirpg::make-character "tom" '("str" "dex")))))
(is (string= "Tom" name))))Obviously, it fails.
Failure Details:
--------------------------------
MAKE-CHARACTER-TESTS []:
NAME
evaluated to
"tom"
which is not
STRING=
to
"Tom"
..
--------------------------------We can understand it, but put yourself in the position of
someone who isn't all that familiar with the make-character
function. Imagine that person just got the above output while testing
the entire game. They're probably really scratching their head trying
to piece this together. Let's make life easy for them. Attempt number 2:
(test make-character-tests
:description "Test the `make-character` function."
(let ((name (quasirpg::name (quasirpg::make-character "tom" '("str" "dex")))))
(is (string= "Tom" name)
"MAKE-CHARACTER should capitalize the name \"tom\", but we got: ~s" name)))We use the &rest reason-args parameter of the is check. You can use format
directives and pass it arguments, just like in a format call. Now the test result
is much easier to interpret:
Failure Details:
--------------------------------
MAKE-CHARACTER-TESTS []:
MAKE-CHARACTER should capitalize the name "tom", but we got: "tom".
--------------------------------Reorganizing
Let's imagine what happens when the project grows. For one thing, we'll probably write many more tests, until having all of them in one file looks rather messy.
We'll also probably eventually end up reorganizing the code. roll-dice
might eventually end up a part of a collection of utilities for generating
randomized results, while make-character could get moved to chargen.lisp.
It would be good if the hierarchy of our tests reflected those changes
and let us test only random-utils.lisp or chargen.lisp if we want to.
So above all of our dice-testing code we tuck this in:
(def-suite random-utils-tests
:description "Test the random utilities."
:in all-tests)
(in-suite random-utils-tests)Now all-tests contains random-utils-tests, which in turn contains
dice-tests.
Let's do the same for character generation:
(def-suite character-generation-tests
:description "Test the random utilities."
:in all-tests)
(in-suite character-generation-tests)
(test make-character-tests
:description "Test the `make-character` function."
(let ((name (quasirpg::name (quasirpg::make-character "tom" '("str" "dex")))))
(is (string= "Tom" name)
"MAKE-CHARACTER should capitalize the name \"tom\", but we got: ~s" name)))You can check that running (asdf:test-system 'quasirpg) still runs all
of our tests, since it launches the parent suite all-tests. But we can
also do (fiveam:run! 'quasirpg-tests::make-character-tests).
The next logical step is moving the test suites to separate files. If you wish to see how I've done it, just look at this commit or at the end result in the test branch.
What else is there?
A few different kinds of checks and a way to customize the way test results and statistics are presented.
So far, we've always used run! to run all the tests, which is really a wrapper
for (explain! (run 'some-test)). You can, therefore, replace the explain! function
with your own.
How can you learn about those things? The best I can do is point you to the FiveAM documentation and possibly source code.
Happy hacking!
9th European Lisp Symposium in Kraków
posted on 2016-05-11 10:37 by Daniel "jackdaniel" Kochmański
The 9th European Lisp Symposium in Kraków has ended. It was my second ELS (first time I've attended it just a year before in London). It is really cool to spend some time and talk with so knowledgeable people. The European Lisp Symposium is a unique event because it gathers people from all around the world who are passionate about what they're doing. The mixture was astonishing – the university professors, professional programmers, individual hackers, visionaries, students.
I'm glad that I have met in person many people with whom I had only the contact over the internet. I've heard about various exciting projects and ideas either during the sessions and the breaks. I have even an autograph from miss Kathleen Callaway on my Lisp in Small Pieces book. I'm also very excited that this year there was a Clojure talk, which is a modern incarnation of the Lisp idea.
During the event I had a chance to stand in front of this "angry crowd" (actually crowd of a very nice people – I still was very stressed though) during my lightning talk. I was talking about my opinions on how the contributing to the Common Lisp ecosystem should look like, how people can get involved in a productive and efficient way.
Yesterday we were on a banquet which officially closed the symposium. The food was delicious and the company was great. The only thing is that it could last a little longer. In fact many of the attendees moved to some other place to continue the meeting. I've heard they've ended late in the night (or early in the morning). I was too sleepy, so I've left to my kind host's flat.
I want to thank all the organizers and speakers for the effort they've put to make the symposium happen. Michał Psota did a tremendous job as a local chair – he managed all the local stuff and it was all perfect, Irène Durand and Didier Verna were managing things very well – everything went very smooth and only a few people got shot during the lightning talks by Didier for exceeding the time frame. I hope that I'll be able to attend the next ELS which will probably take place in Brussels.
Kraklisp and hackerspace in Kraków
posted on 2016-05-03 by Daniel "jackdaniel" Kochmański
During the last weekend I've been in Kraków to visit some friends. Since I know that there is a lisp group kraklisp on the Jagiellonian University and I know a few people in there (mainly from IRC channel #lisp-pl @ freenode), I've arrived in Kraków a bit earlier than I have originally planned to give a talk about the reader macros in Common Lisp. You may find it here (it's in polish):
Michał "phoe" Herda has met us on the bus station near the university buildings and lead us to the destination. Jagiellonian University has very beautiful buildings reminding these on Morasko in Poznań. When we've arrived at the destination we have entered KSI students association room, waited a few moments and started the workshops.
I was surprised that so many people arrived. Eleven people is a lot given that Common Lisp is considered a niche language. kraklisp brings together not only students but also lisp enthusiasts who work with Scala, Java, NetBSD and many other technologies outside the university. That's great to know, that we have an active group of lisp hackers here in Poland!
My talk took about an hour and from the feedback I infer it was just fine but the pace was a little too fast. People were actively listening and asking questions. That was fun. After me Jacek "TeMPOraL" Złydach lead a workshop about the web scrapping in Common Lisp. He has shown very nicely how to build programs interactively in the bottom-up manner. The video is available here (also in polish):
After the meeting the group has split and we have headed to the Hackerspace Kraków headquarters. Amazing place – group of people who tinker with the hardware and software in their free time. A lot of electronic devices, soldering irons, computers and boxes with an unknown content. We've chatted a little and we've learned about some nice projects they have developed – like a device mounted on your chest to locate objects in front of you. It has been created to help blind people to get around the room. Theirs location was in a little bustle, because they are currently moving to the new location. Hackerspace in Kraków is definitely worth seeing, and if feasible – cooperating with.
Finally we've moved to our stay due to the late hour (hackerspace is open 24h!). Kraków seems to be a great place to engage in a CS hobbies like programming and electronics or just to hang around with a smart people in general. I'm glad I'll arrive there again soon to attend the European Lisp Symposium.
Creating a project homepage with the SCLP
posted on 2016-04-22 by Daniel "jackdaniel" Kochmański
Introduction
In this short tutorial I'll describe how to bootstrap easily a project website. In fact that's what I did today with the Embeddable Common-Lisp website in order to provide the RSS feed and make putting there the news easier.
Additionally I'm showing here, how to create a standalone executable
for coleslaw with clon after providing quicklisp independant
bundle of systems.
Quick start
First clone the repository:
$ cd /home/p/ecl
$ git clone https://gitlab.common-lisp.net/dkochmanski/sclp.git website
$ cd websiteNow you should adjust the appropriate files. Edit .coleslawrc (file
is self-explanatory), static pages and posts.
Each file with the extension *.page is a static
page. pages/main.page is an example template with a static page –
don't forget to link it in the .coleslawrc's sitenav
section. Exact URL of the page is declared in the file's header.
Files named *.post represent blog/news posts which appear in the RSS
feed. They are indexed and accessible from the root URL. Supported
file formats are markdown, html and cl-who (if enabled).
When you're done, you could just load coleslaw with your favorite CL
implementation, using Quicklisp load coleslaw and call the function
main on the website directory:
(ql:quickload 'coleslaw)
(coleslaw:main "/home/p/ecl/website/")We will take more ambitious road – we'll create a standalone executable with a proper command line arguments built from a clean bundle produced by Zach Beane's Quicklisp. CLI arguments will be handled by Clon – the Command-Line Options Nuker, an excellent deployment solution created by Didier Verna.
Creating the bundle
Bundle is a self-containing tree of systems packed with their
dependencies. It doesn't require internet access or Quicklisp and is
a preferred solution for the application deployment.
Some dependencies aren't correctly detected – Quicklisp can't
possibly know, that our plugin will depend on the cl-who system, and
it can't detect cl-unicode's requirement during the build phase –
flexi-streams (this is probably a bug). We have to mention these
systems explicitly.
Clon is added to enable the clonification (keep reading).
(ql:bundle-systems '(coleslaw flexi-streams
cl-who cl-fad
net.didierverna.clon)
:to #P"/tmp/clw")Clonifying the application
(in-package :cl-user)
(require "asdf")
(load "bundle")
(asdf:load-system :net.didierverna.clon)
(asdf:load-system :coleslaw)
(asdf:load-system :cl-fad)
(use-package :net.didierverna.clon)
(defsynopsis (:postfix "DIR*")
(text :contents "Application builds websites from provided directories.")
(flag :short-name "h" :long-name "help"
:description "Print this help and exit."))
(defun main ()
"Entry point for our standalone application."
(make-context)
(when (getopt :short-name "h")
(help)
(exit))
(print (remainder))
(handler-case (mapcar
#'(lambda (p)
(coleslaw:main
(cl-fad:pathname-as-directory p)))
(remainder))
(error (c) (format t "Generating website failed:~%~A" c)))
(terpri)
(exit))
(dump "coleslaw" main)You may generate the executable with sbcl and ccl (ecl has some
problems with the coleslaw dependency - esrap, I'm working on
it). I have used ccl, because it doesn't "derp" on the symbol exit
and produces slighly smaller executable than sbcl.
Issue the following in the bundle directory (/tmp/clw):
ccl -n -l clonify.lispThis command should create native executable named coleslaw in the
same directory. On my host ccl produces binary with the approximate
size 50M.
Executable usage
This is a very simple executable definition. You may extend it with new arguments, more elaborate help messages, even colors.
To generate a websites with sources in directories /tmp/a and
/tmp/b you call it as follows:
./coleslaw /tmp/a /tmp/bThat's all. Deployment destination is set in the .coleslawrc file in
each website directory.
Adding GIT hooks
You may configure a post-receive hook for your GIT repository, so your
website will be automatically regenerated on each commit. Let's
assume, that you have put the coleslaw standalone executable in
place accessible with the PATH environment variable. Enter your bare
git repository and create the file hooks/post-receive:
cd website.git
cat > hooks/post-receive <<EOF
########## CONFIGURATION VALUES ##########
TMP_GIT_CLONE=$HOME/tmp-my-website/
########## DON'T EDIT ANYTHING BELOW THIS LINE ##########
if cd `dirname "$0"`/..; then
GIT_REPO=`pwd`
cd $OLDPWD || exit 1
else
exit 1
fi
git clone $GIT_REPO $TMP_GIT_CLONE || exit 1
while read oldrev newrev refname; do
if [ $refname = "refs/heads/master" ]; then
echo -e "\n Master updated. Running coleslaw...\n"
coleslaw $TMP_GIT_CLONE
fi
done
rm -rf $TMP_GIT_CLONE
exit
EOFThat's all. Now, when you push to the master branch your website will
be regenerated. By default .gitignore file lists directory
static/files as ignored to avoid keeping binary files in the
repository. If you copy something to the static directory you will
have to run coleslaw by hand.
Conclusion
Coleslaw is a very nice project simplifying managing project website
with easy bootstrapping the site without any need to maintain working
lisp process on the server (this is static content which may be served
with nginx or apache) and allowing easy blogging (write a post in
markdown and push to the repository).
Sample Common-Lisp Project is a pre-configured website definition
with a theme inspired by the common-lisp.net projects themes with
some nice features, like RSS feed and blog engine (thanks to
coleslaw).
We have described the process of creating a simple website, creating a standalone executable (which may be shared by various users) and chaining it with git hooks.