#+author: Daniel Kochmański
#+date: [2026-08-17 Mon]
#+title: Selection protocol
#+status: draft
#+version: 1.0

* Introduction

This protocol proposal replaces the ad-hoc synchronous protocol that was used in
McCLIM until now. Its purpose is to be general enough to be cleanly mapped to
existing clipboard implementations and to add robust extension mechanism, so
that sophisticated clients do not need to modify the backend:

- present itself as asynchronous abstraction (subsumes synchronous clipboards)
- specify a default list of formats that are expected to work (string, design)

Currently this protocol does _not_ specify semantics for defining capabilities,
managing permissions, negotiating transfer method, publishing multiple items,
adding backpressure and size limits and (probably) more. These concerns are left
for backends to figure out.

* Formats and translators

When an object is published or requested it may have multiple formats. For
example a text may be plain or may have additional styling. We distinguish lisp
formats and port formats. For example:

- lisp format ::  :STRING, :DESIGN
- wire format  :: 'FOO-BACKEND:|text/plain|, #<class foo-backend:image>

Notably formats are not lisp object types. This protocol is specified so that it
is possible to add new formats modifying the backend. To achieve that, it is
necessary to provide a translation function between formats:

  PORT-MATERIALIZE-SELECTION PORT SHEET LISP-OBJECT LISP-FORMAT WIRE-FORMAT
  PORT-INTERNALIZE-SELECTION PORT SHEET WIRE-OBJECT WIRE-FORMAT LISP-FORMAT

Moreover when we publish an object that has multiple available lisp formats that
may be served, we need to be able to tell the corresponding list of wire
formats, and when we request an object, we want to map port-specific advertised
formats to a list understandable by the application:

  PORT-LIST-SELECTION-WIRE-FORMATS PORT SHEET LISP-FORMAT
  PORT-LIST-SELECTION-LISP-FORMATS PORT SHEET WIRE-FORMAT

These ordered lists are not necessarily symmetrical. Then there is finally a
function that defines equivalence of two formats:

  PORT-SELECTION-FORMAT-EQUAL-P PORT FORMAT-1 FORMAT-2

These five functions may be used to define necessary piping between lisp and the
backend and they allow downstream clients to extend the backend behavior.  This
pseudocode illustrates the flow (it is not normative):

#+begin_src lisp
  ;;; Helpers

  (defun compute-wire-formats (port sheet lisp-formats)
    "Computes all wire formats achievable from lisp formats (lisp->wire)."
    (remove-duplicates
     (mapped (curry #'port-list-selection-wire-formats port sheet) lisp-formats)
     :test (curry #'port-selection-format-equal-p port)
     :from-end t))

  (defun compute-internalization-formats (port sheet wire-formats lisp-formats)
    "Computes an assoc list (SOURCE-WIRE-FORMAT . TARGET-LISP-FORMATS)."
    (labels ((match-1 (wire-format lisp-format)
               (member lisp-format
                       (port-list-selection-lisp-formats port sheet wire-format)
                       :test (curry #'port-selection-format-equal-p port)))
             (matcher (wire-format)
               (cons wire-format
                     (remove-if-not (curry #'match-1 wire-format) lisp-formats))))
      (stable-sort (remove nil (mapcar #'matcher wire-formats) :key #'cdr)
                   #'<
                   :key (lambda (pair)
                          (position (second pair) lisp-formats)))))

  ;;; Publish selection and serve external request

  (defun fulfill (port sheet object lisp-formats wire-format)
    (dolist (lisp-format lisp-formats nil)
      (when (member wire-format
                    (port-list-selection-wire-formats port sheet lisp-format)
                    :test (curry #'port-selection-format-equal-p port))
        (multiple-value-bind (wire-object successp)
            (port-materialize-selection port sheet object lisp-format wire-format)
          (when successp
            (%send wire-object wire-format)
            (return-from fulfill t))))))

  (defun publish (port sheet object &rest lisp-formats)
    "Advertises OBJECT to be available under specified lisp formats."
    (let ((wire-formats (compute-wire-formats port sheet lisp-formats)))
      (%port-advertise port wire-formats)
      (fulfill port sheet object lisp-formats (%wait-for-request port))))

  ;;; Request selection

  (defun negotiate (port sheet requested-lisp-formats)
    (let ((advertised-wire-formats (%port-list-advertised-formats port)))
      (compute-internalization-formats port sheet
                                       advertised-wire-formats
                                       requested-lisp-formats)))

  (defun request (port sheet &rest requested-lisp-formats)
    (dolist (path (negotiate port sheet requested-lisp-formats))
      (destructuring-bind (wire-format . lisp-formats) path
        (let ((wire-object (%recv wire-format)))
          (dolist (lisp-format lisp-formats)
            (multiple-value-bind (lisp-object successp)
                (port-internalize-selection port sheet wire-object wire-format lisp-format)
              (when successp
                (return-from request lisp-object)))))))
    (error "Can't handle the request."))
#+end_src

The real implementation should concern itself with selection boxes, events,
timeouts and asynchronous execution that are described below. The primary
purpose of the code above is to illustrate negotiation and conversion.
translate object between lisp and port formats.

Normative negotiation rules:

1. Requested lisp-format order
2. Advertised wire-format order
3. On translation failures, try the next path
4. On unexpected errors signal SELECTION-CONVERT-ERROR

  SELECTION-CONVERT-ERROR
    selection-failure
    selection-cause

SELECTION-FAILURE is :TRANSLATE on error and :UNSUPPORTED if no translators. 
SELECTION-CAUSE contains the message or a condtion denoting the failure.

* Selection boxes

Selections are published and requested from boxes, that is distinct clipboards
denoted by names. Some predefined boxes have a default treatment:

- :CLIPBOARD :: The clipboard holds a result of the last kill operation which
  may be used in a yank operation.

  When unavailable on the display server, it is implemented only in the local
  image - then the selection box is local to the port.

- :PRIMARY :: Primary selection on a display server. It contains selected
  objects without explicit copy operation.

  When unavailable on the display server, it is implemented only in the local
  image - then the selection box is local to the port.

- :SECONDARY :: Secondary selection on a display server. By default unused, but
  applications may utilise it as a secondary selection that does not overwrite
  the primary selection.

  When unavailable on the display server, it is implemented only in the local
  image - then the selection box is local to the port.

There is also a special selection box that is shared by all ports in the image
that manages direct references to objects:

- :LOCAL-SELECTION :: Active selection inside the image ports. This type of
  clipboard is available regardless of the backend support.

* Selection objects

Selection objects are represented by instances of SELECTION-PUBLISH and
SELECTION-REQUEST:

  SELECTION-PUBLISH
    SELECTION-BOX      selection box
    SELECTION-SHEET    selection source
    SELECTION-TOKEN    backend-specific token (i.e a timestamp)
    SELECTION-STATE    publish state (:CREATED, :ACTIVE or the clear event)
    SELECTION-FORMATS  advertised lisp formats
    SELECTION-VALUE    published object

  SELECTION-REQUEST
    SELECTION-BOX      selection box
    SELECTION-SHEET    selection target
    SELECTION-TOKEN    backend-specific token (i.e a timestamp)
    SELECTION-STATE    request state (:CREATED, :ACTIVE or the termination event)
    SELECTION-FORMATS  requested lisp formats

Reusing the same selection object in multiple publications and requests is
forbidden - each object is its own handler compared with EQ. Selection objects
initially are in the state :CREATED that changes to :ACTIVE when submitted to
the port. When they are terminated, the state contains the termination event.

The selection state is modified by the port. The backend must ensure that the
transition is atomic to prevent races.

** Predefined selection formats

There are a few predefined formats to make cooperation easier:

- :STRING :: string representing text
- :DESIGN :: design representing graphics
- :OBJECT :: arbitrary lisp object

Each backend should specialize :STRING and :DESIGN in PORT-INTERNALIZE-SELECTION
and PORT-MATERIALIZE-SELECTION to match them with adequate backend formats.

The format :OBJECT is meangiful only in the selection box :LOCAL-SELECTION.
Backends should treat this format as :UNSUPPORTED.

When specifying selection formats, both backends and clients should use either
symbols internal to their packages, or provide specializable objects. This is to
ensure that there are no conflicts between toolkits and backends.

** PUBLISH protocol

Only one publication may be published at the same time per box. Publishing a new
overwrites the old one in the box. Release releases the publication only when
the SELECTION-PUBLISH is in the selection box.

  (PORT-PUBLISH-SELECTION PORT SELECTION-PUBLISH)

Publishes the object on the port. SELECTION-PUBLISH uniquely identifies the
selection. Publishing the same object should signal SELECTION-PUBLISH-ERROR.

  (PORT-RELEASE-SELECTION PORT SELECTION-PUBLISH)

Returns true only when the operation released a publication. If the publication
was not inside the box, returns false.

  SELECTION-CLEAR-EVENT
    selection-publish

This event must be distributed by the port when a third party client overwrites
our selection on the display server.

The same event must be synthesized when PORT-RELEASE-SELECTION succeeds and when
PORT-PUBLISH-SELECTION replaces the existing selection.

  SELECTION-PUBLISH-ERROR
    selection-publish
    selection-failure
    selection-cause

A condition signaled when publishing the object fails. Notably when the client
tries to reuse the same object multiple times.

** REQUEST protocol

Requests are cancellable and asynchronous with an optional timeout:

  (PORT-REQUEST-SELECTION PORT SELECTION-REQUEST &key (TIMEOUT 1))
  (PORT-CANCEL-SELECTION-REQUEST PORT SELECTION-REQUEST)

TIMEOUT is specified in seconds. The timeout begins when the function is
called. Zero is allowed, but will succeed only on selection boxes that have data
immediately available (notably local selection, otherwise depends on the actual
backend). Timeout must be non-negative real number.

Requesting the same object should signal SELECTION-REQUEST-ERROR.

The function PORT-CANCEL-SELECTION-REQUEST returns true if the cancellation was
successful, otherwise it returns NIL when the request already terminated.

The request will result in one of two events being (eventually) dispatched:

  SELECTION-PASTE-EVENT
    selection-request
    selection-value
    selection-format

  SELECTION-ABORT-EVENT
    selection-request
    selection-failure
    selection-cause

SELECTION-FAILURE contains the best-effort match of the issue to this set:

- :NAUGHTY     :: the client did something naughty
- :TIMEOUT     :: the timeout was reached
- :ABORTED     :: the request was cancelled
- :TRANSLATE   :: error was encountered during translation
- :DENIED      :: the peer denied us the request
- :TOO-LARGE   :: the backend refused because the selection is too big
- :UNSUPPORTED :: the backend could not handle the available request
- :UNAVAILABLE :: the peer could not satisfy the request
- :UNKNOWN     :: the failure happened due to an unspecified reason

SELECTION-CAUSE contains the underlying condition.

When cancellation is requested with PORT-CANCEL-SELECTION-REQUEST, it is still
possible that the request was finished. In a case of such race:

- request finished :: PORT-CANCEL-SELECTION-REQUEST returns false
- cancel committed :: PORT-CANCEL-SELECTION-REQUEST returns true

The same applies to timeouts - the first committed completion state wins. Every
submitted request results in exactly one terminal state and dispatched event.

  SELECTION-REQUEST-ERROR
    selection-request
    selection-failure
    selection-cause

A condition signaled when requesting the object fails. Notably when the client
tries to reuse the same object multiple times, but also in synchronous requests
in the simplified sheet API when the request can't be fulfilled.

* Simplified sheet API

For convenience we provide sheet protocols that map directly to port protocols
and provide additional convenience functions at the expense of some limitations.

  PUBLISH-SELECTION sheet box object format

Constructs and puts the publication in the box. The backend maps the format to
port-specific formats and advertises them. Returns SELECTION-PUBLISH instance
that serves as an identity.

  RELEASE-SELECTION sheet selection-publish

Removes the publication from the box. When the box does not contain the
publication returns false.

  REQUEST-SELECTION sheet box format &key (timeout 1)

This is a synchronous operation that waits for the selection of a specified
format. This function will advance the event queue in single-threaded builds.

On success this returns the requested object, otherwise signals a condition
SELECTION-REQUEST-ERROR.

* Constructors

  MAKE-SELECTION-PUBLISH (box sheet object lisp-format &rest lisp-formats)
  MAKE-SELECTION-REQUEST (box sheet        lisp-format &rest lisp-formats)

* Reference index

Conditions

  SELECTION-ERROR
    SELECTION-PUBLISH-ERROR
    SELECTION-REQUEST-ERROR
    SELECTION-CONVERT-ERROR

Events

  SELECTION-EVENT
    SELECTION-CLEAR-EVENT
    SELECTION-PASTE-EVENT
    SELECTION-ABORT-EVENT

Objects

  SELECTION-ACTION
    SELECTION-PUBLISH
    SELECTION-REQUEST

Port protocol

  PORT-SELECTION-FORMAT-EQUAL-P
  PORT-LIST-SELECTION-WIRE-FORMATS
  PORT-LIST-SELECTION-LISP-FORMATS
  PORT-INTERNALIZE-SELECTION
  PORT-MATERIALIZE-SELECTION  

  PORT-PUBLISH-SELECTION
  PORT-RELEASE-SELECTION

  PORT-REQUEST-SELECTION
  PORT-CANCEL-SELECTION-REQUEST

Sheet protocol (consumer API)

  PUBLISH-SELECTION
  RELEASE-SELECTION
  REQUEST-SELECTION

* Limitations

** TODO Transfer modes

Currently protocol supports only an immediate transfer mode. This section is
left unfinished for the future protocol work.

Backends are allowed to transparently construct immediate objects from other
transfer modes or fail with a reason :UNSUPPORTED.

- :IMMEDIATE :: the request is fulfilled with a single event
- :STREAMING :: the request is fulfilled incrementally
- :NATIVE-HANDLE :: the request is fulfilled using a backend-specific method

** TODO Multiple items

Currently the protocol operates always on a single item, although some systems
provide an API that allows managing multiple items per selection. The main
problem is that these systems vary with capabilities and negotiation strategies.
That also complicates the specification.