pixel (pinterface) wrote,

Persisting Simple Types with cl-perec

Don't forget to check out the other articles in this series:

  1. Getting Started with cl-perec
  2. Persisting Simple Types with cl-perec (you are here)
  3. Sensible Serializing with cl-perec
  4. Peering Down the Rabbit Hole with cl-perec

You're busy defining your data classes, when without thinking you do something like this:

(deftype scale-1-10 () `(integer 1 10))

(defpclass* pointy-haired-boss (generic-guy)
  ((hair-color :black :type (member :black :blue :green))
   (intelligence 5 :type scale-1-10)
   (moxy 5 :type scale-1-10)
   (grumpiness 5 :type scale-1-10)))

Unfortunately, while :type (integer 1 10) would have worked fine, your named type does not, instead resulting in some unhelpful output:

WARNING: Could not process type SCALE-1-10 specified for slot INTELLIGENCE, falling back to type T. The error was: Unknown type specifier SCALE-1-10
WARNING: Could not process type SCALE-1-10 specified for slot MOXY, falling back to type T. The error was: Unknown type specifier SCALE-1-10
WARNING: Could not process type SCALE-1-10 specified for slot GRUMPINESS, falling back to type T. The error was: Unknown type specifier SCALE-1-10

Not even real conditions, just stuff liable to get lost in miles of scrolling compilation notes.¹ Seemingly, this is easily solved by changing 'deftype' to 'cl-perec:defptype'.

(cl-perec:defptype scale-1-10 () `(integer 1 10))

Sadly, this comes with one rather annoying and unmentioned caveat: your package must :use cl-perec, or at least import the right symbols. You see, defptype uses defclass-star, which when generating accessor methods from slot-names does not use the slot-name's home package, no-no. It uses the current *package*, potentially resulting in undefined methods where there shouldn't be.

The defptype expansion includes this:

(defclass-star:defclass* scale-1-10-type
  ((cl-perec::name 'scale-1-10)
   (cl-perec::args 'nil)
   (cl-perec::body '('(integer 1 10)))
   (cl-perec::substituter …)
   (cl-perec::parser …))
  (:export-accessor-names-p t))

which expands into:

(defclass scale-1-10-type
  ((cl-perec::name … :accessor name-of …)
   (cl-perec::args … :accessor args-of …)
   (cl-perec::body … :accessor body-of …)
   (cl-perec::substituter … :accessor substituter-of …)
   (cl-perec::parser      … :accessor parser-of …)))

Notice how name-of, args-of, body-of, substituter-of, and parser-of are not prefixed by the cl-perec package. That's because they aren't in the cl-perec package, even though they should be, which doesn't really do us much good.

There are a few ways to work around this:²

  1. Import the necessary symbols from cl-perec into our own package.
  2. Create a secondary package which imports all the necessary symbols and exports just the stuff we want in our main package.
  3. Put an (in-package #:cl-perec) form just before our defptype and refer to our symbols package-qualified.
Basically, so long as defptype is macroexpanded in a package that includes symbols from cl-perec, it will generate code that actually does what it's supposed to. Our options are narrowed considerably, however, when we discover upon macroexpansion an assortment of symbols 'helpfully' exported, all related to the internal machinations of defptype. Because we don't want those symbols infesting our primary package, the path of least resistance—and least code—is option number 3: a well-placed in-package.

Not to be deterred from causing trouble, defptype will 'helpfully' try to export your type-name and fail because the symbol it's trying to export isn't available in the cl-perec package, so you'll also need to add an (import '(type-name)) after the in-package.³

To shield ourselves from as many problems as possible, let's package up a combination of options 2 and 3. First, a defstar-shield package.

(defpackage #:defstar-shield
  (:documentation "Beware proton torpedoes.")
  (:use #:cl #:cl-perec)
  (:shadowing-import-from #:cl-perec #:set #:time)
  (:shadow #:defptype)
  (:export #:defptype))

Then a wrapper around cl-perec:defptype.

(in-package #:defstar-shield)
(defmacro defptype (name args &body body)
  "Wrapper around cl-perec:defptype."
  (let ((package (package-name *package*)))
    `(eval-when (:compile-toplevel :load-toplevel :execute)
       (in-package #:defstar-shield)
       (import '(,name)) ; lame
       (cl-perec:defptype ,name ,args ,@body)
       (in-package ,package))))

This is a bit messy, but fairly simple: it arranges *package* so cl-perec:defptype gets macroexpanded within the defstar-shield package, making sure not to leak that package change into your surrounding code.

That finally out of the way, you continue on.

(defun string-has-even-length-p (str)
  (evenp (length str)))

(cl-perec:defpclass* tps-report ()
     :type (and string
                (satisfies string-has-even-length-p)))))

And once again, you are met with frustrating failure. cl-perec doesn't really support (and) type specifiers, and so when you try to create a tps-report you end up with errors thorough investigation reveals to be related to the string type. You see, cl-perec's type mapping is a bit awkward. For instance, the string mapping machinery gets called for an '(and string …) type, and then promptly mistakes (and …) for (string n), and tries to determine the string's length, which goes over about as well as you might expect.

At this point, we've got a couple of options: we can dig into the string mapping and fix that, or we can give our type a name and trick cl-perec into thinking it's a primitive type. I've tried to go both ways, and the latter is less work (and more in line with using a library), so that's what I'll show you.

First, we give our type a name.

(defstar-shield:defptype even-string ()
  `(and string (satisfies string-has-even-length-p)))

Then we push it onto the list of canonical types.

(pushnew 'even-string cl-perec::*canonical-types*)

This ensures cl-perec won't convert 'even-string into '(and string …), which was causing our woes. At this point, cl-perec will treat even-strings as though they were regular strings—quite possibly by accident, but it works.

(cl-perec:defpclass* tps-report ()
  ((report-title :type even-string)
   (report-text  :type string)))

And now we can generate tps-reports, with the strange and arbitrary requirement that their titles have an even number of characters.

Later, our boss comes by and says each TPS report should be barcoded for easy automatic identification.

(defun calculate-checkdigit (barcode)
  "UPC-A barcode checkdigit."
  (- (nth-value
       (loop :for digit :across barcode
             :for pos :from 1
             :for num = (digit-char-p digit)
             :sum (if (oddp pos)
                      (* 3 num)

(defun valid-barcode-p (barcode)
  "Returns true if the given string is a valid barcode."
  (and (= 12 (length barcode))
       (every #'digit-char-p barcode)
       (= (calculate-checkdigit (subseq barcode 0 11))
          (digit-char-p (char barcode (1- (length barcode)))))))

(defstar-shield:defptype barcode ()
  `(and (string 12) (satisfies valid-barcode-p)))
(pushnew 'barcode cl-perec::*canonical-types*)

(cl-perec:defpclass* tps-report ()
  ((barcode      :type barcode)
   (report-title :type even-string)
   (report-text  :type string)))

You get all that coded up only to discover your barcode type isn't creating a fixed-length field in the database like it should. All our fancy workarounds are coming back to haunt us!

We could backtrack, figure out how to fix the combination of (and string) types in cl-perec, and go from there. Or, we could trudge along delving further into the scary guts of cl-perec.

Delving farther in than a library user should probably go, we discover cl-perec::defmapping, which looks promising. Copying things from the default string defmapping and modifying slightly, we come up with something that works.

(cl-perec::defmapping barcode
  (cl-rdbms::sql-character-type :size 12)

It's pretty scary that everything we use is unexported, but what we're doing is pretty simple: this says the lisp-type 'barcode equates to an SQL string 12 characters long. identity-reader and identity-writer are the functions that convert a string from SQL into a lisp object—and vice versa—in this case, by doing nothing.

Regrettably, this all feels pretty hacky. After all, we've just gone to a lot of trouble for no other reason than that cl-perec doesn't properly support (and string satisfies) types. But we've got an application to write, so it'll have to do for now.


  1. a comment inside cl-perec notes due to the MOP we must not fail when this is called, otherwise the entire (sblc) [sic] image breaks. So much for using the debugger!
  2. I don't include fixing defstar-class in this list. For all I know, it does this stuff intentionally.
  3. This means your type name cannot conflict with any symbol in your defstar-shield package. If this might be a problem, be wary of which packages are used by your defstar-shield package, possibly limiting yourself to importing only necessary symbols.
  4. It also breaks the return value, but you didn't want that anyway.
  5. Common Lisp lacks any standardized facility to perform type inspection, so we can't really blame cl-perec for the defptype requirement.
  6. Common Lisp type specifiers are turing-complete, which makes full automatic conversion to database types impossible. (and string) may not seem difficult, but it isn't far to a fairly hairy type specifier.

Continue to Sensible Serializing with cl-perec

Tags: cl-perec, lisp, programming, tutorial
  • Post a new comment


    default userpic