pixel (pinterface) wrote,

CFFI Tricks: Creating a Lisp Version of a Missing C Function

Working on burgled-batteries, I found that a non-trivial portion of Python's C API isn't actually exported functions, but preprocessor macros disguised as functions. In the interests of continuing to work if they go back and forth on any given function, I thought it would be a good idea to automatically detect if the function exists, and use an alternative Lisp implementation if it didn't.

Now, the thing about replacing C code with Lisp code is that I want the Lisp code to run as if it were C. In other words, for all of the type-translation of CFFI to have already occurred by the time the Lisp code has been called, and to also occur on the return value. CFFI, naturally, does not anticipate this use-case. Fortunately, Lisp is not a bondage-and-discipline language; also fortunately, CFFI's internals are perfectly aligned for this particular, ah, "creative restructuring".

(let* ((realfn (symbol-function 'cffi::defcfun-helper-forms))
       (lisp-args ...)
       (alt-body ...)
       (ourfn (make-defpyfun-helper-forms lisp-args alt-body decl))
       (doc+decl ...))
  ;; We temporarily replace cffi::defcfun-helper-forms and call
  ;; cffi::%defcfun to create the defun so we can ensure the body
  ;; of our alternate function runs having been given pointers as
  ;; if it were C.
         (setf (symbol-function 'cffi::defcfun-helper-forms) ourfn)
         (cffi::%defcfun lisp-name c-name return-type c-args nil doc+decl))
    (setf (symbol-function 'cffi::defcfun-helper-forms) realfn)))

If you're not familiar with CFFI's internals, #'cffi::%defcfun is the function which produces the macroexpansion for DEFCFUN. Quite handy that it was already split out for us to call. #'cffi::defcfun-helper-forms is the function #'%defcfun uses to determine what form to use to call a C function. Fate smiles upon us: #'%defcfun produces the type translation code, and wraps it around the form returned by #'defcfun-helper-forms. Since we want to run code in place of calling a C function, all we have to do is get D-H-F to spit out the code we want to run.

So we replace D-H-F with a call to our own function (produced by #'make-defpyfun-helper-forms) and let #'%defcfun do its thing none the wiser.

#'make-defpyfun-helper-forms isn't particularly complicated. It binds the gensym'd variables from CFFI back into the user-provided variables, and includes any relevant user-specified declarations.

(defun make-defpyfun-helper-forms (wrapped-args wrapped-forms declarations)
  "When called, produces a function which is intended to take the place of
#'CFFI::DEFCFUN-HELPER-FORMS within a CFFI:DEFCFUN expansion.  This is used when
creating an alternate definition for a C function which either doesn't exist in
the currently-included Python library (e.g., because it belongs to a newer
version of Python, or because it's really a preprocessor macro)."
  (lambda (name lisp-name rettype args types options)
    (declare (ignore name lisp-name rettype types options))
    (values '()
            `(let ,(mapcar #'cl:list wrapped-args args)
               ,@(%filter-declarations declarations :for wrapped-args)

It's somewhat awkward to write C-equivalent code within Lisp; largely because it's necessary to keep in mind that the return value must also be a C value (e.g., 1 and 0 for true and false, rather than t and nil). But it also means library users have less to worry about, because some of the minor differences between Python versions can be papered over.

Lots of scary internal machinations allow us to provide a (hopefully) straightforward interface to create Lisp-side reimplementations of C code:

(defcvar ("Py_Py3kWarningFlag" *err.warn-py3k* :read-only T) :boolean)
(defpyfun "PyErr_WarnPy3k" 0-on-success ((message :string) (stacklevel :int))
   (if *err.warn-py3k* (err.warn-ex +exc.deprecation-warning+ message stacklevel) 0)))
(defpyfun "PyErr_NewExceptionWithDoc" object ((name :string) (doc :string) (base object) (dict dict))
   (declare (ignore doc))
   (err.new-exception* name base dict)))
Tags: burgled-batteries, cffi, lisp

  • CL-FTP Bugs Squashed

    One of the perks of being unemployed are that I have a reasonable amount of downtime again, so I've been spending some of it updating libraries…

  • Clojure Is For Me, Go Is For My Team

    The Basic Problem The Method Inheritance Is… Uh… Missing? Clojure Go Subjective Comparisons Recommendation Available…

  • I'm Giving A Talk Today!

    Apologies for the short notice--I've been rushed just trying to get the darn thing ready--but I will be giving a talk at Iowa Code Camp on…

  • Post a new comment


    default userpic
    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.