A Tangential Discourse on Lisp and the Lineage of Languages

[Levi Pearson]

On Mar 13, 2006, at 11:35 AM, Bryan Sant wrote:
>
> When I run sbcl and then the load instruction, I get the following  
> error:
>
> debugger invoked on a SB-INT:STREAM-DECODING-ERROR in thread  
> 3085018816:
>   decoding error on stream
>   #<SB-SYS:FD-STREAM for "file \"/usr/share/dict/words\"" {9075801}>
>   (:EXTERNAL-FORMAT :UTF-8):
>     the octet sequence (243 110 10 65) cannot be decoded.
>
> Why can't lisp parse the words file?  UTF8 problem?


That's what it looks like, but it works on my Debian box with sbcl  
0.9.9 and a /usr/share/dict/words with extended chars, but it's  
encoded in latin-1, afaict.  What versions of things are you using?   
Is the words file actually encoded in utf-8, or did it get confused  
about it somehow?  You might try replacing the (defvar *words* form  
with this, to force latin-1:

(defvar *words*
   (with-open-file (words *words-file* :direction :input :external- 
format :latin-1)
     (let ((hash (make-hash-table :test 'equal)))
       (loop (let ((line (read-line words nil)))
               (if line
                   (setf (gethash line hash) t)
                   (return))))
       hash)))

		--Levi