Compilation Speed

I have this code:
http://rghost.net/4410616

And it kills SBCL. But if I replace my-lst with

(defmacro my-lst (&rest xs)
  `(list ,@xs))
it compiles much faster.

A good amount of lambdas make SBCL crawl. Why doesnt it do lambda reduction before all?

Without lambdas,

(eval (cons 'list (loop as i from 0 below 200 collect (list 'vector 1))))

gives no problem.

{m functions considered harmful}
Everything should be inside a big {code main} function, and we should just use goto and ifs.

Basically, the question is:

why, given

(defun vecs1 (n)
  (cons 'vector (loop as i from 0 below n collect (list 'vector 1))))

(defun vecs2 (n)
  (let ((gs (loop as i from 0 below n collect (gensym ""))))
    (labels ((r (xs)
               (if xs
                   `((lambda (,(car xs)) ,(r (cdr xs)))
                     (vector 1))
                   `(vector ,@gs))))
      (r gs))))
, (eval (vecs2 400)) is much slower than (eval (vecs1 400))?

>>5
vecs1 is just a simple vector of vector code generation function and it's trivial to compile and evaluate (it will just compile to an in-memory constant), on the other hand vecs2 invokes the compiler to do a much more complex job involving compiling multiple lambdas (functions in their own right), in which case over 400 functions in your second case, as opposed to just one in the first case. Of course having your compile compile 400+ functions will be slower than having it compile a single function (if at all).

EVAL is traditionally handled in two ways. Native-code compilers like SBCL (and others) will treat eval similar to
(defun eval (expression) (funcall (compile nil `(lambda () ,expression)))) (this is not 100% accurate as SBCL does do plenty of quick optimizations where possible, however such an implementation is usually the gist of it, if it encounters a lambda or something not trivially evaluatable). Which means that the interpreter is unneeded and can just be replaced to a call to the compiler. This is very wasteful in the second example, but not too wasteful in the first. (SBCL also has an interpreter, but I'm not sure how maintained or usable it is, I've never tried using it).

If you were to run EVAL in an implementation that just implements it metacircularily (such as how it's usually done in Scheme), or merely implementations which have full-fledged interpreters (CLISP and possibly AllegroCL have one), you would find that the second piece of code would be ran faster than it takes SBCL to compile it. I still do expect the second piece of code to run a bit slower, regardless of wether a real interpreter is used or not as it does involve more argument passing and setting up stack frames (function calling) as opposed to the first one.

tl;dr: Have a lot of random and complex code to execute at runtime that can't wait to run through a full-fledged compiler? Use an interpreter - either provided by the implementation or your own. If not? Enjoy your implementation provided eval which will most likely just compile your code first then run it.

>>6
>a lot of random and complex code
No. The sequence of lambdas in vecs2 is pretty simple and linear enough, so that with a fast lambda reduction could be reduced to that of vecs1 (meaning no lambdas).

>>7
And BTW, vecs2 causes exponential slowdown, compared to vecs1

>>6
compile your code first then run it.
Java also compiles code, but compilation speed isnt a problem, because that has JIT and knows about "Hot Spots". This leads to conlusion, that from perfomance point Java superior.

>>7
Less babbling, more understanding. >>6 was recommending that if you have lots of random and complex code to execute, you should use an interpreter. And lambda reduction doesn't help here, since it doesn't happen before compilation (duh), which is the pointless step you're trying to avoid.

>>9
Lolwut. If you tryed to compile Java source at runtime for lots of one-off code it sure would be a problem. Especially considering javac will compile, then the JVM will undo a chunk of that work on loading to make it easier to work with. Then the actual JIT will probably never happen, since you're only running once.

compile my dubs

<-- dirty mutha fukken dubs yo!

>>11
You, sir, are a bastard.

>>10
>Lolwut. If you tryed to compile Java source at runtime for lots of one-off code it sure would be a problem
It would just generate unoptimized assembly.

In my case, I get |vecs2| code from a big config file in the form of [a=[name1=value1] b=[name2=value2]...] it runs only on startup and needs recompilation every time, changes are made.

>>10
>And lambda reduction doesn't help here, since it doesn't happen before compilation (duh), which is the pointless step you're trying to avoid.
Lambda reduction is fast by itself. For mere 400 lambdas you wont notice speed difference from vecs1.

too lisp; didn't read

loop print; eval read

SBCL is 100 times slower than CLISP


$ clisp
[1]> (time (eval (vecs2 400)))
Real time: 0.16027 sec.
Run time: 0.16001 sec.
Space: 483744 Bytes


$ sbcl
* (time (eval (vecs2 400)))
 
Evaluation took:
  16.87 seconds of real time

>>18
Only if you decide to use the default EVAL which compiles all code before running it. If you use a real interpreter, it may end up even faster (there's one for SBCL, afaik).

/prog/ will be spammed continuously until further notice. we apologize for any inconvenience this may cause.