Return Styles: Pseud0ch, Terminal, Valhalla, NES, Geocities, Blue Moon.

Pages: 1-

Lisp

Name: Anonymous 2011-02-04 8:39

So I was making a kind-of-real-time 4chan interface in C# and then I though "holy fuck, why am I not using Lisp?"

I'm hoping that Lisp has thread and regexii support and good HTML parsing/XPath libraries, and I'd be grateful if you guys could describe the various ways in which I could enhance the syntax or use an embedded DSL or whatever in Lisp (macros, I suppose? There are different types of them, aren't there?).
And also, what are the differences between various Lisps?

Name: Anonymous 2011-02-04 8:46

Why not just use Greasemonkey?

Name: Anonymous 2011-02-04 8:49

I'm hoping that Lisp has thread and regexii support and good HTML parsing/XPath libraries

You are looking for Racket.

http://racket-lang.org/

Name: Anonymous 2011-02-04 9:06

>>2
Because it's also going to be an archive, and a will be mainly used for data mining (stalking).

>>3
Thanks.

Name: Anonymous 2011-02-04 9:18

>>4
I think it's possible to invoke wget (or any other program) from within Javascript code using XPConnect, or something. If it is indeed possible, perhaps it would save you some trouble.

Name: MADE IN LISP 2011-02-04 9:18

>cancel download

Name: >>5 2011-02-04 9:19

On Firefox, I mean.

Name: Doctor Racket !RACKET/.HY 2011-02-04 9:20

You're pretty vague about what kind of Lisp you need.
Just ``Lisp'', today, is used either as an umbrella term for all the Lisp derivates or to mean Common Lisp, I'll assume the first, as you also asked the difference between them.

Lisp, in the years, always spawned various derivatives for being a simple language and trivial to parse, but these days there are only two main branches: Common Lisp and Scheme.

Now, let me introduce them.
Common Lisp is born to be a, well, ``common'' Lisp, to unificate various features from the other Lisps, being ultimately based on MacLisp.
You surely have heard that Common Lisp (abbreviated CL, never CLISP) is a ``Lisp-2'' or ``Lisp-n'', it means that symbols can have multiple meanings.
For example, you can do
(setq foo 2)
(defun foo (x) x)
(foo foo)

and will call the function `foo' with the previously setqed variable, also named `foo', without name clashes.
You also know that Lisp is famous for being able to pass functions as argument, so to disambiguate it from a variable, you need to use (function ...) or #' prefixed to the function's name, and then call it with (funcall ...):

(defun foo (x) x)
(defun bar (f x) (funcall f x))
(bar #'foo 3) ; this really expands to (bar (function foo) 3)

You may think this like a ``quote for functions''.
The need for funcall and #' may appear unnecessary, but has its advantages.

Scheme born to be simple and elegant, and it's a ``Lisp-1'', which means that your functions and your variables share the same namespace, but it's usually simpler to use for newcomers to Lisp.

(define (foo x) x)
(define (bar f x) (f x))
(bar foo 3)

Will do exactly what you think. In fact, function definitions just bind a procedure to a variable, and the (define (foo x) x) syntax is just a shorthand to (define foo (lambda (x) x)).
This simplify things, but that means you can't name your variables `list', `string', etc, without shadowing those functions.

Common Lisp has an ANSI standard and much more mature implementations than Scheme, which has gained some serious and fast implementations just recently.
CL code, having a quite big standard, is mostly portable between all the implementations.
Scheme, instead, has a 55 pages long standard (the R5RS, the most widespread one), so implementations are usually self-contained and non-portable. There are libraries that are portable between the R5RS Schemes (like SLIB). This doesn't mean that you can't use Scheme, it just means that you have a wider choice.
I suggest you Chicken, which can be as fast as C (and it's also compilated to C), and has a excellent library support and FFI, and Racket, which is JIT compilated and provides many nice features and a big library (and the FFI is simply fantastic).

Here some libraries for what you asked:
http://www.cliki.net/thread
http://www.cliki.net/CL-PPCRE
http://www.cliki.net/CL-HTML-Parse

http://docs.racket-lang.org/reference/regexp.html
http://docs.racket-lang.org/reference/threads.html
http://docs.racket-lang.org/html/index.html
http://docs.racket-lang.org/xml/index.html

http://wiki.call-cc.org/man/4/Unit%20regex
http://wiki.call-cc.org/man/4/Unit%20srfi-18
http://wiki.call-cc.org/chicken-projects/egg-index-4.html#web

The choice between CL, Scheme (Chicken, Racket), depends on what you want, I can't decide for you.

Name: Anonymous 2011-02-04 9:24

>>8
oh u

Name: Doctor Racket !RACKET/.HY 2011-02-04 9:26

>>9
Sorry, only quality posts on my /prog/.

Name: Anonymous 2011-02-04 9:27

>>8
Oh gosh, I haven't expected to get such a in-depth reply! Thanks!!

Name: Anonymous 2011-02-04 9:32

>>10
gtfo namefag

Name: Anonymous 2011-02-04 9:35

>>11
You're welcome.

Name: Anonymous 2011-02-04 9:36

Check my base 13 dubs.

Name: Anonymous 2011-02-04 9:41

>>12
And then /prog/ imitates /b/.

Name: Anonymous 2011-02-04 9:42

>>15
Change name to /prob/, we are here. That is all!

Name: Anonymous 2011-02-04 10:44

Might as well put my earlier work to use while you're at it

http://dis.4chan.org/read/prog/1232487233

Name: Anonymous 2011-02-04 11:31

Name: Anonymous 2011-02-04 13:12

Name: Anonymous 2011-02-04 13:43

You can also interact with Java code from Javascript in Firefox. Since you were already considering C# anyway, Greasemonkey + Java might get you a working prototype in a shorter time.

Name: Anonymous 2011-02-04 16:48

>>20
I see. But originally I intended the SYSTEM to incorporate a separate archive+interface that will run on a remote server and (possibly several types of) clients.

I have the archiver done and I'm just adding an interface to it now, but since I'm so close to completion I'm losing most of my motivation. Also since the interface is sort of "tacked on" to the archive, the design could be cleaner. Also I'll have to bother with MONO, so I thought I might as well rewrite the thing in Lisp eventually.

>>17
What I'm doing now is just the server, which won't download any images because lol possible illegal content.
I tried to optimize the number of fetches it needs to do, what the main loop does is:
  * download and parse the 0th page
  * if possible, update the 10/15 threads with just the data from the front page
  * keep track of all post numbers on the board, check which ones were missed
  * use the sys.4chan.org interface to see which threads the missed posts are in
  * full-refresh those threads
I see keeping the data up-to-date as more more important, so the archive will full-update a thread even if the only thing missing is a single filename or a comment.
Also I haven't considered gzip compression, I'll do that, thanks!

The interface exposed by the archiver will allow for "subscribing" to a number feeds, such as "new threads" or "new posts" or "new images"; the client can use the data however it wants, and can also specify which data about, say, posts it wants.
The client can also query the SQL database with all the archived data.

Name: Anonymous 2011-02-04 19:47


Don't change these.
Name: Email:
Entire Thread Thread List