I'm implementing a Lisp

Discuss how the perfect module/namespace should behave.  Keep in mind things like multimethods (which should be merge rather than shadow).

>>40

this, this, I mean this!

>>3
Lambda-calculus already gravitates towards a simple "environment" argument, which holds "dynamic" state. Why would you implement some external module/namespace hack? Do you miss C++ so much?
So "importing stuff in other files" should work as C's #include directive?

The more I think about it, the more confused I get.

>>45

I think ey was talking about this. include might be a different issue. In java, definitions from other classes defined in other files are resolved by looking for class files in a certain place, and the inclusion dependency isn't specified by the programmer using includes.

>>30
mind = blown

>>46
I just wanna know how they do it in CL (which is considered to be a good language that scales well), preferably without reading a 1000 page standard.

>>48
you could check out one of the module systems in one of the scheme implementations. They'll give you all the details, and are usually simpler than lisp in general, although racket's looks pretty sophisticated.

Here is chicken scheme's

http://wiki.call-cc.org/man/4/Modules

It actually looks about as complex as I remember CL's being.

You should think about what your needs will be and design around that. As long as you are developing the language, you can mess around with different approaches and see what works and what fails when.

>>48
Not sure what you want to know about CL, but it has:
- value and function namespaces (separate), so Lisp-1 vs Lisp-2, or to put it simpler, a symbol has both a value and a functional value
- many other namespaces also exist, such as symbols may define types, classes, structures, ... ; A good implementation will allow you to view all this interactively, but you can also inspect all of these programatically and likely portably for most cases. Defining some extra namespace on a symbol can be done rather easily and is a typical 'pattern' the CL programmer uses (it may be too simple to be called a pattern) - it's like defining some documented interface to access some namespace, then just using some hashtable behind the scenes to associate symbols to values.
- symbols belong to packages, but a symbol can also be uninterned (such as gensyms, but you can also make your own) if it belongs to no package. Symbol comparison is a pointer comparison (eq), and symbols in the same package are equal on virtue of the interning process and how the reader works.
- packages themselves can be modified at runtime, the same with symbols and various things you can access about a symbol, they're just objects in their own right
- there's also the whole lexical and dynamic scope, which can be controlled very exactly using declares (but also has reasonable defaults)
- fancier forms of handling the build-process is not described in the standard, you have your compile and compile-file functions which let you compile either functions or larger units of code (there's a lot of other functions, macros and special forms allowing very tight control of evaluation, compilation, read-time and compilation units). Typically you implement the build process via systems such as ASDF which build on top of CL, but may also integrate with the implementation or environment (mostly keeping portability, but some unportable stuff is to be accepted as yoy might also want to be dealing with a bit of system-specific stuff when compiling large projects)
- Executable files and other binary modules are not defined in the standard, but some implementations support either compiling/linking FASLs (Lisp's object files, post-compilation) into executables, while others support dumping the in-memory image after some sanitizations to an executable (after specifying an entrypoint function)
- multimethods? CLOS supports them and allows great flexibility in their ordering, execution, wrapping, etc. CLOS is implemented metacircularily in MOP, which is much wider than CLOS and allows defining OO systems in general. It's a really wonderful idea and you should read AMOP to understand it better.

tl;dr: symbols, packages, scope, systems, compile/read/run-time, executables/object-files, CLOS/MOP are all separate. You have fine control over all of them, but there's also known conventions about handling them and most CL programmers use them when writing larger projects. MOP, systems (more complicated building/compilation of files), executables are not defined in the standard, but MOP is in most implementations, systems are also supported by most implementations and system support may even be defined portably(not that it is in most cases, I suggest looking at ASDF, but it's hardly the only one, there's also another which allow parallel building, and some older ones which you might use out of habit)), and executable/module-generation is completly implementation specific.

tl;tl;dr: systems and packages are most certainly not one and the same, even if in some languages they are mixed. While it's possible to do what you do in C with #include, in CL you tend to have compilation units for larger projects with specific build orders. Some people allergic to systems may choose to just concatenate all files, but that's a bit silly, however if you want an example, look at: http://www.cs.utexas.edu/~mfkb/km/
If you want an example of how to write larger projects, just look at any serious library, they'll all follow the pattern of a package file, a system file and specific files for the actual code (sometimes with more fancier dependences between them).

>>49-50
Oh my, so you can have two symbols that have the same string representation yet are not eq? Symbols are not just plain strings? Oh my, oh my, I'm definitely going to have to sleep on this one.

>>50
Your amazing post will probably take a while for me to digest, so pardon my lack of response for a while. I'm merely a poor disoriented soul trying to reconcile lexical/dynamic(/global?) scope and environments in a way that will allow writing in some form of module system later on, if (?) necessary.

(Warning: deranged rambling from this point on) If a module has a large amount of complexity and its interface is rather small, it's probably better off being separated in its own independent process/environment, and being accessed via some form of IPC. I can't really think of an architecture so large that it would necessitate the importing of a huge amount of modules in the same environment, effectively creating a need for a rather more complex package system. Or maybe it's just me not seeing the big picture. Or maybe it's the alcohol and the sleep deprivation. Anyway, I leave you in peace, my friend, and thank you again for your excellent and detailed post.

Oh my, so you can have two symbols that have the same string representation yet are not eq? Symbols are not just plain strings? Oh my, oh my, I'm definitely going to have to sleep on this one.
Yes. Strings are read by the reader and depending on (readtable-case *readtable*), the reader try to find the symbol in a package (if specified, or default), and if not found, it will create a symbol which is interned in that package (if not specified). Fresh, uninterned symbols can be made using (make-symbol string), and they should be regarded as a new object that just happens to have a field which contains the string. Symbols having the same name within the same package are eq because the reader will just find the same symbol (or you would find the same symbol, if you searched directly in the package):

CL-USER> (eq '#:test '#:test)
NIL
CL-USER> (eq '#:test (make-symbol "test"))
NIL
CL-USER> (eq 'test 'test)
T


I'm merely a poor disoriented soul trying to reconcile lexical/dynamic(/global?) scope and environments in a way that will allow writing in some form of module system later on, if (?) necessary.
I probably put too much stuff in that post. CL does let you control whatever you want, but that doesn't always make for a simple language. Most languages tend to opt for less.

Dynamic vs lexical scope isn't too difficult, but it can be a bit confusing because the same forms are used to handle both, despite them being very different internally (as to how they are implemented). I've even seen people advocate using different form names so as to not confuse newcomers. Dynamic scope works like a (thread-local) stack which pushes a new value each time you bind something to the 'special' (name) variable to some value, and pops it when it goes out of the scope of the binding form (it will also obey all non-local transfers of control-flow, such as conditions, among others).
Whenever you access or set the value of some dynamicly bound variable, you act on the top of the variable's stack (which encompasses the entire thread or process, if not multi-threaded). It tends to be useful as far as parametrizing code goes, but it's not really meant as a 'global' in the way you use globals in C (despite that you could use it like that if you never bound it and just set it). CL doesn't support globals, but implementations do, and you can portably implement them if you want using symbol macros - I can't say I ever needed 'pure' globals, most of the time I actually wanted dynamic variables.

While dynamic scope affects all the functions that are called (if you want to access some dynamicly scoped variable), lexical scope is the default and any normal variable is lexically scoped, that is, a variable will be accessible within the textual scope where it was defined - it's also possible to shadow lexical variables, but this shouldn't be seen as anything more than a convenient trick as the compiler/interpreter would just rename each variable and even if you used the same name for 2 lexically scoped variables, they'd be different ones (and the compiler/interpreter knows it), in the sense:

>(let ((x 1))
   (print x)
   (let ((x 2))
     (print x)
     (setf x 3))
   (print x))

1
2
1 1


is equivalent to


(let ((x 1))
  (print x)
  (let ((y 2))
    (print y)
    (setf y 3))
  (print x))


Lexical scope isn't completly trivial though, you can also have closures, either on lambda's that are returned, on local functions, or even as a wrapper around an entire top-level function - almost anything goes, as long as the variable is visible lexically.


Back to the module system: I don't know what the goals of your language are - there's so many options and it's really about trade-offs between simplicity and flexibility/features. CL is very flexible, but it can get a bit hairy at times, while some other languages have simpler systems, but don't always tweaking that one special thing that you might want.

*s/don't always tweaking/don't always allow tweaking//

I like how in Lua, files are functions. So I just have my Lua files return functions and tables and rarely put anything in _G.

>>52
>setf
imperative AIDS

>>55
It was only to demonstrate that the value is not changed when exiting the scope because they are different variables. Not that SETF is bad as far as imperative features go, it allows great amount of customization as far as how accessors expand into code.

>>56
if a function takes or returns nothing and isn't directly involved in IO then it is AIDS. Fact.

>>57
(set! (vector-ref v 69) 'fuck-off-and-die)

>>57
SETF actually returns the result of the argument that is assigned to the variable, but it also causes a potential side-effect of changing something (this is not guaranteed, you could for example have SETF do I/O in the expansion, it's far too generic to assume that it does no I/O).

Not that I have anything against some imperative code where it makes sense, but you assume too much about the behavior of some macro. SETF is a generic place function setter, but what it means to ``assign something to a generic place'' is only defined by a user, so if you only wanted to use it for I/O, you could do that.

>>57
Just because 99% of functions in a well-written project happen to be pure functions (which may use side effects internally) doesn't justify making the entire language immutable, and relying on hacks like monads instead. Fuck off and die now, and take your dead dog with you.

>>60
monads are more about getting around laziness than immutability. At any rate, they're just a really cool design pattern -- basically a functional "Command pattern" to use OOPAIDS terms.

"temporary mutations for the sake of performance" on the other hand are done with things like http://clojure.org/transients.

>>61
Clojure is almost cool enough to break my rule about not using JVM languages. It's a shame the CLR version was dropped.

>>62

if you want lisp + jvm then just do ABCL

>>61
Oh, so you think I'm an OOP advocate? Fuck you, faggot, you don't know shit. Go back to the academic shithole you came from.

>>62
Clojure is cool
No. No. ^No. _No. No. NO. Your opinion is wrong and invalid.

>>61
Transients are shit, the only reason they exist is because the compiler writers aren't smart enough to do shit.

>>63
No, I don't want JVM. I don't especially want a Lisp (but a Lisp is fine too.)

>>65
It's not an opinion, it's a fact. Big ups for having a decent concurrency model and doing a great job of immutability. The transients nice to have support for although they shouldn be unnecessary in the example provided.

>>67
Your fact is wrong and unproven.
But seriously, it's got a decent concurrency model, and there are some more good ideas in it, but that's it. There are some major design flaws I'm not even going to list anymore that make it absolutely disgusting.

>>67
Transients are shit.

>>66
I don't see them as being much different from type-hints.

>>68
Your fact is wrong and unproven.
Naturally, these are the best facts.

Aside from running on JVM, what's allegedly wrong with it? I heard some ruby dropouts bitching about having to refactor things to make simple changes, but FP will do that to any web developer.

Mutate structures everyday

Side effects must be legalized.

>>71
/prog/scrape my posts on Clojure, I'm not going to type that shit again.

>>74
I'll pass.

>>46
ey
Fuck off and die in a fire you feminist piece of faggot shit.

Lisp
GC is shit.