Script interpreter time-jumping.

I'm writing a "game"-engine for visual novels in python and have the following problem:

My engine includes a script-interpreter for a language without 'if' and 'for' statements. I save the interpreter-state for each line in a cache, so I can jump to a line without executing the whole file up to that point. When the sourcecode changes, the engine sets the cached interpreter-states for the involved lines to None. The engine rebuilds outdated states by going downwards from each None and filling newly calculated states into the cache until the calculated state matches the one that is already in the cache (before overwriting it), because it's safe to assume that all further states are not affected by the change. Example:
1 {}                       a=0
2 {"a": 0}                 b=3
3 {"a": 0, "b": 3}         #comment
4 {"a": 0, "b": 3}         a=1
5 {"a": 1, "b": 3}         c=7
6 {"a": 1, "b": 3, "c": 7} #commentIf line 1 is changed to "a=1" the engine rebuilds lines 1-5. Because at line 5 the calculated state and the cached one are the same. Now there is a problem with that approach. When I have a 5000 line file and I change a variable at the top, the changes propagates through the whole file and the engine starts lagging. A solution I'm considering is using something like {"a": [0]} instead of {"a": 0} and using references to the same 1-element-list from all states wherein the variable in question isn't changed. In the example that "a=0" is changed to "a=1" this would lead to all 0s from 1-4 to be 1s afterwards when actually only one number was changed. The rebuild stops at 2 already because the calculated and cached state match. This is definetly a speed-up, but only works with assignments. If line 3 had something like a=a+10 the rebuild-cursor would never reach the line 3 because the references across states break the assumption mentioned earlier. At least when not used correctly. I can't even make special treatment for this kind of instructions because commands can be user-defined too. A different approach would be to store the data like cache[var][linenr] instead of cache[linenr][var]. I haven't thought about that much yet. Also, I can only support simple types yet, like ints. I need to find a way to have (recursively) versioned python-objects.

It's quite difficult to put all this into words. There is still more, but I don't even know if /prog/ is able to help me so I won't bother to type it down (yet)

I've been reading >>1 plea a couple of times and I still don't get whats going on.

1. You have a `language' you're making that doesn't have if and for (and implies I assume, that there are no alternative primitives): how can it have anymore than one state for each tick of the time line, isn't the `language' equivalent to a linear movie screenplay, how can the visual novels be interactive?

2. Somehow you've crammed in a couple of fibs and an Ackermann every tick, I don't see how scripting the events behind a visual novel can be so slow, does it have a physics engine?

3. Because of slowness you make a precomputed choice-point table, so you can backtrack Prolog-style, for this (supposedly?) choice-point-free language.

4. Changing of the source code is a common use case that needs to be optimized, so you're making a home made diff engine that dynamically patches the choice-point table. wtf?

If the root of 3 and 4's raison d'être is constant time execution costs of running an interpreter in a slow interpreter and not of some algorithms you haven't mentioned. You really should either implement your interpreter in C and FFI it, or implement a meta circular evaluator (at least use a language that allows this) with a couple custom procedures for its environment to maintain speed.

With the latter route, write a translator that translates each line to the equivalent in your language of
(lambda (input-data)
  (your-instructions) ...
  (values new-data (wait-on-input)))
where input-data and new-data are immutable hash-tables and wait-on-input saves the continuation into a lambda and returns from eval, and then call the returned function to resume continuation to go to the next tick. To `backtrack' just cache the continuation functions and execute the one you want (immutable data at that point will be reused and further data will be GC'd) or go clean slate and re-run the script and skip waiting until the wanted location. Even if it passes through 5000 lines this really should be nothing, some Lisps and Schemes are JITted, they may also JIT the sandboxed evaluation, it would be a bonus if your language supports this. Decision points (whether you have them or not) are easily integrated with this approach, just by memoizing user input.

>>12
It's easier to implement my own language
Famous last words.