Immutable String Buffers

Why is this telling me to concatenate small strings to the ends of large ones, but not in reverse, and then concatenate all remaining strings in the table?
http://www.lua.org/pil/11.6.html

Wouldn't it just be easier to put all the strings in a table then concatenate them all at once, as in the following?

function filetostring(file)
    local t == {}
    for line in io.lines(file)
        if line == "" then
            break
        end
        table.insert(t, line .. '\n')
   return table.concatenate(x)
end

I'm confused as fuck.

Wait, that's broken.

function filetostring(file)
    local t == {}
    for line in io.lines(file)
        if line == "" then
            break
        end
        table.insert(t, line .. '\n')
    end
    return table.concatenate(x)
end

Still broken, using my text editor this time.

function filetostring(file)
    local t == {}
    for line in io.lines(file)
        if line == "" then
            break
        end
        table.insert(t, line .. '\n')
    end
    return table.concatenate(t)
end

Because otherwise you end up allocating and copying excessively. The best thing you can possibly do is to use a StringBuffer/StringBuilder/whatever-the-fuck-it's-called object that tries to minimize extra copying and allocation.

>>4
If I have to concatenate every line together and I'm doing it line by line why not just do it at once.

I can see why this is wrong.
 -- WARNING: bad code ahead!!
    local buff = ""
    for line in io.lines() do
    buff = buff .. line .. "\n"
    end

But how is this any better then what I wrote? If you need to allocate a new piece of memory every time you assign a string to a variable, then why wouldn't this be worse?
function addString (stack, s)
    table.insert(stack, s)    -- push 's' into the the stack
    for i=table.getn(stack)-1, 1, -1 do
        if string.len(stack[i]) > string.len(stack[i+1]) then
            break
        end
        stack[i] = stack[i] .. table.remove(stack)
    end
end

>>5
The best you can do in pure Lua without any external C code is to read every string into a table (without ever actually concatenating anything) and call table.concatenate at the very end. That will still take twice as much memory than it should, but at least the worst run-time is linear.

>>6
That's what I thought, but then why is "Programming in Lua", by the Chief Architect of the language, telling me to do something less efficient. Is it just to prove a concept? It reads as if he thinks it's faster.

>>7
I'd say you should just benchmark the two approaches and tell us what results you get.

>>8
The authors was seven hundredths of a second slower. Well, I hope this doesn't look the same in the second edition.

>>9
That's not how you publish benchmarks. First you show the source codes of both programs, then you run them in a way that each takes at least 20 seconds (possibly by running the benchmarked function in a loop). Then you publish the exact timings of each program.

Wait, I've failed at reading comprehension. I was so confused by the earlier program I didn't realize he made a new one at the end. The one we've been talking about.

I just wasted several hours because I couldn't read the last paragraph without thinking. "But this is something else, it doesn't explain the last thing at all, better go back!"

...

table.concatenate is much faster (and more understandable, I think) than concatenating each string, yes.

However, you are incorrect in assuming that the size of the strings matters much for concatenation. In Lua, concatenation is pretty uniformly slow because all strings are interned symbols.

concatenation is really only horribly inefficient if done inside loops

as far as internal details of the standard vm go, the concat instruction can actually work over a range of registers, so if the compiler sees a clusterfuck of a concatenation like this:

d = ""
local a, b, c

return "y" .. a .. ("xxxxxxxxx"):sub(1, 7) .. c .. b .. "2" .. 1 .. tostring(true) .. "" .. d

it will be quite anal about getting all of that shit into registers (in the correct order), so that it can concatenate all of it in just one operation. on top of that, concat uses a buffer internally so you don't end up with a ton of intermediate strings like you might think

so if you're using loops or recursion or whatever to build strings, use a table + table.concat. otherwise just use the operator