I want to make my own textboard in C fastcgi using files instead of a sql database.
Does it sound like a good fast idea?
I just want it to be blazingly fast.
What do you think /prog/?
Name:
Anonymous2007-06-05 17:54 ID:vFwRrJHg
lopl, you got the idea right
Name:
Anonymous2007-06-05 18:07 ID:u59cwM3T
fast? find what works best - SQL isn't bad, especially if you have a small set of servers (such as lighthttpd - I'm sure there are SQL servers of this sort). The primary advantage over a filesystem-based db is that your data is always easy to access and not dependent necessarily on complex directory structures. Also, cahcing is your friend for speed!
Protip: You think using files and directories is great? Think again. First, directories are lists, thus file access is linear with the number of files. Database indexes are trees, thus logarithmic. Second, files seem like the easy thing to use at first. But then you need to implement file locking, and you better come up with a good mechanism. Locking is a standard feature of databases. Some will allow safe reading while you write — even safe writing while you write other rows.
Name:
Anonymous2007-06-05 19:11 ID:qnqx4xmb
Having a database actually simplifies a lot of things. If you want something lightweight it's still better to go with sqlite than flat files.
Although, now that I think about it, with a textboard you only ever really need to append. I guess you could try files, it might work.
Name:
Anonymous2007-06-06 6:34 ID:k2H1dO/B
Use SQLite. Databases are just file libraries, implementing what you would implement by hand if you didn't have them. Most of them work in a client-server fashion, but embedded servers like SQLite work just like any other library, which happens to have a command interface called SQL. SQLite is like 250 KB, blazingly fast, and very easy to use.
Name:
Anonymous2007-06-06 6:50 ID:pwbLLqBE
>>7 speaks the truth. SQLite is very simple yet efficient.
Name:
Anonymous2007-06-06 6:59 ID:gxfhyc/M
The obvious answer is: HAppS
Name:
Anonymous2007-06-06 7:21 ID:ECZkpIP9
BBcode, some duck tape, and a rickrolling is all you need to write a textboard.
>>12
So true. Use a real filesystem, get real performance. Use a toy OS with a toy filesystem, get shit performance.
Name:
Anonymous2007-06-07 3:36 ID:6yHe9Pfn
fuck sql and databases, make your own solution it will be 100x faster and more reliable.
people use databases way too much. databases are supposed to be for multiple users to keep concurrency. if you have only one user(the webpage) accessing it, its a waste.
Name:
Anonymous2007-06-07 4:22 ID:FTSaHxQ9
>>14
PROTIP: Most text boards have multiple users who read and post on them. >>7
SQLite is made of slow and lack of features.
>>18
PROTIP: both 2ch and Kareha also use flatfiles. Shiichan being shit is unrelated.
Name:
Anonymous2007-06-07 12:56 ID:6yHe9Pfn
>>15
yes i know there are multiple people visiting, but when i say user i mean entities accessing the data. a text board requires just requires one writing "user"(the script that adds a post to the data file) and one reading "user" (the script that writes the page), they reading "user" only has to read once after each write so in effect there is only one "user" and concurrency/locking is irrelevant.
Name:
Anonymous2007-06-07 12:57 ID:4GuhcJds
If your web server is limited to one connection, sure.
>>22
AND THEN TWO PEOPLE POST AT THE SAME TIME AND SHIT FUCKING BLOWS UP.
Name:
Anonymous2007-06-07 17:20 ID:6yHe9Pfn
>>25
the write script queues all the incoming posts and handles them one at a time, there is still only one user accessing the data.
Name:
Anonymous2007-06-07 17:33 ID:wDmTvpTH
>>26
You moved concurrency handling from the file to the write script! GJ
Name:
Anonymous2007-06-08 1:23 ID:jkBr1q4Q
First, directories are lists, thus file access is linear with the number of files.
lol wut? I wonder if even the first versions of UNIX had this problem.
SQLite is made of slow and lack of features.
From what I've seen, SQLite is a lot faster than other OSS databases for small workloads, which seems to be its niche.
Name:
Anonymous2007-06-08 1:48 ID:RKkZ2Jlj
>>28
Connecting to the database takes longer with SQLite than with MySQL. Maybe if i was doing thousands of SELECTs on every connection that wouldn't matter, but for a simple text board it does end up being pretty significant. Also, lack of features.
Name:
Anonymous2007-06-08 2:12 ID:b+1W54TM
>>30
OP said it'd be a FastCGI program, which means he can just keep the database connection open since the program keeps running between requests; thus, the time it takes to connect is irrelevant. And SQLite has more than enough features for a simple text board.
Name:
Anonymous2007-06-08 3:46 ID:ia8lUjoV
don't bother with files, just limit yourself to one fastcgi process, and keep all the data in memory. maybe have a dump/import so you can shutdown and restart without losing data.
apache's fastcgi stuff should buffer input / output such that you're not waiting around for clients, which means that a simple while(true) { handle_request(); } should perform just as well as multiple processes/threads. unless you have multiple cores, i guess, in which case threads+locking would be the way to go.
you win for lack of filesystem overhead, i would advise you to design your data structures such that paging out to swap will work reasonably well.
>>32
It's actually not too bad an idea. If you don't mind the slight risk of total data loss, you could get a massive performance boost.
Name:
Anonymous2007-06-08 6:13 ID:li0J5Z9H
If you don't mind the slight risk of total data loss, you could get a massive performance boost.
VROOM VROOM HAY GENTOO BUDDY
Name:
Anonymous2007-06-08 11:31 ID:7541SlLQ
Maybe you could keep everything in memory for fast searching, but also save a file for each thread in the board. Then, when one of the threads is changed/added to, write that to a new file, delete the old file, and rename the new file to match the name of the old one. This prevents huge read-write times if you have a lot of threads, and also prevents anything from being lost if the system shuts down unexpectedly in a power outage or something. And of course, I reccomend getting saving your files on a bunch of cheap drives in a RAID 5 array.
Also, for searching, how about using merge sort and binary search?
Name:
Anonymous2007-06-08 11:39 ID:GFZdltC7
>>35
Shut up. Open that existing file and try a little something called "appending" to it