(Double) Linked Lists

I've been trying to write a little forum application that scales to million of threads in a category and million posts in a thread. The idea is that viewing a category or thread on any page will only take n+const key lookups in a database (like berkeleydb) where n is the number of items (threads or posts) on the page.

Implementing threads and posts as double linked lists makes it really easy to view the oldest and newest items (and adding new items) but obviously pagination doesn't work at all and also deletion is going to be a bitch because you have to "find" the item first.

I've been thinking about adding "shortcuts" to every 10th, 100th, 1000th, etc item to the next 10th, 100th, 1000th, etc item, but then deleting becomes very very expensive because you possibly have to update thousands of shortcuts. Is there a solution that doesn't suffer from having one really bad operation (or requires in-memory indices)? Or maybe a n+log(items)+const solution...

This is just a thought experiment about data structures but it has been driving me kinda nuts lately :p

jesus fuck you fucking nigger twats any self-respecting RDBMS can handle this stupid shit fine without you having to wrangle you fucking brains to figure out how to bastardize the incorrect data structure for the job.

``SELECT p.* FROM posts p WHERE p.thread_id=4085 ORDER BY p.post_id LIMIT 99 OFFSET 5837''

Obviously you're not going to be able to display all million posts of a thread at once, but that's going to break other shit (ie, your fucking pipes) before the DB collapses.

If you're seriously in a situation where you're generating enough content to NEED such a system, you're ass is gonna be fucking toast -- you're not going to be able to handle the entire fucker on a single machine. Fuck, just look at the goddamn storage requirements of such a system:

1000000 threads * 1000000 posts/thread * 200B/post
= 2*10¹⁴B
= 181TB

Get your goddamn head out of the clouds and just fucking implement something, then see how well it performs, THEN try to figure out goals. Don't just bullshit numbers and then decide to use a stupid ass bullshit datastructure you learned about in Computer Science III. Read SICP.

I don't give a shit if IHBT and IHBTE -- I'm drunk as shit right now and you niggers are the closest thing I have to a goddamn friend in this stupid fucking world. fuckking sage.