char buf[LINE_MAX]

Why do programmers still code in arbitrary limits like these?

It breaks when you do
#define LINE_MAX UINT_MAX

Why not redesign so that you use an O(1) space algorithm, or if it can't be avoided, dynamically allocate as much as needed?

It also breaks when you randomly remove characters and shuffle variables around.
Why not redesign it so there's no code at all?

>>1
Everything breaks when you do stuff that's fucking retarded.
Solution: Don't do stuff that's fucking retarded.

>>1
If you're hitting the UINT_MAX limit in your number of lines, you're doing something wrong.

UINT_MAX? Don't you mean SIZE_MAX?

Let's clear this up once and for all. Dynamic languages such as Python, Ruby, etc, all will malloc the memory when you do a readline, so there is no reason why you couldn't/shouldn't do it in C.

>>6
So write your own readline function in C which mallocs the memory for you. It's not that hard. If you don't want to do low-level programming you shouldn't use a low-level programming language.

>>7
What if you are expecting to receive a command over the network but someone sends you a bunch of data without any new line!

>>8
Then use a maximum request size.

How often do you know how much space is needed or even an upper bound?

>>9
No, do not arbitrarily limit the software. You should dynamically allocate as much memory as required.

>>11
Dynamic allocation is too slow. You should statically allocate as much memory as possible.

>>4
If your program gives up at some number far below the capabilities of the hardware, it's not scalable enough. If I have a 32-bit machine I expect e.g. uniq to easily deal with files containing lines several hundred million characters long, if I have enough memory (virtual or not). And if I'm on a 64-bit machine with a terabyte of RAM, lines in the gigacharacter range should be possible to process -- what's the point of having an extra 32 bits if you can't even use it?

>>8,9
Just resize the buffer dynamically, up to a certain size. Networking is different -- you need to set a maximum, or someone will DoS not by overflowing the buffer, but by making it grow as big as possible to consume all memory. And that's also what ulimits are for.

Statically allocated buffers are instantly available. On most architectures I know of, the compiler generates no implicit instructions to handle initialization. The OS is never even bothered to set it up, unless for whatever reason you need more blocks of memory to hold your stack or something. It's a fast and easy way to get a reasonably sized block of data out of a function.
char buf[4096];
unsigned int line_no;
FILE *file = fopen("foo", "r");
for(line_no = 0; fgets(buf, 4096, file); line_no++) {
  if(buf[0] != '\0') {
    if(buf[strlen(buf) - 1] == '\n') {
      /* handle it */ }
    /* ... */ } }
I claim that if your text file has 'lines' over 4096 bytes, maybe it's time to think of a better way to parse it. Perhaps you should read it in and handle it X bytes at a time, rather than line-by-line, or maybe you'll want to figure out how long your lines can get before reading it.

ITT Cfags with their "640 KB RAM will be enough for everyone" mentality.

>>14
static variables are cleared by the startup code. However, in your example your array is in the automatic storage class, which means it goes on the stack. Stacks have size limits.

>>16
As long as you aren't calling something like that recursively, or a large-ish sequence of functions that all use these buffers, or you're developing for a PDP-11 running Plan 9 on Mars or something, I don't think you'd run into stack size limit problems easily when using stack-dynamic buffers that are reasonably big enough to hold the amount of data you're buffering.

And if you are in such a situation, as mentioned, it's time to realize that and reevaluate what the hell you're doing.

>>13
If your network protocol allows DOS attacks, then you have more problems than just dynamic allocation.

>>17
Some of us are writing more than toy programs.

>>1,6,11,15
Oh god.  It's like we've been invaded by /code/.
You are all seriously clueless.  GTFO my internet.

>>20
Of course, I would force a limit on network packets and pretty much any input in any serious Python program, but Cfags tend to impose more/shorter limits than necessary, and it's always good to troll them regardless.

>>21
Segmentation fault (core dumped)

>>22
I said Python, not C

>>19
If you know roughly the format of a single row-element of data you're trying to buffer, you can safely use a stack-dynamic buffer that's about as big to store it. Why make it any bigger than necessary? For one-off shit, in case you're wrong.

Also, if you're dealing with data X bytes at a time, and X is a reasonably small number, and you don't have a ton of functions that call each other in sequence, or a ton of threads threads, to parse up to a ton of X-bytes of data at a time for whatever fucking reason, there is absolutely no need ever to malloc X bytes to temporarily hold it, at least in C. I have no idea why people still write code like int *x = calloc(50, sizeof(int)); to store a fixed quantity of data in situations where int x[50] would be fine.

Sometimes you find yourself in situations where, your data is structured in such a way that you can't really tell what is happening at pos X until you hit pos Y later on. When you have, say, files, and they're all rather small, and you only read one at a time, it makes sense to just malloc the whole thing in. When you have gigantic files, it makes sense to use the file stream to look ahead a bit manually. When you have huge streams of data coming from network traffic, it's time to start using resizing buffers.

My point is that it's not a useless concept that's merely used to demonstrate to script kiddies how to perform file input; like with everything, you use it when appropriate.

>>23
Yes, and after you profile your Python code wondering why the shit it's so fucking slow, you end up rewriting and embedding parts of it in C, you make all your limits exactly what you think is necessary to hold the data it'll contain, and then a month later you post to /prog/ whining about not being able to track down your
Segmentation fault (core dumped)

>>25
Despite being a FIOC lover, I can do C. I had been doing C for years before I got tired of working like a nigger and discovered dynamic languges.

>>26

All stream processing should work with some arbitrarily-sized chunk of the data at a time, without relying on the entire stream to fit within that chunk.  Dumb shits.

I claim that if your text file has 'lines' over 4096 bytes, maybe it's time to think of a better way to parse it.
OP here, read the last line of the post --- but what if "parsing it" entails needing the entire line and there is no O(1) space algorithm that exists?

>>28
what if the stream has arbitrarysize+1 amount of data in it? then you parse arbitrarysize pieces of data??????????

>>29
You never need the entire line at once.  lrn2hold state.

>>30
Yes.  Dumb shit.

Unless you want to optimize for the special case of having arbitrarysize + 1 amount of data, but of course premature optimization etc etc.

>>31
What. Dick lips.

Suppose you are parsing an incoming message, for example, the following message comes into your socket:


1 512 1024 I am humping a goat. The Sussman is humping a goat.

And you have a buffer of size 16, you read in the first 16 characters, so your buffer is filled with this:


1 512 1024 I am

How do you plan to save the state so that the full message can be read and understood after the whole thing has been read and processed?

>>32
Have you never written real code before?  The easiest is a continuation, but in C or ENTERPRISE OO TUR_^NKEY SOLUTIONS you encapsulate the state of processing in an object (you know, just those same damn variables you'd have bound up inside your OMG OPTIMIZED parsing loop) and hop right back where you left off when more data comes in.  You don't even need a full state machine, though if you're generating those or have a protocol generator it's the easiest.

Fuck, you people are inexperienced.

>>33
The state is contained in the string I am humping a goat. The Sussman is humping a goat. You need to pass this on to the user. How? You can't use malloc.

>>34
No, moron. The state is the current string so far, flags stating that you've parsed the initial integers, etc etc.

I have a feeling IHBT.

>>35
The string is random. It could be anything. There is no way to parse it, there is no grammar for it. You have to malloc space for it so that the person requesting the data can actually use it for something.

How the fuck do you expect to do that dick licker?

>>33,35
I dare you to write a program that reverses each line of input, given arbitrarily long lines (for the purposes of this paragraph, lines are defined as any sequence of bytes terminated with '\n'), in O(1) space, without seaking backwards in the input stream.

>>37
Sure, reverse each substring that comes along, tack it on the front of what's already been accumulated, sending it out and starting a new accumulator when '\n' is hit.  Takes the exact same space as yours (given that you don't know how much to preallocate either), but doesn't need to be processed all at once.

>>37
Fuck yeah, Seaking!

http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2008q3/002389.html

>>40
Interesting.

>>40
Status: RESOLVED FIXED
Comment: Increased buffer size to 8192 bytes.

>>1
This is because some programmers aren't bright enough to pick a language that includes real string support, so rather than do all the work required to be safe, they implement lazy hacks. And then the Inter was a Bot.