programming is one giant mess

I'm just an amateur programmer.  I have no trouble understanding programming languages though.  In fact I think I understand them better than most.  Languages are just rules and I am very good at building a correct mental model and logically organizing everything. 

What is extremely hard for me to understand is everything besides programming itself, meaning everything that comes after writing the source code.  Getting shit to run is very hard.  No one has laid out all the rules in one place.  Compiling, linking, assembling and this crap just 'works', unless of course it doesn't work in which case it's time to learn a new rule that you never heard of before.  You have different OSes, different hardware, different libraries, different everything each with their own standards and incompatibilities.  And you can't just decide to figure it all out and make one big mental model because the information you need is - in one way or another - almost secret;  the information is either locked up in closed source, or is in documentation that you have no idea how to find, or maybe it just plain doesn't exist.

I know something is wrong because part of my brain believes I should just say 'fuck it' start all over from the bottom up; program in assembly (intimidating as it may be, it is just a language) and slowly build all the abstractions I'm used to on top of that.  I know that is stupid though, but it sums up my frustrations.

Okay, so how many people can relate to this?  And - dare I ask - is there a way to find any of this information laid out in one place?

Do you remember the 1980s? Veteran users of free software on Unix could testify that though there were a lot of programs distributed as source code back then (over USENET), there was not a lot of consistency in how to compile and install it. The more complicated a package was, the more likely it was to have its own unique build procedure that had to be learned first. And there were no widely used approaches to portability problems. Each software author handled them in a different way, if they did at all.

 Fast forward to the present. A de facto standard is in widespread use for solving those problems, and it's not just free software packages that are using it; some proprietary programs from the largest computer companies are built using this software. It even does Windows.

 As it evolved in the 1990s it demonstrated the power of some good ideas: sharing expertise, automating repetitive work, and having consistency where it is helpful without sacrificing flexibility where it is helpful.

 What is "it"? The GNU Autotools, a group of utilities developed in the 1990s for the GNU Project. The authors of this book and I were some of its principal developers, but it turned out to help solve many other peoples' problems as well, and many other people contributed to it. It is one of the many projects that developed by cooperation while making what is now often called GNU/Linux. The community made the GNU Autotools widespread, as people adopted it for their own programs and extended it where they found that was needed. The creation of Libtool is that type of contribution.

 Autoconf, Automake, and Libtool were developed separately, to make tackling the problem of software configuration more manageable by partitioning it. But they were designed to be used as a system, and they make more sense when you have documentation for the whole system. This book stands a level above the software packages, giving the expertise of its authors in using this whole system to its fullest. It was written by people who have lived closest to the problems and their solutions in software.

 Magic happens under the hood, where experts have tinkered until the GNU Autotools engine can run on everything from jet fuel to whale oil. But there is a different kind of magic, in the cooperation and sharing that built a widely used system over the Internet, for anyone to use and improve. Now, as the authors share their knowledge and experience, you are part of the community, too. Perhaps its spirit will inspire you to make your own contributions.

http://sources.redhat.com/autobook/