Physics: where to start?

Hi /sci/,

I'm a math nerd but of late have become exasperated with how little my field seems to apply to reality.  Plus what I've seen and heard about theoretical physics (NOVA specials lol) is interesting as fuck.

My background is limited: 2 semesters of shitty "general physics" (collegiate plug and chug) and I've read Einstein's "Relativity: The Special & General Theory". 

So my question is, where do I start?  Are there any good, comprehensive sources for actually learning physics?  Any books suitable for self-study (if this is even possible) but still hard science?  I want to learn more about a lot of this stuff - QM, E&M, astrophysics, classical mechanics - but I don't want a Stephen Hawking-esque "Oh lol I understand physics" feeling, I want to understand the motivation for the theory and how and why the experimental data supports it.  I feel like an emphasis on the math would be helpful for me, too.

So yeah, if anyone can help a brother out, I'd appreciate it.  Thanks in advance.

>>1
What's your math background? May seem redundant, but pure and applied math majors can have totally different areas of knowledge.
 
Agreed with the review of basic physics though. Then, if your interests are as outlined, you should grab a modern physics text and/or a sophomore/junior level mechanics or E&M book from the library or something. If you're really interested in the experimental side, see if you can find some lab manuals or something. You probably won't be able to do the experiments on your own, but whatever.