Wasn't that good at math in school

Is there any chance of me getting into quantum physics? I'm interesting in what makes up the universe, but...

To get the basics of QM down you really just need linear algebra.

Unless you are completely retarded, which you probably are not, you can probably make some good progress if you put the work into it.