Too Old

I'm 24 and I've studied a bit of math in university. I took a linear algebra and intro calc class and did alright in those. I've always been good in math because I've always found it deeply interesting.

My question is, does anyone think it is too late in my life to try to get into math seriously and major in it? Right now, I've just been taking general science courses trying to get a feel for what interests me. Physics/mathematics are what I find most appealing.

Does anyone have any advice on this?

who gives a fuck? if you wanna do it then do it. you can be fucking 60 or something and major in engineering if you will it. the difference is that at 60 you may not plan to be around that much longer to reap that much more of the benefits of the extra education throughout your life.
i know some guy who went to bed school to become an orthopedic surgeon at like 50. did good at it too.