Scheme/Common-Lisp

I had decided to learn Scheme, but discovered that it lacks decent libraries, so i decided to learn Common-Lisp instead. It seems less elegant (more complicated) than Scheme, so, considering the risk of brain damage if I continue, am I doing it wrong or should i proceed?

By brain damage, I was referring to the fact that, because the languages are so similar, but still different, it could be troublesome to learn one after getting used to the other.