The thing that continues to bother me about quaternions is that algebraic equations have too many solutions. For example, consider x^2 + 1 = 0. Obviously, i, j, k, -i, -j, -k are all solutions. That's too many already, but it's not too bad.
But then you realize that (i cos t + j sin t) is also a solution for any real t at all, and you go totally insane.