Prejudice against speculative fiction

do you think that there is prejudice against fantasy and science fiction? (Fantasy, Science fiction etc')
If so, why? Is it justified?

i wonder if i'll get trolled if i bring up the fact that i really enjoy Heinlein.