Can't Delete Directory (Folder)

Sometimes when I try to delete a folder, it tells me I can't because it's being used by another program.  But it's NOT being used by any other program I have running.  What gives?

rm -rf for the win