>>4
Clang is open source and compiles OpenCL C kernels, with target support for all of the major CPUs and intermediate bytecode output for both nVidia and AMD GPUs (PRX and AMD ISA). Intel, AMD, ARM, and nVidia all use Clang for their commercial OpenCL products already and contribute to Clang. You just need to roll your own OpenCL runtime, which isn't that hard, the spec is fairly small compared to say OpenGL. There are several open source OpenCL implementations being developed around Clang already that you can use as a starting point.
https://launchpad.net/pocl
http://libclc.llvm.org/
http://cgit.freedesktop.org/~steckdenis/clover/
And OpenCL isn't just for GPUs. It's for all types of compute devices. CPUs, GPUs, FPGAs, heterogenous cores like the PS3's Cell SPUs, or AMD's next-gen APUs which will have a multi-core x86-64 CPU, an ARM Cortex A5 core, and next-gen Radeon GPU all on the same die.
The best way to use it is as a task-oriented and grid-oriented threading engine for your CPU code. OpenCL can call into your host C/C++ functions, you just pass it a function pointer when scheduling a task.
OpenCL makes stuff like OpenMP or POSIX Threads/C11/C++11 threads obsolete.