a touhou face recognizer

how difficult would it be to create a pipeline that can detect and recognize touhou faces in images?

I was thinking of building something very simple like this using opencv:
1. face detector that returns bounding boxes around possible faces. copying code from one of them generic face detector will probably do.

2. extract sift features from the cropped images in the bounding  boxes. again copying code will be the easiest way

3. use a svm trained on 250+ images for each touhou downloaded from danbooru or something to classify the sift features from the cropped images

Thats all I can think of for now. Any suggestions for improving this very basic pipeline would be helpful.

Like this:

require 'nokogiri'
require 'net/http'
require 'uri'
require 'open-uri'

touhous = []

doc = Nokogiri::HTML(open('http://danbooru.donmai.us/wiki/show?title=List_of_Touhou_Characters').read)
found_reimu = false

doc.css("a").each do |i|
  next unless i["href"] =~ /wiki.show/
  found_reimu ||= i["href"] =~ /reimu/
  next unless found_reimu
  touhous << URI.unescape(i["href"].split('=')[1])
end

image = ARGV[0]

response = Net::HTTP.post_form URI('http://iqdb.org/'), { "url" => image }
doc = Nokogiri::HTML(response.body)
tags = doc.css("img")[1]["alt"].split[3..-1]

raise "Image does not contain touhous!" unless tags.include? "touhou"

puts (tags & touhous)