Wireless hacker

Hi /comp/
I own a DLink-Di524 wireless router, and even though I've set MAC filters, disabled SSID broadcast and enabled WPA2, someone keeps trying to connect to it (light flashing repeatedly, logs report denied access from mac address xxx...)

My question is, is there anyway to simple stop this guy from connecting in the first place? Or should I just rely on the MAC filters and WPA2? It's really frustrating to see this person hammering my router >:(

OP here.
I actually do leave my router turned off during the day, and only turn it on at night, but still every morning when I wake up I get 60 pages of ``Denied Access from...'' in the router status page. Is there anyway I can screw this person bad? Like maybe let him connect so I can sniff his passwords or whatever?
>>5 sounds like an interesting idea, though I have no idea how to implement it ;o