Just wanted to update and say that I did end up filing in small claims court, and although we tried to go through mediation to work it out (to be a decent person, you know how it is) he refused to give back any more than half of what he originally took. So it went to trial and the judge did indeed rule in my favor and I will get my $400 back! Double or triple damages weren't awarded here, so that's a bummer - but at least I'm getting what he took, and court costs.
The judge spoke rather harshly to him about privacy laws, after the landlord came in waving like a crazy man that damm calander with all the times and dates on it. He didn't seem to listen or care so I don't know if it will do any good for his next tenants, but at least it's not my problem anymore. This is the first time I've ever fought any bad service, but $400 was too much to just lose. So all's well that ends well. :)