So apparently yesterday he said "fuck it" and stopped trying.
When I went to the mailbox at six last night (and he was nowhere in sight), what did I see but a US Postal Service mail bin. And in it? People's effing MAIL. I searched through to make sure none of it was mine and found that it belonged to several people at the complex. It was like he just stopped delivering in the middle and left the mail there to be stolen! We've got huge identity theft issues in our area, and now at least I have some idea why.
At least when I called the USPS, they took the complaint really seriously. Very nice service on the phone.
Edit: they forwarded it to the local post office, who just called. They asked lots of questions, so it seems like they're doing a real investigation. Unfortunately, he did ask whether I just left the mail out there, which I did (the apartment office was closed and I wasn't comfortable putting it in my apartment) and now I feel kinda guilty.