Oh, to be young again! I haven't seen the front end of 5 a.m. in over a decade. But aside from the skin and the metabolism, I don't think I could handle being young again, or even surrounded by young people. They're so baffling.
Anyway, the scenario you described sounds like a half-dozen scenarios I experienced in my early 20s. You go over to someone's place, on a bike or on foot or on the bus or some other inconvenient mode of transport, and once you get there he goes immediately to sleep, or starts talking on the phone ( we didn't have texting back then), or wants to watch some incomprehensible movie instead of having sex or even just talking to you, or gets into a fight with his roommate and completely forgets you're there. And then you have to go trudging back home in the wee hours because you just can't stay in a place where you're being treated like a houseplant.
I never took it too personally, though. The thing is, a lot of people are too self-involved to see that other people in the room are having feelings different from their own; they feel an impulse or concern and just automatically assume that you share it. They're not bad, they're just dumb, missing an essential part of the human education.
In your position, I wouldn't have gone over in the first place, even if he was promising mind-blowing sex instead of a movie and a snuggle...biking at night is dangerous. Sleep deprivation is even more dangerous.
But once there, I would have been gracious about leaving ("Look, I can see you've got something important on your mind, I'll get out of your way and we can do this some other time, OK?") and I would have left the minute the movie was over. By leaving in a huff, after sitting there so docilely for four hours, you put him on the defensive... and like most people on the defensive, he tried to find a way to make it your fault instead of his.
But having put him on the defensive, the next step is to guilt-trip him ("I'm sorry I got so huffy, but you really hurt my feelings when you ignored me all night; I know your friend needed you, but it hurt anyway")...see, he knows he's in the wrong, but he probably doesn't have a defense mechanism for admissions of pain, and it will make him learn something from what he did.
Anyway, just chalk it up to "experience" and don't let it bother you.