Well, as a college student, that's not that bad. three days is usually what i do. and if it's at night, not interfering with work, school, whatever during the day, it's not that bad in my opinion
BUT i do not know the situation. does he get belligerent? hit girls? any more info?
What you should do is sit him down and tell him your feelings. That's the first step. just talk to him like friends should