I've said this before, but I don't think the ugly mugs section should have the word ugly in it.
You wouldn't call the section "faggot list" or "scarfaced" or "skinheads" or "small penis men" and then if someone questioned it say: "oh well it's only metaphorical" or something like that. Obviously there are handsome men that are bastards just as much (maybe ugly people sometimes have more issues or were brought up the wrong way/have drug habits and look bad so are slightly worse sometimes, I don't know... that would be no excuse anyway for painting them all with the same brush).
Whether metaphorical or not, words matter. In politics you wouldn't say someone performed a "girly job" on something, and then if questioned say oh it's only metaphorical, because that would be seen as very sexist. True, ugly is subjective, but it's not THAT subjective, some people really are ugly.
I'm not saying I'm ugly or handsome, I just don't see the reason for this. I know that E-I got it from some other "system" somewhere else originally.