I think it's mainly because most of us men are jealous and possessive over what & who we truly love.
I couldn't imaging living with an escort in a loving relationship because if I were in love so to speak I simply couldn't share. What I am hearing is that that is a character flaw on my behalf..fair enough.