To a point, I am sorry you had to deal with this guy, however you did reply to his text and proceed to except at least 3 phone calls from him, all at a time when you say you were not working and with family.
You have to except a lot of responsibility here, the decision to turn off your phone is yours.