You are absolutely correct Tom — it most certainly would not be good manners to walk away — unless there is a good reason to do so!
At the end of the day this is purely a business transaction between you and the lady.
If you went to purchase a car that was advertised as a late model Ferrari and in perfect condition but when you arrived it turned out to be an old battered Fiesta what would you do? Pay the asking price of the Ferrari and drive off in the battered Fiesta so as not be rude?
I wouldn't! Firstly I'd be very annoyed that someone was wasting my time and trying to con me — that alone would make me want to walk! But apart from that there is no way I would want to pay for something I didn't want so I would have no option but to walk away! I would explain to the would be con artist exactly my reasons for doing so!