I suggest liver treats! I've yet to meet a dog who doesn't love them and won't run through a busy two-way main road to get to one. Not that I've tested that part of the theory, but still...you get the idea. If the dog likes treats, he'll melt in the palm of your hand with a liver treat.
I have to say, that really is awful for the poor dog. I can understand why he's scared. Dogs also tend to go through a little bit of trauma with certain people too. I once knew a dog who was an absolute sweetheart, but the family who owned her, (I lived with them for a while) had an employee that came around and he had a full-fledged beard. She absolutely HATED him, and we knew that as a puppy, she was found beaten and abused. It wasn't until he shaved off the beard, that she became friendly towards him. A dog I used to have also, was beaten as a puppy too, and would run and hide whenever you would bring a knife out of the drawer just to prepare a meal.
Some rescuers and trainers believe that dogs can associate "things" that remind them of their abuse and trauma, so perhaps there also (just a guess, of course) might be something about you from the dog's past that he's reminded of. Could be anything from a hair, hair color, or even cologne/after-shave scent. Perhaps maybe even shampoo or conditioner.
I can understand why he's scared. Dogs also tend to go through a little bit of trauma with certain people too. I once knew a dog who was an absolute sweetheart, but the family who owned her, (I lived with them for a while) had an employee that came around and he had a full-fledged beard. She absolutely HATED him, and we knew that as a puppy, she was found beaten and abused. It wasn't until he shaved off the beard, that she became friendly towards him. A dog I used to have also, was beaten as a puppy too, and would run and hide whenever you would bring a knife out of the drawer just to prepare a meal. 
