I can't change the fight scenes as they are a huge part of my book. Not sure how else to describe the people the main characters are fighting other than "the first man", "the man" etc...
Have you tried describing the men in the fight scene? It's difficult for a reader to imagine 'the first man'. I also had the same problem but managed to get round it like this:
The two men escorted Giles round the side of the building to where he had been told someone was waiting with a message. Worried that something had happened to Emma, he turned expectantly as a beefy, red-faced man emerged from the shadows. But instead of a message, the man delivered a heavy punch into Giles' stomach, the searing pain taking his breath away and paralysing his thoughts.
As he doubled over the second man brought his knee up, catching Giles under the chin, forcing his teeth into his tongue. Groggy with shock, he reeled back, grabbing his assailant's jacket as he fell. Confused, he had no idea why he was being attacked.
"I've no money," he said, the words coming in ragged gasps as he struggled to summon his strength. In reply the man swung his fist, but this time Giles was ready and blocked it with his left hand, swinging with his right. His arms were strong and muscular from years of moulding dough and lifting heavy trays, and Giles felt the crunch of bone and the warm spurt of sticky blood and heard a satisfying roar of pain as his fist came in contact with his opponent's nose. But before he could do further damage, two beefy hands grabbed his collar and hauled him to his feet.
Fear coursed through him as he realised he was outnumbered three to one. He stood on unsteady feet, desperately searching for a way to escape. But two of them pinned his arms behind his back, and the third swung a punch. He tried to duck, but he was held firm and the heavy fist caught him square in the face.
Blood trickled warmly down his chin, and the salt taste of it was in his mouth. For a terrible moment he thought he was going to be killed, and he kicked out furiously. There was a moan of pain as his foot caught one of them in the groin. In retaliation the beefy one slammed him against the wall, and he felt a sickening crack as his head hit the brickwork. His vision became a red haziness as his knees sagged. Dimly he heard the shouts of Happy New Year, and the wail of the sirens and hooters as the band struck up Auld Lang Syne. He slid to the ground and everything went black.
With their heavy boots, the men kicked him in the ribs a few times for good measure before stealing off into the shadows, where Laurence waited with their money, a satisfied smirk on his long vengeful face.