The attack on Chatterton's Hill now began in earnest. The brigade holding the position had been weakened by sending one battalion down to cover the retreat of the militia brigade. Encouraged by the fire of the British skirmishers this battalion had been swept up in the retreat of the militia and had sought refuge in the woods behind the hill. The skirmishers followed the retreating Americans and the line battalions now concentrated on assaulting the hill. The lack of a wall on the American right flank now caused problems. The Americans had had no problem with British fire from the front, but now they were also subjected to volleys from the flank. With no protection from this direction losses began to rise. Seeing the wall of bayonets advancing towards them the morale of the right hand American battalion broke and they too sought the protection of the woods behind the hill.
Fortunately for the American brigadier there was a transverse wall on the hill and he redeployed one of his two remaining battalions to line this in preparation for the expected British advance.
In the centre Clinton was slowly assembling his battalions ready to attack the right flank of the American position in White Plains. He was hampered by fire from the American artillery and also the fire of the American defenders. He was constantly employed rallying his troops as they fell back under the American volleys. However, he had positioned his cavalry on the road, ready to exploit any shakiness in the opposition line. To his right, Archer's brigade was exerting real pressure on the American position. The Hessian fusiliers had now crossed the works driving off the militia who attempted to attack them, began firing on the defenders of White Plains. Their supports were now crossing the works and the artillery was moving at its best speed towards that flank.
Brown's battalions crested the brow of Chatterton's Hill and were met by a volley from the defenders. Undaunted they stopped, fired a volley and continued their advance. The right flank battalions of Brown's brigade now advanced and began to fire at the battalion lining the wall. Flinching from this flanking fire and eyeing the advancing bayonets the Americans decided it was time to take to the woods as their comrades had done and soon all resistance on the hill ceased.
Howe sensed that a major victory may be within his grasp; with luck he could 'net' a substantial part of the American force if Brown cut the road behind White Plains. However, Clinton's men were almost 'played out'. Their advance had slowed as officer casualties mounted and they were passed by Brown's grenadiers who swept forward towards the American held works. A volley from the grenadiers drove off the artillerymen who were attempting to withdraw their guns and a second forced the battalion holding the works to withdraw. As the red line of grenadiers flowed into the American position there seemed to be no opponents left to dispute possession of White Plains.
Brown's success had not been duplicated by Archer. He had seemed to be on the brink of victory as his fusiliers moved against White Plains; but the American commander called upon his last reserves to oppose him. The fusiliers were stopped and then driven back by volleys from a militia unit. The supporting battalion was fired on before it could deploy and had to fall back to reform. A third battalion was also stopped in its tracks by accurate fire from the defenders. Suddenly the pressure on the Americans eased and, taking the opportunity the order to retreat was given. The British pincers failed to close and so the Americans were able to slip away. On the road Clinton's cavalry commander fumed as the order came to hold his position. Surely, now was the time for the cavalry to advance and turn retreat into rout.
Clinton explained to Howe that is brigade was in no shape to advance and Howe confirmed the order to halt.
Although the British had carried the position two of their three brigades required time to reorganise. For the Americans Two brigades has been driven from the field and a third was in a bad way; Washington felt fortunate that he was able to withdraw unhindered.