Did Doflamingo run away from Luffy to do it? Then its def not part of the fight. Are you referring to Rebecca and Violet? That indeed was not a part of the fight.Luffy vs Doffy not a real fight because Doffy performed a move that target the weak confirmed.
Are you referring to when Luffy was fighting Bellamy underground? Again not part of the fight.
So yea it holds.