He can do it. He broke a few branches in part 1 with ease imagine if he continuosly destroyed the hands with Enma
Logically, he should be able to keep on using his taijutsu prowess together with Enma pole to get rid of that fake statue, but plot says no o>o