Considering it took Zoro quite a long time to finally cut Pica, and even then required the help of Elizabello, yeah, Pica did make it difficult for him.
That doesn't change the diff of the fight though. You don't need to inflict a lot of damage for a fight to high diff. If person A is so weak and slow that they can't land a single hit on person B, but it takes person B a long time to defeat them because they are super durable, then it's a high diff fight. Vergo vs Law was Law high diff even though he had the ability to defeat Vergo in one swing because Vergo was squeezing his heart. Luffy vs Usopp was mid diff because Usopp was so tricky and had prep in the terrain. The problem here is that you looking at diff in a very one dimensional manner. Someone doesn't need to take any damage at all to defeat someone high diff. Difficulty is just how hard was it to defeat your opponent, which can be determined and influenced through many ways and factors.
That doesn't make what I said about Bones vs Zoro an issue at all. The fight was high diff because Zoro lacked an ability needed for the fight, causing him to be unable to defeat his opponent while sustaining heavy damage. He learned the ability while fighting, allowing him to finally end it. The fact that Bones landed many, many hits on Zoro but failed to defeat him, while Zoro defeated Bones with one hit once his metal was broken, is actually proof that Bones was far weaker than Zoro. It was still high diff because Bones' ability made it harder for Zoro to cut him. Same happened with Pica, the difference just being tat Pica did so through evasive maneuvers and didn't damage Zoro.