I think the trees died as part of Mom's ability. She takes souls and transplants them. They only have a set amount of life available. Maybe fear causes the implanted spirits to "flee" or be used up faster than real time. Which goes back to when she attacked her son and they kept telling him to not be afraid, because fear affects the souls she puts into objects.
THIS! I've been theorizing something similar. I think this will play a vital role in the story. I think fear also plays a big role in counteracting Linlin's abilities. When and if the time comes for Luffy to fight her, the only way it makes sense to me for Luffy to win is to exploit the loophole in this theory. Since Luffy will NOT be afraid of her, perhaps her powers will have less of or even no effect on him. Today's chapter basically confirmed that fear causes the souls affected by Big Mam to become more easily detached, but we will have to wait and see if Oda elaborates on it more.