Always take fillers with a grain of salt. Danzo is a little more sneaky than that, Kakashi definitely would have told Hiruzen, and Hiruzen might have actually done something about that in that incident. But Hiruzen was often thought of as lenient leader when you consider what happened between him and Oro before so I'm guessing the anime went on with that.