The loli genre was created after the publication of a book called "Lolita" which is about a middle aged man becoming sexually obsessed with a 12 year old girl.
Loli is slang for Lolicon, which is a genre defined by works that portray child characters or characters that appear like young children in erotic and sexual manners. It also refers to people who have a sexual attraction to such characters.
If you're a fan of Loli, you might not be a pedophile, but you're still supporting a genre where the crutch is the sexualization of children's bodies.