Can I sue for defamation of character?

To start off, I already know my situation is complicated before anyone responds, but ultimately I am happy and just want the drama to end. So I live with my 1 year old daughter and her father that I have been with for 2 years. He has 7 other children with 4 other mothers (ranging from ages 2-16)...yes, I know.... Anyway, during our relationship, he cheated on me with 2 of those four mothers, and got caught. When the whole situation came to its screeching halt, one of the mothers decided that since I didn't leave him, she was gonna try to get rid of me. She started making accusations that I was abusing her 3 year old when he would come to visit. When the 3 year old started visiting us for 2 weeks at a time, he was 1. I was his caretaker during the day and we absolutely never had any issues up until the "big blowout". She is saying this occurred at least 6 months ago, but never said anything up until the blow out. Now, this has obviously caused much drama not only in our home where our relationship is in jeapordy because her father of course has to question it, but also with the other kids who now think that I have abused their little brother and want nothing to do with us. Not only that, but now she has filed official paperwork with the court regarding this to take away the fathers visitation because of it. To make it worse, my goal has been to open a daycare for years and now with people getting wind of the situation, I have heard that many are aware of the situation and I am sure that it will affect peoples judgement of me now and whether or not I would be capable of doing this job without inflicting harm on their kids. Before this, I could have called anyone and everyone that knows me to say what kid of person I am, but now, I am not so sure. Any time a person makes an allegation as she has especially with a toddler it should be taken seriously, but what can I do!? I am innocent!!!