warning - this is a long and drama filled rant

So I have to rant. I have to get this situation out of my mind as it just keeps playing over and over and over! I am really mad, bewildered and just don't know what to do.
It all started a couple of months ago after church on Sunday. A mom, we will call her "Crazy" brought her son, who we will call "Joe" to me. Crazy told Joe to tell me what Austin did. Joe said that Austin hit him and he started crying. Crazy then told me to get control of my son or next time she would discipline him. I was astounded and totally blindsided. I responded that I would talk to Austin and see what he had to say. Then Crazy disappeared. I talked to Austin and found out that, yes he had hit Joe but Joe had hit him first. Not surprising on either part as they are both energetic little six year old boys. (Her son has hit/kicked me several times in primary. She thinks he is an angel and can do no wrong.)
I thought the whole incident had blown over. I had seen Crazy at church, at birthday parties and various functions. She always acted fine, dare I say even friendly?
3 weeks ago
At church, the primary prez and I are sitting in the back of the primary room having a discussion about the new primary sharing time/sacrament meeting program guide for next year (which are awesome by the way!). Crazy comes in and leans across 2 other people to tell me to make sure to tell my son not to kick her (Austin was in his class at this time so I had no clue what had happened). Then she disappeared. Maybe I should have tracked her down right then and there and then it would never have gone any further.
The primary prez and I just looked at each other, both a little shocked. I told her that this was the not the first time this had happened and that Crazy seems to have a grudge against my son. She confirmed that this was so and went on to tell me that Crazy calls her on a regular basis to complain about "the Austin issue", how he is a menace and a threat to her son, that he is jealous because Joe is friends with another little boy at church, that her son should be protected at church, etc. She has gone so far as to complain to the bishopric as well. After church I asked Austin what had happened and he told me that he was kicking his legs and looking behind him when she walked in front of him and that he kicked her on accident. I believed him as that is completely something Austin like.
2 Sundays ago
I was talking to the primary prez again and she told me that no one knew what to do about "Crazy". That she had told the bishopric counselor over primary that because of Austin kicking her she now requires 6 months of physical therapy and she is going to bill us. The primary prez assured me that I am not the crazy one in this situation, that no one else understands why Crazy is doing this. I decide that week to send an email to Austin's teacher to see if she had witnessed what had happened. I had held off on doing this because I trust Austin and I did not want to drag anyone else into the middle of this. She sent an email back assure me that she had witnessed it and Austin had only kicked Crazy on accident, and that he apologized several times. She also emphasised that there was no way that Austin could have done any real injury.
Today
I get a call from the Bishop. He felt the need to give me a heads up about what is happening. Today Crazy brought the medical bill to him and wanted to know if he was going to give it to the Danenhauer's or would she have to. He replied that neither would be happening as it was policy that the church would pay for it. She was not happy with that response and wanted to give the bill to us. He tried to get the bill so he could write her a check to cover the expenses but instead she left in quite a huff. So the Bishop is concerned that she will be trying to bill us and wanted to assure me that the church would cover the bill.
I am not really concerned about the bill. I do not feel that we are responsible for it and if she cannot cover it the church will do so. What I am concerned about is being at church around this crazy lady and being in constant fear that she is going to threaten my son or come up with some more trumped up charges against us. Today I had to leave Austin alone during opening exercises so I could take Danny to nursery and set up for sharing time. I was a nervous wreck the whole entire time, shaking and nauseous. I was so relieved when Crazy snatched Joe out of primary and left.
I am not the kind of person to confront people and cause drama. I hate drama. I tend to try to understand and, when I can, just let things go. I realize that a lot of the time people don't mean to be hurtful or give offense. I am sure that there have been many times when I have unintentionally hurt someone and I would hope to have the same understanding. But what do you do when someone is maliciously out to get you? Especially when this is happening at church! I don't feel it is fair to drag other people into this and make them choose sides. I don't think that this should be happening at all. I just don't get it! So anyway, maybe now that I have that off my chest I can get it off my mind. Ugh!