If my accusation is false, nothing will be done to waffle anyway.
Except it puts you in a bad light to put forward accusation which has no proof whatsoever. People, myself included, take it as a sign of you being vindictive, paranoid or something along these lines. In order to be productive, you should look at the matter from the other person's point of view and assume that they are onto something, and first try to see that POV. Otherwise, you're not showing that you're taking them seriously. Sure, it does nothing to me (except making me laugh), but it does something to your own credibility as someone who is willing to take others seriously and discuss while considering multiple points of view.
You accusing me of downvoting you with alts is a possible explanation, sure, but it is wildly implausible, has no evidence whatsoever, and was totally ridiculous to begin with. You have accomplished nothing good by putting it forward.