I've long used them in PvP so I'm guilty of it myself, but honestly it's mostly due to how game changing and broken Magic Potions are (and charms to some extent). In modes where you have to kill another player to achieve an objective (capturing a Windmill in GT or scoring in Arena), it creates a massive problem because some players are literally unkillable with Charms and basically EVERY player is unkillable with a Magic Potion unless you can 1 hit them (which won't happen in Arena usually because everyone is a similar level). Arena weapons are pretty overwhelmingly broken in themselves, but add on Charms and mpots and how is someone without one even supposed to have a chance? The balance difference is far too much. A player with an Arena weapon (in Arena) usually has to wear the same gear as everyone else, and that would essentially be their weakness. Add charms and mpots on that and the weakness no longer exists, and they still have far more damage than another player that doesn't have any Arena weapon.
At any rate, I think Charms and Magic Potions are great for open world fighting (like a guild war) and PvE, but for instanced PvP such as CTF Arena and Guild Tournament, they simply create an overwhelming problem that just can't be overcome even with the most extreme means (AKA doing it yourself). I mean, I even be willing to call it balanced if two equal players could kill each other charmed and mpotting, but even that isn't possible. Even two charmed Legendary users fighting each other with mpots cannot kill each other, which is pretty outrageous and seems quite illogical. Once again, in PvE I can totally understand, but in PvP where players are meant to kill each other there shouldn't be any means for anyone to become completely invincible.. which is essentially what a mpot allows players to do.