People are removing a common plant from their gardens as it's said to imply they could be swingers. Many are stepping into action in April to remedy the supposed issue