Can't unban user unless you ban yourself first

This bug was inherited from Conduit. See also https://gitlab.com/famedly/conduit/-/issues/452.