Jokingly in turn, "Hey, there's people I'd side with you over. Even if. Y'know. Most of them are probably the ones we both don't like."
Other options:
no subject
Jokingly in turn, "Hey, there's people I'd side with you over. Even if. Y'know. Most of them are probably the ones we both don't like."