Conundrum

Seems like you want to still be friends....just not with benefits, since you said "outside of the lifestyle"...yes? If so, are you comfortable both approaching the other couple and sharing exactly how you feel? Something along the lines of, "we still would love to be friends, but we're not interested in a sexual relationship." That way you're avoiding getting into the negatives about attraction, him, etc. If not, I'm not seeing how a friendship could be maintained and it would eventually fizzle out either due to the lack of sex or honesty if you're both avoidant sexually and they sense it. What are your thoughts?

Finding a perfect 4-some is one of the toughest things in the lifestyle, nearly impossible to luck into a situation where 4 individuals are equally attracted.

I think that "taking one for the team" is misused and over-simplified. The reality is that in every couples relationship we've been involved with, whether vanilla or horizontal, someone was probably not quite as eager as the rest in the group. As the guy, I'm rarely one to put on the brakes, but in our circle the women tend to be the social directors. Some people call it "pussy rules" - the ones with the pussy make the rules. YMMV.

Sometimes personal pronouns cause problems in figuring out the problem. If I understand this correctly, the female half of the couple is asking the question. There is a connection between your husband and the other wife and the other husband has an attraction to you and that's an attraction that you don't share.

So, unless you're willing to take one for the team (whatever that means), either explain your lack of interest if it comes up, politely indicate same without using your words or ghost them. You can't control how they will react to your lack of interest and some couples will not want to continue a friendship where there isn't a mutual attraction, so be prepared.

Explain to him that you have no male sexual desires. That's it. Pretty simple.

If it is a deal breaker for him, you lost them. If it isn't, congrats, you found a couple.