To some extent, it depends on what else is happening - nerves, condoms, discomfort having sex with other men nearby, overstimulation, concern about your wife's pleasure or jealousy . But if you're like most men with occasional situational ED issues, it's mostly that it happened once (because of nerves, condoms, discomfort having sex with other men nearby, overstimulation, concern about your wife's pleasure or jealousy) and forever after you've had that niggling concern that it'll happen again, pretty much guaranteeing that it will sometimes.
So take Viagra or Cialis, recruit a partner of the moment that is okay that you sometimes have ED issues, and instead of thinking about your dick, think about doing what feels good for the two of you. Maybe your dick will get hard, maybe it won't, but sex that doesn't involve a hard dick can be amazing if the guy doesn't make it some kind of drama event.