Souder has admitted to an affair and is now leaving Congress. Let this be a reminder that we are all human and that we all make mistakes. This why the amount of power that we give any frail and imperfect human should be limited.
My opinion is that adultery is a private matter that should be resolved between the cheater and the aggrieved spouse. I would not rule out voting for a candidate based on an affair anymore than I would make a decision to hire or fire someone on that same basis (unless it was a relationship that presented a conflict of interest). To fire a man over an affair can potentially cause financial harm to his wife if she decides to stay with him, or leaves him and demands alimony or child support.