I like it, but I don't think it should become a romance rule--it does disturb me when I see submission calls stating that no infidelity will be accepted or when readers say they will stop reading the book at that point.

I think you put it very well, that it can - done correctly - be a valid character trait and how the book deals with that is often the reason I enjoy to read them.