I'll make an appeal to authority (kernel developer working on memory management):
Disabling swap does not prevent disk I/O from becoming a problem under memory contention, it simply shifts the disk I/O thrashing from anonymous pages to file pages. Not only may this be less efficient, as we have a smaller pool of pages to select from for reclaim, but it may also contribute to getting into this high contention state in the first place.
And then he goes on to say what I said, that it can make the OOM killer quicker to react.