I think I did it at five. It was about the last thing I did before moving on to act 2. I started it at 2 or 3 and it was just too hard. By the time I went back I had the amulet that gives an extra magical missile and it trivializes it. I could just magic missile the clones and they'd instantly be gone. I probably could have done it the first time, but there was plenty of other things to do and the hardest difficulty is unforgiving.