Because, for a touch screen, the screen itself IS the user interface. Imagine while holding with one hand, you want to reach your thumb to the opposite corner to hit a button. Even if the body of the phone is the same, a larger screen will need a bigger reach for your thumb. That is primary issue.