Caching only if some number of your own users upvote might work.
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Caching only if some number of your own users upvote might work.