You're on kbin.social, so I'm gonna make some assumptions about how it works, because I really only have a decent understanding of lemmy - though I'm assuming this is how other activitypub federated services work too.

While it may seem like you (on kbin.social) are replying to this post (on lemmy.nz), that is not the case. You server has a copy of the reply and all the other posts and comments. When you comment, it gets posted to the kbin.social community (magazine?) called !newzealand@lemmy.nz.

Behind the scenes, your server will syncronise your copy with the official lemmy.nz copy.

A user on (say) lemmy.world will also have a copy on their server. Your server will not send your comment directly there, rather it sends it to the only source of truth (the community host server), then the host server will send it out to all servers where there is a subscriber.

By defederating, they will no longer get any updates for any lemmy.nz communities, but will still see their own comments (and other users on their server will see them).

Honestly, I'm not sure what happens when e.g. a lemmy.nz user posts to lemmy.ml who are one of the few large servers still federated with hexbear - do the comments go to hexbear? Not sure. Do hexbear comments made to lemmy.ml federate back to lemmy.nz? Again, not sure (I suspect not).

they can still see your posts, they can even comment and vote on them. You just don't see it anymore.

This would actually be ideal, as long as their downvotes don't get federated back to us.