As @cwagner@lemmy.cwagner.me commented, the official definitions of free software and Open Source actually overlap quite heavily; the concerns made by many - including Stallman/the FSF and even Bruce Perens (author of the Open Source Definition) - involve the belief that Open Source has detached from the values associated with the free software movement.

If you are in fact specifically addressing the fairly small subset of open-source-but-not-free software, I would guess that the overlap is great enough for it to not detract from discussions, and "open source" is simply more commonly used.

Just a note, I'm also pretty sure some people in the comments have mixed up free-as-in-libre software for free-as-in-beer software, which is why I prefer to say "libre" instead.