somewhere in between
Depends. Since legalization, between two neighbours I have who seem to smoke regularly, I had to keep the windows closed to our kitchen and bedrooms over the summer months. We're talking about early morning to midnight. Working on my bike in my own backyard is a challenge because of that.
On walks/rides, this can happen in passing at least a few times per outing. At parks, sometimes, someone will be there smoking weed while they watch their kid, which means I can't take my grandkids to the park at that moment.
For me, it's not just a smell to be annoyed with. I get persistent headaches/migraines when exposed to cigarette or cannabis smoke, for hours. Effectively, it can ruin my day, and I've had to return from hikes on more than several occasions because of this.
I'm not arguing against people's choice to do drugs, but when it affects others around them, it's problematic. It's honestly disappointing that my original comment has been downvoted, considering this is something that I'd think any reasonable person would agree with. My experience has been profoundly negative.