One problem with almonds is that they can't grow just anywhere.

It's often grown in regions where water is already an issue, with pretty significant irrigation, with significant environmental consequences.

Meanwhile, it is possible to raise cows in a humid climate where water isn't a major issue. Though it's also totally possible to feed them alfalfa grown with water from the Colorado River, which is way way worse than anything else.

I'm not saying there aren't any problems with cow milk at all. There absolutely are, and overall I'm pretty sure it's worse than any other option.

But personally, especially considering I like the taste of the vast majority of dairy alternatives anyway, getting one with a lower environmental impact compared to other alternatives is a factor in my decision.