thought it was a religion

It absolutely started as a religious practice. There are various forms of yoga that are used as mediation practices in Hinduism.

That said, Westerners almost never practice yoga that way. They'll sprinkle in a generous helping of, "namaste" but they're basically doing it as a form of light exercise.

It's like if a bunch of Indians saw a Catholic ceremony and said, "That's a pretty good workout. Stand up, sit down, kneel, sit, kneel, stand. It's not a religious thing, I just go to mass for leg day."