Not always. And I don't see the key distinction you mean. Can you explain further.

Using the example of my city again, there's no difference between the buses and taxis here in terms of contracted or not. The various bus companies are all privately owned. Some still have their own liveries. Some have the city council liveries. Some bus services don't have regular contracts with the council/government at all and just run various private services. Sometimes the council will contract them for one off services. Regardless of how they look or the contract (or lack thereof), they're all privately owned.

All the taxis are private owned. But the government/council contracts them for certain purposes. For example, if you are injured and unable to drive, ACC will pay for a taxi to take you to and from health services.

All of these companies are for-profita nd make profit from their contracts.

The profit made by some of the public services by private companies is a regular issue of contention in this country. As is the selling off by state owned public interest facilities (such as the rail system, power generation, communications, etc.).