(I summarised the following explanation from Quora and it sounds pretty accurate)

The main reason most people think is to prevent a revolt at the ballot box by upper-middle class voters.

The first non-govt large scale school system in Australia was set up by the Catholic Church. Many of Australia’s earliest colonists were Irish-Catholic and poor and so the Church set up a school system for them.

When the first public school system was set up, it started by appropriating a large number of Catholic schools. The Catholic school system survived, however, and has remained powerful in Australia.

In the past, the main purpose of governments providing funds to private schools wasn’t to keep the Catholic schools open, but rather to fund an Anglican equivalent for the children of English settlers - particularly out of fear that the Irish-Australians might have an advantage over them. Catholic vs Protestant factionalism is largely dead in modern Australia, but our current school funding system owes a great deal to it.

Nowadays, whenever politicians start pointing out that it is a bit crazy that the states pay for public education but the federal government also gives large grants to private schools, they encounter massive opposition. The last time this was suggested, many upper-middle class parents spoke out loudly, saying they were concerned that the funding to their children’s schools was the “only thing they got back” from the large amounts of tax they paid. They are very wrong about this, but it seems to be a widely held belief.

So, the system of using public money to support private education in Australia started for historical reasons that are no longer relevant, but continues because a powerful bloc of voters more-or-less demands it do so.