The score matters as far as what jobs you qualify for, and it also tests different aptitudes. For example, two people could have the same overall score (say a 70), but one person could show mechanical aptitude and be pushed toward a Machinist position, while the other could do poorly on mechanical but do well in electrical stuff and become an electrician.
They also don't want to waste their smarter people as basic grunts or cooks, so a higher ASVAB score can mean you're less likely to get the job you want if, say, you score 90+, but always wanted to be a chef. If you have a specific job you want to go in for, you basically have to get it in writing that you're joining for that job, otherwise you're at the mercy of the "needs of the {branch name}" - you will be what they need most that you're very good at.