I wouldn't say that.
Just because he lost is seat doesn't mean the conservatives didn't gain any ground. It's quite the opposite. And chances are, another MP is going to step down to have a partial election to elect him in another riding where he'll be more popular.
Best case scenario, he retires at the sweet young age of 45 with his cushy government pension and we never hear from him ever again.