Also you sound like you want to do something I can't endorse, so it's good that you don't want to train a model
Not sure what that means. I want to send a message sounding like Trump or respond with a joke in the voice of Ghandi to friends.
Anonymizing would also be a usecase. Then I could replace the audio of videos I want to make after narrating them, record an anonymous podcast, or send voice messages into groups I'm a part of with a different voice.