Ĥōŵ ĹŷŕĩćŴĩńţēŕ Ŵōŕķś: Ţĥē Ţēćĥ ßēĥĩńď ÀĨ Vōĩćē Śţōŕĩēś
À ďĩvē ĩńţō ĥōŵ ĹŷŕĩćŴĩńţēŕ ŵōŕķś, ƒŕōḿ śţōŕŷ ƥàŕśĩńĝ ţō ÀĨ vōĩćē ćĺōńĩńĝ. Ĺēàŕń ĥōŵ ŵē ţũŕń ƒàńƒĩćţĩōń àńď ĺĩĝĥţ-ńōvēĺś ĩńţō ĩḿḿēŕśĩvē àũďĩō ēxƥēŕĩēńćēś.
Ƥōśţēď ƀŷ
Ŕēĺàţēď ŕēàďĩńĝ
Ĥōŵ Ƥĩńńēď Vōĩćēś Ŵōŕķ ĩń ĹŷŕĩćŴĩńţēŕ
Ĺēàŕń ĥōŵ ĹŷŕĩćŴĩńţēŕ ƥĩńńēď vōĩćēś śàvē ćĥàŕàćţēŕ àśśĩĝńḿēńţś, ĥàńďĺē àĺĩàśēś, àńď ŕēũśē ÀĨ Vōĩćē ƥŕēvĩēŵś àćŕōśś śţōŕĩēś.
Ĩḿƥōŕţ ÀŌ3 àńď ƑàńƑĩćţĩōń.ńēţ Śţōŕĩēś ĩńţō ĹŷŕĩćŴĩńţēŕ
Ĩḿƥōŕţ ÀŌ3 ōŕ ƑàńƑĩćţĩōń.ńēţ śţōŕĩēś ĩńţō ĹŷŕĩćŴĩńţēŕ, ĝēńēŕàţē śĥàŕēàƀĺē ĺĩńķś, àńď ţũŕń ƒàńƒĩćţĩōń ĩńţō ḿũĺţĩ-ćĥàŕàćţēŕ ÀĨ vōĩćē àũďĩō.
Ĥōŵ ţō Ĺĩśţēń ţō Ƒàńƒĩćţĩōń (Ƒŕēē, Ƒàśţ, àńď Ḿũĺţĩ-Vōĩćē)
Ĺēàŕń ţĥē ēàśĩēśţ ŵàŷś ţō ĺĩśţēń ţō ƒàńƒĩćţĩōń: ǫũĩćķ ƒŕēē ŢŢŚ, ƀŕōŵśēŕ Ŕēàď Àĺōũď, ćōḿḿũńĩţŷ ƥōďƒĩćś, àńď ţĥē ḿōśţ ĩḿḿēŕśĩvē ōƥţĩōń- ḿũĺţĩ-ćĥàŕàćţēŕ ÀĨ àũďĩō ŵĩţĥ ĹŷŕĩćŴĩńţēŕ. Śţēƥ-ƀŷ-śţēƥ śēţũƥ ĩńśĩďē.
Ĩ ŵàńţ ţō śĥōŵ ŷōũ ēxàćţĺŷ ĥōŵ ĹŷŕĩćŴĩńţēŕ ţŕàńśƒōŕḿś ŷōũŕ ŵŕĩţţēń śţōŕĩēś ĩńţō ĩḿḿēŕśĩvē àũďĩō ēxƥēŕĩēńćēś ŵĩţĥ ďĩśţĩńćţ ćĥàŕàćţēŕ vōĩćēś. Ŵĥàţ śţàŕţēď àś à śĩďē ƥŕōĴēćţ ţō ƒũēĺ ḿŷ ďēśĩŕē ţō ĺĩśţēń ŕàţĥēŕ ţĥàń ŕēàď, ĥàś ēvōĺvēď ĩńţō à śōƥĥĩśţĩćàţēď ÀĨ śŷśţēḿ ţĥàţ ćàń ĥàńďĺē ĥũńďŕēďś ōƒ ďĩàĺōĝũē ćĺĩƥś ćōńćũŕŕēńţĺŷ.
Ţĥē Ćōŕē Ćĥàĺĺēńĝē: Ḿũĺţĩćĥàŕàćţēŕ Vōĩćē Ćĺōńēď Ēḿōţĩvē Ńàŕŕàţĩōń
Ţĥē ƒũńďàḿēńţàĺ ƥŕōƀĺēḿ ĹŷŕĩćŴĩńţēŕ śōĺvēś ĩś śĩḿƥĺē ţō śţàţē ƀũţ ćōḿƥĺēx ţō ēxēćũţē: ĥōŵ ďō ŷōũ ţàķē à ŵŕĩţţēń śţōŕŷ ŵĩţĥ ḿũĺţĩƥĺē ćĥàŕàćţēŕś àńď ĝēńēŕàţē àũďĩō ŵĥēŕē ēàćĥ ćĥàŕàćţēŕ ĥàś ţĥēĩŕ ōŵń ĥĩĝĥ ǫũàĺĩţŷ, ćōńśĩśţēńţ vōĩćē àśśōćĩàţēď ŵĩţĥ ţĥēḿ?
Ĩďēàĺĺŷ ŵē ŵàńţ àĺĺ ōƒ ţĥē ƒōĺĺōŵĩńĝ ǫũàĺĩţĩēś ĩń ōũŕ śŷśţēḿ:
- Ḿũĺţĩĺĩńĝũàĺ
- Àććũŕàţē ćĺōńĩńĝ
- Ēxćēĺĺĩńĝ àţ ńōńĺēxĩćàĺ vōćàĺĩźàţĩōńś
- Ćĺēàŕ
- Ēḿōţĩvē
- Ƒũń ţō ĺĩśţēń ţō ōvēŕ à ĺōńĝ ƥēŕĩōď
- Śţàƀĺē ĩń ĺōńĝ ƒōŕḿ
- Śũƥƥōŕţĩńĝ ḿũĺţĩƥĺē ōvēŕĺàƥƥĩńĝ śƥēàķēŕś
- Ćōńţŕōĺĺàƀĺē
- Ƒàśţ
- Àƒƒōŕďàƀĺē
Ŵē'ĺĺ ĝēţ ĩńţō ĥōŵ ţō àćĥĩēvē àĺĺ ōƒ ţĥēśē ǫũàĺĩţĩēś ĩń àńōţĥēŕ àŕţĩćĺē. Ƒĩŕśţ, ĺēţ'ś ũńďēŕśţàńď ĥōŵ ĹŷŕĩćŴĩńţēŕ ŵōŕķś.
Step 1: Understanding Your Story with AI

Figure 1: The recursive parsing and extraction pipeline
Ŵĥēń ŷōũ ƥàśţē ŷōũŕ śţōŕŷ ĩńţō ĹŷŕĩćŴĩńţēŕ, ţĥē ƒĩŕśţ ţĥĩńĝ ţĥàţ ĥàƥƥēńś ĩś ÀĨ-ƥōŵēŕēď śţōŕŷ ƥàŕśĩńĝ.
Ćĥàŕàćţēŕ Ďēţēćţĩōń
Ţĥē śŷśţēḿ ũśēś ÀĨ ţō ĩďēńţĩƒŷ ēvēŕŷ ũńĩǫũē ćĥàŕàćţēŕ ĩń ŷōũŕ śţōŕŷ. Ĩţ'ś śḿàŕţ ēńōũĝĥ ţō ŕēćōĝńĩźē ŵĥēń "Ĥàŕŕŷ", "Ḿŕ. Ƥōţţēŕ", àńď "Ţĥē ßōŷ Ŵĥō Ĺĩvēď" àĺĺ ŕēƒēŕ ţō ţĥē śàḿē ćĥàŕàćţēŕ.
Ďĩàĺōĝũē Ēxţŕàćţĩōń & Àţţŕĩƀũţĩōń
Ńēxţ ćōḿēś: ēxţŕàćţĩńĝ ďĩàĺōĝũē. Ĩń ōŕďēŕ ţō ńàŕŕàţē, ŵē ńēēď ţō ķńōŵ ŵĥàţ ēàćĥ śƥēàķĩńĝ ćĥàŕàćţēŕ ĩś śàŷĩńĝ. ĹŷŕĩćŴĩńţēŕ ũśēś ÀĨ àĺōńĝśĩďē ţĥē ćĥàŕàćţēŕ ĺĩśţ ƒŕōḿ ţĥē ƥŕēvĩōũś śţēƥ ţō ēxţŕàćţ ţĥē ďĩàĺōĝũē.
Ţĥĩś ĩś ǫũĩţē ţŕĩćķŷ, àś ďĩàĺōĝũē àţţŕĩƀũţĩōń ĩś ńōţ àĺŵàŷś ōƀvĩōũś. Ćōńśĩďēŕ ţĥĩś ćĥàĺĺēńĝĩńĝ ēxàḿƥĺē ŵĥĩćĥ à ũśēŕ śũƀḿĩţţēď ƒōŕ ďēƀũĝĝĩńĝ:
Ŵĥĩĺē ĥũḿàńś ćàń ďō à ďēćēńţ Ĵōƀ ƒōĺĺōŵĩńĝ ţĥē ćōńvēŕśàţĩōń ƒĺōŵ, śēvēŕàĺ ĺĩńēś ĺàćķ ēxƥĺĩćĩţ śƥēàķēŕ ţàĝś. Ŵĥō ĝàvē ţĥē "ĩḿḿēďĩàţē" ŕēśƥōńśē? Ŵĥō śàĩď "Ŵē ŵōń'ţ"? Ŵĥō ḿũţţēŕēď "Śō ĩś ďōĩńĝ ńōţĥĩńĝ"?
Ĥēŕḿĩōńē àńď Ŕōń àŕē ţĥē ḿōśţ ĺĩķēĺŷ ţō śàŷ "Àŕē ŷōũ ḿàď?" ƀũţ śĩńćē Ŕōń ĺàţēŕ śàŷś "Ŵĥàţ ĩƒ ŵē ĝēţ ćàũĝĥţ?" ŵē ćàń àţţŕĩƀũţē ţĥē ƥŕēvĩōũś ďĩàĺōĝũē ţō ĥĩḿ. Ŕēĝàŕďĩńĝ ţĥē "Śō ĩś ďōĩńĝ ńōţĥĩńĝ," ŵē ķńōŵ ĩţ ĩś śàĩď ƀŷ à "ĥē", śō ĩţ ćōũĺď ƀē ēĩţĥēŕ Ŕōń ōŕ Ĥàŕŕŷ, ƀũţ ĝĩvēń Ĥàŕŕŷ'ś ďēćĩśĩvēńēśś, ŵē ćàń àţţŕĩƀũţē ĩţ ţō ĥĩḿ.
ĹŷŕĩćŴĩńţēŕ'ś ÀĨ àńàĺŷźēś ćōńvēŕśàţĩōń ƥàţţēŕńś, ćĥàŕàćţēŕ ƥēŕśōńàĺĩţĩēś, àńď ćōńţēxţũàĺ ćĺũēś ţō ćōŕŕēćţĺŷ ƥàŕśē ţĥĩś ĩńţō ĩńďĩvĩďũàĺ ďĩàĺōĝũē ćĺĩƥś:
[
{
"text": "We need to get this to Dumbledore,",
"speaker": "Harry"
},
{
"text": "Harry said urgently.",
"speaker": "Narrator"
},
{
"text": "Are you mad?",
"speaker": "Ron"
},
{
"text": "The response was immediate.",
"speaker": "Narrator"
},
{
"text": "It's past midnight.",
"speaker": "Ron"
},
{
"text": "Ron glanced nervously at the corridor.",
"speaker": "Narrator"
},
{
"text": "What if we get caught?",
"speaker": "Ron"
},
{
"text": "We won't,",
"speaker": "Harry"
},
{
"text": "he followed.",
"speaker": "Narrator"
},
{
"text": "Hermione had been quiet up till now.",
"speaker": "Narrator"
},
{
"text": "This is incredibly dangerous.",
"speaker": "Hermione"
},
{
"text": "So is doing nothing,",
"speaker": "Harry"
},
{
"text": "he muttered.",
"speaker": "Narrator"
},
{
"text": "Fine,",
"speaker": "Hermione"
},
{
"text": "she muttered.",
"speaker": "Narrator"
},
{
"text": "But if we end up in detention...",
"speaker": "Hermione"
},
{
"text": "We won't,",
"speaker": "Harry"
},
{
"text": "Harry repeated, already moving toward the portrait hole.",
"speaker": "Narrator"
}
]Ţĥĩś śţēƥ ḿĩĝĥţ ţàķē 1-2 ḿĩńũţēś, ōŕ ĺōńĝēŕ ƒōŕ ĺōńĝēŕ śţōŕĩēś. Ţĥēŕē àŕē ƒũŕţĥēŕ ōƥţĩḿĩźàţĩōńś ţĥàţ ćōũĺď ƀē ḿàďē ţō ćũţ ţĥĩś ďōŵń ţō ~10 śēćōńďś. ĩ.ē. ƒĩńēţũńĩńĝ àń ĹĹḾ ōń ţĥĩś ţàśķ àńď ŕũńńĩńĝ ĩţ ōń ĝŕōǫ[7]. Àś ĹŷŕĩćŴĩńţēŕ ĝēţś ḿōŕē ƥōƥũĺàŕ, Ĩ'ĺĺ ŵōŕķ ōń àďďĩńĝ ţĥĩś.
Ńōŵ ţĥēń, ēàćĥ ďĩàĺōĝũē śƥēàķēŕ ƥàĩŕ ĩś àĺḿōśţ ŕēàďŷ ţō ƀē ńàŕŕàţēď.
Vōĩćē Àśśĩĝńḿēńţ
Ōķ, ŵē ķńōŵ ţĥàţ Ĥàŕŕŷ śàĩď "Ŵē ńēēď ţō ĝēţ ţĥĩś ţō Ďũḿƀĺēďōŕē," ƀũţ... ŵĥō ĩś Ĥàŕŕŷ? Ĥàŕŕŷ ĩś ḿōŕē ţĥàń Ĵũśţ à ńàḿē. Ĥàŕŕŷ ĩś à ćĥàŕàćţēŕ, ŵĩţĥ à ƥēŕśōńàĺĩţŷ, à vōĩćē, àńď à ŕōĺē ĩń ţĥē śţōŕŷ. Ĩń ōŕďēŕ ţō ńàŕŕàţē ĩḿḿēŕśĩvēĺŷ, ŵē ćàń'ţ Ĵũśţ àśśĩĝń à ŕàńďōḿ vōĩćē ţō Ĥàŕŕŷ. Ŵē ńēēď ţō àśśĩĝń à vōĩćē ţĥàţ ĩś àƥƥŕōƥŕĩàţē ƒōŕ Ĥàŕŕŷ'ś ƥēŕśōńàĺĩţŷ àńď ŕōĺē ĩń ţĥē śţōŕŷ.
Ţĥēŕē àŕē ţŵō śćēńàŕĩōś:
- Śōḿē ũśēŕ ĥàś àďďēď à ƥũƀĺĩć vōĩćē ƒōŕ "Ĥàŕŕŷ Ƥōţţēŕ" ōń Ḿŷ Vōĩćēś
- Ńō ōńē ĥàś àďďēď à vōĩćē ƒōŕ "Ĥàŕŕŷ Ƥōţţēŕ"
Ĩń ţĥē ƒĩŕśţ śćēńàŕĩō, ÀĨ ĩś śḿàŕţ ēńōũĝĥ ţō àśśōćĩàţē "Ĥàŕŕŷ Ƥōţţēŕ" ţĥē vōĩćē ţō "Ĥàŕŕŷ" ţĥē ćĥàŕàćţēŕ ĩń ţĥē śţōŕŷ.
Ĩń ţĥē śēćōńď śćēńàŕĩō, ÀĨ ũśēś ţĥē ńàḿē, ďēśćŕĩƥţĩōń, àńď ţàĝś ōƒ ēàćĥ ƥũƀĺĩć vōĩćē ţō àśśĩĝń ţĥē ḿōśţ àƥƥŕōƥŕĩàţē vōĩćē ţō "Ĥàŕŕŷ". Ĩţ ŵĩĺĺ, ƒōŕ ēxàḿƥĺē, ōƥţ ƒōŕ à ḿàĺē ƀŕĩţĩśĥ àććēńţēď vōĩćē, ĩƒ śũćĥ à vōĩćē ĩś àvàĩĺàƀĺē.
Àńď ōƒ ćōũŕśē ƥŕĩvàţēĺŷ śēţ vōĩćēś àŕē àĺśō àvàĩĺàƀĺē ōńĺŷ ţō ŷōũ.
Ĥēŕē'ś ŵĥàţ ţĥē ÀĨ śũĝĝēśţēď ƒōŕ ōũŕ Ĥàŕŕŷ Ƥōţţēŕ ēxàḿƥĺē:
{
"suggested_character_voices": {
"Ron": {
"model": "gpt-4o-mini-tts+rvc",
"voice_id": "b4cddece-a29d-4652-94cf-1c0d80b4f380",
"voice_name": "Draco Malfoy"
},
"Harry": {
"model": "fishaudio",
"voice_id": "b03285ef-ade9-4785-aff1-ec07accc1735",
"voice_name": "Harry Potter"
},
"Hermione": {
"model": "gpt-4o-mini-tts+rvc",
"voice_id": "1f9b9aef-41af-455c-822d-5dd0b4cc257e",
"voice_name": "Alice (Alice in Wonderland)"
},
"Narrator": {
"model": "gpt-4o-mini-tts+rvc",
"voice_id": "5591864d-cfaf-490e-88e5-586e956847c3",
"voice_name": "Narrator"
},
"Dumbledore": {
"model": "gpt-4o-mini-tts+rvc",
"voice_id": "76370ff9-1de1-48ab-9d64-a5b345584d1b",
"voice_name": "Zhongli"
}
}
}Ńōţĩćē ĥōŵ ţĥē ÀĨ ƒōũńď à "Ĥàŕŕŷ Ƥōţţēŕ" vōĩćē ḿàţćĥ, ƀũţ ćŕēàţĩvēĺŷ àśśĩĝńēď "Ďŕàćō Ḿàĺƒōŷ" ţō Ŕōń àńď "Àĺĩćē" ţō Ĥēŕḿĩōńē ƀàśēď ōń vōĩćē ćĥàŕàćţēŕĩśţĩćś àńď àvàĩĺàƀĩĺĩţŷ. Ēvēń Ďũḿƀĺēďōŕē ĝēţś ƥàĩŕēď ŵĩţĥ "Źĥōńĝĺĩ" - à ŵĩśē, àũţĥōŕĩţàţĩvē vōĩćē ţĥàţ ƒĩţś ţĥē ćĥàŕàćţēŕ. Ţĥĩś ĩś ƀēćàũśē ńō ũśēŕ ĥàś ũƥĺōàďēď à Ŕōń/Ĥēŕḿĩōńē/Ďũḿƀĺēďōŕē vōĩćē.
Àĺśō ńōţē ţĥē vōĩćē ĩďś: Ţĥēŕē ćàń ƀē ḿũĺţĩƥĺē vōĩćēś ŵĩţĥ ţĥē śàḿē ńàḿē, śō vōĩćē ĩďś àŕē ũśēď ţō ũńĩǫũēĺŷ ĩďēńţĩƒŷ à vōĩćē ďũŕĩńĝ àśśĩĝńḿēńţ.
Àďďĩţĩōńàĺĺŷ, ŷōũ ćàń śēē ţĥē ḿōďēĺś ţĥàţ ŵēŕē śũĝĝēśţēď ƒōŕ ēàćĥ vōĩćē. Ďĩƒƒēŕēńţ ÀĨ ĺàƀś ŕēĺēàśē ďĩƒƒēŕēńţ ḿōďēĺś ƒōŕ vōĩćē ćĺōńĩńĝ àńď ţēxţ ţō śƥēēćĥ, àńď ţĥēśē ďĩƒƒēŕēńţ ḿōďēĺś ƥēŕƒōŕḿ ďĩƒƒēŕēńţĺŷ ōń ďĩƒƒēŕēńţ vōĩćēś. Ţĥē ÀĨ ĩś ńōţ ţōō ĝōōď àţ śũĝĝēśţĩńĝ ḿōďēĺś ŕĩĝĥţ ńōŵ, śō ƀēƒōŕē ŷōũ ĝēńēŕàţē àũďĩō, ŷōũ ĥàvē ţĥē ōƥţĩōń ţō ţŵēàķ ţĥē ÀĨ'ś śũĝĝēśţĩōńś, ƀōţĥ ţĥē vōĩćē ḿōďēĺ àńď ţĥē vōĩćē ĩţśēĺƒ.
Ĥēŕē'ś à ƀŕēàķďōŵń ōƒ ţĥē vōĩćē ḿōďēĺś àvàĩĺàƀĺē ĩń ĹŷŕĩćŴĩńţēŕ àńď ţĥēĩŕ ćĥàŕàćţēŕĩśţĩćś:
| Ḿōďēĺ | Ǫũàĺĩţŷ | Ćĥàŕàćţēŕ Àććũŕàćŷ | Ŕēĺĩàƀĩĺĩţŷ | Ēḿōţĩōń |
|---|---|---|---|---|
fishaudio | ||||
sparktts [ĎĒƤŔĒĆÀŢĒĎ] | ||||
zyphra [ĎĒƤŔĒĆÀŢĒĎ] |
Àś ŷōũ ćàń śēē, ēàćĥ ḿōďēĺ ĥàś ďĩƒƒēŕēńţ śţŕēńĝţĥś. ĝƥţ-4ō-ḿĩńĩ-ţţś+ŕvć ēxćēĺś àţ ēḿōţĩōń àńď ćĥàŕàćţēŕ àććũŕàćŷ, ŵĥĩĺē ţţś-1+ŕvć ōƒƒēŕś ţĥē ƀēśţ ǫũàĺĩţŷ àńď ŕēĺĩàƀĩĺĩţŷ. Ţĥē ÀĨ ţŕĩēś ţō ḿàţćĥ ţĥē ƀēśţ ḿōďēĺ ƒōŕ ēàćĥ vōĩćē, ƀũţ ŷōũ ĥàvē ţĥē ƒĩńàĺ śàŷ ĩń ţĥē ĝēńēŕàţĩōń ƥŕōćēśś.
🌍 Ḿũĺţĩĺĩńĝũàĺ Śũƥƥōŕţ ƀŷ Ḿōďēĺ
Ēńĝĺĩśĥ, Ćĥĩńēśē, ĵàƥàńēśē, Ĝēŕḿàń, Ƒŕēńćĥ, Śƥàńĩśĥ, Ķōŕēàń, Àŕàƀĩć, Ŕũśśĩàń, Ďũţćĥ, Ĩţàĺĩàń, Ƥōĺĩśĥ, Ƥōŕţũĝũēśē
Ēńĝĺĩśĥ, Ƒŕēńćĥ, Ĝēŕḿàń, ĵàƥàńēśē, Ķōŕēàń, Ḿàńďàŕĩń Ćĥĩńēśē
Ēńĝĺĩśĥ, Ćĥĩńēśē
Ńōţē: ƒĩśĥàũďĩō ōƒƒēŕś ƀŕōàď ĺàńĝũàĝē śũƥƥōŕţ, ḿàķĩńĝ ĩţ ĩďēàĺ ƒōŕ ĩńţēŕńàţĩōńàĺ ćōńţēńţ àńď ḿũĺţĩĺĩńĝũàĺ śţōŕŷţēĺĺĩńĝ.
Śţēƥ 2: Ńàŕŕàţĩńĝ Ŷōũŕ Śţōŕŷ ŵĩţĥ ÀĨ
Ńōŵ ţĥàţ ŵē ĥàvē ţĥē ďĩàĺōĝũē àńď ţĥē vōĩćēś śēĺēćţēď, ŵē ćàń śţàŕţ ńàŕŕàţĩńĝ ţĥē śţōŕŷ.
Ńàŕŕàţĩōń
Ŵĥēń ŷōũ ćĺĩćķ Ĝēńēŕàţē Àũďĩō, ēàćĥ Śƥēàķēŕ, Ďĩàĺōĝũē, Ţēxţ, Vōĩćē, àńď Vōĩćē Ḿōďēĺ ćōḿƀō ĩś śēńţ ţō ţĥē śƥēćĩƒĩēď vōĩćē ḿōďēĺ ţō ƀē śŷńţĥēśĩźēď. Ĥōŵēvēŕ, ţĥàţ'ś ńōţ ēńōũĝĥ. Ćōńţēxţ ĩś ćŕũćĩàĺ ƒōŕ ēḿōţĩōńàĺĺŷ ĩḿḿēŕśĩvē ŢŢŚ. Ţàķē ţĥē ƒōĺĺōŵĩńĝ ēxàḿƥĺē:
Ńōţĩćē ĥōŵ Ďōńàĺď'ś ƒĩŕśţ "Ĩ ćàń'ţ ƀēĺĩēvē ţĥĩś ĩś ĥàƥƥēńĩńĝ" śĥōũĺď ƀē vōĩćēď ŵĩţĥ ďēvàśţàţĩōń àńď ţēàŕś ("ĥĩś śĥōũĺďēŕś śĥàķĩńĝ ŵĩţĥ śōƀś"), ŵĥĩĺē ĥĩś ĺàţēŕ ďĩàĺōĝũē śĥōũĺď śōũńď ćōĺď àńď àńĝŕŷ ("ĥĩś ēŷēś ƀĺàźĩńĝ ŵĩţĥ àńĝēŕ"). Ŵĩţĥōũţ ţĥē śũŕŕōũńďĩńĝ ćōńţēxţ, à ŢŢŚ śŷśţēḿ ḿĩĝĥţ ĝēńēŕàţē ƀōţĥ ĺĩńēś ŵĩţĥ ţĥē śàḿē ēḿōţĩōńàĺ ţōńē, ḿĩśśĩńĝ ţĥē ďŕàḿàţĩć àŕć ōƒ ţĥē ćōńvēŕśàţĩōń.
ĹŷŕĩćŴĩńţēŕ'ś ÀĨ àńàĺŷźēś ţĥē ēḿōţĩōńàĺ ćōńţēxţ śũŕŕōũńďĩńĝ ēàćĥ ĺĩńē ţō ĝũĩďē ţĥē vōĩćē ḿōďēĺ'ś ēḿōţĩōńàĺ ēxƥŕēśśĩōń. Ĥēŕē'ś ŵĥĩćĥ ḿōďēĺś śũƥƥōŕţ ēḿōţĩōńàĺ śţēēŕĩńĝ:
Models WITH Emotional Control:
- źŷƥĥŕà źōńōś-v0.1 [Ďēƥŕēćàţēď][3]: Ũśēś 8-ďĩḿēńśĩōńàĺ ēḿōţĩōń ŵēĩĝĥţś ĩńćĺũďĩńĝ ĥàƥƥĩńēśś, śàďńēśś, àńĝēŕ, ƒēàŕ, śũŕƥŕĩśē, ďĩśĝũśţ, ńēũţŕàĺ, àńď ōţĥēŕ. Ţĥē ÀĨ àńàĺŷźēś śţōŕŷ ćōńţēxţ ţō àũţōḿàţĩćàĺĺŷ śēţ àƥƥŕōƥŕĩàţē ēḿōţĩōń ĺēvēĺś ƒōŕ ēàćĥ ďĩàĺōĝũē ĺĩńē. Ōńē śĥōţ vōĩćē ćĺōńĩńĝ.
- ĝƥţ-4ō-ḿĩńĩ-ţţś[1]: Ēḿƥĺōŷś ćōńţēxţũàĺ ĩńśţŕũćţĩōńś ƀàśēď ōń śţōŕŷ àńàĺŷśĩś. Ţĥē śŷśţēḿ ēxàḿĩńēś śũŕŕōũńďĩńĝ ďĩàĺōĝũē àńď ńàŕŕàţĩvē ţō ĝēńēŕàţē ńàţũŕàĺ ĺàńĝũàĝē ĩńśţŕũćţĩōńś ţĥàţ ĝũĩďē ēḿōţĩōńàĺ ďēĺĩvēŕŷ. Ţŕàĩńĩńĝ ƀàśēď vōĩćē ćĺōńĩńĝ.
Models WITHOUT Emotional Control:
- ţţś-1+ŕvć[1][4]: Ţŕàĩńĩńĝ ƀàśēď vōĩćē ćĺōńĩńĝ
- ƒĩśĥàũďĩō[2]: Ōńē śĥōţ vōĩćē ćĺōńĩńĝ
- śƥàŕķţţś [Ďēƥŕēćàţēď][8]: Ōńē śĥōţ vōĩćē ćĺōńĩńĝ. Ŵĥĩĺē ţĥē ũńďēŕĺŷĩńĝ ţēćĥńōĺōĝŷ śũƥƥōŕţś 24 ēḿōţĩōń ćàţēĝōŕĩēś, ĹŷŕĩćŴĩńţēŕ'ś ćũŕŕēńţ ĩḿƥĺēḿēńţàţĩōń ďōēśń'ţ ũśē ćōńţēxţũàĺ ēḿōţĩōń àńàĺŷśĩś.
Ţĥàţ'ś ńōţ ţō śàŷ ţţś-1 ōŕ ƒĩśĥàũďĩō àŕē ƀàď ƀŷ àńŷ ḿēàńś ƀēćàũśē ţĥēŷ ďōń'ţ ĝēţ ćōńţēxţ-ďēƥēńďēńţ ēḿōţĩōńàĺ śţēēŕĩńĝ, ţĥēŷ àŕē ĝŕēàţ. Ţĥēŕē àŕē ţŕàďēōƒƒś ţō ƀē ḿàďē. Ĩ ḿŷśēĺƒ ƥŕēƒēŕ ţţś-1 ƒōŕ ţĥē Ńàŕŕàţōŕ ƒōŕ ţĥē ćàďēńćē ćōńśĩśţēńćŷ.
Ēŕŕōŕ Ĥàńďĺĩńĝ
Ŵĥàţ ďō ŷōũ ďō ŵĥēń ţĥē vōĩćē ḿōďēĺ ḿēśśēś ũƥ? Źŷƥĥŕà źōńōś-v0.1 [Ďēƥŕēćàţēď] ƒōŕ ēxàḿƥĺē ōƒţēń ćũţś ōũţ śŷĺĺàƀĺēś ńēàŕ ţĥē śţàŕţ ōŕ ēńď ōƒ ţĥē ĺĩńē. Ŵē ũśē ŵĥĩśƥēŕ[5] ţō ţŕàńśćŕĩƀē ţĥē àũďĩō àńď ćĥēćķ ĩƒ ţĥē ĝēńēŕàţĩōń ŵàś śũććēśśƒũĺ. Ĩƒ ńōţ, ŵē ţŕŷ àĝàĩń à ƒēŵ ţĩḿēś. Ĩƒ ĩţ ćōńţĩńũēś ţō ƒàĩĺ, ŵē ƒàĺĺ ƀàćķ ţō à ďĩƒƒēŕēńţ vōĩćē ḿōďēĺ. Ţĥàţ'ś ţĥē ćōŕē -- ţŕŷ àĝàĩń, àńď ƒàĺĺƀàćķ ĩƒ ńēēďēď.
Ŕēţŕĩēś śĺōŵ ţĥĩńĝś ďōŵń, ƀũţ ũĺţĩḿàţēĺŷ ŵē ŵàńţ ĥĩĝĥ ǫũàĺĩţŷ ŕēśũĺţś. Ũśũàĺĺŷ ĩţ'ś ńōţ ţōō ƀàď. Ţĥōũĝĥ źōńōś [Ďēƥŕēćàţēď] ĩś ǫũĩţē ũńŕēĺĩàƀĺē. Ĩ àćţũàĺĺŷ ĩńţēŕvĩēŵēď ŵĩţĥ ţĥēḿ -- śĥōũţōũţ ţō ţĥēḿ! Ţĥēŷ ƀũĩĺţ à ĝōōď ƥŕōďũćţ, ţĥōũĝĥ ţĥēŷ ţōţàĺĺŷ śĥōũĺď ĥàvē ĥĩŕēď ḿē ţō ḿàķē ţĥĩńĝś ŕēĺĩàƀĺē.
Àńď ōƒ ćōũŕśē, ēŕŕōŕ ţŕàćķĩńĝ! Ēŕŕōŕś àŕē ĺōĝĝēď ƒōŕ ŕēvĩēŵ.
Śƥēēď
Śōḿē ōƒ ţĥēśē ḿōďēĺś ŕũń ōń ēxţēŕńàĺ śēŕvĩćēś, śōḿē ŕũń ōń śēŕvēŕĺēśś ĩńƒŕàśţŕũćţũŕē. Ńōńēţĥēĺēśś ŕēǫũēśţś àŕē ďĩśţŕĩƀũţēď ƒōŕ śƥēēď! Śĥōũţōũţ ţō Ḿōďàĺ[6]. Ōŕĩĝĩńàĺĺŷ Ĩ ũśēď Ŕũńƥōď àńď Ĝōōĝĺē Ćĺōũď Ŕũń, ƀũţ --- ţĥēŷ ŵēŕē ţōō śĺōŵ. + Ḿōďàĺ ĝàvē ḿē à ƀũńćĥ ōƒ ƒŕēē ćŕēďĩţś. Ĩ ĺōvē ţĥēḿ. Ŵĥēń ēàćĥ ŕēǫũēśţ ćōḿƥĺēţēś, ţĥē ćĺĩēńţ (ŷōũŕ ƀŕōŵśēŕ) ƥũĺĺś ĩń ţĥē àũďĩō àńď ćōńćàţēńàţēś ĩţ. Àś śōōń àś ţĥē ƒĩŕśţ ďĩàĺōĝũē ćĺĩƥ ĩś ćōḿƥĺēţē, ŷōũ ćàń śţàŕţ ĺĩśţēńĩńĝ!
Ţĩḿē ţō ĺĩśţēń!
Ŷōũŕ àũďĩō ĩś ŕēàďŷ ţō ĺĩśţēń ţō, àńď ŷōũ ćàń ōƥţĩōńàĺĺŷ ďōŵńĺōàď àś à .ŵàv ōŕ ďōŵńĺōàď ţĥē .śŕţ ƒĩĺē àś ďēśĩŕēď.
Ţŕŷ Ĩţ Ŷōũŕśēĺƒ!
Ţĥē ƀēśţ ŵàŷ ţō ũńďēŕśţàńď ĹŷŕĩćŴĩńţēŕ ĩś ţō ēxƥēŕĩēńćē ĩţ. Ĥēàď ōvēŕ ţō ţĥē Ĝēńēŕàţē ƥàĝē, ƥàśţē ĩń ŷōũŕ ƒàvōŕĩţē ƒàńƒĩćţĩōń, ĺĩĝĥţ ńōvēĺ, ōŕ śĥōŕţ śţōŕŷ, àńď ŵàţćĥ àś ÀĨ ƀŕĩńĝś ŷōũŕ ƒàvōŕĩţē ćĥàŕàćţēŕś ĩńţō ţĥē àũďĩō ďōḿàĩń ŵĩţĥ ĩḿḿēŕśĩōń. Ŵē ĥàvē à ĝēńēŕōũś Ēńĝĺĩśĥ-śũƥƥōŕţēď ƒŕēē ţĩēŕ ŵĩţĥ ŚƥàŕķŢŢŚ [Ďēƥŕēćàţēď], śō ŷōũ ćàń ēxƥēŕĩḿēńţ ŵĩţĥōũţ àńŷ ćōḿḿĩţḿēńţ.
Ŵĥēţĥēŕ ŷōũ'ŕē à ƒàńƒĩćţĩōń ŵŕĩţēŕ ŵàńţĩńĝ ţō ĥēàŕ ŷōũŕ śţōŕĩēś àĺōũď, à ĺĩĝĥţ ńōvēĺ ŕēàďēŕ ĺōōķĩńĝ ƒōŕ ńàŕŕàţĩōń ţōōĺś, ōŕ Ĵũśţ ćũŕĩōũś àƀōũţ ÀĨ vōĩćē ţēćĥńōĺōĝŷ, ĹŷŕĩćŴĩńţēŕ ōƒƒēŕś śōḿēţĥĩńĝ ũńĩǫũē: ţĥē àƀĩĺĩţŷ ţō ţŕàńśƒōŕḿ àńŷ ţēxţ ĩńţō àń ĩḿḿēŕśĩvē àũďĩō ēxƥēŕĩēńćē ŵĩţĥ ďĩśţĩńćţ ćĥàŕàćţēŕ vōĩćēś, àĺĺ ĩń Ĵũśţ à ƒēŵ ćĺĩćķś.
Ĥàƥƥŷ śţōŕŷ ţēĺĺĩńĝ! Àńď ĩƒ ŷōũ ĥàvē àńŷ ǫũēśţĩōńś ōŕ ƒēēďƀàćķ, ƒēēĺ ƒŕēē ţō ŕēàćĥ ōũţ. Ĩ'ḿ àĺŵàŷś ēxćĩţēď ţō ĥēàŕ ĥōŵ ƥēōƥĺē àŕē ũśĩńĝ ĹŷŕĩćŴĩńţēŕ àńď ŵĥàţ ƒēàţũŕēś ţĥēŷ'ď ĺĩķē ţō śēē ńēxţ.
Ńōţē
Ĩḿƥēŕśōńàţĩōń ōŕ ńēƒàŕĩōũś ũśē ĩś ńōţ àĺĺōŵēď! Ţĥĩś ĩś ƒōŕ ŕēćŕēàţĩōń àńď ƥēŕśōńàĺ ũśē. ĹŷŕĩćŴĩńţēŕ ĩś ńōţ àƒƒĩĺĩàţēď ŵĩţĥ, ńōŕ ďōēś ĩţ ćĺàĩḿ ōŵńēŕśĥĩƥ ōƒ, àńŷ ũśēŕ-ũƥĺōàďēď vōĩćēś. Àś ĥĩĝĥĺĩĝĥţēď ĩń ţĥĩś àŕţĩćĺē, ĹŷŕĩćŴĩńţēŕ ĩś ĩńƥũţ àńď vōĩćē àĝńōśţĩć śŷśţēḿ. Ēxàḿƥĺēś ĝĩvēń àŕē ƒōŕ ĩĺĺũśţŕàţĩōń ƥũŕƥōśēś ōńĺŷ.
References
- [1]OpenAI Platform - Text-to-Speech Guide
- [2]Fish Audio - Text-to-Speech Platform
- [3]Zyphra - Zonos v0.1
- [4]RVC-Project - Retrieval-based Voice Conversion WebUI
- [5]OpenAI Whisper - Automatic Speech Recognition
- [6]Modal - Serverless GPU Computing Platform
- [7]Groq - Fast AI Inference Platform
- [8]SparkAudio - Spark TTS