# EmotiVoice | A Multi-Voice And Prompt-Controlled TTS Engine

**URL:** <https://onehack.st/t/emotivoice-a-multi-voice-and-prompt-controlled-tts-engine/291238>\
**Category:** Tools & Scripts\
**Tags:** programming, freebies, audio, ai\
**Created:** [September 12, 2024, 9:22pm UTC](https://onehack.st/t/emotivoice-a-multi-voice-and-prompt-controlled-tts-engine/291238 "2024-09-12T21:22:31Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Prometheus](https://onehack.st/user_avatar/onehack.st/prometheus/32/34950_2.png) [@Prometheus](https://onehack.st/u/Prometheus)\
**Post date:** [September 12, 2024, 9:22pm UTC](https://onehack.st/t/emotivoice-a-multi-voice-and-prompt-controlled-tts-engine/291238/1 "2024-09-12T21:22:31Z")

</div>

# EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine

[![](https://camo.githubusercontent.com/3eb53d2b84bd52b8f07490c7d30eff7d3e0ca56075215f2b2e9f6b2bf5911ddb/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f524541444d452de4b8ade69687e78988e69cac2d726564)](https://github.com/netease-youdao/EmotiVoice/blob/main/README.zh.md) [![](https://camo.githubusercontent.com/d5cda8d6e45fef7ffc4d1e7f1cebab0984063d759064ea2ec7c58c4bf991d957/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4170616368652d2d322e302d79656c6c6f77)](https://github.com/netease-youdao/EmotiVoice/blob/main/LICENSE) [![](https://camo.githubusercontent.com/faa222165d59019efb10db6e446b64af88e8a156318a07439ea4ce3e8f0a007d/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f666f6c6c6f772d25343059444f70656e536f757263652d3144413146323f6c6f676f3d74776974746572267374796c653d7b7374796c657d)](https://twitter.com/YDopensource)

**EmotiVoice** is a powerful and modern open-source text-to-speech engine that is available to you at no cost. EmotiVoice speaks both English and Chinese, and with over 2000 different voices (refer to the [List of Voices](https://github.com/netease-youdao/EmotiVoice/wiki/%F0%9F%98%8A-voice-wiki-page) for details). The most prominent feature is **emotional synthesis** , allowing you to create speech with a wide range of emotions, including happy, excited, sad, angry and others.

An easy-to-use web interface is provided. There is also a scripting interface for batch generation of results.

Here are a few samples that EmotiVoice generates:

- emotivoice\_intro\_cn\_im.1.mp4
- emotivoice\_intro\_en\_im.1.mp4
- emotivoice\_intro\_en\_fun\_im.1.mp4

## Demo

A demo is hosted on Replicate, [EmotiVoice](https://replicate.com/bramhooimeijer/emotivoice).

## Hot News

- Tuning voice speed is now supported in ‘OpenAI-compatible-TTS API’, thanks to [@john9405](https://github.com/john9405). [#90](https://github.com/netease-youdao/EmotiVoice/pull/90) [#67](https://github.com/netease-youdao/EmotiVoice/issues/67) [#77](https://github.com/netease-youdao/EmotiVoice/issues/77)
- [The EmotiVoice app for Mac](https://github.com/netease-youdao/EmotiVoice/releases/download/v0.3/emotivoice-1.0.0-arm64.dmg) was released on December 28th, 2023. Just download and taste EmotiVoice’s offerings!
- [The EmotiVoice HTTP API](https://github.com/netease-youdao/EmotiVoice/wiki/HTTP-API) was released on December 6th, 2023. Easier to start, faster to use, and with **over 13,000 free calls**. Additionally, users can explore more captivating voices provided by [Zhiyun](https://ai.youdao.com/).
- [Voice Cloning with your personal data](https://github.com/netease-youdao/EmotiVoice/wiki/Voice-Cloning-with-your-personal-data) has been released on December 13th, 2023, along with [DataBaker Recipe](https://github.com/netease-youdao/EmotiVoice/tree/main/data/DataBaker) and [LJSpeech Recipe](https://github.com/netease-youdao/EmotiVoice/tree/main/data/LJspeech).

## Features under development

- Support for more languages, such as Japanese and Korean. [#19](https://github.com/netease-youdao/EmotiVoice/issues/19) [#22](https://github.com/netease-youdao/EmotiVoice/issues/22)

EmotiVoice prioritizes community input and user requests. We welcome your feedback!

## Quickstart

### EmotiVoice Docker image

The easiest way to try EmotiVoice is by running the docker image. You need a machine with a NVidia GPU. If you have not done so, set up NVidia container toolkit by following the instructions for [Linux](https://www.server-world.info/en/note?os=Ubuntu_22.04&p=nvidia&f=2) or [Windows WSL2](https://github.com/nyp-sit/it3103/blob/main/nvidia-docker-wsl2.md). Then EmotiVoice can be run with,

docker run -dp 127.0.0.1:8501:8501 syq163/emoti-voice:latest

The Docker image was updated on January 4th, 2024. If you have an older version, please update it by running the following commands:

docker pull syq163/emoti-voice:latest docker run -dp 127.0.0.1:8501:8501 -p 127.0.0.1:8000:8000 syq163/emoti-voice:latest

Now open your browser and navigate to [http://localhost:8501](http://localhost:8501/) to start using EmotiVoice’s powerful TTS capabilities.

Starting from this version, the ‘OpenAI-compatible-TTS API’ is now accessible via [http://localhost:8000/](http://localhost:8000/).

### Full installation

conda create -n EmotiVoice python=3.8 -y conda activate EmotiVoice pip install torch torchaudio pip install numpy numba scipy transformers soundfile yacs g2p\_en jieba pypinyin pypinyin\_dict python -m nltk.downloader “averaged\_perceptron\_tagger\_eng”

### Prepare model files

We recommend that users refer to the wiki page [How to download the pretrained model files](https://github.com/netease-youdao/EmotiVoice/wiki/Pretrained-models) if they encounter any issues.

git lfs install git lfs clone [https://huggingface.co/WangZeJun/simbert-base-chinese](https://huggingface.co/WangZeJun/simbert-base-chinese) WangZeJun/simbert-base-chinese

or, you can run:

git clone [https://www.modelscope.cn/syq163/WangZeJun.git](https://www.modelscope.cn/syq163/WangZeJun.git)

### Inference

1. You can download the [pretrained models](https://drive.google.com/drive/folders/1y6Xwj_GG9ulsAonca_unSGbJ4lxbNymM?usp=sharing) by simply running the following command:

git clone [https://www.modelscope.cn/syq163/outputs.git](https://www.modelscope.cn/syq163/outputs.git)

1. The inference text format is `<speaker>|<style_prompt/emotion_prompt/content>|<phoneme>|<content>`.

- inference text example: `8051|Happy|<sos/eos> [IH0] [M] [AA1] [T] engsp4 [V] [OY1] [S] engsp4 [AH0] engsp1 [M] [AH1] [L] [T] [IY0] engsp4 [V] [OY1] [S] engsp1 [AE1] [N] [D] engsp1 [P] [R] [AA1] [M] [P] [T] engsp4 [K] [AH0] [N] [T] [R] [OW1] [L] [D] engsp1 [T] [IY1] engsp4 [T] [IY1] engsp4 [EH1] [S] engsp1 [EH1] [N] [JH] [AH0] [N] . <sos/eos>|Emoti-Voice - a Multi-Voice and Prompt-Controlled T-T-S Engine`.

1. You can get phonemes by `python frontend.py data/my_text.txt > data/my_text_for_tts.txt`.
2. Then run:

TEXT=data/inference/text python inference\_am\_vocoder\_joint.py \ –logdir prompt\_tts\_open\_source\_joint \ –config\_folder config/joint \ –checkpoint g\_00140000 \ –test\_file $TEXT

the synthesized speech is under `outputs/prompt_tts_open_source_joint/test_audio`.

1. Or if you just want to use the interactive TTS demo page, run:

pip install streamlit streamlit run demo\_page.py

### OpenAI-compatible-TTS API

Thanks to @lewangdev for adding an OpenAI compatible API [#60](https://github.com/netease-youdao/EmotiVoice/issues/60). To set it up, use the following command:

pip install fastapi pydub uvicorn[standard] pyrubberband uvicorn openaiapi:app --reload

### Wiki page

You may find more information from our [wiki](https://github.com/netease-youdao/EmotiVoice/wiki) page.

## Training

[Voice Cloning with your personal data](https://github.com/netease-youdao/EmotiVoice/wiki/Voice-Cloning-with-your-personal-data) has been released on December 13th, 2023.

## Roadmap & Future work

- Our future plan can be found in the [ROADMAP](https://github.com/netease-youdao/EmotiVoice/blob/main/ROADMAP.md) file.
- The current implementation focuses on emotion/style control by prompts. It uses only pitch, speed, energy, and emotion as style factors, and does not use gender. But it is not complicated to change it to style/timbre control.
- Suggestions are welcome. You can file issues or [@ydopensource](https://twitter.com/YDopensource) on twitter.

## WeChat group

Welcome to scan the QR code below and join the WeChat group.

[![qr](https://private-user-images.githubusercontent.com/49354974/283381522-cc3f4c8b-8369-4e50-89cc-e40d27a6bdeb.jpg?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNzYzMDksIm5iZiI6MTcyNjE3NjAwOSwicGF0aCI6Ii80OTM1NDk3NC8yODMzODE1MjItY2MzZjRjOGItODM2OS00ZTUwLTg5Y2MtZTQwZDI3YTZiZGViLmpwZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MTIlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTEyVDIxMjAwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVkOWM5OThmNWE5MTY2Njg3NjY2ZTc5OTQ2NmUyM2QwN2JhYzg0YzFkZTk4OGE4ZGViYWQ5OGMxODRjNTRmNmQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.5t0J8yRSVlF2JESD0K8huK6obwN21s5Ge44zEhlhJsU)](https://private-user-images.githubusercontent.com/49354974/283381522-cc3f4c8b-8369-4e50-89cc-e40d27a6bdeb.jpg?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjYxNzYzMDksIm5iZiI6MTcyNjE3NjAwOSwicGF0aCI6Ii80OTM1NDk3NC8yODMzODE1MjItY2MzZjRjOGItODM2OS00ZTUwLTg5Y2MtZTQwZDI3YTZiZGViLmpwZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA5MTIlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwOTEyVDIxMjAwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVkOWM5OThmNWE5MTY2Njg3NjY2ZTc5OTQ2NmUyM2QwN2JhYzg0YzFkZTk4OGE4ZGViYWQ5OGMxODRjNTRmNmQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.5t0J8yRSVlF2JESD0K8huK6obwN21s5Ge44zEhlhJsU)

## Credits

- [PromptTTS](https://speechresearch.github.io/prompttts/). The PromptTTS paper is a key basis of this project.
- [LibriTTS](https://www.openslr.org/60/). The LibriTTS dataset is used in training of EmotiVoice.
- [HiFiTTS](https://www.openslr.org/109/). The HiFi TTS dataset is used in training of EmotiVoice.
- [ESPnet](https://github.com/espnet/espnet).
- [WeTTS](https://github.com/wenet-e2e/wetts)
- [HiFi-GAN](https://github.com/jik876/hifi-gan)
- [Transformers](https://github.com/huggingface/transformers)
- [tacotron](https://github.com/keithito/tacotron)
- [KAN-TTS](https://github.com/alibaba-damo-academy/KAN-TTS)
- [StyleTTS](https://github.com/yl4579/StyleTTS)
- [Simbert](https://github.com/ZhuiyiTechnology/simbert)
- [cn2an](https://github.com/Ailln/cn2an). EmotiVoice incorporates cn2an for number processing.

## License

EmotiVoice is provided under the Apache-2.0 License - see the [LICENSE](https://github.com/netease-youdao/EmotiVoice/blob/main/LICENSE) file for details.

The interactive page is provided under the [User Agreement](https://github.com/netease-youdao/EmotiVoice/blob/main/EmotiVoice_UserAgreement_%E6%98%93%E9%AD%94%E5%A3%B0%E7%94%A8%E6%88%B7%E5%8D%8F%E8%AE%AE.pdf) file.

## GitHub:

> **[GitHub - netease-youdao/EmotiVoice: EmotiVoice 😊: a Multi-Voice and Prompt-Controlled...](https://github.com/netease-youdao/EmotiVoice)**
>
> EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine