NeuralCompanion is a Windows desktop application built on Python 3.11 and a native Qt Quick/QML interface. The main runtime is required. MuseTalk, Ditto, and PocketTTS are optional isolated runtimes for local talking avatars and local speech.
Installation only puts components on disk. It does not choose a chat model, activate a speech provider, or select an avatar. Those are separate steps inside the application; see First run.
Before you begin
- Windows and Python 3.11. The installer checks for Python 3.11 exactly and can rebuild an isolated environment that uses another version. Do not point it at Python 3.12 or newer.
- A chat provider for your first reply. LM Studio or Ollama needs its server running and a model available. A hosted provider needs its own API credentials and available credit.
- Disk space and time. A text-only setup is much smaller than local chat, speech, and an animated avatar together. MuseTalk models, Ditto models, and prepared avatar packs are substantial downloads.
- An NVIDIA CUDA GPU for local talking avatars. MuseTalk requires a CUDA setup. Ditto uses its own environment and a CUDA 12.8 stack for standard RTX and Blackwell cards. Neither is needed for text chat or ordinary speech.
- Media tools if you plan to use media workflows. Audio Story Mode and some media utilities need FFmpeg and ffprobe. The installer can place bundled tools under
tools/ffmpeg/binwhen they are missing fromPATH.
Obtain the project from the NeuralCompanion repository, or clone it:
git clone https://github.com/Rakile/NeuralCompanion.git
cd NeuralCompanion
Choose components
| Install target | What it adds | Notes |
|---|---|---|
| Main runtime | The application, native QML interface, core providers, local speech stack, and Stockfish chess engine | Required. |
| MuseTalk runtime | Local MuseTalk talking-head engine | Separate environment, NVIDIA CUDA, model weights, and a prepared MuseTalk pack. |
| Ditto runtime | Local Ditto talking-head engine | Separate environment, models, and prepared Ditto packages. Ditto packages are not MuseTalk packs. |
| PocketTTS runtime | Local PocketTTS and PocketTTS Multilingual speech | Separate environment. Reference cloning may need Hugging Face terms and a login. |
| Default avatar packs | The prepared Echo and Eon MuseTalk packs | Downloaded separately from the runtimes. They are not Ditto packages and do not install MuseTalk for you. |
PocketTTS uses the verified local wheel shipped under vendor/pockettts/ rather
than a moving GitHub download. Its source, version, license and checksum are
recorded there. Normal dependency installation and model access remain separate;
this does not make the full installation offline. A damaged/missing wheel bundle
stops before PocketTTS environment changes and must be restored from the same NC
release. An existing package with the same version is refreshed from the bundled
wheel without forcing its dependencies to reinstall.
The main runtime already supports Chatterbox and Whisper-based local speech. Add the isolated runtimes only for the providers you intend to use.
The Windows main-runtime installation also downloads and verifies official Stockfish 19 for the machine’s x64 or ARM64 architecture. It installs the universal executable at tools/stockfish/stockfish.exe inside the NC folder, together with upstream license and source files. This tool download needs internet during installation; a verified existing bundle is reused. It does not use a fixed drive letter or require a separate Python environment.
Companion Chess uses bundled Stockfish by default for new setups. A valid custom engine path is kept; an explicitly empty path still chooses the basic engine. An old missing I:/chess/engines/stockfish.exe shortcut selects the bundled copy instead. Restart NC and the local chess server after updating, then start a new game to apply a changed engine choice. To repair a missing or damaged bundled copy, rerun the main-runtime installer.
Recommended: graphical installer
From the project folder, open INSTALL_NEURAL_COMPANION.bat. From a terminal, the equivalent is:
py -3.11 install_neural_companion_gui.py
- Check the Python card. It shows the detected Python 3.11 interpreter. Use Detect or Browse if needed. Get Python 3.11 opens the Windows download page. Correct the interpreter before installing anything.
- Review Install targets. The current installer preselects the main runtime, MuseTalk, PocketTTS, Echo, and Eon, with Ditto off. Accept or change those choices deliberately; the presets are a starting point, not a recommendation for every computer.
- Choose Default avatar packs separately. Echo and Eon are prepared MuseTalk packs. Installing an avatar engine and installing prepared packs are different operations.
- Run preflight. Choose RUN PREFLIGHT ONLY and read every warning before installing. It can distinguish a missing component from a service that is merely offline.
- Install. Choose INSTALL SELECTED and wait for each selected component. If one component fails, keep the final summary and repair that component before reinstalling everything.
- Handle the expected follow-ups. PocketTTS can offer a Hugging Face login after installation. For RTX 50 / Blackwell cards, check the PyTorch stack selection. The installer can auto-detect the GPU, keep the default CUDA stack, or Force CUDA 12.8 / RTX 50.
The installer may offer FFmpeg tools, PocketTTS model-access links, and a final command summary. It does not start LM Studio, Ollama, VoiceStudio, VSeeFace, ComfyUI, or any hosted account.
Command-line installation
Run commands from the project folder. A minimal main-runtime installation is:
py -3.11 install_neural_companion.py --main --non-interactive
Install optional runtimes individually:
py -3.11 install_neural_companion.py --musetalk --non-interactive
py -3.11 install_neural_companion.py --ditto --non-interactive
py -3.11 install_neural_companion.py --pockettts --non-interactive
--all installs the main app, MuseTalk, Ditto, and PocketTTS. It does not download the default avatar packs:
py -3.11 install_neural_companion.py --all --non-interactive
py -3.11 install_neural_companion.py --avatar-packs --non-interactive
--avatar-packs selects both default packs. --avatar-pack-echo and --avatar-pack-eon select them individually. Combine --all --avatar-packs when you want the four runtimes and both packs. Install the MuseTalk runtime as well if you intend to use those packs.
If interpreter detection is wrong, provide Python 3.11 explicitly:
py -3.11 install_neural_companion.py --python-exe "C:\Path\To\Python311\python.exe" --main --non-interactive
To inspect setup without installing components:
py -3.11 install_neural_companion.py --doctor-only
For the main and/or MuseTalk RTX 50 path, add --torch-stack cu128 to the install command when automatic selection misses the card:
py -3.11 install_neural_companion.py --main --musetalk --torch-stack cu128 --non-interactive
Prefer the installer to combining every requirements file in one environment. requirements.txt is a small baseline, while requirements.companion.txt and the separate runtime requirements serve different environments.
Launch and verify
Open run_neural_companion.bat. It uses the project’s installed environment. In a Git checkout, the launcher may offer to update the project; read that prompt before accepting an update.
For a direct launch from the project folder:
.\.venv\Scripts\python.exe qt_app.py
Opening the window does not load models. Follow First run to configure an active language provider, choose Text Only, select None for both speech output and avatar, then use the header power control, Initialize NC. A successful text reply verifies the basic path before you add speech or animation.
The important distinction is:
- Installing the main runtime does not make a chat provider active.
- Opening a provider’s settings does not activate it.
- Installing MuseTalk does not select a pack or make MuseTalk the active avatar.
- Installing a voice runtime does not assign a voice to a persona.
Voices, avatars, and personal data
Voice references belong in voices/; prepared MuseTalk packs belong in avatar_packs/<pack_id>/ and include a manifest.json. NC includes a limited, explicitly approved set of voice samples and persona assets. Most additional voice samples, model weights, prepared avatar packs, and generated outputs are not shipped in the source repository.
MuseTalk setup checks the size and SHA256 of its eight weights and five required JSON configurations against musetalk-model-manifest.json. The inventory pins the revisions already used by the existing runtime. Setup reads existing files to verify them, downloads only missing or invalid files into staging, and verifies every staged repair before replacing any destination. Download or verification failure preserves the existing files. Publication is atomic per file; a later filesystem error can leave some files repaired and others unchanged, and rerunning setup verifies and completes the remaining repairs.
See the release asset policy for the exact bundled exceptions and the avatar-pack guide for pack layout. Use assets you have permission to use.
Do not delete voices/, avatar_packs/, runtime/, or MuseTalk/runtime/ as a general cleanup step. They can contain your recordings, packs, personas, and conversation state. Cleaning generated caches is different from deleting a pack or a saved chat.
Next steps
- First run for a text-only verification.
- Chat and speech for speech output and microphone input.
- PocketTTS for its isolated runtime and cloning access.
- Troubleshooting if a component is missing or initialization fails.
