Abort when python demo_toolbox.py

[Interfish]

Hi:
I am trying to run your code on a centos server with X11 forwarding open. But when I try python demo_toolbox dataset , it prints

Arguments:
    datasets_root:    dataset
    enc_models_dir:   encoder/saved_models
    syn_models_dir:   synthesizer/saved_models
    voc_models_dir:   vocoder/saved_models
Aborted

I believe I installed all required packages. Looks like the error is not caused by python but some low level call. So is there any way to print more error message? Or is there any way to run without GUI ? (I think although I open X11 forward on this server but it still might not fit as good as a pure GUI machine).

Thanks!

[CorentinJ]

I had planned to make a GUI-less demo inference script for a while, but really its only purpose would be to show you how to interface the models with your code. Essentially it comes down to making calls to the three <model>/inference.py scripts I wrote for each model. You can read their current documentation and refer to how they're used in the toolbox to implement it yourself.

Either way I'll be writing that script in two days I think (currently have exams to pass). I'm still thinking about how I want to do it.

As for the GUI problem, I'm sorry but I just don't know enough on that topic to help you. Your guess is a good as mine.

[CorentinJ]

Hey @Interfish, I've added this stub to allow for quick debugging without a GUI. Can you test and see if it runs? I'll implement an interactive way of doing inference later on.

[DrStoop]

Hi @CorentinJ, I am facing exactly the same problem as @Interfish on my local machine. and I tested your demo_cli.py. The test passes though, see attachment below.

I tracked the error with faulthandler down to class UI(QDialog).__init__():

Connected to pydev debugger (build 191.7479.30)
Arguments:
    datasets_root:    /home/developer/data/datasets
    enc_models_dir:   data/models/encoder/saved_models
    syn_models_dir:   data/models/synthesizer/saved_models
    voc_models_dir:   data/models/vocoder/saved_models

Fatal Python error: Aborted

Thread 0x00007ff44b7c9700 (most recent call first):
  File "/conda/envs/rapids/lib/python3.6/threading.py", line 299 in wait
  File "/conda/envs/rapids/lib/python3.6/threading.py", line 551 in wait
  File "/opt/pycharm-2019.1.3/helpers/pydev/pydevd.py", line 128 in _on_run
  File "/opt/pycharm-2019.1.3/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 321 in run
  File "/conda/envs/rapids/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/conda/envs/rapids/lib/python3.6/threading.py", line 884 in _bootstrap

Thread 0x00007ff44bfca700 (most recent call first):
  File "/opt/pycharm-2019.1.3/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 383 in _on_run
  File "/opt/pycharm-2019.1.3/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 321 in run
  File "/conda/envs/rapids/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/conda/envs/rapids/lib/python3.6/threading.py", line 884 in _bootstrap

Thread 0x00007ff44c7cb700 (most recent call first):
  File "/conda/envs/rapids/lib/python3.6/threading.py", line 299 in wait
  File "/conda/envs/rapids/lib/python3.6/queue.py", line 173 in get
  File "/opt/pycharm-2019.1.3/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 460 in _on_run
  File "/opt/pycharm-2019.1.3/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 321 in run
  File "/conda/envs/rapids/lib/python3.6/threading.py", line 916 in _bootstrap_inner
  File "/conda/envs/rapids/lib/python3.6/threading.py", line 884 in _bootstrap

Current thread 0x00007ff460371740 (most recent call first):
  File "/home/developer/toolbox/ui.py", line 344 in __init__
  File "/home/developer/toolbox/__init__.py", line 38 in __init__
  File "/home/developer/demo_toolbox.py", line 31 in <module>
  File "/opt/pycharm-2019.1.3/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18 in execfile
  File "/opt/pycharm-2019.1.3/helpers/pydev/pydevd.py", line 1147 in run
  File "/opt/pycharm-2019.1.3/helpers/pydev/pydevd.py", line 1752 in main
  File "/opt/pycharm-2019.1.3/helpers/pydev/pydevd.py", line 1758 in <module>

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

The error occurs in File "/toolbox/ui.py", line 344 in __init__ due to QDialog or QApplication initialization:

class UI(QDialog):
...
...
...
    def __init__(self):
        ## Initialize the application
        self.app = QApplication(sys.argv)
        super().__init__(None)
        self.setWindowTitle("SV2TTS toolbox")

I don't know PyQt5 so I couldn't debug it so far. Any ideas? Still working on it...

Cheers!

Attachment:

This is a UI-less example of interface to SV2TTS. The purpose of this script is to show how you can interface this project easily with your own. See the source code for an explanation of what is happening.

Arguments:
    enc_model_fpath:   data/models/encoder/saved_models/pretrained.pt
    syn_model_dir:     data/models/synthesizer/saved_models/logs-pretrained
    voc_model_fpath:   data/models/vocoder/saved_models/pretrained/pretrained.pt
    no_sound:          False

Found 1 GPUs available. Using GPU 0 (GeForce GTX 1080) of compute capability 6.1 with 8.5Gb total memory.

Loading the encoder, the synthesizer and the vocoder. This should take a few seconds. The synthesizer will output a lot of stuff. Tensorflow is like that.
Loaded encoder "pretrained.pt" trained to step 1564501
Constructing model: Tacotron
WARNING:tensorflow:From /home/developer/synthesizer/models/tacotron.py:86: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, use
    tf.py_function, which takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    
WARNING:tensorflow:From /conda/envs/rapids/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /home/developer/synthesizer/models/modules.py:112: LSTMCell.__init__ (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.LSTMCell, and will be replaced by that in Tensorflow 2.0.
WARNING:tensorflow:From /home/developer/synthesizer/models/modules.py:421: conv1d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.conv1d instead.
WARNING:tensorflow:From /home/developer/synthesizer/models/modules.py:422: batch_normalization (from tensorflow.python.layers.normalization) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.batch_normalization instead.
WARNING:tensorflow:From /home/developer/synthesizer/models/modules.py:425: dropout (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dropout instead.
WARNING:tensorflow:From /home/developer/synthesizer/models/modules.py:236: bidirectional_dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.Bidirectional(keras.layers.RNN(cell))`, which is equivalent to this API
WARNING:tensorflow:From /conda/envs/rapids/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py:443: dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.RNN(cell)`, which is equivalent to this API
WARNING:tensorflow:From /conda/envs/rapids/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py:626: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From /home/developer/synthesizer/models/modules.py:305: MultiRNNCell.__init__ (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.StackedRNNCells, and will be replaced by that in Tensorflow 2.0.
WARNING:tensorflow:From /home/developer/synthesizer/models/modules.py:269: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dense instead.
WARNING:tensorflow:From /conda/envs/rapids/lib/python3.6/site-packages/tensorflow/python/keras/layers/core.py:143: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
initialisation done /gpu:0
Initialized Tacotron model. Dimensions (? = dynamic shape): 
  Train mode:               False
  Eval mode:                False
  GTA mode:                 False
  Synthesis mode:           True
  Input:                    (?, ?)
  device:                   0
  embedding:                (?, ?, 512)
  enc conv out:             (?, ?, 512)
  encoder out (cond):       (?, ?, 768)
  decoder out:              (?, ?, 80)
  residual out:             (?, ?, 512)
  projected residual out:   (?, ?, 80)
  mel out:                  (?, ?, 80)
  <stop_token> out:         (?, ?)
  Tacotron Parameters       28.439 Million.
Loading checkpoint: data/models/synthesizer/saved_models/logs-pretrained/taco_pretrained/tacotron_model.ckpt-278000
WARNING:tensorflow:From /conda/envs/rapids/lib/python3.6/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
Loaded synthesizer "pretrained" trained to step 278000
Building Wave-RNN
Trainable Parameters: 4.481M
Loading model weights at data/models/vocoder/saved_models/pretrained/pretrained.pt

All models succesfully loaded!

Testing your configuration with small inputs.
	Testing the encoder...
	Testing the synthesizer...
	Testing the vocoder...
All test passed! You can now synthesize speech.

Process finished with exit code 0

[DrStoop]

Enable Qt-Debug $ export QT_DEBUG_PLUGINS=1 ==> reproduce error ==> re/install the No such file or directory-library listed in debug message ==> repeat!

System

I am running my PyQt5-application in a Docker container with shared /tmp/.X11-unix/ socket and display for GUI visualization:

$ nividia-docker run --interactive --tty --env DISPLAY=$DISPLAY --volume /tmp/.X11-unix/:/tmp/.X11-unix/ <docker_iamge>

Error

Initializing PyQt5.QtWidgets.QApplication led to following error:

Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5.QtWidgets import QApplication
>>> app = QApplication([])
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.

Aborted (core dumped)

In PyCharm Debug mode the error returned:

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

Solution

General method:

• set Qt-debug environement variable in docker container terminal:

   $ export QT_DEBUG_PLUGINS=1

• reproduce error in the docker terminal (or in the IDE), e.g.:

$ python
Python 3.6.8 |Anaconda, Inc.| (default, Dec 30 2018, 01:22:34) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
KeyboardInterrupt
>>> from PyQt5.QtWidgets import QApplication, QLabel
>>> app = QApplication([])

• read debug messages printed to the terminal, e.g.:

QFactoryLoader::QFactoryLoader() checking directory path "/conda/envs/rapids/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms" ...
QFactoryLoader::QFactoryLoader() looking at "/conda/envs/rapids/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqeglfs.so"
Found metadata in lib /conda/envs/rapids/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqeglfs.so, metadata=
{
    "IID": "org.qt-project.Qt.QPA.QPlatformIntegrationFactoryInterface.5.3",
    "MetaData": {
        "Keys": [
            "eglfs"
        ]
    },
...
...
...
Got keys from plugin meta data ("xcb")
QFactoryLoader::QFactoryLoader() checking directory path "/conda/envs/rapids/bin/platforms" ...
Cannot load library /conda/envs/rapids/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so: (libxkbcommon-x11.so.0: cannot open shared object file: No such file or directory)
QLibraryPrivate::loadPlugin failed on "/conda/envs/rapids/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so" : "Cannot load library /conda/envs/rapids/lib/python3.6/site-packages/PyQt5/Qt/plugins/platforms/libqxcb.so: (libxkbcommon-x11.so.0: cannot open shared object file: No such file or directory)"
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl, xcb.

Aborted (core dumped)

• find the <No such file or directory>.so.* and <coud not be loaded>-packages, here e.g. libxkbcommon-x11.so.0 and libxcb. Then re/install the corresponding packages/libraries (finding the packages works with apt-file --package-only search <filename> or conda/pip search ...). In my case the following libs were required:

### lib no.1 ###
$ sudo conda install --name <env_name> --force-reinstall libxcb    # or pip install ...
### lib no. 2 ###
$ apt-file --package-only search libxkbcommon-x11.so.0
libxkbcommon-x11-0
$ sudo apt install libxkbcommon-x11-0 

After repeating this process for all sequentially reproduced debug messages and installing the 2 libs I can now run PyQt5-apps from inside the Docker container on my local machine desktop.

[CorentinJ]

Hey @DrStoop, sorry for the late reply, I was away for a while.

What I understand is that you managed to get the toolbox to work through X11? If so, I could link your comment in the readme for other users who might face the same issue. It looks to me that there isn't much that I can change to the code that would have helped, what do you reckon? @Interfish, does this solve your issue as well?

[DrStoop]

@CorentinJ that's correct, I did not touch your code and I am running SV2TTS-toolbox in a docker container with the GUI through shared X11 on my host desktop. Whereas I should mention that I connected the mic and speakers through shared /dev/snd with the sounddriver packages alsa-base and also-utils installed in the container (plus appending audio-group to the container user). So together with the speaker & mic I got full functionality... sure, feel free to link the comment :).

[Interfish]

@CorentinJ Sorry for late reply, i will try it now and feed back the result

[Interfish]

@CorentinJ By the way, what's the total GPU memory this code consume? I currently run on a 4gb GPU but CUDA out of memory?

[CorentinJ]

You're fine with 4gb if you don't put too long sentences. I'm currently experimenting with implementing low-memory inference to work around that but it's always a tricky thing to know in advance how much VRAM you'll need for an operation and how much is available...

[Interfish]

@CorentinJ Hi, I try your new code and it worked! Thanks for the great job, I really like your art of coding and necessary comments. I am now trying to manipulate the code to run fine-tuning. Maybe I can finally submit a version to run the full function as demo_toolbox.py does with CLI.

[MorganCZY]

I am trying to run demo_toolbox.py on a server(Ubuntu 16.04.4 LTS), but the same problem occurs to me.

I didn't get the method to fix it even after reading your discussions. Could you simply tell me what should I do or what else should be installed?

[DrStoop]

Hi @MorganCZY, have you set the environment variable in your terminal with the command export QT_DEBUG_PLUGINS=1? Your output still does not include any pyqt-debug messages which tell you what libraries couldn't be found and need installing (e.g. proposed here or here). Just set the qt-debug-flag in your terminal with the command

$ export QT_DEBUG_PLUGINS=1

and rerun the demo_toolbox.py in the same terminal. This should return a lot of debug messages as listed above and you can proceed with the solution also described there (finding missing libs, installing missing libs, rerun, repeat). Hope that helps!

[MorganCZY]

@DrStoop Following your guidance, I didn't get more debug info.

Do I correctly and fully get your point?

[MorganCZY]

I wrote a piece of simple code to create a window, but getting an error"qt.qpa.screen: QXcbConnection: Could not connect to display Could not connect to any X display."

Does it mean this server has no tools for displaying GUI codes. If so, what should I install?