undefined reference to `vmaf_cpu_cpuid

[thompon]

Hi,

I am trying to build libvmaf for Windows, using msys2.

I get this error several times:

undefined reference to `vmaf_cpu_cpuid'

This function is called in src\x86\cpu.c but it does not exist in the code (I have checked in all files).

Could someone fix this? How can you compile the source code with such a missing function?

Thanks in advance.

[1480c1]

The function comes from

vmaf/libvmaf/src/x86/cpuid.asm

Line 30 in e237326

cglobal cpu_cpuid, 0, 5, 0, regs, leaf, subleaf

Can you check to make sure you have nasm or yasm installed?

[thompon]

Thank you for your reply.

I have both nasm and yasm installed.

But for some reasons, meson does not use the compiled cpuid.obj file during linking.

Do you have any idea about how to fix that? (I am using the latest version of the source code: v2.3.0)

In case it could help you, here is the output I get:

$ meson install -C libvmaf/build
ninja: Entering directory libvmaf/build' [2/6] Linking target tools/vmafossexec.exe FAILED: tools/vmafossexec.exe c++ -o tools/vmafossexec.exe tools/vmafossexec.exe.p/main.cpp.o tools/vmafossexec.exe.p/read_frame.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -Wl,-O1 -Wl,--start-group src/libvmaf.a -pthread -lm -Wl,--subsystem,console -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x2e): undefined reference to vmaf_cpu_cpuid'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x5c): undefined reference to vmaf_cpu_cpuid' /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xbb): undefined reference to vmaf_cpu_xgetbv'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xe0): undefined reference to vmaf_cpu_cpuid' collect2: error: ld returned 1 exit status [3/6] Linking target tools/vmaf.exe FAILED: tools/vmaf.exe c++ -o tools/vmaf.exe tools/vmaf.exe.p/vmaf.c.o tools/vmaf.exe.p/cli_parse.c.o tools/vmaf.exe.p/y4m_input.c.o tools/vmaf.exe.p/vidinput.c.o tools/vmaf.exe.p/yuv_input.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -Wl,-O1 -Wl,--start-group src/libvmaf.a -pthread -lm -Wl,--subsystem,console -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x2e): undefined reference to vmaf_cpu_cpuid'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x5c): undefined reference to vmaf_cpu_cpuid' /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xbb): undefined reference to vmaf_cpu_xgetbv'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xe0): undefined reference to vmaf_cpu_cpuid' collect2: error: ld returned 1 exit status [4/6] Linking target test/test_context.exe FAILED: test/test_context.exe c++ -o test/test_context.exe test/test_context.exe.p/test.c.o test/test_context.exe.p/test_context.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -Wl,-O1 -Wl,--start-group src/libvmaf.a -pthread -lm -Wl,--subsystem,console -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x2e): undefined reference to vmaf_cpu_cpuid'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x5c): undefined reference to vmaf_cpu_cpuid' /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xbb): undefined reference to vmaf_cpu_xgetbv'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xe0): undefined reference to vmaf_cpu_cpuid' collect2: error: ld returned 1 exit status [5/6] Linking target test/test_model.exe FAILED: test/test_model.exe c++ -o test/test_model.exe test/test_model.exe.p/test.c.o test/test_model.exe.p/test_model.c.o test/test_model.exe.p/.._src_dict.c.o test/test_model.exe.p/.._src_svm.cpp.o test/test_model.exe.p/.._src_pdjson.c.o test/test_model.exe.p/.._src_read_json_model.c.o test/test_model.exe.p/.._src_log.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -Wl,-O1 -Wl,--start-group src/libvmaf.a -pthread -lm -Wl,--subsystem,console -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x2e): undefined reference to vmaf_cpu_cpuid'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x5c): undefined reference to vmaf_cpu_cpuid' /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xbb): undefined reference to vmaf_cpu_xgetbv'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xe0): undefined reference to vmaf_cpu_cpuid' collect2: error: ld returned 1 exit status [6/6] Linking target test/test_predict.exe FAILED: test/test_predict.exe c++ -o test/test_predict.exe test/test_predict.exe.p/test.c.o test/test_predict.exe.p/test_predict.c.o test/test_predict.exe.p/.._src_dict.c.o test/test_predict.exe.p/.._src_feature_feature_collector.c.o test/test_predict.exe.p/.._src_feature_alias.c.o test/test_predict.exe.p/.._src_model.c.o test/test_predict.exe.p/.._src_svm.cpp.o test/test_predict.exe.p/.._src_log.c.o test/test_predict.exe.p/.._src_read_json_model.c.o test/test_predict.exe.p/.._src_pdjson.c.o test/test_predict.exe.p/.._src_feature_feature_name.c.o test/test_predict.exe.p/.._src_feature_feature_extractor.c.o -Wl,--as-needed -Wl,--allow-shlib-undefined -Wl,-O1 -Wl,--start-group src/libvmaf.a -pthread -lm -Wl,--subsystem,console -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x2e): undefined reference to vmaf_cpu_cpuid'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0x5c): undefined reference to vmaf_cpu_cpuid' /usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xbb): undefined reference to vmaf_cpu_xgetbv'
/usr/lib/gcc/x86_64-pc-msys/10.2.0/../../../../x86_64-pc-msys/bin/ld: src/libvmaf.a(x86_cpu.c.o):cpu.c:(.text+0xe0): undefined reference to `vmaf_cpu_cpuid'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
Could not rebuild libvmaf/build

[1480c1]

Can you provide all of the steps you have taken to reproduce your issue?

[1480c1]

Perhaps you have your subsystems mixed? the x86_64-pc-msys part is implying you are compiling for msys2's MSYS base, you might be looking for the MINGW64 or one of the other *64 build environments

[thompon]

Thank you again for your help.

I have done as indicated here: https://github.com/Netflix/vmaf/blob/master/resource/doc/windows.md

It means I have done that:

pacman -S --noconfirm --needed mingw-w64-x86_64-nasm mingw-w64-x86_64-gcc mingw-w64-x86_64-meson mingw-w64-x86_64-ninja
meson setup libvmaf libvmaf/build --buildtype release --default-library static --prefix C:/vmaf-install
meson install -C libvmaf/build

In order to avoid compilation errors:

• in vmaf-master\libvmaf\meson.build,I had to replace 'c_std=c11' by 'c_std=gnu11'
• in vmaf-master\libvmaf\src\svm.cpp, I had to implement the function char* strdup (const char* s)
• in vmaf-master\libvmaf\src\feature\ciede.c, I had to define M_PI

But now I am facing linking errors

[1480c1]

Which "shell" are you opening?

The one you should use with those packages is

Also, can you post your linking errors? If they are really long, you can instead post them to https://gist.github.com/ and link the url to here, as some people have these issues subscribed to their email, and it can get pretty unreadable real quick.

For the compilation errors, I so far haven't had any, although I have had compilation warnings, and plenty of them

[thompon]

Thank you VERY MUCH because that was the cause of the problem. I don't know how I have not seen that by myself. Sorry for the inconvenience.

Now compilation works fine.

But I cannot find any libvmaf.lib file (that's the file I am looking for because I want to include it in a C++ program in Visual Studio).

There is a file libvmaf.a in C:\vmaf-install\lib but it's a small file (467 KB) whereas the libvmaf.lib I compiled several months ago from version 1.3.14 had a file size of 15714 KB so I am expecting a bigger file. I have even tried to rename it to libvmaf.lib but Visual Studio says it's an "invalid or damaged" lib file.

Do you have any idea?

[1480c1]

Oh, what you compiled was a static library, but for mingw-w64 toolchains specifically, not one that you can easily import into Visual Studio due to many reasons including incompatible C runtime library (mingw-w64 uses msvcrt while individual Visual Studios get packaged with their own dlls that you have to link against).

I personally have never tried compiling for MSVC as I don't have MSVC installed on any of my machines, but you could try the instructions mentioned in https://mesonbuild.com/Using-with-Visual-Studio.html however, I am not sure if you would have meson available inside your PATH by default, so it's possible even if you open the development console, you wouldn't be able to run meson. You could try inside the msvc development command line

rem or wherever your msys2 is installed to
set "PATH=%PATH%;C:\msys64\mingw64\bin"
cd %VMAF_REPO%
meson setup --backend vs visualstdio

and I think that should get you a visual studio solution project

[thompon]

Thanks again for your help.

Unfortunately I have an error.

I type this:

meson setup libvmaf libvmaf/build --reconfigure --backend vs --buildtype release --default-library static --prefix C:/vmaf-install

It gives me some warnings:

_Regenerating configuration from scratch: Build directory has been generated with Meson version 0.59.2, which is incompatible with the current version 0.61.0.
The Meson build system
Version: 0.61.0
Source dir: C:\VMAF230\vmaf-master\libvmaf
Build dir: C:\VMAF230\vmaf-master\libvmaf\build
Build type: native build
Project name: libvmaf
Project version: 2.3.0
C compiler for the host machine: cl (msvc 19.29.30136 "Compilateur d'optimisation Microsoft (R) C/C++ version�19.29.30136 pour x86")
C linker for the host machine: link link 14.29.30136.0
C++ compiler for the host machine: cl (msvc 19.29.30136 "Compilateur d'optimisation Microsoft (R) C/C++ version�19.29.30136 pour x86")
C++ linker for the host machine: link link 14.29.30136.0
Auto detected Visual Studio backend: vs2019
Host machine cpu family: x86
Host machine cpu: x86
Check usable header "stdatomic.h" : NO
Configuring version.h using configuration
Program nasm found: YES (C:\Program Files\Meson\nasm.EXE)
WARNING: You should add the boolean check kwarg to the run_command call.
It currently defaults to false,
but it will default to true in future releases of meson.
See also: mesonbuild/meson#9300
Configuring config.asm using configuration
Program xxd found: NO
Configuring config.h using configuration
WARNING: extract_all_objects called without setting recursive
keyword argument. Meson currently defaults to
non-recursive to maintain backward compatibility but
the default will be changed in the future.

Run-time dependency threads found: YES
Library m found: NO
WARNING: extract_all_objects called without setting recursive
keyword argument. Meson currently defaults to
non-recursive to maintain backward compatibility but
the default will be changed in the future.

WARNING: extract_all_objects called without setting recursive
keyword argument. Meson currently defaults to
non-recursive to maintain backward compatibility but
the default will be changed in the future.

Checking for function "strsep" : NO
Program doxygen found: NO
WARNING: extract_all_objects called without setting recursive
keyword argument. Meson currently defaults to
non-recursive to maintain backward compatibility but
the default will be changed in the future.

WARNING: extract_all_objects called without setting recursive
keyword argument. Meson currently defaults to
non-recursive to maintain backward compatibility but
the default will be changed in the future.

WARNING: extract_all_objects called without setting recursive
keyword argument. Meson currently defaults to
non-recursive to maintain backward compatibility but
the default will be changed in the future.

Build targets in project: 21

libvmaf 2.3.0

User defined options
backend : vs
buildtype : release
default_library: static
prefix : C:/vmaf-install

WARNING: msvc does not support C++11; attempting best effort; setting the standard to C++14_

So far it's only warnings but when I type:

vmaf-master>meson install -C libvmaf/build

I get this error:

_Only ninja backend is supported to rebuild the project before installation.
Traceback (most recent call last):
File "mesonbuild\mesonmain.py", line 146, in run
File "mesonbuild\minstall.py", line 808, in run
File "mesonbuild\minstall.py", line 586, in do_install
File "mesonbuild\minstall.py", line 715, in install_targets
RuntimeError: File 'src\libvmaf.a' could not be found

ERROR: Unhandled python exception

This is a Meson bug and should be reported!_

[1480c1]

You might need to remove the libvmaf/build folder before you run meson since it's going to be conflicting with ninja that you previously used

[thompon]

I have removed the folder libvmaf/build
I have also tried to empty it only

Each time I have the same error:

meson setup libvmaf libvmaf/build --reconfigure --backend vs --buildtype release --default-library static --prefix C:/vmaf-install
Directory does not contain a valid build tree:
C:\VMAF230\vmaf-master\libvmaf\build

I think it would be better if I started a new thread about "How to build libvmaf for MS Visual Studio". Do you agree?

[1480c1]

I realized, you have your builddir and sourcedir wrong, it should be meson libvmaf/build libvmaf and not the other way around

[thompon]

I have tried to swap the 2 arguments:

meson setup libvmaf libvmaf/build --reconfigure --backend vs --buildtype release --default-library static --prefix C:/vmaf-install

But it gives the same issue when I run meson install -C libvmaf/build :

Only ninja backend is supported to rebuild the project before installation.
Traceback (most recent call last):
File "mesonbuild\mesonmain.py", line 146, in run
File "mesonbuild\minstall.py", line 808, in run
File "mesonbuild\minstall.py", line 586, in do_install
File "mesonbuild\minstall.py", line 715, in install_targets
RuntimeError: File 'src\libvmaf.a' could not be found
ERROR: Unhandled python exception
This is a Meson bug and should be reported!

I have noticed that meson created a solution file (*.sln) for Visual Studio in libvmaf/build. The file is called libvmaf.sln

Unfortunately, when I load in Visual Studio 2019 (I have VS2010, 2015, 2017 and 2019 - I chose 2019 because VS2019 supports c++11), I get many compilation errors. For example there are many include statements for which the file to included is not found. I have manually added their directories and it seems to fix the problems. But there are some files to be included (like in #include <unistd.h>) that just don't exist in the source code. There are also many include errors concerning pthreads. It seems the libvmaf.sln file is not really well generated.

I think it would be better if I started a new thread about "How to build libvmaf for MS Visual Studio". Do you agree?