Singularity matrix

[ZinovievVladimir]

Hi!

I have a hard function, that includes max and min and some conversions. I am getting the following error after few steps:

 File "/home/vzinoviev/ttf_DALI/training_toolbox/ssd_detector/blackbox_opt.py", line 124, in <module>
   main()
 File "/home/vzinoviev/ttf_DALI/training_toolbox/ssd_detector/blackbox_opt.py", line 113, in main
   resfile='output.csv')  # text file where results will be saved
 File "/home/vzinoviev/ttf_DALI/training_toolbox/ssd_detector/blackbox.py", line 123, in search
   fit = rbf(points, T)
 File "/home/vzinoviev/ttf_DALI/training_toolbox/ssd_detector/blackbox.py", line 212, in rbf
   sol = np.linalg.solve(M, v)
 File "/home/vzinoviev/ttf_DALI/venv/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 390, in solve
   r = gufunc(a, b, signature=signature, extobj=extobj)
 File "/home/vzinoviev/ttf_DALI/venv/lib/python3.6/site-packages/numpy/linalg/linalg.py", line 89, in _raise_linalgerror_singular
   raise LinAlgError("Singular matrix")
numpy.linalg.linalg.LinAlgError: Singular matrix```

[paulknysh]

Hi,

What do you mean by "includes max and min and some conversions"?

Also, does it happen only for this specific function? Have you tried with other functions?

[ZinovievVladimir]

@paulknysh,
My function is conversing parameters: size and aspect ratio of bounding box to width and height and third parameter is [0.0, 1.0], that I am interpreting as True or False by 0.5 threshold. There are many parameters like that (180). Then, it computing Intersection over Union for that boxes and Ground Truth's, get maximum by all boxes for each GT, and then get minimum by all GT, then it subtracted by some regularization parameter and return it.

It happen on that function. I tried your example, it's working. Also I tried to simplify my function, and it worked, but I need to compute really heavy function....

[paulknysh]

I recommend to simplify the function as much as you can, make sure that method works well for it, and then start adding complexity step by step and see at which step it starts failing. Also if you can figure out which part of the code fails, that will be useful.

[ZinovievVladimir]

Hi, @paulknysh !

I had have some initial point, close to maximum of function, and i have done the next thing: my function parameters now not the same that parameters I want, my parameters are coefficients, that I use to multiply for initial point and, it seems, it's good for me. Thank you, I think this issue can be closed. But can you please explain, why do you not processing cases, when that matrix is singular or close to singular?

[ZinovievVladimir]

@paulknysh

Sorry, I missed your question about simplify the function. Yes, I tried. I can say, that it was my mistake in first situation, because i tried to find the minimum instead of maximum. But in second situation, I discovered that if m is big (400, for example) that error is going on too.

[paulknysh]

I just looked again at your error log, it looks like error occurs at line 212 (solving linear system for RBF fit). That is a built-in numpy solver, so I don't know how it works and what's wrong.

I found that people had similar issues in the past:

https://stackoverflow.com/questions/13795682/numpy-error-singular-matrix

They suggest using linalg.lstsq instead of linalg.solve. Feel free to try that.

[ZinovievVladimir]

@paulknysh
What parameters n, m can you to recommend in different cases of function types? I want to understand how parameters affect computing and results.

[ZinovievVladimir]

@paulknysh
How can I understand, numpy.lstsq solves the system of equation close to true solution. Will it affect your algorithm?

[paulknysh]

As for n and m, there are no specific rules. n regulates initial function evaluations (global search), m - subsequent evaluations (local search). Typically, choosing n = m works reasonably well.

Here is the description of what numpy.lstsq does:
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.linalg.lstsq.html

I'm probably going to add this into the code as a backup solver. This shouldn't affect optimization algorithm.

[paulknysh]

Added np.linalg.lstsq as a backup solver. Feel free to test.