Feature: New tehcnique to compute the gaze vector

[vladmandic]

Discussed in #465

^{Originally posted by saeidn95 May 16, 2024}
Here is what I propose as an alternative to compute the gaze vector. The results are much better than the current technique. The current technique actually does not produces the right results (by a wide margin in the opposite direction for some samples.)

a. Select one of the eyes that is closer to the camera (same as the current technique)

b. Pick four points around the each eye
leftEyePoints = [33, 133, 159, 145]; //left, right, up, down, (we can replace 159, 145 with 27 and 23)
rightEyePoints = [362, 263, 374, 386]; //left, right, up, down, (we can replace 374, 386 with 257 and 253 )

compute the x and y coordinates of the centers of the eye by taking the averages of x and y coordinates of these four sets of points.

c. With the x, y, z coordinates of these four points build a linear system of equations as f = Ac, and solve for c (the center of the eyeball and the radius of the eyeball). The procedure is explained in (https://jekel.me/2015/Least-Squares-Sphere-Fit/)

d. Use the opencv linear solver or something similar to solve for c cv::solve(A, f, c, cv::DECOMP_SVD);

e. With x, y, z coordinates of center of the eyeball, the radius of the center of the eyeball, and x, y coordinates of c enter of the eye, compute the z component of the center of the eye. Change coordinates so that everything is with reference with the eyeball center (simple subtraction).

repeat the same procedure for iris. Note that for iris we might have to choose an offset. The offset I have chosen is (-5.0f, -10.0f) for the left eye, and (5.0f, -10.0f) for the right eye.

Note that we do sqrt to compute the z component of eye and iris center, we need to choose a right sign. For leftEyeCenter.z and leftIrisCenter.z, the sign is +ve and for the right eye righEyeCenter.z and rightrisCenter.z the sign is -ve.

f. convert from x, y, z for the eye center and the iris center to polar coordinate using the equation
leftEyeCenterHor = atan2(leftEyeCenter.x, - leftEyeCenter.z);
leftIrisCenterVer = asin(leftIrisCenter.y / leftEyeSphereRadious);

Finally
verAngle = leftEyeCenterVer - leftIrisCenterVer;
horAngle = leftIrisCenterHor - leftEyeCenterHor;

That is all!

[xhc-code]

Big shot, it's too powerful.

[vladmandic]

can you provide actual examples - image and results before/after?

d. Use the opencv linear solver or something similar to solve for c cv::solve(A, f, c, cv::DECOMP_SVD);

"or something similar" - to make human portable, whatever is used must be a pure js solution.
do you know of any such solution?

[xhc-code]

I also really want to participate, but I don't have the ability and knowledge to participate in such projects, it's really uncomfortable.😶