Linux & OS X Deviation Errors

[moflo]

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.01766666672968616.

Safari Version 10.0.1 (12602.2.14.0.7)
MacOS Sierra 10.12.1
MacBook Pro late 2012
Intel HD Graphics 4000 1536 MB

[turbo]

Thanks for reporting. I'll look into that. Also, I'd be interesting in test results from stable Chrome on your hardware.

[moflo]

Keep up the good work! Yes, the test works fine on Chrome on the same machine:
Chrome Version 54.0.2840.71 (64-bit)

Test results:
Pure JS @ 1.09
JS + Turbo.JS @ 3.04

[turbo]

I originally designed this as a chrome experiment. There are some issues with other WebGL pipelines in other browsers (Chrome being the fastest, followed by FireFox, then Edge), that are intermittent and almost impossible to debug. Thanks for the test data, that gives me a broader idea of the problems (sharing helps with data collection ;-))

[evanc]

I also received this error, deviation for me was 0.673000000262012. Chrome Version 54.0.2840.71 (64-bit) on a MacBook Air (13-inch, Early 2014).

[turbo]

@evanc That doesn't seem to be the same issue, as the benchmark works for @moflo in Chrome.

[micahscopes]

I'm getting this same issue on linux kernel 4.7.6, running Chromium 53.0.2785.143 with Intel i5-3320M CPU + HD Graphics 4000 GPU

[turbo]

@micahscopes Can you also provide the console output? There should be diagnostic data there,

[micahscopes]

@turbo sure:

Values are:
0.003528135799570009
and
2.006000000052154

Values are:
0.006395667718607001
and
0.012000000453554094

Values are:
1.001881273201434
and
2.018000000156462

[turbo]

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.3373333334457129.
console:
Values are: 0.00900000031106174 and 1.0210000006482005

Chrome on Ubuntu 14.04 64 bit on Intel NUC i3-4010U

from #2

[turbo]

Deviation was 0.020666667027398944.

Values are:
0.0020000000949949026
and
0.06400000117719173

on Firefox Developer Edition 51.0a2 (2016-10-17) (64-bit)

from #2

[turbo]

CC @andreapaiola

[turbo]

I also got an error:

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 10220674220032.67

The meter shows a 0.98 improvement

On the console:

Values are:
-30662022660096
and
2.012000000104308

My Software/Hardware:

Firefox 49.0.2
Ubuntu 16.04
Zenbook UX305CA
Processor: Intel® Core™ m3-6Y30 CPU @ 0.90GHz × 4
Graphics: Intel® HD Graphics 515 (Skylake GT2)

Another note: I've successfully used web workers to x4 the performance, but that's just for the CPU and not the GPU. A fallback would be nice though (:

From #3, CC @franciscop

[turbo]

In the https://turbo.github.io website, under the JAVASCRIPT & TURBO.JS section, I got an "Ah snap!" error message with deviation of 0.6786666667709748

In my JavaScript console, I got the following message:

Values are:
-0.010000000707805157
and
2.0259999996051192

Browser: Google Chrome Version 54.0.2840.87 (64-bit)
Hardware: MacBook Air (early 2014)
Processor: 1.4 GHz Intel Core i5
Memory: 4 GB 1600 MHz DDR3
Graphics: Intel HD Graphics 5000 1536 MB
Software: OS X El Capitan 10.11.4

From #6. CC @kritixilithos

[taneliva]

Another oh snap, deviation: 8290839237929643

Values are:
-24872517713788930
and
1.0100000003585592

Ubuntu 16.04, Chromium 53.0.2785.143, CPU is "AMD E-450 APU with Radeon(tm) HD Graphics", GPU is Radeon HD 6320 "Chipset: "AMD Radeon HD 6300 Series Graphics" (ChipID = 0x9806)".

(EDIT: similar result with Firefox 49.0.2.)

[somebody1234]

Wildly varying errors (Ubuntu 16.10, Vivaldi 1.5.655.3 (Official Build) dev (64-bit) (uses Chromium 54.0.2840.89)) - worked fine the first time. Experimental WebAssembly, Experimental Validate Asm.js and convert to WebAssembly when valid, experimental canvas features, 2D canvas dynamic rendering mode switching, Extensions on chrome:// URLs, Save Page as MHTML, Experimental JavaScript and GPU rasterisation enabled, GPU rasterisation MSAA sample count is 0 (in case any of this is somehow useful).

Celeron N2840 @ 2.16GHz x 2 processor, Bay Trail graphics

Note: Worked fine the first time, which is before I updated Vivaldi and with all flags off.

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.01733333357454588.
Values are:
0.0050000002374872565
and
0.0570000009611249

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.009332046611234546.
Values are:
0.0000038603320717811584
and
0.02800000016577542

Values are:
1
and
3.00400000018999
Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.6680000000633299.

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.0033333333752428498.
benchmark.js:44 Values are:
0.003000000026077032
and
0.01300000015180558

[jemisa]

Ubuntu 14.04 - CPU AMD Athlon(tm) 64 X2 Dual Core Processor 6000+ - GPU Gallium 0.4 on AMD RV635
For FF 49.0.2
Deviation was 0.33666666674738127.

Values are:
0
and
1.0100000002421439

For Chrome Version 54.0.2840.90 (64-bit)
Deviation was 0.3356666666998838.

(index):1 [.Offscreen-For-WebGL-0xd51cf4da000]GL ERROR :GL_INVALID_ENUM : GetIntegerv: <- error from previous GL command
benchmark.js:44 Values are:
0
and
1.0070000000996515

http://webglreport.com/ for FF says WebGL 1 supported, WebGL2 not supported
details:

Platform:   Linux x86_64
Browser User Agent:     Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0
Context Name:   webgl
GL Version:     WebGL 1.0
Shading Language Version:   WebGL GLSL ES 1.0
Vendor:     Mozilla
Renderer:   Mozilla
Antialiasing:   Available
ANGLE:  No
Major Performance Caveat:   No
Supported Extensions:
    ANGLE_instanced_arrays
    EXT_blend_minmax
    EXT_color_buffer_half_float
    EXT_frag_depth
    EXT_sRGB
    EXT_texture_filter_anisotropic
    OES_element_index_uint
    OES_standard_derivatives
    OES_texture_float
    OES_texture_float_linear
    OES_texture_half_float
    OES_texture_half_float_linear
    OES_vertex_array_object
    WEBGL_color_buffer_float
    WEBGL_compressed_texture_s3tc
    WEBGL_depth_texture
    WEBGL_draw_buffers
    WEBGL_lose_context
    MOZ_WEBGL_lose_context
    MOZ_WEBGL_compressed_texture_s3tc
    MOZ_WEBGL_depth_texture

[turbo]

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 137920146396501.34.

Chromium Version 53.0.2785.143 Built on Ubuntu , running on LinuxMint 18 (64-bit)
Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
Intel® HD Graphics 530

Let me know if you need anything else!

from #9

[rafaelspring]

MacBook Air (13-inch, Mid 2013)
1.3 GHz Intel Core i5
8 GB 1600 MHz DDR3
Intel HD Graphics 5000 1536 MB
Mac OS X 10.11.6.

On Safari 10.0.1. Deviation was 0.007000000138456623.
On Chrome 54.0.2840.71 (64-bit): Deviation was 0.006666666944511235.

[gravypod]

I got one too:

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.011576077551580966.day

I'm on the following:

• uname -a: Linux katzdesktop 4.7.10-1-MANJARO Linux & OS X Deviation Errors #1 SMP PREEMPT Sat Oct 22 19:15:11 UTC 2016 x86_64 GNU/Linux
• cat /proc/cpuinfo: http://hastebin.com/beyadazaka.txt
• chrome://version: http://hastebin.com/owosapojuk.txt

I'm using the AMD APU as my graphics chip. I can't think of any other pertinent information that I could provide. I'm on Manjaro so everything in my graphics stack should be relatively up to date.

Edit: I don't know if this is helpful but I've just done a test with my laptop and got a deviation of similar magnitude despite it the only difference between these two systems being their hardware. They are both running Manjaro (my desktop and laptop).

My laptop is a Thinkpad x220 so I'm not sure if that helps narrow down where the issue is coming from.

[Akkowicz]

Deviation was 0.6604074428711708.

• Linux UAMF-HQ 4.8.6-1-ARCH Linux & OS X Deviation Errors #1 SMP PREEMPT Mon Oct 31 18:51:30 CET 2016 x86_64 GNU/Linux
• Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
• AMD Radeon HD7870 on Mesa 13.0.0 drivers
• Chromium 54.0.2840.90 (Developer Build) (64-bit)

[rjdlee]

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.003999999957159162.

Safari Version 10.0 (11602.1.50.0.10)
MacOS Sierra 10.11.6
MacBook Pro 13" Mid-2010
NVIDIA GeForce 320M 256 MB

[palotasb]

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.671666666942959 [or other seemingly nondeterministic number on different runs].

• Chrome 54.0.2840.71 (64-bit)
• Safari 10.0.1 (10602.2.14.0.7)
• Firefox 46.0.1

All browsers give the same errors.

[jpopesculian]

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.33186534373089666.

• Dell XPS 13.3
• Ubuntu 16.04.1
• Intel Core i7-6560U
• Intel Iris Graphics 540
• Chrome 56.0.2906.0
• Firefox 51.0a2

Numbers varied wildly

[msimpson]

I noticed something interesting on my desktop. If I keep calling reBenchmark(), everything seems fine. But if I open and close a few OpenGL applications between executions, I begin to get deviations outside the threshold. This makes me think there may be a problem in your testTurbo code, here:

float y0 = ipt.g, x, y, xt, c;

If I initialize those floats, the errors stop under the same circumstances:

float y0 = ipt.g, x = 0., y = 0., xt = 0., c = 0.;

I think you may be getting garbage.

Reading a variable before writing (or initializing) it is legal, however the value is undefined.
The OpenGL® Shading Language, Version 4.40 (16-Jun-2014)

Desktop Specs
Arch Linux (Kernel 4.4.1-2)
Intel i7-3930K
nVidia GeForce GTX 670 (Driver 361.18-2)
Chromium 48.0.2564.103 (64-bit)

Update
I was still able to get some deviations outside the threshold with those floats initialized, albeit after many tries:

Values are:
0.11800000676885247
and
0.12099999748170376

Deviation was 0.000999996904283762.

Values are:
0.5280000198399648
and
0.5129999745404348

Deviation was 0.005000015099843352.

It appears the deviation is still random.

[micahscopes]

@msimpson,
That sounds like a very likely all-around issue. I have some experience with writing GLSL shaders for fragmentarium (basically doing gpgpu stuff), and I vaguely remember needing to make sure I initialized my variables for the shaders to work consistently. Since I was drawing fractals, the differences became obvious very soon. Fractals made debugging a lot easier in some ways, and a lot more "complex" in other ways.

[micahscopes]

P.s., when this stabilizes you will probably find me using it to draw fractals.

[turbo]

Variables are initialized now. I always forget this, I'm just used to the much more relaxed AMD GLSL flavor.

Please test again and see if it does anything for you 🎱 .

[turbo]

@msimpson That deviation is pretty good actually. If it doesn't massively exceed this, I might dare to call it usable.

[turbo]

Well. That fixes it for me:

• archlinux
• Chrome stable, Chrome, FF

Please just leave "fixed" if that resolved the issue for you.

[micahscopes]

"fixed"

looks like I'm getting ~5x speedups on the front page 👍 👍 👍 👍

[somebody1234]

fixed

[andreapaiola]

works for me, I get 4.15

[msimpson]

I modified reBenchmark to track all individual deviations above the threshold as well as the highest deviation overall. Then, I ran reRun through ten thousand iterations, instead of three, on the same desktop I described above. The results were interesting:

Total Iterations: 10,000

Average Deviation: 0.000005499842064465854
Combined JS Sanity: 2259.9060005557258
Combined GLSL Sanity: 2259.851002135081

Highest Deviation: 0.2930000424385071
JS Sanity: 0.593999981880188
GLSL Sanity: 0.8870000243186951

Trials Above Threshold (0.001): 22 (0.22%)

So that's an accuracy of 99.78% to within plus or minus one thousandth of each other, where no individual iteration drifted further than plus or minus three tenths overall. Not too bad, considering the type conversions and variance of IEEE 754 at play.

[turbo]

@msimpson,

That's seems to be pretty much exact. I actually expect to be the GPU results to be more precise. Thanks for the test.

[msimpson]

@turbo
I agree, and my pleasure.

[jemisa]

fixed in FF and Chrome.
thanks

[malonehedges]

I also had a low deviation on Safari Version 10.0 (12602.1.50.0.10) on macOS Sierra 10.12 (16A323).

[turbo]

In Safari 10.0.1 (12602.2.14.0.7) on macOS Sierra I get

Ah snap! There was no error compiling the test kernel or running it, but the numbers don't check out. Please report your browser (+ version) and hardware configuration to us, so we can try to fix this. Deviation was 0.337999999950019.

From #17.

[mroderick]

Safari 10.0.1 (12602.2.14.0.7), macOS Sierra
MacBook Pro (15-inch, Mid 2010), NVIDIA GeForce GT 330M 256 MB, Intel HD Graphics 288 MB

Deviation was 0.0026666667157163224.

Values are:
0
and
0.008000000147148967

Chrome 54.0.2840.98 (64-bit) and Firefox 49.0.2 are both working, Firefox getting the best score.

[kritixilithos]

It works for me now.

[AnalyzePlatypus]

A mysterious warning appears in the console:

WebGl: INVALID_ENUM 
readPixels - invalid type

Relevant line turbo.js:184:

gl.readPixels(0, 0, size, size, gl.RGBA, gl.FLOAT, ipt.data);

Config

• Safari 10.1.1 (macOS Sierra)
• MacBook Pro (Retina, 15-inch, Mid 2015)
• Intel Iris Pro 1536 MB

Deviation was 0.3426666664890945.