_verify_waveform fails due to data type

[wlter]

Hi there,

first of all thank you very much for this great project. I just discovered it and it works like a charm with an AFG3022C now.
I'd like to report a minor bug for this system here though which is in the "_verify_waveform" function:

The bug occurs straight away when running the "Arbitrary waveforms" example. When the waveform is set and verify is active, it fails. As far as I can see the reason is the following: in this line

if not instrument_waveform[i] == waveform[i]

the two arrays have different datatypes. In the setup here, the datatypes are

'>u2' and 'uint16'

Even though they carry the same values, the comparisons fails (for exactly one entry in the array).
This can be visualized in the following way:

I assume a simple cast is needed but since I am not familar with the topic, I don't know into which data type. Maybe a different comparison operator would also do it.
Again, thanks for the great project.

[asvela]

Hello and thanks for reporting this!

Sorry for my late reply.

It seems like you found the problem and perhaps the solution too. Casting them to the same type will probably do the trick. I wonder what is going on in that difference plot though, is there exactly one data point that has a value of 2^16 = 65536? If so we might solve it with the casting, but I am not sure why it would give this, as both the datatypes you report are uints.

Did you try some casting? I am no expert on this either, but it makes sense to me to cast the instrument waveform into a standard numpy format in get_custom_waveform()

Perhaps we should insert

waveform = waveform.astype(np.uint16)

before the current line 533 so we cast it to the same format as the waveform was cast before transferring to the instrument?

Do you still have access to the instrument to try this fix?

(Maybe a plot like the one you've made would be a nice addition to have before the RuntimeError is raised in line 631 so you get a visual for a failed test. We can insert this too in a fix.)

I'm glad you are finding the module useful, and hope we can solve this issue!

Andreas