Error in running defined anomaly

[satkalra1]

Hey,
I was trying the cue observe on the test dataset, to understand its working properly but after defining the anomaly, I found this particular error:

{"stackTrace": "Traceback (most recent call last):\n File "pandas/_libs/lib.pyx", line 2062, in pandas._libs.lib.maybe_convert_numeric\nValueError: Unable to parse string "null"\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/code/ops/tasks.py", line 49, in anomalyDetectionJob\n dimValsData = prepareAnomalyDataframes(datasetDf, anomalyDefinition.dataset.timestampColumn, anomalyDefinition.metric, anomalyDefinition.dimension, anomalyDefinition.top)\n File "/code/access/utils.py", line 17, in prepareAnomalyDataframes\n datasetDf[metricCol] = pd.to_numeric(datasetDf[metricCol])\n File "/opt/venv/lib/python3.7/site-packages/pandas/core/tools/numeric.py", line 155, in to_numeric\n values, set(), coerce_numeric=coerce_numeric\n File "pandas/_libs/lib.pyx", line 2099, in pandas._libs.lib.maybe_convert_numeric\nValueError: Unable to parse string "null" at position 1715\n", "message": "Unable to parse string "null" at position 1715"}

Could you please tell me , where the things are going wrong?

[ankitkpandey]

Hi @satkalra1,
It's because the dataset contains "null" in string format in the metric column in certain rows. If you look into the code, you'll find that for aggregation we convert the metric column to numeric form using the pandas API.

pd.to_numeric(datasetDf[metricCol])

You could make sure that the test dataset doesn't have strings("null" in this case) in the metric column. That would be a fix.

Also, if you'd like you could contribute by updating the code to handle strings

[satkalra1]

Hi @ankitkpandey,
I figured it out later. If you can please point it out where exactly this string handling has to be done like the exact file, I can try and fix this issue, will be happy to help.

[ankitkpandey]

Hey @satkalra1, It's in the function prepareAnomalyDataframes in api/access/utils.py
I would encourage you to just go through the code once as well to get an idea of all the modifications a dataframe goes through.
Thanks for contributing