Editing Log data?

[nborof]

Hey all -

I've been using Fermentrack for several months now, and it's been going great - thanks so much for all the time and work you've put into it!

My one question is - is there any way to directly edit the log files? A couple of times, there has been an errant piece of data (e.g., the fermenter getting bumped and the Tilt hydrometer recording a wildly high or low value) which then skews the whole display of the chart (e.g., the gravity appears as an almost flat line in order to fit the one errant data value). I would love to go and edit the log file so that this can be corrected.

I have fermentrack installed via Docker on an Ubuntu Linux machine. I am reasonably bash/linux proficient (although I haven't done much with docker before). Let me know if this is at all possible.

Thanks again!

Nate

[lbussy]

I don't know where the data is on the Docker install, but if you are decent in the shell you should be able to find it. Here's what the data looks like in BrewPi, and until @thorrak logs in here to argue with me, my assertion is he wasn't motivated to change it a whole bunch. 😁

pi@brewpi:/home/brewpi/data/Test $ ls -al
total 1868
drwxrwxr-x 2 brewpi brewpi   4096 Oct 31 00:00 .
drwxrwxr-x 5 brewpi brewpi   4096 Oct 13 16:31 ..
-rw-r--r-- 1 brewpi brewpi  60748 Oct 30 23:58 Test-20211030.json
-rw-r--r-- 1 brewpi brewpi  22521 Oct 31 08:47 Test-20211031.json
pi@brewpi:/home/brewpi/data/Test $

pi@brewpi:/home/brewpi/data/Test $ cat Test-20211031.json
{"cols":[{"type":"datetime","id":"Time","label":"Time"},{"type":"number","id":"BeerTemp","label":"Beer Temp."},{"type":"number","id":"BeerSet","label":"Beer Setting"},{"type":"string","id":"BeerAnn","label":"Beer Annot."},{"type":"number","id":"FridgeTemp","label":"Chamber Temp."},{"type":"number","id":"FridgeSet","label":"Chamber Setting"},{"type":"string","id":"FridgeAnn","label":"Chamber Annot."},{"type":"number","id":"RoomTemp","label":"Room Temp."},{"type":"number","id":"State","label":"State"}],"rows":[
{"c":[{"v":"Date(2021,9,31,0,0,26)"},null,null,null,null,null,null,null,{"v":0}]},
{"c":[{"v":"Date(2021,9,31,0,2,27)"},null,null,null,null,null,null,null,{"v":0}]},
{"c":[{"v":"Date(2021,9,31,0,4,27)"},null,null,null,null,null,null,null,{"v":0}]},
{"c":[{"v":"Date(2021,9,31,0,6,28)"},null,null,null,null,null,null,null,{"v":0}]}]}

Prettified, it looks like this:

{
  "cols": [
    {
      "type": "datetime",
      "id": "Time",
      "label": "Time"
    },
    {
      "type": "number",
      "id": "BeerTemp",
      "label": "Beer Temp."
    },
    {
      "type": "number",
      "id": "BeerSet",
      "label": "Beer Setting"
    },
    {
      "type": "string",
      "id": "BeerAnn",
      "label": "Beer Annot."
    },
    {
      "type": "number",
      "id": "FridgeTemp",
      "label": "Chamber Temp."
    },
    {
      "type": "number",
      "id": "FridgeSet",
      "label": "Chamber Setting"
    },
    {
      "type": "string",
      "id": "FridgeAnn",
      "label": "Chamber Annot."
    },
    {
      "type": "number",
      "id": "RoomTemp",
      "label": "Room Temp."
    },
    {
      "type": "number",
      "id": "State",
      "label": "State"
    }
  ],
  "rows": [
    {
      "c": [
        {
          "v": "Date(2021,9,31,0,0,26)"
        },
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        {
          "v": 0
        }
      ]
    },
    {
      "c": [
        {
          "v": "Date(2021,9,31,0,2,27)"
        },
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        {
          "v": 0
        }
      ]
    },
    {
      "c": [
        {
          "v": "Date(2021,9,31,0,4,27)"
        },
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        {
          "v": 0
        }
      ]
    },
    {
      "c": [
        {
          "v": "Date(2021,9,31,0,6,28)"
        },
        null,
        null,
        null,
        null,
        null,
        null,
        null,
        {
          "v": 0
        }
      ]
    }
  ]
}

So, the technical answer is: yes, you can edit it, I think you are quickly going to figure out it doesn't bother you that much.

[thorrak]

What @lbussy said is correct. The files he cites are the “almost JSON” files which are used for annotations. The data points themselves are saved to CSV files and can be edited using whatever tools you prefer.

If I ever learn to use pandas, I’ll add the ability to filter/delete points that meet certain criteria, but that is not a tool I have spent much time learning, unfortunately.

[nborof]

Thank you both for writing back!

After some fiddling around, I got it to work. Here's what worked for me (in case anyone else ever wants to do something like this).

To list the active docker containers, type:
[docker container ls]

Find the name of the container with "nginx" in the name (aka "fermentrack-tools_nginx_1"). Then type:
docker exec -ti [CONTAINER NAME] ls -al data

This will list all of the logs. Find the filename of the log you want to edit that ends with "_graph.csv". The only text editor I could see in the container was vi - which I always have to re-learn how to use every time. :-)

Then type:
docker exec -ti [CONTAINER NAME] vi "data/[FILE NAME]"

Find the lines you want to remove, and save.

This seems to do the trick. The graphs redraw perfectly without the erroneous data.

Thanks again for your help!