Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added database feature. #935

Open
wants to merge 111 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
111 commits
Select commit Hold shift + click to select a range
1c07249
Added database feature.
RaBa64 May 19, 2023
abc4b83
Added code for JS and HTML (as comment at the end of the file).
RaBa64 May 21, 2023
15c6b31
Merge pull request #1 from RaBa64/Database
RaBa64 May 21, 2023
a785074
added new card to LiveView page with chart for last 25 hours
RaBa64 May 21, 2023
c26b10a
fixed formatting of the DatabaseChart
RaBa64 May 21, 2023
7133a4b
fixed delayed load of google libraries after page refresh
RaBa64 May 21, 2023
de9d471
GChart
RaBa64 May 21, 2023
7d96c52
Delete database.png
RaBa64 May 22, 2023
a45b3d2
removed serial number from screenshot
RaBa64 May 22, 2023
d8646b8
Merge branch 'Database' of https://github.com/RaBa64/OpenDTU into Dat…
RaBa64 May 22, 2023
268d492
use vue-google-charts wrapper
RaBa64 May 22, 2023
8070121
fixed wromg position of chart
RaBa64 May 23, 2023
6e8c157
updated screenshot
RaBa64 May 23, 2023
4f663c9
updated webapp
RaBa64 May 24, 2023
d0beca3
added calendar chart
RaBa64 Jun 1, 2023
7ae87bc
added calendar chart
RaBa64 Jun 1, 2023
23454a5
Merge branch 'master' into Database
RaBa64 Jun 1, 2023
695f021
updated screenshot
RaBa64 Jun 1, 2023
e955226
Merge branch 'tbnobody:master' into master
RaBa64 Jun 1, 2023
5447e55
Merge branch 'master' into Database
RaBa64 Jun 1, 2023
ccc9d55
used new interface
RaBa64 Jun 1, 2023
78a82e0
Merge branch 'tbnobody:master' into master
RaBa64 Jun 2, 2023
6d52b7f
Fixed problem that charts are not shown after page refresh.
RaBa64 Jun 3, 2023
7824a06
made Vue compile
RaBa64 Jun 3, 2023
10397d1
changed chart options
RaBa64 Jun 3, 2023
d7bb820
Merge branch 'origin/HEAD' into Database
RaBa64 Jun 3, 2023
d5b354a
fixed height
RaBa64 Jun 3, 2023
4281c67
updated webapp
RaBa64 Jun 3, 2023
df2c353
changed chart options
RaBa64 Jun 3, 2023
a6d64a8
updated webapp
RaBa64 Jun 3, 2023
40cf097
updated Webapp and screenshot
RaBa64 Jun 3, 2023
b176a7a
new screenshot
RaBa64 Jun 3, 2023
64c6bd6
create database when it does not exist
RaBa64 Jun 3, 2023
1f89ada
fixed name of days in calendar chart
RaBa64 Jun 3, 2023
3cdab45
added debug messages
RaBa64 Jun 4, 2023
5f1664a
added more debugging messages
RaBa64 Jun 5, 2023
d6b8f26
hooked to LiveView instead of MQTT
RaBa64 Jun 5, 2023
4b23b38
moved loop handler
RaBa64 Jun 6, 2023
9b417f3
fixed time handling
RaBa64 Jun 6, 2023
831cecc
fixed first datapoint
RaBa64 Jun 6, 2023
196c983
fix hanging of GUI
RaBa64 Jun 7, 2023
7c8b5bb
updated Webapp
RaBa64 Jun 7, 2023
7d9d2bd
Merge branch 'tbnobody:master' into master
RaBa64 Jun 9, 2023
f5b5d7e
some cosmetic changes
RaBa64 Jun 9, 2023
60a3350
Merge https://github.com/RaBa64/OpenDTU into Database
RaBa64 Jun 9, 2023
c45d339
updated Webapp
RaBa64 Jun 9, 2023
3768ae8
Merge branch 'tbnobody:master' into master
RaBa64 Jun 12, 2023
0da01d7
Merge branch 'master' of https://github.com/RaBa64/OpenDTU into Database
RaBa64 Jun 13, 2023
ec66df4
used chunked data transfer
RaBa64 Jun 13, 2023
3710591
used chunked data transfer
RaBa64 Jun 13, 2023
ec788d6
fixed loading flag
RaBa64 Jun 13, 2023
925c434
removed lambda function
RaBa64 Jun 13, 2023
a5ac646
Merge branch 'tbnobody:master' into master
RaBa64 Jun 15, 2023
660aaf3
Merge branch 'tbnobody:master' into master
RaBa64 Jun 21, 2023
2994ff9
Merge branch 'master' of https://github.com/RaBa64/OpenDTU into Database
RaBa64 Jun 21, 2023
8f70391
RaBa
RaBa64 Jun 21, 2023
e6cf6b0
fixed chart refresh after reload
RaBa64 Jun 25, 2023
d2a15f8
fixed wrong time for first database entry on a day
RaBa64 Jun 26, 2023
4a8078a
added autorefresh every hour
RaBa64 Jun 26, 2023
4b88413
optimized autorefresh
RaBa64 Jun 26, 2023
83df4e0
Merge branch 'tbnobody:master' into master
RaBa64 Jun 28, 2023
bf7be21
fixed formatting of card
RaBa64 Jun 28, 2023
92f491a
Merge branch 'master' of https://github.com/RaBa64/OpenDTU into Database
RaBa64 Jun 28, 2023
a762fcf
html change
RaBa64 Jun 28, 2023
372f7f7
Merge branch 'tbnobody:master' into master
RaBa64 Jun 30, 2023
c289c27
fixed date comparison
RaBa64 Jun 30, 2023
8030a99
Merge remote-tracking branch 'origin/master' into Database
RaBa64 Jun 30, 2023
0ee3233
new screenshot
RaBa64 Jun 30, 2023
2d31456
Update README.md
RaBa64 Jun 30, 2023
08579b0
Database (#2)
RaBa64 Jun 30, 2023
884c114
Update README.md
RaBa64 Jun 30, 2023
8c0d43c
updated ReadMe
RaBa64 Jun 30, 2023
1319225
text changes
RaBa64 Jun 30, 2023
0509952
Merge branch 'GoogleCharts' of https://github.com/RaBa64/OpenDTU into…
RaBa64 Jul 17, 2023
5ef5621
updated WebApp
RaBa64 Jul 17, 2023
dd3c12d
Merge remote-tracking branch 'origin/master' into GoogleCharts
RaBa64 Aug 13, 2023
2fbc9bc
merged OpenDTU master
RaBa64 Aug 13, 2023
2c4e2c8
Merge branch 'GoogleCharts' into Database
RaBa64 Aug 14, 2023
9ebcb4c
new WebApp
RaBa64 Aug 14, 2023
a2a2e2a
Merge branch 'master' of https://github.com/RaBa64/OpenDTU into Database
RaBa64 Sep 16, 2023
c5016a3
fixed one bug reports and one feature request
RaBa64 Sep 16, 2023
772c3c4
Merge branch 'master' of https://github.com/tbnobody/OpenDTU into Dat…
RaBa64 Dec 20, 2023
a5fc54e
fixed problem in initialization
RaBa64 Dec 21, 2023
dce06e3
update lockfile
RaBa64 Dec 21, 2023
0f12353
update lock file
RaBa64 Dec 21, 2023
e087060
updated yarn.lock file
RaBa64 Dec 21, 2023
095bfee
Merge https://github.com/RaBa64/OpenDTU-Database into Database
RaBa64 Jan 2, 2024
0c288e8
show 2 years
RaBa64 Jan 2, 2024
f901793
pre update
RaBa64 Jan 7, 2024
06f4e64
Merge branch 'master' of https://github.com/tbnobody/OpenDTU into Dat…
RaBa64 Jan 7, 2024
1db3956
first version with new caleandar
RaBa64 Jan 7, 2024
cbfe01c
changed font size
RaBa64 Jan 16, 2024
d784e14
fixed text size
RaBa64 Jan 16, 2024
05d7bff
round corners
RaBa64 Feb 18, 2024
1a30c26
round corners
RaBa64 Feb 18, 2024
db3b5a6
Merge branch 'master' of https://github.com/tbnobody/OpenDTU into Dat…
RaBa64 Feb 18, 2024
b5a526c
added Database restore
RaBa64 Feb 18, 2024
0ea19c3
fixed changes in the scheduler
RaBa64 Feb 21, 2024
53fbbe2
Add files via upload
RaBa64 May 23, 2024
9d26888
Rename Screenshot 2024-05-23 131208.png to Screenshot_2024-05-23_1312…
RaBa64 May 23, 2024
7a25063
Update README.md
RaBa64 May 23, 2024
ff492af
Update README.md
RaBa64 May 23, 2024
ac88d4f
Merge branch 'tbnobody:master' into master
RaBa64 May 27, 2024
d6c183c
Merge https://github.com/RaBa64/OpenDTU-Database into Database
RaBa64 May 27, 2024
33a33fc
v24.5.27
RaBa64 May 27, 2024
7d1a3c6
Merge branch 'tbnobody:master' into master
RaBa64 Jun 13, 2024
04881da
Merge branch 'master' of https://github.com/tbnobody/OpenDTU into Dat…
RaBa64 Jun 18, 2024
18f64a4
Merge branch 'tbnobody:master' into master
RaBa64 Jul 6, 2024
ceef11a
Merge branch 'tbnobody:master' into master
RaBa64 Aug 6, 2024
ac716b5
Merge remote-tracking branch 'origin/master' into Database
RaBa64 Aug 6, 2024
7573688
RaBa64 Aug 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
{
"C_Cpp.clang_format_style": "WebKit"
}
"C_Cpp.clang_format_style": "WebKit",
"cmake.sourceDirectory": "C:/git/OpenDTU-Database/.pio/libdeps/generic_esp32/ArduinoJson/src"
}
29 changes: 26 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,35 @@
# OpenDTU
# OpenDTU-Database-Database
# OpenDTU-Database

One year OpenDTU-Database
![Image](docs/screenshots/Screenshot_2024-05-23_131208.png)

[![OpenDTU Build](https://github.com/tbnobody/OpenDTU/actions/workflows/build.yml/badge.svg)](https://github.com/tbnobody/OpenDTU/actions/workflows/build.yml)
[![cpplint](https://github.com/tbnobody/OpenDTU/actions/workflows/cpplint.yml/badge.svg)](https://github.com/tbnobody/OpenDTU/actions/workflows/cpplint.yml)
[![Yarn Linting](https://github.com/tbnobody/OpenDTU/actions/workflows/yarnlint.yml/badge.svg)](https://github.com/tbnobody/OpenDTU/actions/workflows/yarnlint.yml)

## !! IMPORTANT UPGRADE NOTES !!
OpenDTU-Database adds an ESP32 LittleFS Database and two energy charts, a column chart of the last 25 hours and a full calendar chart.

![Screenshot](https://github.com/RaBa64/OpenDTU/blob/Database/docs/screenshots/23_Database.png)

OpenDTU-Database adds an ESP32 LittleFS Database and two energy charts, a column chart of the last 25 hours and a full calendar chart.

There are 3 new APIs available, returning JSON strings:

| API | returned values |
|-------------------|-----------------|
| /api/database | returns all recored data points from the database with total energy value |
| /api/databaseHour | returns the energy per hour for the last 25 hours |
| /api/databaseDay | returns the energy for each day |

Each data point has the following format:
[ _year (00-99)_, _month (1-12)_, _day (1-31)_, _hour (0-23)_, _energy (Wh)_ ]

Example: [23,6,30,15,132.995605]

The 192KB LittleFS in OpenDTU can store around 6 years of data, because each data point needs only 8 bytes of memory.

If you are upgrading from a version before 15.03.2023 you have to upgrade the partition table of the ESP32. Please follow the [this](docs/UpgradePartition.md) documentation!
![Screenshot](docs/screenshots/23_Database.png)

## Background

Expand Down
Binary file added docs/screenshots/23_Database.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions docs/screenshots/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,7 @@ here are some screenshots of OpenDTU's web interface.
***

![Console](18_Console.png)

***

![Console](23_Database.png)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions include/WebApi.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
#include "WebApi_webapp.h"
#include "WebApi_ws_console.h"
#include "WebApi_ws_live.h"
#include "WebApi_database.h"
#include <AsyncJson.h>
#include <ESPAsyncWebServer.h>
#include <TaskSchedulerDeclarations.h>
Expand Down Expand Up @@ -65,6 +66,7 @@ class WebApiClass {
WebApiWebappClass _webApiWebapp;
WebApiWsConsoleClass _webApiWsConsole;
WebApiWsLiveClass _webApiWsLive;
WebApiDatabaseClass _webApiWsDatabase;
};

extern WebApiClass WebApi;
37 changes: 37 additions & 0 deletions include/WebApi_database.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
// SPDX-License-Identifier: GPL-2.0-or-later
#pragma once

#include <Arduino.h>
#include <ESPAsyncWebServer.h>
#include <TaskSchedulerDeclarations.h>

#define DATABASE_FILENAME "/database.bin"

class WebApiDatabaseClass {
public:
WebApiDatabaseClass();
void init(AsyncWebServer& server, Scheduler& scheduler);
bool write(float energy);

struct pvData {
uint8_t tm_year;
uint8_t tm_mon;
uint8_t tm_mday;
uint8_t tm_hour;
float energy;
};

private:
void onDatabase(AsyncWebServerRequest* request);
void onDatabaseHour(AsyncWebServerRequest* request);
void onDatabaseDay(AsyncWebServerRequest* request);
static size_t readchunk(uint8_t* buffer, size_t maxLen, size_t index);
static size_t readchunk_log(uint8_t* buffer, size_t maxLen, size_t index);
static size_t readchunkHour(uint8_t* buffer, size_t maxLen, size_t index);
static size_t readchunkDay(uint8_t* buffer, size_t maxLen, size_t index);

AsyncWebServer* _server;

Task _sendDataTask;
void sendDataTaskCb();
};
1 change: 1 addition & 0 deletions src/WebApi.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ void WebApiClass::init(Scheduler& scheduler)
_webApiWebapp.init(_server, scheduler);
_webApiWsConsole.init(_server, scheduler);
_webApiWsLive.init(_server, scheduler);
_webApiWsDatabase.init(_server, scheduler);

_server.begin();
}
Expand Down
Loading