Improvement of the dragfactor calculation by using Linear TS estimator #159
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Looks like dead code Dead code????? * Improvement of the MovingAverager startup Improvement of the MovingAverager's startup-behaviour, to allow use for the dragcalculation. In this approach, the defaultValue is only used when the array isn't sufficiently filled (less then two pushed values), but after that it is directly released and the average only depends on pushed values. This is especially important for the drag calculation, as the drag calcuation is sufficiently robust in handling errornous values, and drag factors might make big jumps in the beginning (i.e. from the middle setting to a maximum value). * Allow the settings to set a specific priority The nice-level of -20 is arbitrary and very counterproductive, as it dwarfs OS processes including the internal timer. This results in a more noisy measurement, even on a high-speced Pi 4. As the priority is dependent on the hardware/software configuration of the Pi anyway, I made the configuration setting more granular. * Added Concept2 PM5 wiring diagram Added wiring diagram for the PM5 (with 18V generator) * Update server.js * Delete app/engine/averager directory * Creation of the series.js Creation of a series of values, as an abstraction away from the array, with added standard functionality. This is the basis for the more complex functions like the regression series. * Update the server.js to adapt to new architecture Server.js has been adapted to fit the new architecture. It is now in control of the session, directing others how to proceed. * Deleted Timer.js as it has become obselete Deleted Timer.js as it has become obselete * Updated to more abstract Flywheel.js Renamed and rewritten to Flywheel.js. It provides an abstraction from the individual currentDt's and provides a more abstract view of the flywheel and its current state, which can be interrogated by Rower.js * Created an initial testset for Flywheel.js An initial set of tests to check whether Flywheel.js works as expected. * Renaming variables and added new functionality Renamed variables to fit the naming scheme of Rower.js. Added the option to create a RowingData file, hich can be read by RowingData, RowsAndAll and Excel. Also added new fields in the notes of a TCX, including HRR. * Update Pm5Constants.js * Update to the constants Update to the constants, to make them closer to the actual firmware options. Also I added the Row to the name, as Concept2 also seems to do this on their current firmware. * Added additional defensive programming Added additional defensive programming * updated variable names Updated variable names to keep them in sync with RowingStatistics * Added metrics and updated variable names Added new metrics, like projected time and distance, stroke calories Updated variable names to keep them in sync with RowingStatistics * Added metric, states and variable rename Added the dragfactor metric, aded the state for the workoutType, workoutDurationType, workoutState, RowingState and strokeState Renamed most variable to allign them with RowingStatistics * Added metrics, rename variables Added several metrics (driveLength, driveTime, strokeRecoveryTime, strokeDistance, peakDriveForce, averageDriveForce and workPerStroke) Alignment of variable names with RowingStatistics.js * Renamed variables Renamed variables to align them with the RowingStatistics naming convention * Update variable definitions and naming Update to the variable definition of speed: as almost all consumers use the m/s unit, we now use this internally, and only convert to km/h for the bike Alignment with the naming scheme of RowingStatistics * Update based on PM5 spec workoutDurationType was a more complex enum * Complete redesign Complete redesign: * Switched to a Finite State machine to maintain session state * To reduce complexity, it directly inspects Rower.js, instead of awaiting messages to be processed * Small snippets process all data, where updates are grouped based on their update moment * Renaming all variables to clearly identify when they are updated (as determined by the state machine) * Introduced a first version of interval management, as a preperation for future solutions with more complex training scheme's This should help in reducing the code base while making the code easier to understand. * Rename and adoptation to the new architecture Split RowingEngine into two blocks: * Rower (which represents the key metrics from the rower) and * flywheel (which represents the key metrics of the flywheel, formally known as MovingFlankDetector). This should simplify the reading of the code, as Flywheel reports key metrics without the need to many specific configuration parameters. Also added many metrics (including force curve, power curve and handle speed). * Update and rename RowingEngine.test.js to Rower.test.js Update to fit Rower.js better, includes testing with known rowers * Introduction of VO2Max (Beta) Introduction of a Beta-version of VO2-Max, which will be calculated each time a tcx-file is created. It introduces two versions of VO2-Max calculation: * extrapolation-based VO2Max: it extrapolates the power to maxpower based on MaxHR and the relation between HR and power in the session * interpolation based VO2Max: it projects the session onto a 2K, and then calculates the VO2Max based on Concept2's research * Beta version Beta version of the BucketLinearSeries to accomodate VO2Max * Create CurveAligner.js CurveAligner for cleaning up the (power, force and handle velocity) curves * Create Series.test.js Added test-class to series.js, as it is so fundamental to OpenRowingMonitor * Create StreamFilter.js Streamfilter implements a running streamfilter. It is designed to be robust to outliers, and replaces all averaging functions. * Create StreamFilter.test.js Tests the steamfilter * Create OLSLinearSeries.js Creation of the OLSLinearSeries, to allow for drag calculation, VO2Max calculation and projecting calories, distance and time. * Create OLSLinearSeries.test.js Test of the Simple Linear Regression method * Create FullTSQuadraticSeries.js FullTSQuadraticSeries delivers Full Theil-Senn Quadratic regression, which is essential to determine Angular Velocity and Angular Acceleration in a noise-resistant way. These metrics are essential for force curve, power curve, handle velocity curve and stroke detection (as it is triggered by the force on the handle). * Create FullTSQuadraticSeries.test.js Test of the FullTSQuadraticSeries * Create curveMetrics.js Object to manage the specifics of curves * Delete curveMetrics.js * Create curveMetrics.js Object to create curves and the associated curve metrics * Update rowerProfiles.js Update to accomodate the new RowingEngine, and its parameters * Renaming variables Renaming variables to align with the variable renaming in RowingStatistics.js * Rename variables Update variables to align with RowingStatistics naming conventions * Update GpioTimerService.js * Added example CurrentDt curve Added example CurrentDt curve for explanation of good and bad noise. * Testfile for Concept2 RowErg Testfile for Concept2 RowErg, 2000 meters, for validation purposes * Increasing consistency in logging Increased consistency in logging specific messages * Improved logging to facilitate configuration Improved the logging of specific data to facilitate an easier setup for new rowers. * Update backlog.md * Update rower_settings.md * Adaptation to the newly developed physics model Adaptation of the text to the newly developed physics model. * Removed the settings analysis as it isn't needed Removed this tool, as it isn't up-to-date with the current physics model, and logging does a better job in helping the analysis. * Updated to reflect the new approach Updated the setup procedure to: * Adapt it to the new physics model * Adapt it to the new logging-based setup procedure. * Added the Concept2 RowErg Added the Concept2 RowErg configuration * Added the Concept2 as test-object Added the Concept2 as test-object * Added the Concept2 as test-object Added the Concept2 as test-object * Update Rower.test.js * Adaptation to C2 Flywheel Inertia change Adaptation to C2 Flywheel Inertia change * Created to explain the current architecture Created an architecture document to explain the current architecture, as well as the major components in Open Rowing Monitor to maintainers and reviewers of the code. * Create hardware_setup_Concept2_RowErg.md * Added Concept 2 manual Added concept 2 RowErg manual, as some minor updates * Added performance improvement guide Added a guide for improving the performance (Latency) of the Raspberry Pi. * Fixed typos, filled in some blanks Fixed some typos, improved some text for clarity and added some text to complete this description. * Update to reflect new capabilities Update to reflect new capabilities of OpenRowingMonitor and make the text easier to parse for novice users (make it less technical). * Updated with new settings Updated with the new settings available * Update default.config.js * Update to allow for the 64Bit PREEMPT kernel Update of the install script to allow for the 64Bit PREEMPT kernel of Raspberry Pi, which handles low latency much better. * Fixed a typo in a file reference Thanks to @Abasz , dound a typo which i now fixed. * Fixed typo Fixed typo Co-authored-by: Lars Berning <[email protected]> * Fixed a typo Fixed a typo Co-authored-by: Lars Berning <[email protected]> * Fixed a cut-copy-paste error Fixed a cut-copy-paste error in the infinity symbol * Fixed error in repo-name Fixed an error in the repo-name when moving for the experimental branch to the PR branch * Fixed a copy error in the Repo-name Missed updating the repo-name when moving the code from private experimental branch to a production PR repo * Updated ToDo descriptions Updated ToDo descriptions * Added more detailed ToDo's Added more detailed ToDo's * Better explanation of design issue A much better (and English) and updated explanation of the open design issues in Flywheel.js * Fixed a bug in the strokecount reinitialization Fixed a bug in the stroke count reinitialization (great find by @Abasz), which was inconsistent with the normal init. * Graphic showing systematic error File showing systematic (periodic) deviations in the magnet placement * Added an image of misaligned magnets * Removed non-working code Removed the trimmedMedian function, as its design contains fundamental mathematical errors. The initially proposed way is the only good way of doing it. * Added observation about noise overfitting Added a note about Noise overfitting, based on the Concept2 and the observation of Gordon_Shumway (see laberning#87) * Added the pigpio library Added the pigpio library * Added an explanation into the GPIO settings Added an explanation of the GPIO settings * Added the settings for the new GPIO library Added the settings to configure the new GPIO library * Replaced onoff with pigpio, added debounce param. Replaced the onoff library (goodbye my old loyal friend) with pigpio, to facilitate much more accurate timing calculations and add more flexibility to the flank detection (including debounce) * Replaced onoff with pigpio Replaced onoff with pigpio library * Replaced onoff with pigpio Replaced onoff with pigpio * Install and config pigpio dependency Added the installation of the pigpio C-library and setting the key parameters. * Added a bit clarification about the setup guide Added a bit better description about the setup guides. * Removed and added spaces.... * Remove an unnecessary space.... * Resolved open design issue in noise filter In the previous design, we were uncertain that the noise filter would handle all cases. With pigpio, we are quite certain it will handle all cases internally, and here we detect any deviations. Please note, startup-behaviour after a cold or warm (re)start still has to be handled as time since the privious pulse tends to be in the minutes, throwing off all timekeeping functions. * Removed obsolete variables Removed two obsolete variables: lastKnownGoodDatapoint and numberOfErrorsAllowed * Removed obsolete variable Removed the now defunct variable numberOfErrorsAllowed * Changes in startup and config Due to changes in the gpio library, the startup behaviour of flywheel.js has changes, as have the Concept2 configuration settings. These changes modify the results of the testcase accordingly. * Updated testcases after update pigpio Due to changes in the gpio library, the startup behaviour of flywheel.js has changed, as have the Concept2 configuration settings. These changes modify the results of the testcase accordingly. * Update testcase to accomodate lib change Update testcase to accomodate lib change * Update due to lib modification Update due to lib modification * Update Rower.test.js * Update Rower.test.js * Update Rower.test.js * Update Rower.test.js * Update due to gpio library change * Update due to removal obsolete parameter Update due to the removal of numberOfErrorsAllowed * Removed obsolute parameter Removed the numberOfErrorsAllowed parameter * Performance improvement A small performance improvement: by moving the evaluation of the flywheel.isPowered(), flywheel.isUnpowered() and flywheel.isDwelling() backwards, we move these potentially expensive operations further in the evaluation. As Javascript is using lazy evaluation, it won't be called unless all other conditions are true. Especially flywheel.isDwelling() checks if the entire flank is above maximumTimeBetweenImpulses, which is expensive (flywheel.isPowered() and flywheel.isUnpowered() are actually quite simple, but are moved along to maintain symmetry). * Fixed a missing bracket Fixed a missing bracket * Allignment with main branch (64Bit support) Alignment with the improvement of the install script of the main branch to support the 64 bit version of the Lite kernel. * removes some obsolete variables * adds a missing async statement Signed-off-by: Lars Berning <[email protected]> * pin dependency Signed-off-by: Lars Berning <[email protected]> * default setting should not fiddle with nice levels of os, otherwise we require root permissions by default Signed-off-by: Lars Berning <[email protected]> * prettyfies a log message Signed-off-by: Lars Berning <[email protected]> * unifies VO2max namings Signed-off-by: Lars Berning <[email protected]> * Introduced appPriority as setting Added the appPriority setting, to allow a clear entry to set the priority for the main process. * Removed some obscure priority behaviour Removed the obscure approach of using the gpio-priority to set the main app priority. * Added a distinction between stop and pause Based on a bugreport by @carlito1979 about the pause behaviour, added a distinction between the stopMoving() and pauseMoving() command, as a pause would lead to an undesired stop state. * Added a distinction between stop and pause Based on a bugreport by @carlito1979 about the pause behaviour, added a distinction between the stopMoving() and pauseMoving() command, as a pause would lead to an undesired stop state. * Improved description of the various states Based on bugs reported, and some internal soul searching, improved the description of states. * Removed a trailing space * Added profiles, put them in alphabetic order Added the Force USA R3 Air Rower, and put the rowers in alphabetic order * Removal of surplus comma * Improvement of pause-behaviour Improvement of the pause behaviour based on feedback from @carlito1979 * Improved RestingHR behaviour, clarifications Improvement of the resting HR functionality, it will now also trigger when rowing is paused Clarification of "Stopped" sessionState behavior: this is a less than obvious but deliberate approach. * Change to const postExerciseHR Due to Lint error, changed postExerciseHR from "let" to "const" * Create and advertising data builder Since Bleno library does not provide a way to set/tweak appearance data implement a builder that enables to add name and appearance information to the advertisement data. * Expose time of last completed stroke For the CPS and CSC BLE Services the time stamp of the last stoke event is necessary. This is available in the Rower as `drivePhaseStartTime` so expose it * Add 32bit capability to BufferBuilder Since the wheel revolution count in the CSC and CPS measurement service is a 32bin UINT extend the BufferBuilder to be able to handle it. * Add static data and functions used in CSCS and CPS Implement device information service (necessary for garmin to recognise the device as sensor) a wrapper around static read characteristics to ease creation. * Add cycling speed and cadence profile Add necessary services and related characteristics. Implement CscPeripheral interface to be able to properly communicate to the PeripheralManager. * Add cycling power profile Add necessary services and related characteristics. Implement CpsPeripheral interface to be able to properly communicate to the PeripheralManager. * Wire up PeripheralManager with the new profiles Add the new CSCP and CPP profiles to the PeripheralManager. Update comments in settings and front end web view with the new profiles * Add new Rower Profile Zoco Body Fit Add a rowing profile for a Concept2 clone air rower * Update FTMS profile with new services Add device information service to the FTMS peripheral and move to the new advertising data builder. * Improvement of startup behaviour Modification of the startup behaviour: now unrealistic values (i.e. above maximumTimeBetweenImpulses) are ignored. The rower will only start when all values in the flank are above the minmumSpeed (i.e. all currentDt's are below maximumTimeBetweenImpulses). * Improvement of startup behaviour Modification of the startup behaviour: now unrealistic values (i.e. above maximumTimeBetweenImpulses) are ignored. The rower will only start when all values in the flank are above the minmumSpeed (i.e. all currentDt's are below maximumTimeBetweenImpulses). * Improvement of startup behaviour Modification of the startup behaviour: now unrealistic values (i.e. above maximumTimeBetweenImpulses) are ignored. The rower will only start when all values in the flank are above the minmumSpeed (i.e. all currentDt's are below maximumTimeBetweenImpulses). * Improvement of startup behaviour Modification of the startup behaviour: now unrealistic values (i.e. above maximumTimeBetweenImpulses) are ignored. The rower will only start when all values in the flank are above the minmumSpeed (i.e. all currentDt's are below maximumTimeBetweenImpulses). * Finalize Generic Air Rower profile * Change name of displayed name of new profiles * Rename lastStrokeTime to driveLastStartTime * Fixed some small bugs Fixed * the missing pause command * a small typo in the totalmovingtime * Fixed a small bug Fixed a small bug * Improvement of browser performence Improved some startup settings to increase browser performance (disables some security settings as they are not needed at a local system). Please note these settings are NOT RECOMENDED for general purpose browsers, but as we run on a localhost machine only, showing our GUI, this is quite acceptable. * Update due to increased BLE profiles Thanks to @Abasz, we can now use BLE Cycling Power Profile and the Cycling Speed and Cadence Profile. * Added Abasz to the list of great contributors * More expanded explanation of BLE profiles * Update of the testfile to align with new GPIO conf Update of the testfile to reflect the new GPIO-module * Introduction of supporting Linear TS Introduction of the Linear Theil-Senn algorithm to improve the Quadratic Theil-Senn algorithm * Improvement of the Quad. Theil-Senn Algorithm Improvement of the accuracy and robustness of the Quadratic Theil-Senn algorithm, as well as improving (reducing) the CPU load. It employs the Linear Theil-Senn Algorithm to calculate the optimal B and C for the selected A (instead of a marginally related B and C in the older algoithm). * Update of the stroke detection algorithm Update of the stroke detection algorithm: as the Qudratic Theil-Senn algorithm is more robust, we can rely on the sole combination of dragslope being gone and seeing torque. * Updated to reflect more accurate algorithm Updated the testscript to reflect the much more accurate and robust algorithm. * Updated to reflect more improved algorithms Updated test scripts to reflect the improvements in the Quadratic Theil-Senn algorithm and the Flywheel stroke detection algorithm. * Updated to reflect more improved algorithms Updated test scripts to reflect the improvements in the Quadratic Theil-Senn algorithm and the Flywheel stroke detection algorithm. * Removed dead code Removal of dead code * Improvement of Code quality * Improvement of code quality Improvement of code quality * Added space to survive Lint Added space to survive Lint * Addition of the isAboveMinimumSpeed function Addition of the isAboveMinimumSpeed function * Removed unused variable * Add modification for new startup behaviour Add modification for new startup behaviour * Update to fix regression issues Update to fix regression issues * Update Rower.test.js * Update of the C2 testcase Update of the C2 testcase due to GPIO update * Update of the C2 testcase due to GPIO changes Update of the C2 testcase due to changing the testdata due to GPIO changes * fixes some review findings Signed-off-by: Lars Berning <[email protected]> Co-authored-by: Lars Berning <[email protected]> Co-authored-by: Abász <> Co-authored-by: Abasz <[email protected]>
Fixed a bug in the tick rollover (which seems to happen every 60 minutes of processtime).
Bugfix: Fixed a bug where a new year or month would crash the workoutrecorder due to a missing data directory Several improvements: * Added possibility of only creating RowingData file * Improved HRR data when restarting a workout
A bit more defensive programming as one bad (NaN) measurement of CurrentDt will kill all metrics for the entire session.
To add insult to injury: by a typo in the error logging function, the app would crash if there was a recoverable error
Addittion of the interval type, as a preperation of the more complex workout schedules
Replaced the horrible ested If...then....else with a much more readable case statement to implement the Concept 2 table
Added a lot of sanity checks to the loading of the config.
Improvement of code quality
Fixed Lint errors
Removed sanity checks in RowingStatistics, as the configmanager centralises the parameter check
Removed sanity checks in RowingStatistics, as the configmanager centralises the parameter check
Added the ability to automatically fix forgotten or implausible settings when possible.
Added logic for more complex workouts: RowingStatistics will handle this completely independently
Update to facilitate more complex workouts. RowingStatistics will now manage the intervals completely independently and alert server.js only when the session has ended.
Added Intervals.icu to the list of compatible users of tcx files.
Updates to fix review comments
Added workout management to the role of RowingStatistics
Fixed Line error
Checked that the sanity check would fix all mew variables, allowing a 0.8.4 config file to be used with V1Beta without any crashes or issues.
Added a restart limit to the service definition to prevent an infinite bootloop
Added a restart limit to the service definition to prevent an infinite bootloop
On hindsight, the approach of letting the FSM handle all state transitions explicitly is much clearer
On hindsight, the approach of letting the FSM handle all state transitions explicitly is much clearer
Added a paragraph about handle-based sensors.
Added reason to modify hardware setup in documentation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
An improvement of the dragfactor calculation by using Linear Theil-Sen estimator to estimate it, which makes the drag calculation more robust against outliers (flanks) and thus more stable.