design review for Jan 26 #856
Replies: 15 comments 11 replies
-
|
Here is the readme about plantUML |
Beta Was this translation helpful? Give feedback.
-
|
matsPlot sequence
process data sequence
|
Beta Was this translation helpful? Give feedback.
-
|
Keeping a few notes here: |
Beta Was this translation helpful? Give feedback.
-
|
New scPlot Sequence
|
Beta Was this translation helpful? Give feedback.
-
|
Rather than putting images in the discussion it is much better to go to |
Beta Was this translation helpful? Give feedback.
-
|
This is a potential list of issues, some of these will need subtasks. I am not creating the actual task list until after we review it.
scMatsPlot:
|
Beta Was this translation helpful? Give feedback.
-
|
Given that we want to persist and share these scorecards between users, it seems like the scorecard-related stuff should go in its own long-lived mongo database or in couchbase. We'll need to back these settings/configurations up in case something happens to the database. (Especially given that the current Mongo database has to be regularly purged) Having them in a long-lived database would also make it easier to update the scorecards in the background so that everything is up-to-date when users view the scorecard. |
Beta Was this translation helpful? Give feedback.
-
|
Absolutely. The scorecards themselves (the thing that holds the params and the data) is a document in the couchbase. Just retrieve the records i.e. "select * from vxdata._default.SCORECARD" (This only works against adb-cb2.gsd.esrl.noaa.gov or adb-cb3... or adb-cb4....) because the SCORECARDS only live on the cluster. The two entities that I don't yet have a good feel for are the settings (which won't be specific to scorecard) and the user account data. User account passwords are encrypted and I am wondering if I save the account data documents and reload them on another server do they still work? Will the bcrypt decryption still work. I doubt it. So I am wondering if we have to use login.gov and offload the distribution of password problem to them. |
Beta Was this translation helpful? Give feedback.
-
|
An example of a significance calculation. It demonstrates the complexity of the calculation. I haven't verified this, I just borrowed it from some code. // use a student's t-test to see if a point on a contour diff is statistically significant
// see https://en.wikipedia.org/wiki/Student's_t-test
// The t-test's most common application is to test whether the means of two populations are different.
// x1,x2, are means of two populations, s1,s2, are standard deviations, and n1, and n2 are the number of individuals in the populations.
// We would possibly offload the means calculations to the database query and optimize the query with indexing.
function (x1, x2, s1, s2, n1, n2, sigType) {
const t = Math.abs(x1 - x2) / Math.sqrt((Math.pow(s1, 2) / n1) + (Math.pow(s2, 2) / n2));
const df = Math.pow(((Math.pow(s1, 2) / n1) + (Math.pow(s2, 2) / n2)), 2) / ((1 / (n1 - 1)) * Math.pow((Math.pow(s1, 2) / n1), 2) + (1 / (n2 - 1)) * Math.pow((Math.pow(s2, 2) / n2), 2));
return isStudentTTestValueSignificant(t, df, sigType);
};
const isStudentTTestValueSignificant = function (t, df, sigType) {
const sigThreshs = {
1: 12.706,
2: 4.303,
3: 3.182,
4: 2.776,
5: 2.571,
6: 2.477,
7: 2.365,
8: 2.306,
9: 2.262,
10: 2.228,
11: 2.201,
12: 2.179,
13: 2.160,
14: 2.145,
15: 2.131,
16: 2.120,
17: 2.110,
18: 2.101,
19: 2.093,
20: 2.086,
21: 2.080,
22: 2.074,
23: 2.069,
24: 2.064,
25: 2.060,
26: 2.056,
27: 2.052,
28: 2.048,
29: 2.043,
30: 2.042
};
var sigThresh;
if (sigType === 'standard') {
if (df <= 30) {
sigThresh = sigThreshs[df];
} else if (df <= 40) {
sigThresh = 2.021;
} else if (df <= 60) {
sigThresh = 2.000
} else if (df <= 120) {
sigThresh = 1.980
} else {
sigThresh = 1.960
}
} else {
sigThresh = 1.960
}
return t > sigThresh
}; |
Beta Was this translation helpful? Give feedback.
-
|
Comment: Make text cell backgrounds white. |
Beta Was this translation helpful? Give feedback.
-
|
sure |
Beta Was this translation helpful? Give feedback.
-
|
Comment: Ceiling and visibility do need separate metadata, so instead of changing that there, we need a dictionary to interpret app names or something. |
Beta Was this translation helpful? Give feedback.
-
|
This is a sticky wicket, but it must be played. |
Beta Was this translation helpful? Give feedback.
-
AWS Lambda vs an on-prem systemFollow up from the 1/26 meeting Given that this has to be working in some fashion by April, we decided that AWS Lambdas are probably not possible for the initial DDRF. However, if we were to use Lambdas we would most likely want to make heavy use of a message queue. It'd be good to keep in mind how we could refactor the backend's processes to fit that model. |
Beta Was this translation helpful? Give feedback.
-
|
I like the idea of a message queue. I think that we have to consider that the director and its workers might not be colocated. |
Beta Was this translation helpful? Give feedback.
-
Project ScopeFollow up from the 1/26 meeting Matt had a great question: Given that this needs to be done by April - are there ways to reduce the scope or to make the implementation easier? E.g. - can we only process certain variables or can we limit the flexibility of the application (by say only processing set intervals) to make our lives easier? |
Beta Was this translation helpful? Give feedback.
-
|
I spent quite a bit of time thinking about this. The primary functionality is repeatable, I mean if it works for one variable it will work on 10. So limiting the datasources, regions, variables, or levels may not help. Limiting the apps might help. And we may have to say something like "if you make a huge scorecard it might take awhile". |
Beta Was this translation helpful? Give feedback.
-
Scorecard updatingFollow up from the 1/26 meeting It was unclear to me how scorecard updates were being scheduled. Is this something that will be triggered on a schedule by the backend? Or will these calculations be triggered when a user loads the scorecard page? Or something else? The first seems preferable unless someone wants a one-off scorecard. |
Beta Was this translation helpful? Give feedback.
-
|
Do you mean updates from the CalculateService? That happens when the director process gets something from the workers worth updating. By worth updating I mean when enough has been done to be useful but not so often that it hamstrings the database. The update method is likely a server side route that updates the scorecard in couchbase and the mongo record, If it exists. |
Beta Was this translation helpful? Give feedback.
-
SchedulingFollow up from the 1/26 meeting Scheduled/batched operations tend to happen at the same time. E.g. - cron jobs tend to run at the top of the hour, etc... When the scheduler makes a large number of requests how will we ensure the requests aren't lost? This seems like a great place for a message queue. Otherwise, the backend could queue up the requests to hand off to the workers. (Until it ran out of memory) If we didn't want to do a queue, we could start by having multiple backends running in the lab's Kubernetes cluster and using a Kubernetes service/load balancer to distribute requests between them. |
Beta Was this translation helpful? Give feedback.
-
|
Correct, except that the director has to know it's resource limits. For the ingest code that limit is number of threads. I suspect that for this it won't be memory, it will also be the number of allowed workers that can simultaneously run. Each worker does this...
|
Beta Was this translation helpful? Give feedback.
-
Persisting SchedulesFollow up from the 1/26 meeting Where does the NPM scheduler persist its schedules? (I.e. - what happens when we destroy and recreate the app and mongo containers) |
Beta Was this translation helpful? Give feedback.
-
|
I don't remember where Bree has its persistence. I think there are options for it. I am thinking it is file based, actually. I know it persists over restarts. |
Beta Was this translation helpful? Give feedback.
-
|
Gotcha. If there are options to persist it out to couchbase or an external store somewhere, that'd be helpful. GSL's Kubernetes cluster has PVC's but I'd be a little less comfortable relying on them for stuff that can't be easily recreated. (Like our current usage of Mongo) |
Beta Was this translation helpful? Give feedback.
-
|
Anything that has text? It’s hard to read on the textured background.
… On Jan 26, 2023, at 6:17 PM, randytpierce ***@***.***> wrote:
sure
Just the cell background?
—
Reply to this email directly, view it on GitHub <#856 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHBWGZPJS6EIFQJO25WXXSDWUMAXLANCNFSM6AAAAAAUG4QPOM>.
You are receiving this because you commented.
|
Beta Was this translation helpful? Give feedback.
-
|
I defer to your judgement. Feel free to spend as much time as you like on that stylesheet! |
Beta Was this translation helpful? Give feedback.
-
The primary goal of this review is to discuss the approaches for a few things.
The mechanism for the scorecard display to open a new window to a specific mats application and a specific preconfigured plot that corresponds to the significance value in the scorecard (table cell) that the user clicked on.
The mechanism for the processScorecard backend matsMethod to calculate all the required significance values and populate the scorecard couchbase data record. The assumption is that this will be a very parallel worker set.
There are preliminary UML sequence diagrams in the github scorecard_display branch in the apps/scorecard/model folder. There is a README.md that explains how to use plantUml (which was used to generate the sequence diagrams).
Beta Was this translation helpful? Give feedback.
All reactions