Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does the implementation of sample overlap correction in METAL need to be amended? #43

Open
freeseek opened this issue Jul 23, 2024 · 0 comments

Comments

@freeseek
Copy link

I was trying to understand how sample overlap correction is implemented in METAL, as described here. The full method description from Sebanti Sengupta is detailed here and here and it seems to match the implementation from Daniel Taliun here:

if (studyOverlap) {
    double n12 = 0.0;
    for (int i = 0; i < backupStatistics.Length(); ++i) {
        n12 = rho[((int) (samples[i] + backupSamples[i])) / studyOverlapStep] * sqrt(weights[i] * backupWeights[i]);
        samples[i] += backupSamples[i] - n12;
        weights[i] += backupWeights[i] + 2 * n12;
        statistics[i] += backupStatistics[i];
    }
}

Following the notation from Sebanti Sengupta, suppose we have two studies with n_1 = 400 and n_2 = 900 samples with n_{12} = 100 overlapping samples and define

w_1 = sqrt(n_1) = 20
w_2 = sqrt(n_2) = 30
r_{12} = n_{12} / sqrt(n_1 * n_2) = 1/6

Then Ω as the covariance matrix for the β's or the Z/sqrt(N)'s, as defined in Lin and Sullivan 2009, would be equal to

1/w_1^2           r_{12}/(w_1*w_2)
r_{12}/(w_1*w_2)           1/w_2^2

The optimal weights for the meta-analysis, following Lin and Sullivan 2009, would be

3/11 and 8/11

While following the formulas from Sebanti Sengupta and Daniel Taliun they would be

4/13 and 9/13

The effective sample size, following Lin and Sullivan 2009, would be

(n_1 + n_2 - 2n_{12}) / (1 - r_{12}^2) = (400 + 900 - 200) / (35 / 36) = 7920 / 7 ~ 1130

While following the formulas from Sebanti Sengupta and Daniel Taliun it would be

n_1 + n_2 - n_{12} = 400 + 900 - 100 = 1200

It seems to me as if the mathematics behind METAL's meta analysis with sample overlap correction is incorrect and, while claiming to follow the optimal weights in Lin and Sullivan 2009, it does something other than that. In particular, while METAL does not perform sample overlap correction for inverse variance based meta-analysis, it does not seem to me that the sample size based meta-analysis implementation computes a weighted average of the Z/sqrt(N)'s statistics. Should METAL be avoided when using sample overlap correction?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant