Multimer model scores

[s-kyungyong]

Hi!

Alphafold's multimer model produces iptm scores that are saved within the pickle file as below:

distogram
experimentally_resolved
masked_msa
predicted_aligned_error
predicted_lddt
structure_module
plddt
aligned_confidence_probs
max_predicted_aligned_error
ptm
iptm
ranking_confidence

However, it seems like fastfold's output does not contain this data

msa
pair
single
sm
final_atom_positions
final_atom_mask
final_affine_tensor
lddt_logits
plddt
distogram_logits
masked_msa_logits
experimentally_resolved_logits
tm_logits
predicted_tm_score
aligned_confidence_probs
predicted_aligned_error
max_predicted_aligned_error

Is it possible to get the iptm scores?

Thanks!

[Shenggan]

hello, I add iptm for results in branch add_iptm_in_heads, can you help to check if the value is the same as AlphaFold?

[s-kyungyong]

Hi @Shenggan. Thanks for the update! I used precomputed MSAs for both fastfold and alphafold, so I know the inputs are the same. But I haven't checked whether other parameters are variable at default. There are some variations in the iptm scores. Is this a reasonable amount of differences you are expecting?

Fastfold:
AT1G52380_and_AT5G48810_model_1_multimer.pkl: iptm 0.11; plddt 48.45; ptm 0.32
AT1G52380_and_AT5G48810_model_2_multimer.pkl: iptm 0.09; plddt 45.71; ptm 0.3
AT1G52380_and_AT5G48810_model_3_multimer.pkl: iptm 0.08; plddt 44.54; ptm 0.3
AT1G52380_and_AT5G48810_model_4_multimer.pkl: iptm 0.08; plddt 45.12; ptm 0.29
AT1G52380_and_AT5G48810_model_5_multimer.pkl: iptm 0.08; plddt 43.84; ptm 0.29



Alphafold:
result_model_1_multimer_v3_pred_0.pkl: iptm 0.16; plddt 46.66; ptm 0.3
result_model_2_multimer_v3_pred_0.pkl: iptm 0.16; plddt 47.08; ptm 0.31
result_model_3_multimer_v3_pred_0.pkl: iptm 0.16; plddt 45.26; ptm 0.3
result_model_4_multimer_v3_pred_0.pkl: iptm 0.15; plddt 45.1; ptm 0.29
result_model_5_multimer_v3_pred_0.pkl: iptm 0.12; plddt 45.02; ptm 0.28

[Gy-Lu]

Hi, it seems not so reasonable.
Could you please share your input to help us reproduce it?

[s-kyungyong]

Hi @Gy-Lu! I uploaded the precomputed alignments I used and model 1 outputs from fastfold and alphafold here.

For fastfold, I used:
python /global/scratch/users/skyungyong/Software/FastFold/inference.py --output_dir ./ --model_preset multimer --use_precomputed_alignments Alignments --enable_workflow --inplace --param_path /global/scratch/users/skyungyong/Software/FastFold/data/params/params_model_1_multimer_v3.npz --model_name model_1_multimer AT1G52380_and_AT5G48810.fasta /global/scratch/users/skyungyong/Software/alphafold-multimer-v2.2.2-080922/Database/pdb_mmcif/mmcif_files/

For alphafold, I allowed it to use the precomputed MSAs, without relaxation. For both, top 4 templates were used regardless of their release dates.

Thank you!

[Gy-Lu]

Hi, I have reproduced this problem. And it would be fixed soon.

[s-kyungyong]

Hi!

Is there any update on this? Would it also be possible to compute the iptm score given the pickle file produced during the inference? I have several thousand multimer structures, and it would be a waste to rerun the prediction to get the score.

Thank you!

[Gy-Lu]

Hi, I have made some experiments.
And I found it is the preprocessing led to the difference, seems there is some randomness in preprocessing. Can you rerun alphafold with precomputed aligments for us to see if the result is consistent?

Besides, to my understanding, you want to calculate iptm score for given structures?