dgl.heterograph

[maryamag85]

I need to create a hetrograph on a data with over 4M relationship. When I am running graph = dgl.heterograph({...}) in a loop over my relations, it works fine to 50K or so samples after that it starts to write null and nothing meaningful comes out of it. I guess that is a memory issue. Currently I am running on 64G Ram. How can I get over this issue?

[classicsong]

Do you mean we have want to use dgl.heterograph({...}) when the {...} is a dict of 4M objects?
How large is the graph?

[maryamag85]

Yes, the graph has around 4.6M edge types and when I am filling the graph dictionary it works fine up until it gets over 800K or so samples and over that it only keeps empty dictionary. I understand that it might be the limit of the python dictionary on a 64G ram system but I am wondering what can be the solution to get over this? How to input graphs with huge relation types into DGL?