How can I make prediction faster with only training data?

[gurokeretcha]

Hello,

When I train a model such as General models, I use full_sort_topk funtion to make top k prediction. However this function takes valid_data as an argument. My first question is, how I can make prediction If I do not have any validation set and I have only training dataset?

My solution is to create a Interaction object of all interactions of (user, item) and make prediction with model_2.predict(), however I am facing two issue.

1. Memory issue.
   I have 1m users and 20k items so all combination is 1m*20k which takes too much run.
2. Time complexity
   It takes too much time when I make prediction of each user item interaction.

Interesting fact is that full_sort_topk executes too fast (up to 10 min) on the whole dataset to make topk prediction for all the users, however model.predict takes too much time (2days) for all users.

Thank you in advance, for your help.

[Ethan-TZ]

@gurokeretcha Hello, thanks for your attention to RecBole!
You can only create an interaction object with all users, and utilize model.full_sort_predict() function, which will give the scores of all items for a certain user. Then you can select topk item based on it.

[gurokeretcha]

@chenyuwuxin Thank you for you quick reply.
I used model.full_sort_predict() and it returns scores of the list with the size of total_items+1. I am wondering what is the mapping of the scores and its internal item_ids? so which probability represents which item_id?

[Ethan-TZ]

@gurokeretcha Hi! '0' is used as the padding item, which leads to the size of scores list is total_items+1. The column indexes of topk items are their internal item_ids.