FQE support .d3 file model?

[CastleImitation]

Dear Sir,
Is our FQE evaluation method support .d3 file model?
When I'm going to run my trained DDQN .d3 file model on FQE, an error occurred saying "RuntimeError: Invalid magic number; corrupt file?"
It seems that FQE does not support .d3 file model.
Or, should I save the model to .pt file?

Looking forward your kind reply!

[takuseno]

@CastleImitation Hi, thanks for the issue. FQE does support .d3 file as well. Could you share the minimal code that I can reproduce your issue?

[CastleImitation]

Dear Takuseno,
Many thanks for you kind reply.

Since I'm not sure whether FQE can support d3. I have save my model with .pt.
by fqe.save_model(f"{label}fqe.pt") after read the official documents from https://d3rlpy.readthedocs.io/en/v2.3.0.

haha

[CastleImitation]

Dear Takuseno Thanks for your kind reply. I have checked the official document of d3rlpy and found the way to save the model as .pt. Regards! | | Felix Li | | ***@***.*** | ---- Replied Message ---- | From | Takuma ***@***.***> | | Date | 3/3/2024 19:42 | | To | ***@***.***> | | Cc | ***@***.***>, ***@***.***> | | Subject | Re: [takuseno/d3rlpy] FQE support .d3 file model? (Issue #381) | @CastleImitation Hi, thanks for the issue. FQE does support .d3 file as well. Could you share the minimal code that I can reproduce your issue? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[CastleImitation]

Dear Takuseno, By the way, could I have one additional question? I have trained my model. But when I tried to load my model and build the test dataset for the model, it reported error : raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for ModuleList: size mismatch for 0._fc.weight: copying a param with shape torch.Size([343, 256]) from checkpoint, the shape in current model is torch.Size([340, 256]). size mismatch for 0._fc.bias: copying a param with shape torch.Size([343]) from checkpoint, the shape in current model is torch.Size([340]). I also tried to build the trained dataset for the trained model, but it didn't report any error. Here is my code config_BCQ = DiscreteBCQConfig() config_SAC = DiscreteSACConfig() config_DDQN = DoubleDQNConfig() # model_BCQ = DiscreteBCQ(algo=self.policy,config=fqe_config,device="cuda:0") device = "cuda" model_BCQ = config_BCQ.create(device=device) model_SAC = config_SAC.create(device=device) model_DDQN = config_DDQN.create(device=device) POLICIES = { 'BCQ':[ [f'{FINAL_POLICIES_PATH}/raw_intermediate/BCQ/run_0/DiscreteBCQ_train.pt',model_BCQ], [f'{FINAL_POLICIES_PATH}/raw_intermediate/BCQ/run_1/DiscreteBCQ_train.pt',model_BCQ], [f'{FINAL_POLICIES_PATH}/raw_intermediate/BCQ/run_2/DiscreteBCQ_train.pt',model_BCQ], [f'{FINAL_POLICIES_PATH}/raw_intermediate/BCQ/run_3/DiscreteBCQ_train.pt',model_BCQ], [f'{FINAL_POLICIES_PATH}/raw_intermediate/BCQ/run_4/DiscreteBCQ_train.pt',model_BCQ] ], 'SAC': [ [f'{FINAL_POLICIES_PATH}/raw_intermediate/SAC/run_0/DiscreteSAC_train.pt', model_SAC], [f'{FINAL_POLICIES_PATH}/raw_intermediate/SAC/run_1/DiscreteSAC_train.pt', model_SAC], [f'{FINAL_POLICIES_PATH}/raw_intermediate/SAC/run_2/DiscreteSAC_train.pt', model_SAC], [f'{FINAL_POLICIES_PATH}/raw_intermediate/SAC/run_3/DiscreteSAC_train.pt', model_SAC], [f'{FINAL_POLICIES_PATH}/raw_intermediate/SAC/run_4/DiscreteSAC_train.pt', model_SAC] ], 'DDQN': [ [f'{FINAL_POLICIES_PATH}/raw_intermediate/DQN/run_0/DoubleDQN_train.pt', model_DDQN], [f'{FINAL_POLICIES_PATH}/raw_intermediate/DQN/run_1/DoubleDQN_train.pt', model_DDQN], [f'{FINAL_POLICIES_PATH}/raw_intermediate/DQN/run_2/DoubleDQN_train.pt', model_DDQN], [f'{FINAL_POLICIES_PATH}/raw_intermediate/DQN/run_3/DoubleDQN_train.pt', model_DDQN], [f'{FINAL_POLICIES_PATH}/raw_intermediate/DQN/run_3/DoubleDQN_train.pt', model_DDQN] ] } for P in POLICIES.keys(): # for i in range(len(POLICIES[P])): # 5个 data = load_data(states='raw', rewards='intermediate',index_of_split=i)[1] # 测试集数据 循环到最后一个算法的最后一轮时，data是最后一个数据集的数据 POLICIES[P][i][1].build_with_dataset(data) # 把类添加到字典的时候就已经实例化了，即便没有显式地赋予一个变量名 POLICIES[P][i][1].load_model(POLICIES[P][i][0]) data = load_data(states='raw', rewards='intermediate',index_of_split=i)[1] here is my dataset. [1] indicates the test data. In my experiment, there are totally 7*7*7=343 action possibilities. By the way, since you have willingly helped me a lot, would you mind if I add your name in the authors when I issue my paper? Awaiting your kind reply. | | Felix Li | | ***@***.*** |