Update MTVRP #176

FeiLiu36 · 2024-05-13T10:00:51Z

Changes

Avoid checking the time window of the last node to the depot if it is an open route
Set the max time window to 4.6 instead of inf
Create MTVRPContext and MTVRPInitEmbedding

fedebotu

Awesome! Added some comments~

fedebotu · 2024-05-13T11:09:15Z

rl4co/models/nn/env_embeddings/init.py

+        #durations = td["durations"][..., 1:]
+        time_windows = td["time_windows"][..., 1:, :]
+        # embeddings
+        demands =  td["demand_linehaul"][..., None] - td["demand_backhaul"][..., None]


It makes sense; basically, if it's "-", the model will understand it is a backhaul. I was thinking about having a flag, but this is also good

A flag is also a good idea!

fedebotu · 2024-05-13T11:14:09Z

rl4co/models/nn/env_embeddings/context.py

+
+        capacity = super()._state_embedding(embeddings, td)
+        current_time = td["current_time"]
+        current_length = td["current_route_length"]


How does the model understand whether there is a limit?
In case there is no limit (say CVRP), then it will be the same as having VRPL, since the model does not know whether the constraint will be enforced or not

You are right. The route length in the state_emdding should be the rest length, i.e., length limit-current length, instead of current length... It seems to be a mistake...

fedebotu · 2024-05-13T11:27:22Z

rl4co/envs/routing/mtvrp/generator.py

@@ -256,7 +256,8 @@ def _default_open(td, remove):
    @staticmethod
    def _default_time_window(td, remove):
        default_tw = torch.zeros_like(td["time_windows"])
-        default_tw[..., 1] = float("inf")
+        #default_tw[..., 1] = float("inf")
+        default_tw[..., 1] = 4.6 # max tw


Doesn't this influence the solution? If default time window is 4.6, the problem should not be a CVRP but a "relaxed" VRPTW.

The reason why I thought having "inf" is because it can generalize to any scale - for the embedding, this can be set as:

time_windows = torch.nan_to_num(td["time_windows"][..., 1:, :], posinf=0.0)

So it shouldn't influence the calculation as describe in Section 4.1 (Attribute composition) in your paper.
What do you think?

I agree that the default should be float("inf"), as T=4.6 should only apply as default value to the environments where we actually want to model time windows!

OK, that makes sense!

fedebotu · 2024-05-13T11:31:04Z

rl4co/envs/routing/mtvrp/env.py

@@ -349,9 +349,14 @@ def check_solution_validity(td: TensorDict, actions: torch.Tensor):
            curr_time = torch.max(
                curr_time + dist, gather_by_index(td["time_windows"], next_node)[..., 0]
            )
+
+            new_shape = curr_time.size()
+            skip_open_end = td["open_route"].view(*new_shape) & (next_node == 0).view(*new_shape)


Makes sense, good catch.
Anyways, I recommend setting check_solution to False when training; otherwise, the solution will be checked at each step and it can be a bit slow. I will add a warning

Actually, I don't think this is necessary. Since skip_open_end will only be true if next_node == 0, and since the depot has the highest time window end, curr_time <= gather_by_index(td["time_windows"], next_node)[..., 1]) should always be True except when curr_time is very close to the max time already and then the duration in that last node is long enough to go over the time limit - is that something we want to allow?

I agree with ngastzepeda as the last node in the route should also satisfy the time window constraints (allow it back it depot even when it is OVRP). However, I find some outliners when training the MTVRP (i.e., the time window of the last node of OVRP route may exceed the max time window). I do not yet know the exact reason, instance generation, or masking procedure.

I will have a check.

cbhua

Good job! 🚀

cbhua · 2024-05-13T11:07:53Z

rl4co/envs/routing/mtvrp/env.py

@@ -281,7 +281,7 @@ def get_action_mask(td: TensorDict) -> torch.Tensor:
            & ~exceeds_dist_limit
            & ~td["visited"]
        )
-
+        #print(can_visit)


[Minor] Debugging comments could be removed.

cbhua · 2024-05-13T14:15:30Z

rl4co/envs/routing/mtvrp/env.py

-        has_tw = (td["time_windows"][:, :, 1] != float("inf")).any(-1)
+        has_tw = (td["time_windows"][:, :, 1] != 4.6).any(-1)


Same as the discussion with the _default_time_window(), Changing this bound in the setting will need to modify this part. Any reason for this hardcode?

Avoid numerical issues during training as it will go through embedding, but I will have a check, the inf would be more general.

ngastzepeda

I agree with @fedebotu and @cbhua that we shouldn't hardcode the duration of 4.6 here!

ngastzepeda · 2024-05-13T11:55:12Z

rl4co/envs/routing/mtvrp/generator.py

@@ -256,7 +256,8 @@ def _default_open(td, remove):
    @staticmethod
    def _default_time_window(td, remove):
        default_tw = torch.zeros_like(td["time_windows"])
-        default_tw[..., 1] = float("inf")
+        #default_tw[..., 1] = float("inf")
+        default_tw[..., 1] = 4.6 # max tw


I agree that the default should be float("inf"), as T=4.6 should only apply as default value to the environments where we actually want to model time windows!

ngastzepeda · 2024-05-13T11:55:40Z

rl4co/models/nn/env_embeddings/context.py

+    """Context embedding for the Capacitated Vehicle Routing Problem (CVRP).
+    Project the following to the embedding space:
+        - current node embedding
+        - remaining capacity (vehicle_capacity - used_capacity)
+    """


Since this also includes backhauls, we should mention this in the docs.

ngastzepeda · 2024-05-13T15:43:38Z

rl4co/models/nn/env_embeddings/init.py

+
+    def forward(self, td):
+        depot, cities = td["locs"][:, :1, :], td["locs"][:, 1:, :]
+        #durations = td["durations"][..., 1:]


Why are the durations not included?

FeiLiu36 added 8 commits April 9, 2024 21:23

four new routing envs

d742f5b

update

1ac4516

Merge branch 'main' of https://github.com/FeiLiu36/rl4co

096fc8e

update mtvrp

044c789

update mtvrp

99af6f9

update mtvrp

991c516

Update init.py

1fd55d6

Update env.py

4e02795

cbhua requested review from fedebotu, LTluttmann and ngastzepeda and removed request for LTluttmann May 13, 2024 11:03

fedebotu reviewed May 13, 2024

View reviewed changes

cbhua reviewed May 13, 2024

View reviewed changes

ngastzepeda requested changes May 13, 2024

View reviewed changes

fedebotu added this to the 0.5.0 milestone Jun 19, 2024

fedebotu removed this from the 0.5.0 milestone Sep 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update MTVRP #176

Update MTVRP #176

FeiLiu36 commented May 13, 2024

fedebotu left a comment

fedebotu May 13, 2024

FeiLiu36 May 14, 2024

fedebotu May 13, 2024

FeiLiu36 May 14, 2024

fedebotu May 13, 2024

ngastzepeda May 13, 2024

FeiLiu36 May 14, 2024

fedebotu May 13, 2024

ngastzepeda May 13, 2024

FeiLiu36 May 14, 2024

FeiLiu36 May 14, 2024

cbhua left a comment

cbhua May 13, 2024

cbhua May 13, 2024

FeiLiu36 May 14, 2024

ngastzepeda left a comment

ngastzepeda May 13, 2024

ngastzepeda May 13, 2024

ngastzepeda May 13, 2024

		has_tw = (td["time_windows"][:, :, 1] != float("inf")).any(-1)
		has_tw = (td["time_windows"][:, :, 1] != 4.6).any(-1)

Update MTVRP #176

Are you sure you want to change the base?

Update MTVRP #176

Conversation

FeiLiu36 commented May 13, 2024

Changes

fedebotu left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cbhua left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ngastzepeda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment