-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to pause world simulation #99
Comments
Interesting, the Can you send a short code snippet of the PythonAPI-based pause functionality you are using so we can reproduce it on out end? Have you tried this with an Editor version of the simulator? If so do you have any logs that you can share. |
Packaged version
Which doesn't end up doing anything. |
Okay, I played around with it and got this which seems to do what you want: import carla
import time
client = carla.Client('localhost', 2000)
client.set_timeout(10.0)
world = client.get_world()
settings = world.get_settings()
settings.fixed_delta_seconds = None # Set a variable time-step
settings.synchronous_mode = True # Set a variable time-step
world.apply_settings(settings)
time_to_wait = 10.0
start_time = world.get_snapshot().timestamp.elapsed_seconds
while True:
# Update the CARLA simulation
world.tick()
# Get the current simulation time
current_time = world.get_snapshot().timestamp.elapsed_seconds
print(f"Tick: {current_time:.2f}s", end='\r', flush=True)
# Check if the desired time has passed
if current_time - start_time >= time_to_wait:
# Pause the simulation
# world.wait_for_tick() <-- only need to not call world.tick() in synchronous mode to pause!
print("Simulation paused at time:", current_time)
break
# Resume the simulation after a delay
time_to_resume = 5.0 # Set the delay to resume the simulation in seconds
time.sleep(time_to_resume)
# Continue the simulation
print("Simulation resumed at time:", world.get_snapshot().timestamp.elapsed_seconds)
while True:
# need to re-enable per-frame ticking
world.tick() Couple things of note:
Hope this helps! |
Thanks, this does pause the simulation. Is it also possible to display text on the screen when the simulation is paused to ask a question to the driver? After which the driver has to press a button on the wheel which will resume the simulation? Since carla wasn't made to be driven by a user using a wheel, we aren't sure if this is a Dreyevr question or a carla question. |
First off, what I think will work best is if you use Carla's built-in DebugHelper (from PythonAPI) This has methods such as (this may be sufficient to answer your question, but if not I have some C++ approaches below) There are several methods of displaying text in UE4 but I'm not super familiar with all of them (ex. HUD, dynamic textures, UMG, components, debug strings, etc.). The carla-built-in method is to use their CarlaHUD, which is supposed to draw text on the server side visualization, but this is known to have issues when in VR (ex. it only displays text in one eye). This is why we elected to use the However, there is a lot more coding necessary to add a new text render element to DReyeVR (example) compared to Carla's HUD, which you should be able to access from anywhere in the simulation. Note that we actually have our own DReyeVRHUD which inherits from CarlaHUD and adds methods for us to draw gaze lines and the reticle easily. You will see this HUD active if you run the simulation in flat-screen-mode (no Feel free to take a look at our DReyeVR hud to see how to integrate it into your pipeline. |
If I'm not mistaken, the debug methods add the text in a specific location, and not on the dashboard for the driver to see like the speedometer. This isn't what we are looking for. You said the DReyeVRHUD only works in flat screen mode. How do you then display the speedometer for the users? You mentioned UTextRenderComponent, but in which file did you write the code for this? Or is it supposed to be used in unreal engine? |
Yes, you are correct. The debug methods are PythonAPI-guided to render in a specific location relative to the world, so this is not what we used for our dashboard internals. The DReyeVRHUD is not the speedometer/dash, the HUD is what displays the crosshair reticle for the eye tracking (if you enabled eye tracking). There is a different HUD (flat vs VR) depending on if you are using VR mode or not. The speedometer, along with all the other dashboard text is drawn (as you said) with the text render components which are relative to the EgoVehicle. All the relevant code for this lies in the EgoVehicle.cpp|h because they can be thought of as properties of the vehicle mesh. |
That certainly helps us, thanks a lot. About displaying text while the simulation is paused, you mentioned it isn't straightforward. But what we had in mind was implementing an overlay, much like when you press menu while playing a game to save for example, where the game is paused but you can navigate using your controller to select the option you want. Do you think that's possible to do, and if so how we could go about doing that? |
Gotcha. So in the current implementation I think it might be difficult to render any new elements on the server side because I worry that the Carla pausing might also pause the render thread of the UE4 server, in which case nothing new can be drawn. But this might not be the case! (simple test for this is: pause the simulator as you do, then try to move your head in VR and if new views are rendered then you're golden!) If (under-the-hood) it doesn't pause the underlying server tick (just freezes all elements) then you should be able to draw new dynamic elements such as a pause menu overlay in the server much like how we draw the dashboard elements like the speedometer. You can even choose to fix these to components to be relative to the user's head (camera position) or relative to the vehicle chassis. Building an entire pause menu can certainly be complicated but if you'd just want simple buttons, you might be able to get away with building the entire system by spawning primitives (ex. cubes, cones, spheres, text elements) and manipulating them to form one cohesive unit. Spawning actors dynamically at runtime is also somewhat tedious, but we try to alleviate this by adding our own CustomActor class which is used for a similar purpose (drawing overlays in the world at runtime). You can check them out here. |
I tried what you suggested, but it seems the render thread is also paused. When the simulation is paused, I can only see what was displayed on the screen just before it paused. Also I get the following output, which shows that the simulation time isn't updated (which I think also reinforces this idea):
How difficult would it be to implement a functionality to freeze all elements instead? Or does this functionality already exist perhaps? |
Hmm. This is what I hoped wouldn't be the case. Unfortunately this is then much more complicated since I don't think Carla has any easy way to do this. Since this is new territory I'd suggest also asking the Carla folks to see if they have any recommendations. But to answer your last question I think something that might work for your use case (if all you want is to stop all current actors in their tracks) is to iterate through all the actors in your world and pause them by disabling their tick functionality. If you do this, you'll probably want to look into // this should be in some kind of "pause" function that takes input
bool bIsPaused = true;
for (TActorIterator<AActor> actor_it(GetWorld()); actor_it; actor_it++)
{
actor->SetActorTickEnabled(!bIsPaused);
} As per the But I'm not sure if this will still affect underlying C++ elements such as the traffic manager etc. But its a start. (Also this is entirely untested by me so be prepared for dragons) |
First of all thanks for your help. We will try the way you described, but maybe we won't have to if the following is possible:
The question boils down to this: If the simulation is paused, can we still register the keys pressed by the user on the logitech wheel using PythonAPI? Or is this also blocked because of the blocked server tick? |
Hm unfortunately I think the entire engine seems to be paused with this script (including the EgoVehicle tick which manages the logi inpnuts), meaning you can't quite do what you want to do (at least not in this way). An alternative approach could be to use the pause functionality provided by a controller (such as the DReyeVR controller) or But the Logi-logic might still be doable if you make the underlying pawn that controls the logitech inputs (here) also tickable during pause. This may be doable using this |
Hi, I'm not sure how to implement the solutions you suggested. I am talking specifically about the following:
and this
I think we have to change some cpp files, but I'm not sure which one. Also, how would we invoke the functions from the pythonAPI? I'm guessing we need to add extra methods to existing classes and just invoke them like the standard methods in pythonAPI? But we are very inexperienced in this stuff and don't really know how everything fits together. So if it's not too much trouble, could you point us in the right direction? |
The first step would be to read through the Development documentation and familiarize yourself with the inner-workings of DReyeVR, since this will require a decent number of code changes to the backend (C++ and Python) code. The first code snippet basically provides a means to iterate through all the actors in the scene and "pause" them by disabling their tick functionality, this may suffice to what you want to accomplish because all the dynamic actors in the scene such as vehicles/walkers would be frozen in time as they could not update their game state. Learn more about the UE4 tick system here. The alternative approach is to instead (of manually disabling all the actor ticks from the previous code snippet) just use the UE4 built-in pause functionality, but ensuring that for the actors you care about they have a special property that permits them to continue "ticking" even while the game is paused. The Carla interface between C++ and Python is a bit messy and convoluted, so it would be a good idea to familiarize yourself with how this works as well. While I can't provide all the implementation details you might need since I don't know them, I can point you into directions that we took to get these kinds of new "features" from PythonAPI (client) -> C++ (server). For instance, consider the simple function Hope this helps. |
Hi. Projecting text on the dashboard seems to work. But I was wondering if it's also possible to draw a kind of grey rectangle that is a bit transparant, over which we can then display the text? Right now it's just hanging there in front of the background, which influences the readability. |
Sure, you should be able to just spawn a cube actor or some other simple mesh and apply a slight transparency to it. You can even try to do this using our own CustomActor API (though its a bit heavy-handed). To see how to make a mesh transparent there is lots of information in the UE4 documentation. |
Thanks, I'll look into that. In the mean time, I was trying to get my additional code to add text compatible with your config file but it doesn't seem to be working. I edited the TeslaM3.ini file to include the enable signal and the location of the text, just like how you did for the GearShift and Turnsignals. I also edited the EgoVechicle.cpp file to get the information, however in the log I get: |
Looks like you are building for package mode? Does this work in editor mode? Did you make sure to follow the documentation on how to add your own custom config-file entry? |
What this error is saying is that in the [Dashboard] section of your .ini file (TeslaM3.ini) the .cpp parser can't find entries for # in DReyeVRConfig.ini
[EgoVehicle]
VehicleType="TeslaM3" # this is the name of the .ini config file in Config/EgoVehicles/ Otherwise it might be reading another config file that doesn't have your saved changes. |
Hi, I was indeed building for the package version. I just copied the code for the speedometer, gear and turnsignals. The problem seems to lie with the packaged version and not with the config files. I changed the transform of the speedometer and the changes were reflected in the editor after executing make launch. After this I rebuild the packaged version again, but the speedometer was again in the default location instead of the one specified through the config file. |
Did you change the speedometer location value in the editor (blueprint) and then save it or in the config file and then re-run it? Try changing it in the editor and saving the blueprint (this cooks the data into the file, and is easier to visualize. |
Hi, I am not sure what you mean by changing it in the editor or blueprint. I thought the location needed to be added in the EgoVehicle.cpp file either directly (how it previously was) or via config file (how it is in the latest dev branch). Adding it directly works, because my additional text does appear. It's via the config file that it doesn't seem to work. I think the config file isn't read correctly through the cooking process. |
Hi, previously you also mentioned that we can attach things such as text to the camera so that it moves relative to the users head. From what I could find from EgoVehicle.cpp, VRCameraRoot seemed to be what you were talking about:
But the cube was still fixed in a position. As if it was fixed to the dashboard itself. Next I looked into DReveVRPawn and thought maybe you were talking about FirstPersonCam. So I did the following:
Now the object just disappeared. I tried a few different locations to be sure, and the object is pretty large, so it should appear in the mirrors at the least. Still I also enabled collision, but the object just seems to be gone. Could you tell me how I'm supposed to fix it to the VR cam? |
Interesting, the attaching to the FirstPersonCam is correct since the VRCameraRoot can be thought of as the default sitting position for the head in the vehicle, while the FirstPersonCam follows the VR headset. I'm not sure why your particular example is not working but I've been successful with another approach as follows: const FVector &CameraLoc = GetCamera()->GetComponentLocation();
const FRotator &CameraRot = GetCamera()->GetComponentRotation();
const FVector InFrontVec = CameraRot.RotateVector(FVector::ForwardVector);
const float Scale = 1.f; // how far away the actor should be
CubeMesh->SetActorLocation(CameraLoc + InFrontVec * Scale);
CubeMesh->SetActorRotation(CameraRot); |
Hi. I tried out your solution, but for some reason this causes the editor to crash before even starting. The CubeMesh is a UStaticComponent and according to the errors I received doesn't have a SetActorLocation or Rotation method. So I tried out 2 ways. First without the attachment to FirstPersonCam by using SetWorldLocation/Rotation, and second with the attachment and using SetRelativeLocation/Rotation. I want to say that when I set the attachment before, I only put it in the constructDash method, and not in the updateDash method as well. I don't think this should make a difference seeing as how attaching it to the dashboard does allow it to move. Also regarding the configFile, I made a mistake last time. Make package does indeed read the right config file. Apparantly make package made a new package with a different name so I forgot to select the right one. So when I changed the location of the speedometer, it did indeed change. However my error still remains, since it still says AtextEnabled not found. |
Well I think the only thing that would be cause for a crash would be that the But also since the approach I mentioned was to simply update the actor (or component's) location/rotation once, you'll need to run this in the |
Now the editor opens without any crashes, and the location seems to be moving, however the locations appears wrong. I had to increase the scale by a 100 again to see anything, and all I saw was a big shadow a few meter away that appeared to move when I rotated the camera. |
Hi Gustavo! Hope you are doing well. |
Hi!! When u tried it, was it able to pause with scenario runner running? |
Btw, how were you able to pause scenario simulation using UE4, could you please tell me, it would be extremely helpful. |
Are there any updates on these functionalities? |
Yes it works only if you don't use the "sync" command |
Hi @Nitro60zeus , what do you mean with "sync" command? |
When you write: "--sync" keeps the traffic synchronized so they don't behave in a wild manner |
We were trying to make the simulation stop and we came across carla issue that stated to first turn on synchronous_mode: carla-simulator/carla#1688
We input the following in jupyter notebook:
world_settings = carla.WorldSettings(synchronous_mode=True)
world.apply_settings(world_settings)
But this makes the simulation hang up. The settings do indeed change because print(world.get_settings()) tells us so.
The only way to resume the simulations is by reloading the world, but this reverts the settings back to default.
Could you help us with this? Our goal is to pause the simulation after a certain amount of time has passed. Pausing the simulation by pressing pause in UE4editor does work, but we would like to program the conditions upon which the simulation should pause on it's own.
The text was updated successfully, but these errors were encountered: