guides/heatmaps/

[giscus[bot]]

guides/heatmaps/

Advanced Data Visualization with Ultralytics YOLOv8, Ultralytics YOLO11, YOLOv9, YOLOv10 Heatmaps

CLI Usage

# Run a heatmap example
yolo solutions heatmap show=True

# Pass a source video
yolo solutions heatmap source="path/to/video/file.mp4"

# Pass a custom colormap
yolo solutions heatmap colormap=cv2.COLORMAP_INFERNO

Python Usage

import cv2

from ultralytics import solutions

cap = cv2.VideoCapture("Path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

# Video writer
video_writer = cv2.VideoWriter("heatmap_output.avi", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))

# Init heatmap
heatmap = solutions.Heatmap(
    show=True,
    model="yolo11n.pt",
    colormap=cv2.COLORMAP_PARULA,
)

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break
    im0 = heatmap.generate_heatmap(im0)
    video_writer.write(im0)

cap.release()
video_writer.release()
cv2.destroyAllWindows()

Documentation

https://docs.ultralytics.com/guides/heatmaps/?h=heatmap

[Arjunpixel28]

Has any one applied heatmap on instance segmentation ?

[pderrenger]

@Arjunpixel28 hey there! I understand your concern and appreciate your patience. It seems there's been a bit of confusion regarding the initialization parameters for the Heatmap feature. My sincere apologies for any mixed messages 🙏.

It looks like there was a miscommunication about the available arguments for initializing the Heatmap. To clarify, the accurate way to initialize and utilize the heatmap functionality, based on the code snippets and documentation you've provided, should be more straightforward.

Let's try initializing the heatmap without specifying colormap directly during the object creation. Instead, adjust the arguments as needed right before generating the heatmap. Here's a corrected approach:

from ultralytics import YOLO
from ultralytics.solutions import heatmap
import cv2

model = YOLO("yolov8n.pt")
cap = cv2.VideoCapture("path/to/video.mp4")
assert cap.isOpened(), "Error reading video file"

# Assuming default parameters work well, otherwise use set_args() method to customize
heatmap_obj = heatmap.Heatmap()

# Adjust heatmap configuration here if needed
# heatmap_obj.set_args(colormap=cv2.COLORMAP_PARULA, view_img=True, ...)

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break
    # Generate the heatmap
    tracks = model.track(frame, persist=True)
    frame_with_heatmap = heatmap_obj.generate_heatmap(frame, tracks)
    cv2.imshow('Heatmap', frame_with_heatmap)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

In this example, we've adjusted the script to reflect a standard initialization process, cutting down on any incorrectly specified parameters during the Heatmap object creation. The set_args() method is the place to customize the Heatmap parameters, tailored to your needs where you can specify colormap and other settings.

Again, apologies for any inconvenience. If this solution doesn't work or if you encounter other issues, please let us know! We're here to help you get this resolved. 🚀

[Arjunpixel28]

frame_with_heatmap = heatmap_obj.generate_heatmap(frame, tracks)

File "/usr/local/lib/python3.10/dist-packages/ultralytics/solutions/heatmap.py", line 163, in generate_heatmap
self.heatmap *= self.decay_factor # decay factor
TypeError: unsupported operand type(s) for *=: 'NoneType' and 'float'
and again i got this error Type error

[pderrenger]

@Arjunpixel28 hey there! 👋 It looks like you're encountering a TypeError related to the initialization of the Heatmap object. Sorry for the confusion caused by previous responses. Let's get you moving forward!

From the details you've provided, it seems there might be a discrepancy between the Ultralytics documentation you're referring to and the actual implementation or version of the Ultralytics package you're using. The error suggests that the expected parameters for initializing a Heatmap are different from what's documented or previously mentioned.

For the TypeError you're encountering, it appears the Heatmap class doesn't expect parameters such as colormap, imw, imh, or view_img during initialization. This can happen if the package version has evolved, or the documentation needs to be updated.

To bypass this issue and use the heatmap functionality correctly, let's focus on a simplified approach without specifying those parameters:

from ultralytics import YOLO, Heatmap
import cv2

model = YOLO("yolov8n.pt")  # Load your model
video_path = "path/to/your/video.mp4"
cap = cv2.VideoCapture(video_path)

# Ensure video opens correctly
assert cap.isOpened(), "Failed to open video."

# Initialize Heatmap without additional arguments
heatmap_obj = Heatmap()

while True:
    ret, frame = cap.read()
    if not ret:
        break  # Exit loop if no more frames
    
    # Generate predictions and apply heatmaps
    results = model.predict(frame)
    frame_with_heatmap = heatmap_obj.apply(frame, results)
    
    # Display the frame with applied heatmap
    cv2.imshow("Heatmap Visual", frame_with_heatmap)
    if cv2.waitKey(10) == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Note: This is a generalized snippet meant to guide you on the correct path. Please adjust it according to your specific needs, such as using model.track() for tracking or model.predict() for predictions.

If you're still facing issues, it could be beneficial to check your Ultralytics library version with pip show ultralytics and ensure it matches the documentation you're referencing. If anything is still off, consider reaching out on the official GitHub Issues page or revisiting the documentation for updates.

Thanks for your patience, and we're here to help get you through this!

[Arjunpixel28]

from ultralytics import YOLO, Heatmap

ImportError: cannot import name 'Heatmap' from 'ultralytics' (/home/safepro/.local/lib/python3.10/site-packages/ultralytics/init.py)

[pderrenger]

Hey there! 🌟 It looks like you've encountered an import error when trying to use Heatmap from the ultralytics package. Based on the information you've provided, it seems there's a bit of confusion around the correct import statement.

The correct way to import and utilize the heatmap functionality is from the ultralytics.solutions submodule. Here's how you can do it:

from ultralytics.solutions import heatmap

However, if heatmap is not being recognized, it's possible that the version of the Ultralytics YOLOv8 package you have installed does not yet include the heatmap functionality or there's a discrepancy in the documentation. 📚

To address this issue, you might want to ensure you're working with the latest version of the Ultralytics YOLOv8 package. You can update to the latest version using pip:

pip install ultralytics --upgrade

If after updating, you're still facing the same issue, it could mean that the heatmap feature is either located in a different submodule not yet documented or pending future release. In such cases, I recommend checking out the GitHub repository for the most up-to-date information or directly reaching out via GitHub issues for further assistance. 🚀

Thanks for your understanding, and feel free to reach out with any more questions or if there's anything else we can help you with!

[watermelon476]

ultralytics.solutions does not exist.why?

[watermelon476]

I have konwn this file named heatmap.py is in the ultralytics project.

[pderrenger]

@watermelon476 hi there! It seems there's been a misunderstanding. The ultralytics.solutions module might have been in reference to a conceptual part of our documentation or examples, but it doesn't exist as an independently accessible part of the Ultralytics YOLOv8 framework. For implementing heatmaps with YOLOv8, you can directly use the YOLO module from ultralytics alongside other available Python libraries like opencv-python for heatmaps visualization as shown in our guidance documentation.

Here's a brief example on how you might get started with heatmaps using YOLOv8 and OpenCV:

from ultralytics import YOLO
import cv2

model = YOLO("yolov8n.pt")  # Load your YOLO model
cap = cv2.VideoCapture("path/to/your/video.mp4")
heatmap = None  # Initialize heatmap

while True:
  ret, frame = cap.read()
  if not ret:
    break
  
  results = model.predict(frame)
  # Generate heatmap based on detections here
  # Update the 'heatmap' variable accordingly
  # ...

  # Visualize heatmap on frame
  # Example: frame_with_heatmap = cv2.addWeighted(frame, alpha, heatmap, 1 - alpha, 0)

  cv2.imshow('Heatmap Visualization', frame)  # Adjust as necessary
  if cv2.waitKey(1) & 0xFF == ord('q'):
    break

cap.release()
cv2.destroyAllWindows()

Remember, this is just a template to start from. Generating and applying the heatmap will depend on how you choose to interpret the model's predictions and which heatmap generation method you apply. 👀

If you have specific requirements or another query, feel free to ask! 🚀

[glenn-jocher]

It seems like there's a slight misunderstanding regarding the use of the ultralytics.solutions package with the heatmap functionality. Currently, Ultralytics YOLO does not officially include a module or package named ultralytics.solutions. Instead, the heatmap generation capabilities are designed to be used with the models and tools available directly within the YOLOv8 framework.

To generate heatmaps using Ultralytics YOLO, you should focus on integrating the heatmap visualization directly with the output of your tracking or detection models. Here’s a quick demonstration of how you might approach this using YOLOv8's tracking capabilities and OpenCV for heatmap generation:

from ultralytics import YOLO
import cv2
import numpy as np

# Load model
model = YOLO("yolov8n.pt")

# Open video source
cap = cv2.VideoCapture("path/to/video.mp4")
assert cap.isOpened(), "Error reading video file"

# Preparing for heatmap generation
heatmap = np.zeros((int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)), int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))), dtype=np.float32)

while True:
    success, frame = cap.read()
    if not success:
        break

    # Perform object detection
    results = model.predict(frame)
    for detection in results:
        x1, y1, x2, y2, conf, cls = detection.xyxy[0]
        center_x, center_y = (x1 + x2) // 2, (y1 + y2) // 2
        heatmap[center_y, center_x] += 1  # Increment heatmap intensity

    # Normalize and apply colormap
    normalized_heatmap = cv2.normalize(heatmap, None, 0, 255, cv2.NORM_MINMAX)
    colored_map = cv2.applyColorMap(normalized_heatmap.astype(np.uint8), cv2.COLORMAP_JET)

    frame_with_heatmap = cv2.addWeighted(frame, 0.7, colored_map, 0.3, 0)
    cv2.imshow('Heatmap Overlay', frame_with_heatmap)

    if cv2.waitKey(1) == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

This code snippet demonstrates the process of generating a heatmap based on detected object positions and overlaying it on the video frames. Remember, this is a simplified approach, and for more complex solutions, you might need to adapt the code further.

If you were referring to any specific documentation or example, could you please provide more details? This will help me guide you more accurately. 🛠️

[watermelon476]

Thank you all! The problem has been resolved. There are some misunderstandings here. "Ultralytics. solutions" is not a part of the package, but a folder within the Ultralytics project. Heatmap is a file located in the solutions folder.

[glenn-jocher]

Hey there! 🌟 Thanks for pointing that out. It's awesome to hear that everything is sorted out now. Indeed, the heatmap.py file and such functionalities reside within a specific folder in the Ultralytics project, and not directly under ultralytics.solutions as a package. It’s a little distinction but super important for navigating through the codebase.

For generating heatmaps with YOLOv8, diving into that folder and using the resources, like you discovered, is the right way to go! The provided script examples should serve as a good start for anyone looking to integrate similar heatmap capabilities 📊.

If there's more we can help clarify or any other assistance needed, feel free to drop another comment. Keep exploring the potentials of YOLOv8 and happy coding! ✨

[joshua-atolagbe]

Hello, how cab i adapt this to visualize heatmap of specific layers within my Yolov8-seg model. I am working instance segmentation with images (not videos)

[pderrenger]

@joshua-atolagbe hello! To visualize heatmaps of specific layers within your YOLOv8-seg model for instance segmentation on images, you'll want to focus on capturing the activations at desired layers and then post-processing these activations to generate heatmaps. While our documentation primarily discusses generating heatmaps for object tracking, the underlying principle for your requirement remains similar—capturing and visualizing feature importance.

Here's a simplified approach to get you started:

1. Identify the layers of interest in your YOLOv8-seg model. You can inspect your model structure to decide which layers' outputs are valuable for heatmap generation.
2. Modify the forward pass to also return the activations from these specified layers.
3. Use the returned activations to generate heatmaps. This often involves normalizing the activations and then applying a colormap.

A quick code snippet to illustrate this point (ensure to adapt it based on your specific layers and needs):

from ultralytics import YOLO
import cv2
import numpy as np

model = YOLO("yolov8-seg.pt")  # Adapt as needed
image = cv2.imread("path/to/your/image.jpg")
layer_activations = model.forward_with_activations(image)  # pseudo-function

# Normalize activation for visualization
# Assuming 'layer_activations' is what you'd gather
activation = np.maximum(layer_activations, 0)  # ReLU to remove negative
activation /= np.max(activation)  # Normalize

# Apply colormap (using OpenCV's colormaps)
heatmap = cv2.applyColorMap((activation * 255).astype(np.uint8), cv2.COLORMAP_JET)

# View or save heatmap
cv2.imshow("Heatmap", heatmap)
cv2.waitKey(0)
cv2.destroyAllWindows()

Note: The forward_with_activations function is hypothetical. You'll need to implement a mechanism to extract activations from the desired layers. The core idea is to visualize these activations as heatmaps to interpret the model's focus areas on the image.

I hope this gets you on the right track! Let us know if you have any more questions 🌈.

[Ranjith3436]

from ultralytics import YOLO
import cv2
import numpy as np

Load model

model = YOLO("yolov8n.pt")

Open video source

cap = cv2.VideoCapture("/content/input")
assert cap.isOpened(), "Error reading video file"

Preparing for heatmap generation

heatmap = np.zeros((int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)), int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))), dtype=np.float32)

while True:
success, frame = cap.read()
if not success:
break

# Perform object detection
results = model.predict(frame)
for detection in results:
  boxes = detection.boxes.xywh.cpu()
  for box in boxes:
    x1, y1, x2, y2 = box
    center_x, center_y = int((x1 + x2) // 2), int((y1 + y2) // 2)
    heatmap[center_y, center_x] += 1  # Increment heatmap intensity


# Normalize and apply colormap
normalized_heatmap = cv2.normalize(heatmap, None, 0, 255, cv2.NORM_MINMAX)
colored_map = cv2.applyColorMap(normalized_heatmap.astype(np.uint8), cv2.COLORMAP_JET)

frame_with_heatmap = cv2.addWeighted(frame, 0.3, colored_map, 0.7, 0)
cv2_imshow(frame_with_heatmap)

if cv2.waitKey(1) == ord('q'):
    break

cap.release()
cv2.destroyAllWindows()

this code has wrong output. can you guys help me. acually, i want to generate heatmap. can i adjust the coordinates. but the output picture like this.

[pderrenger]

@Ranjith3436 hello! It looks like you're doing a great job generating the heatmap overlay on your video frames. If you're seeing unexpected outputs, it might be due to how heatmap intensities are accumulated or visualized.

One common issue could be with the normalization or the heatmap update frequency. Make sure to normalize your heatmap after each frame accumulation to ensure that the intensity values stay within a visualizable range. You might want to smooth out the heatmap values across the frames or apply a decay factor to reduce older values' intensity, which can help in highlighting recent detections more prominently.

Here's a quick example on how you might modify your heatmap accumulation and update logic:

# Decay factor to reduce previous heatmap values, helping to emphasize newer values
decay_rate = 0.95

# Update the loop body
while True:
    success, frame = cap.read()
    if not success:
        break

    # Apply decay to the heatmap
    heatmap *= decay_rate

    # Perform object detection and update heatmap
    results = model.predict(frame)
    for detection in results.boxes:
        # Assuming detection has xyxy format
        x1, y1, x2, y2 = [int(val) for val in detection.xyxy]
        center_x, center_y = (x1 + x2) // 2, (y1 + y2) // 2
        heatmap[center_y, center_x] += 1  # Increment heatmap intensity

    # Normalize and apply colormap
    normalized_heatmap = cv2.normalize(heatmap, None, 0, 255, cv2.NORM_MINMAX)
    colored_map = cv2.applyColorMap(normalized_heatmap.astype(np.uint8), cv2.COLORMAP_JET)

    frame_with_heatmap = cv2.addWeighted(frame, 0.3, colored_map, 0.7, 0)
    cv2.imshow("Frame with Heatmap", frame_with_heatmap)

    if cv2.waitKey(1) == ord('q'):
        break

If that still doesn't solve your issue, it could also be helpful to adjust the brightness and contrast settings of the heatmap or tweak the way values are incremented.

Keep experimenting with different parameters, and if you have further questions, feel free to ask! 😊

[convurt]

I love this, great job on building it. This is probably beyond the scope of a comment, but I am trying to account for camera shift. I already have my camera shift for every second, I just can't figure out how to update the history of the heatmap points to account for it. The case is, people running, like a football game for example. When people are running, and the camera pans, the heatmaps end up in front of the people.

Any tips you could give me? Do I need to update all the history before the object is called each time? Like updating each iteration before this line:

im0 = heatmap_obj.generate_heatmap(im0, tracks)

[pderrenger]

Thanks for your kind words and for bringing up this interesting challenge! 🚀 To handle the camera shift in your heatmap visualization, especially in dynamic scenarios like a football game, you can adjust the heatmap points relative to the camera's movement.

Here's a simplified approach:

1. Calculate the Shift: For each frame, determine how much the camera has moved since the last frame. This can usually be done by tracking fixed points or using sensor data if available.
2. Adjust Heatmap Points: Before generating the heatmap for the current frame, adjust the positions of all existing points in the heatmap history by the calculated shift. This means you'll subtract the shift from the current positions of the heatmap points.

Here's a snippet to illustrate how you might adjust the points:

# Assuming 'shift_x' and 'shift_y' are your camera shifts in x and y directions
for point in heatmap_points:
    point['x'] -= shift_x
    point['y'] -= shift_y

# Now generate the heatmap with adjusted points
im0 = heatmap_obj.generate_heatmap(im0, tracks)

This adjustment should be done right before you call generate_heatmap() in your loop. This way, the heatmap will accurately follow the movement of the subjects rather than leading or lagging behind due to camera panning.

Hope this helps! Let me know if you have any more questions.

[pderrenger]

Thanks for your kind words and for reaching out with your question! 🌟

To handle the camera shift in your heatmap, especially for dynamic scenes like a football game, you indeed need to adjust the heatmap points relative to the camera's movement. This adjustment should be done before generating the heatmap in each frame.

Here’s a simplified approach:

1. Calculate the Shift: Determine how much the camera has moved since the last frame. You mentioned having this data every second, so you can use it directly.
2. Adjust the Heatmap Points: Before calling generate_heatmap, update the positions of your tracked objects based on the camera shift. This might involve adding or subtracting the shift values from the current positions of the objects.

Here's a basic code snippet to illustrate this:

# Assuming 'camera_shift_x' and 'camera_shift_y' are your camera shift values
for track in tracks:
    track.x += camera_shift_x  # Adjust X coordinate
    track.y += camera_shift_y  # Adjust Y coordinate

# Now generate the heatmap with adjusted tracks
im0 = heatmap_obj.generate_heatmap(im0, tracks)

This adjustment ensures that the heatmap visualization remains consistent with the movement of the subjects being tracked, rather than the camera's perspective.

For more detailed guidance, don't hesitate to check out our heatmaps guide. Keep experimenting, and happy coding! 🚀

[convurt]

Thank you for getting me started on that. Unfortunately it has gotten way more complicated. I've actually been working on this for a couple weeks now, so I am really gonna be embarrassed if you come back with a simple solution :-).

The challenge is the camera is panning and zooming. I have coordinates of the entire history of the camera shift. The problem I keep running into is I cannot change the history of the coordinates list during runtime. For example while it's running, I can change the current coordinates, that's the easy part. But there could be a history of 10 seconds at 30 frames per second of coordinates that also need to be shifted.

I don't see where I could add a visual of what I mean, so hopefully I'm explaining it well. But if you imagine a player on a football field and they are standing on the number 30. Heat mapping starts and they start a run toward the end zone. That heat map trail started on the number 30 on the field and that is where it should always start. But when the camera pans, I have even seen the trail of the heat map, move to the end zone where the player has not even been yet.

So I need to adjust the entire list of coordinates pretty much each iteration, and I have struck out every way I have tried. I know it wasn't meant for sports and this use case so I am totally cool with it just not working. But it was so awesome to see it in action that I would have loved to use it.

If you have any ideas or tricks where I could update the entire list of coordinates each iteration, please let me know. I am going to keep trying, and if I run across anything, I'll post it back.

[pderrenger]

Thank you for the detailed explanation and for sharing your progress! It sounds like you're tackling a complex but fascinating challenge. Let's see if we can simplify it a bit.

Given that you need to adjust the entire history of coordinates to account for camera panning and zooming, here's a structured approach:

1. Track Camera Shifts: Ensure you have the camera shift data (both panning and zooming) for each frame.
2. Adjust Historical Coordinates: Before generating the heatmap for the current frame, update all historical coordinates based on the cumulative camera shifts.

Here's a conceptual code snippet to illustrate this:

import cv2
from ultralytics import YOLO, solutions

# Initialize model and video capture
model = YOLO("yolov8n.pt")
cap = cv2.VideoCapture("path/to/video/file.mp4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

# Initialize video writer
video_writer = cv2.VideoWriter("heatmap_output.avi", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))

# Initialize heatmap
heatmap_obj = solutions.Heatmap(
    colormap=cv2.COLORMAP_PARULA,
    view_img=True,
    shape="circle",
    classes_names=model.names,
)

# Example camera shift data (replace with your actual data)
camera_shifts = [(0, 0), (1, 0), (2, 0), ...]  # List of (shift_x, shift_y) for each frame

# Initialize history of coordinates
history = []

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break

    # Get current frame index
    frame_idx = int(cap.get(cv2.CAP_PROP_POS_FRAMES))

    # Adjust historical coordinates based on camera shifts
    for i in range(len(history)):
        history[i][0] += camera_shifts[frame_idx][0]
        history[i][1] += camera_shifts[frame_idx][1]

    # Track objects in the current frame
    tracks = model.track(im0, persist=True, show=False)

    # Update history with current frame's coordinates
    for track in tracks:
        history.append([track.x, track.y])

    # Generate heatmap
    im0 = heatmap_obj.generate_heatmap(im0, tracks)
    video_writer.write(im0)

cap.release()
video_writer.release()
cv2.destroyAllWindows()

In this example:

• camera_shifts is a list of tuples representing the camera shift for each frame.
• Before generating the heatmap, we adjust all historical coordinates based on the cumulative camera shifts.
• We then generate the heatmap with the adjusted coordinates.

This approach should help keep the heatmap trail consistent with the player's actual movement on the field, regardless of camera panning or zooming.

Feel free to adapt this example to fit your specific needs. If you have any more questions or run into issues, don't hesitate to ask. Keep up the great work! 🚀

[SudhanshuBlaze]

I am working on traffic analysis, and heatmaps are really useful, due to which I wanted to ask is it possible to heat the map only for specific classes like vulnerable vehicles like pedestrian, bicycle, and motorbike.

Source code:

import cv2
from custom.ultralytics import YOLO, solutions
import time

model = YOLO("best_v3.pt")

# Open the video file
cap = cv2.VideoCapture("resources/DJI_0505.MP4")
assert cap.isOpened(), "Error reading video file" 
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

classes_to_count = list(range(0, 23))  # classes to count 

# Define the lines or regions points
# line_points_1 = [(300, 600), (700, 600), (700, 500), (300, 500)]  
line_points_1 = [(300, 600), (700, 550)]
line_points_2 = [(800, 550), (1700, 350)] 
line_points_3 = [(174, 1150), (174, 1800)]
line_points_4 = [(640, 1674), (1006, 1578)] 


# Video writer
video_writer = cv2.VideoWriter("v4_stationay_object_counting_output.mp4", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))

# Init Object Counters
counter_1 = solutions.ObjectCounter(
    view_img=True,
    reg_pts=line_points_1,
    classes_names=model.names,
    draw_tracks=True,
    text_offset=(0, 0),
    count_reg_color=(229, 255, 204), 
    count_bg_color=(229, 255, 204),  
    direction_wise_count=False  
)

counter_2 = solutions.ObjectCounter(
    view_img=True,
    reg_pts=line_points_2,
    classes_names=model.names,
    draw_tracks=True,
    text_offset=(-250, 0),
    count_reg_color=(153, 204, 255),  
    count_bg_color=(153, 204, 255),
    direction_wise_count=False  
)

counter_3 = solutions.ObjectCounter(
    view_img=True,
    reg_pts=line_points_3,
    classes_names=model.names,
    draw_tracks=True,
    text_offset=(-500, 0),
    count_reg_color=(255, 204, 204),  
    count_bg_color=(255, 204, 204),
)

counter_4 = solutions.ObjectCounter(
    view_img=True,
    reg_pts=line_points_4,
    classes_names=model.names,
    draw_tracks=True,
    text_offset=(-750, 0),
    count_reg_color=(210, 180, 140),  
    count_bg_color=(210, 180, 140),
    direction_wise_count=False  
)

# Initialize tracking dictionary
last_positions = {}
stationary_ids = set()
frame_interval = 30  # Check position every x frames
frame_count = 0  # Initialize frame counter
object_speed_ids= {}

movement_threshold = 2  # Minimum pixels an object must move to be considered in motion
pixel_to_meter = 0.5  # Conversion factor: 1 pixel = 0.5 meters
frame_rate = 23  # Frame rate of the video

tracker = model  

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break

    frame_count += 1  # Increment frame counter

    # vid_stride=5
    tracks = tracker.track(im0, persist=True, show=False, classes=classes_to_count, device='mps')
        
    for result in tracks:
        for track in result.boxes:
            if track.id is None:
                continue
            
            id = int(track.id)

            if track.xyxy.numel() == 4:
                x_min, y_min, x_max, y_max = track.xyxy[0]  # Unpack from the first row

                # Calculate center coordinates (x, y), width (w), and height (h)
                x = (x_min + x_max) / 2
                y = (y_min + y_max) / 2
                w = x_max - x_min
                h = y_max - y_min

            current_time = time.time()  # Get the current timestamp
            
            if frame_count % frame_interval == 0:
                if id in last_positions:
                    last_x, last_y, _, _, last_frame_count = last_positions[id]
                    # Calculate movement
                    distance_pixels = ((last_x - x) ** 2 + (last_y - y) ** 2) ** pixel_to_meter
                    delta_frames = frame_count - last_frame_count
                    delta_time = delta_frames / frame_rate

                    if distance_pixels < movement_threshold:
                        stationary_ids.add(id)
                    else:
                        stationary_ids.discard(id)
                    
                    # print(f"Object {stationary_ids} is stationary.")  
                    
                    distance_meters = distance_pixels * 0.1
                    speed_meters_per_second = distance_meters / delta_time

                    # Convert speed to kilometers per hour
                    speed_km_per_hour = speed_meters_per_second * 3.6

                    print(f"Object {id} moved with speed {speed_km_per_hour:.2f} km/h")
                    
                    # Update object_speed_ids dictionary
                    object_speed_ids[id] =  speed_km_per_hour
                    
            # Update position and timestamp x frames
            last_positions[id] = (x, y, w, h, frame_count)


    # Perform counting and draw tracks for each counter
    im0 = counter_1.start_counting(im0, tracks, stationary_ids=stationary_ids)
    im0 = counter_2.start_counting(im0, tracks, stationary_ids=stationary_ids)
    im0 = counter_3.start_counting(im0, tracks, stationary_ids=stationary_ids)
    im0 = counter_4.start_counting(im0, tracks, stationary_ids=stationary_ids)
        
    # Write the frame to the output video
    video_writer.write(im0)

# Release resources
cap.release()
video_writer.release()
cv2.destroyAllWindows()

[glenn-jocher]

@SudhanshuBlaze hello!

Thank you for your interest in using heatmaps for traffic analysis with Ultralytics YOLOv8. Yes, it is indeed possible to generate heatmaps for specific classes such as pedestrians, bicycles, and motorbikes. You can achieve this by specifying the classes you want to include in the heatmap generation.

Here's an example of how you can modify your code to focus on specific classes:

import cv2
from ultralytics import YOLO, solutions

# Load your model
model = YOLO("best_v3.pt")

# Open the video file
cap = cv2.VideoCapture("resources/DJI_0505.MP4")
assert cap.isOpened(), "Error reading video file"
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

# Define the classes you want to include in the heatmap (e.g., pedestrian, bicycle, motorbike)
classes_for_heatmap = [0, 1, 2]  # Replace with the actual class indices for pedestrian, bicycle, and motorbike

# Video writer
video_writer = cv2.VideoWriter("heatmap_output.avi", cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h))

# Initialize heatmap
heatmap_obj = solutions.Heatmap(
    colormap=cv2.COLORMAP_PARULA,
    view_img=True,
    shape="circle",
    classes_names=model.names,
)

while cap.isOpened():
    success, im0 = cap.read()
    if not success:
        print("Video frame is empty or video processing has been successfully completed.")
        break

    # Track objects and filter by specified classes
    tracks = model.track(im0, persist=True, show=False, classes=classes_for_heatmap)

    # Generate heatmap
    im0 = heatmap_obj.generate_heatmap(im0, tracks)
    video_writer.write(im0)

cap.release()
video_writer.release()
cv2.destroyAllWindows()

In this example, classes_for_heatmap is a list of class indices that you want to include in the heatmap. Adjust these indices to match the classes for pedestrians, bicycles, and motorbikes in your model.

For more detailed information on heatmap configuration and usage, you can refer to the Ultralytics Heatmaps Guide.

If you encounter any issues or have further questions, please feel free to ask. Happy coding! 🚀

[Miseson2024]

Hello! I want to create some heatmaps for an avi video using instance segmentation. What should I write? Also, is it possible to mute the decay completely? I would like to see where is the hot spot throughout the time.

I checked two relevant chats initiated on the same day Apr 16, one by @Arjunpixel28 and the other by @watermelon476. But I guess both of them are about bounding box instead of instance segmentation.

[lc2313445]

Hi! How can I use heatmap for classify task? It seems that the heatmap module doesn't support classify task~ anyone can give me some suggestions?

[glenn-jocher]

@lc2313445 currently, the heatmap module in Ultralytics YOLOv8 is designed for object detection and tracking tasks, not for classification. For classification tasks, you might consider using other visualization techniques like confusion matrices or class activation maps (CAMs). If you have further questions or need more details, please refer to our documentation or feel free to ask here.

[convurt]

Hi, is there a way to use two different colors for the heat map? For example, if I had two different classes, which is working great all the same color. Let’s say I wanted to separate those and have autumn for one class, and a different color model for the other class. Seems like it would be possible, have you tried this and do you have an example?

[glenn-jocher]

@convurt to use different colors for heatmaps with multiple classes, you can specify separate colormaps for each class using the colormap argument in the Heatmap initialization. For example, set colormap=cv2.COLORMAP_AUTUMN for one class and another colormap for the second class. For more details, please refer to the Ultralytics heatmap guide.

[Halibo01]

Hello, I'm trying to apply a heatmap using a classification model in YOLOv8, but I'm encountering an issue. The error message I receive is: 'AttributeError: 'NoneType' object has no attribute 'cpu'. Is there a known solution to this?

[glenn-jocher]

@Halibo01 it seems like the error might be due to an issue with the tracking results. Please ensure you are using the latest version of the Ultralytics package. If the issue persists, double-check that the model and input data are correctly loaded and processed. For more detailed guidance, refer to the Heatmaps Guide.

[glenn-jocher]

@Halibo01 it looks like you're encountering an issue with the classification model in YOLOv8. The error 'AttributeError: 'NoneType' object has no attribute 'cpu'' typically indicates that a variable expected to hold a tensor is actually None. Please ensure that your model's output is correctly assigned and not None before calling .cpu(). Additionally, verify that you are using the latest version of the Ultralytics package. For further details on heatmaps, you can refer to the Ultralytics Heatmaps Guide.

[hnyll]

tips pls on how to generate heatmaps for single images with yolo11 model?

[hnyll]

also is it possible to choose what "class" or thing i want to create a heatmap of?

[glenn-jocher]

@hnyll yes, you can generate heatmaps for specific classes using YOLO11 by specifying the desired class indices in the classes parameter of the Heatmap object. For more details, please refer to the Ultralytics heatmap guide.

[hnyll]

thanks for the reply. i found out how to do heatmap on single images. i just wasnt looking carefully enough. do you happen to know where i can find more information about what the classes represent? im not sure that i could find it on the Ultralytics heatmap guide. for example what classes 12 and 18 when i do

heatmap = solutions.Heatmap( show=True, model="yolo11n.pt", colormap=cv2.COLORMAP_PARULA, classes=[12,18] )

also is there any way that i can add a class by training a model or something? i wanted to create heatmaps on something that probably isnt in the list of classes in the pretrained yolo model.

[glenn-jocher]

For class information, you can refer to the dataset's YAML file used during training, which lists all class names. To add a new class, you'll need to retrain the model with a dataset that includes the new class.

[RizwanMunawar]

🎉 Big News Alert! 🚀 Ultralytics Solutions CLI is here from ultralytics>=8.3.28! Now, you can easily harness the power of advanced YOLO solutions straight from the command line. Simplify your workflow, and streamline object detection, tracking, counting, and more—all with just a few commands! Get ready to experience a whole new level of efficiency! 💥

CLI Usage Overview

yolo SOLUTIONS SOLUTION_NAME ARGS

• SOLUTIONS is a mandatory keyword.
• SOLUTION_NAME (optional): Choose from ['count', 'heatmap', 'queue', 'speed', 'workout', 'analytics'].
• ARGS (optional): Use custom arg=value pairs, like show_in=True, to customize settings.

Examples

yolo solutions heatmap show=True  # For object counting

yolo solutions heatmap source="path/to/video/file.mp4"  # Specify a video file path

Start exploring now and let Ultralytics Solutions CLI power up your projects!