Instance Segmentation with Object Tracking - Ultralytics YOLOv8 Docs

[giscus[bot]]

Instance Segmentation with Object Tracking - Ultralytics YOLO Docs

Instance Segmentation with Object Tracking using Ultralytics YOLO11, Ultralytics YOLOv8, YOLOv9, YOLOv10

https://docs.ultralytics.com/guides/instance-segmentation-and-tracking/

[shemesheran]

Wow this looks really great!
I noticed that its common that when an object leaves the frame and then returns, its getting a different id
Is there any way to improve this?

[RizwanMunawar]

@shemesheran, absolutely! Give bytetrack a try:

yolo track source="path/to/video/file.mp4" tracker="bytetrack.yaml"

Furthermore, for identification, consider exploring various identification algorithms and integrating them with Ultralytics YOLOv8.

Thanks,
Ultralytics Team!

[hsenzayati]

@RizwanMunawar
I worked with dynamic objects when changing jeans, tracking can put different IDs on the same jeans.
What is the best solution.
code:
import cv2
from ultralytics import YOLO
from ultralytics.utils.plotting import Annotator, colors

from collections import defaultdict

track_history = defaultdict(lambda: [])

model = YOLO("C:/Users/ZAYATI/Downloads/best.pt")
cap = cv2.VideoCapture(0)
while True:
ret, im0 = cap.read()
if not ret:
print("Video frame is empty or video processing has been successfully completed.")
break

annotator = Annotator(im0, line_width=2)

results = model.track(im0, persist=True,imgsz=160)

if results[0].boxes.id is not None and results[0].masks is not None:
masks = results[0].masks.xy
track_ids = results[0].boxes.id.int().cpu().tolist()

for mask, track_id in zip(masks, track_ids):
    annotator.seg_bbox(mask=mask,
                       mask_color=colors(track_id, True),
                       track_label=str(track_id))

cv2.imshow("instance-segmentation-object-tracking", im0)

if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()

[Ambarish-Ombrulla]

Initila track-id is assigned as None

[RizwanMunawar]

@Ambarish-Ombrulla the tracker might take 1-2 frames to assign unique IDs to objects, which can then be utilized for further processing.

Thanks
Ultralytics Team!

[Moizcode]

I am using tracking on my custome_yolo_model train on face dataset, now i want to integrate multiple cameras system, so any direct way by which if one camera detect face and give a track_id then another camera gives same track_id to that person. I don't know how tracking works here, but if possible then give me solutions and i also appreciate if you guide me how i will do this.
Thank you for reading my comment.

[RizwanMunawar]

@Moizcode! Appreciate the comprehensive details you've shared. It appears you're interested in an object reidentification task, an area we currently have no support for!

We may consider providing support for this in the future, for now, you can employ the object reidentification technique to assign consistent tracking IDs to individuals across various camera views.

Hope this helps! Let me know if you have any additional questions.

Thanks and Regards
Ultralytics Team!

[mileseverett]

Any example projects that implement object reidentification that you wouldn't mind sharing?

[RizwanMunawar]

@mileseverett you can explore Strongsort, which I guess supports reidentification!

Thanks
Ultralytics Team!

[yeongnamtan]

I received below error when running the code above ...

AttributeError Traceback (most recent call last)
in
28
29 if results[0].boxes.id is not None:
---> 30 masks = results[0].masks.xy
31 track_ids = results[0].boxes.id.int().cpu().tolist()
32

AttributeError: 'NoneType' object has no attribute 'xy'

[yeongnamtan]

@RizwanMunawar thanks. i tried again with the latest release of ultralytics and still get the error ...

---

AttributeError Traceback (most recent call last)
Cell In[3], line 30
27 results = model.track(im0, classes=[0], persist=True, show=False, tracker="bytetrack.yaml", conf=0.3, stream=False) # class 0 for fish
29 if results[0].boxes.id is not None:
---> 30 masks = results[0].masks.xy
31 track_ids = results[0].boxes.id.int().cpu().tolist()
33 for mask, track_id in zip(masks, track_ids):

AttributeError: 'NoneType' object has no attribute 'xy'

[RizwanMunawar]

@yeongnamtan you are not using the old code, we have updated the code, please use that and it will fix the issue.

import cv2
from ultralytics import YOLO
from ultralytics.utils.plotting import Annotator, colors

model = YOLO("yolov8n-seg.pt")  # segmentation model
names = model.model.names
cap = cv2.VideoCapture("path/to/video/file.mp4")
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

out = cv2.VideoWriter('instance-segmentation.avi', cv2.VideoWriter_fourcc(*'MJPG'), fps, (w, h))

while True:
    ret, im0 = cap.read()
    if not ret:
        print("Video frame is empty or video processing has been successfully completed.")
        break

    results = model.predict(im0)
    annotator = Annotator(im0, line_width=2)

    if results[0].masks is not None:
        clss = results[0].boxes.cls.cpu().tolist()
        masks = results[0].masks.xy
        for mask, cls in zip(masks, clss):
            annotator.seg_bbox(mask=mask,
                               mask_color=colors(int(cls), True),
                               det_label=names[int(cls)])

    out.write(im0)
    cv2.imshow("instance-segmentation", im0)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

out.release()
cap.release()
cv2.destroyAllWindows()

Thanks
Ultralytics Team!

[yeongnamtan]

@RizwanMunawar thank you. It is working now.

[NguyenDucQuan12]

@RizwanMunawar i have a question, i see:

out.write(im0)
cv2.imshow("instance-segmentation", im0)

It only displays and saves the original frames read from the video, but it does not display the frames containing the segment or bbox, id. Is this true, because I'm not a professional developer, I'm just starting to learn

[RizwanMunawar]

@NguyenDucQuan12 If im0 is just an input image, then cv2.imshow will display original frame. If you have process that image with segmentation masks, then masks should be displayed in output image.

Thanks
Ultralytics Team!

[mominali12]

How to save the output results as a .mp4 with the boxes and IDs and tracks?

[RizwanMunawar]

the output feature is already supported in .avi format, If you want to change it to .mp4 you can replace the video writer according to your needs.

out = cv2.VideoWriter('instance-segmentation.avi', cv2.VideoWriter_fourcc(*'MJPG'), fps, (w, h))

[HugeBra1n]

How to save the output results as a TXT or CSV file containing IDs of track and time of detection or start and end of object detecting?

[RizwanMunawar]

@HugeBra1n This feature is not available at the moment. We are committed to releasing it soon. In the meantime, you can experiment by utilizing the Python time module and saving segmentation data in a CSV file.

Hope this helps! Let me know if you have any additional questions.

Thanks and Regards
Ultralytics Team!

[Hug013]

Hi ! I wanted to know if I could save the IDs of track in a txt as save_txt did. Time of tracking does not interest me in my project.
save_txt only allows me to get the bounded box coordinates and the class.
Thanks and Regards !

[pderrenger]

@Hug013 hello! Yes, you can save the track IDs along with the bounding box coordinates and class information to a .txt file. You'll need to modify the code slightly to include the track IDs in the saved output. Here's a concise example of how you might do this:

from ultralytics import YOLO

# Load your trained model
model = YOLO('path/to/your/model.pt')

# Perform tracking on your source
results = model.track(source='path/to/your/source', save_txt=True, save_conf=True)

# Save track IDs to .txt file
for i, result in enumerate(results):
    with open(f'track_{i}.txt', 'w') as f:
        for *box, conf, cls, track_id in result.boxes.tolist():
            f.write(f"{track_id} {' '.join(map(str, box))} {conf} {cls}\n")

This will save the track IDs along with the bounding box coordinates, confidence scores, and class IDs for each detection in your source. The result.boxes.tolist() method converts the Boxes object to a list of lists, where each inner list contains the bounding box coordinates, confidence score, class ID, and track ID.

Let me know if you need further assistance!

[Hug013]

Hello ! thank you for your answer, I tried your code but it does not appear to be the tracking ID but some random ID that can change as boxes appear or disappear.
In my case, some object I detect can have the ID : 4 and if the object with the ID : 3 disappear, the ID : 4 becomes ID : 3. However, when I look at the tracking, the ID remains the same during the whole video, which is what interest me the most.

[glenn-jocher]

@Hug013 hello! I apologize for the confusion earlier. To save the consistent track IDs that remain the same throughout the video, you'll need to access the id attribute from the Boxes object after running the track method. Here's how you can modify the code to save the correct track IDs to a .txt file:

from ultralytics import YOLO

# Load your trained model
model = YOLO('path/to/your/model.pt')

# Perform tracking on your source
results = model.track(source='path/to/your/source')

# Save track IDs to .txt file
with open('tracks.txt', 'w') as f:
    for result in results:
        # Each result contains boxes with .id attribute for track IDs
        for box in result.boxes:
            track_id = box.id
            bbox = box.xyxy
            conf = box.conf
            cls = box.cls
            f.write(f"{track_id} {' '.join(map(str, bbox.cpu().numpy()))} {conf} {cls}\n")

This script will write the track IDs along with the bounding box coordinates, confidence scores, and class IDs to a tracks.txt file. The track IDs should now remain consistent for each object throughout the video, as long as the object is detected by the model.

If you encounter any issues or have further questions, please let me know!

[barutlift]

hi, i trained my model with yolov8. i want to detect and blur so can i

[barutlift]

im ok i succesed so thanks

[barutlift]

results [0].boxes was necessary

[RizwanMunawar]

Great. You can also explore our Object Blurring example: https://docs.ultralytics.com/guides/object-blurring/

Thanks
Ultralytics Team!

[barutlift]

so thanks ultralytics team

[pderrenger]

@barutlift you're welcome! If you have any more questions or need further assistance, feel free to reach out. Happy coding with YOLOv8! 😊🚀

[alexy208]

Hi! may i know if theres a way to get the tracking ID corresponding to each bounding box?

i know that:
detections_results = model.track(frame, persist=True)
for detect in detection_results:
#we can get the coordinates of the bounding box by
detect.boxes.xyxy[0].cpu().numpy()

 #but how do we get the id for this current bounding box?

detect.boxes.id gives the id of all bounding boxes. Does detect.boxes.id[0] correspond to the corrdinates of detect.boxes.xyxy[0].cpu().numpy()?

thanks in advance, have a nice day!

[barutlift]

detect.boxes.id[0] will likely correspond to the ID of the first bounding box, not the coordinates. In most cases, the IDs are assigned independently of the order of the bounding boxes, and they are used to track objects across frames.

So, to associate the ID with the coordinates of a specific bounding box, you should use the same index for both detect.boxes.id[i] and detect.boxes.xyxy[i].cpu().numpy() within the loop.

[barutlift]

detections_results = model.track(frame, persist=True)

for detect in detections_results:
for i in range(len(detect.boxes)):
box_id = detect.boxes[i].id
box_coordinates = detect.boxes[i].xyxy[0].cpu().numpy()
# Now 'box_id' contains the ID corresponding to 'box_coordinates'
# You can use this ID to track the

[alexy208]

ahh, thanks alot for help! =)

[glenn-jocher]

If you have any more questions or need further assistance, feel free to ask. Happy tracking! 😊👍

[barutlift]

hi again
i trained very nice a model because of you. I need more data to train the model much better, but it is difficult to label the data, so can Yolov8 create a txt file for the coordinates of the objects detected from the photographs? From what I researched, this is called active learning.

[RizwanMunawar]

@barutlift Yes, you can use save_txt with prediction command to store results or annotate the dataset.

Thanks
Ultralytics Team!

[NguyenDucQuan12]

I want to customize the annotated_frame to contain text, or remove the id, how do I do that?

[NguyenDucQuan12]

@RizwanMunawar Thank you very much, but when using the code you provided, it doesn't work like before, it doesn't detect the license plate in the video like when I used the previous code. The previous code worked perfectly, I just wanted to customize the labels, but now it shows the error as I said in the previous comment. I want it to be able to trace, draw lines, and customize labels. The code your team provided works for me, but I don't know how to fix the error. Can you show me how to fix the error?

[NguyenDucQuan12]

i change "model", "path to video" then i run code, but it's not working

[RizwanMunawar]

@NguyenDucQuan12, it appears you're addressing object tracking in an instance segmentation thread. I recommend posing your queries in the object tracking discussion section.

Thanks,
Ultralytics Team!

[NguyenDucQuan12]

@RizwanMunawar yes, i'm in the track section, and i see ultralytics provides tracking methods, but when i follow it, i get some error, and i haven't been able to resolve that error. I visit the website: https://docs.ultralytics.com/modes/track/#contribute-new-trackers and follow Plotting Tracks Over Time, how can i fix the error

[RizwanMunawar]

@NguyenDucQuan12 Please open the issue in track discussion. Thanks

[yakumoagito]

How to obtain the mask image of the target and obtain its area size

[RizwanMunawar]

@yakumoagito you can get a mask image of the target by saving the im0 in a png or JPG file i.e

import cv2
from ultralytics import YOLO
from ultralytics.utils.plotting import Annotator, colors

model = YOLO("yolov8n-seg.pt")  # segmentation model
names = model.model.names
cap = cv2.VideoCapture("path/to/video/file.mp4")
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

out = cv2.VideoWriter('instance-segmentation.avi', cv2.VideoWriter_fourcc(*'MJPG'), fps, (w, h))

while True:
    ret, im0 = cap.read()
    if not ret:
        print("Video frame is empty or video processing has been successfully completed.")
        break

    results = model.predict(im0)
    annotator = Annotator(im0, line_width=2)

    if results[0].masks is not None:
        clss = results[0].boxes.cls.cpu().tolist()
        masks = results[0].masks.xy
        for mask, cls in zip(masks, clss):
            annotator.seg_bbox(mask=mask,
                               mask_color=colors(int(cls), True),
                               det_label=names[int(cls)])

    out.write(im0)
    cv2.imwrite("test.png", im0)    # get mask image
    cv2.imshow("instance-segmentation", im0)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

out.release()
cap.release()
cv2.destroyAllWindows()

For area calculation, you can explore: https://www.wikihow.com/Calculate-Area-of-an-Object

Thanks
Ultralytics Team!

[wydxry]

det_label ->label

[glenn-jocher]

To change det_label to label, simply update the annotator.seg_bbox call to use label instead. If you need further assistance, feel free to ask!

[janardan-ds]

@RizwanMunawar, How can I get the dict containg the ids with its mapped detection, is there any way by which I can get all the ids with its detected class and frames passed to it?

[RizwanMunawar]

@janardan-ds! You can generate a dictionary and add detection data for each frame. Subsequently, you can save it in the form of a JSON file.

Best regards,
Ultralytics Team!

[janardan-ds]

Thanks @RizwanMunawar
I also have to ask if I'm tracking and suppose some particlar class is in frame and in the next 50-100 frames it missed and again came so I want to reassign the same id instead of assigning it to the new id or how can I increase the expiry time of the id's so that when it will come again after a certain period of time it reassign the same id instaed of the new one?

[Boaruzhanchik]

Good morning, please tell me how I can get the mask area not in binary display, but also in color (I want to send this area to transform the perspective of the display).

[Boaruzhanchik]

while True:
ret, im0 = cap.read()
if not ret:
break
results = model(im0)
boxes = results[0].boxes.xyxy.tolist()
classes = results[0].boxes.cls.tolist()
names = results[0].names
confidences = results[0].boxes.conf.tolist()
masks = results[0].masks

results = model.predict(im0)
annotator = Annotator(im0, line_width=1)
if boxes is not None and classes is not None and confidences is not None and masks is not None:
    for box, cls, conf, mask in zip(boxes, classes, confidences, masks):
        for box in boxes:
            x_min, y_min, x_max, y_max = map(int, box)
            roi = im0[y_min:y_max, x_min:x_max]

if results[0].masks is not None:
    clss = results[0].boxes.cls.cpu().tolist()
    masks = results[0].masks.xy

for mask, cls in zip(masks, clss):
x_min_mask, y_min_mask, x_max_mask, y_max_mask = map(int, box)
annotator.seg_bbox(mask=mask,
mask_color=colors(int(cls), True),
det_label=names[int(cls)])
roi_2 = im0[y_min_mask:y_max_mask, x_min_mask:x_max_mask]
cv2.imshow('roi_2',roi_2)

[pderrenger]

@Boaruzhanchik good morning! To obtain a color display of the mask area for perspective transformation, you can use the seg_bbox function from the Annotator class, which allows you to specify a mask_color. This color will be used to fill the mask area, and you can then extract this region from the image for further processing.

Here's a simplified example of how you might achieve this:

import cv2
from ultralytics import YOLO
from ultralytics.utils.plotting import Annotator, colors

# Load the YOLOv8 model with segmentation capability
model = YOLO('yolov8n-seg.pt')

# Open the video file
cap = cv2.VideoCapture("path/to/video/file.mp4")

while True:
    ret, im0 = cap.read()
    if not ret:
        break

    # Run inference and tracking on the frame
    results = model.track(im0, persist=True)
    annotator = Annotator(im0, line_width=1)

    # Check if masks are available in the results
    if results[0].masks is not None:
        masks = results[0].masks.xy
        clss = results[0].boxes.cls.cpu().tolist()

        # Iterate over each mask and class
        for mask, cls in zip(masks, clss):
            # Use seg_bbox to draw the mask with a specific color
            annotator.seg_bbox(mask=mask, mask_color=colors(int(cls), True))

            # Extract the mask area for perspective transformation
            # Note: You may need to modify this part to fit your specific use case
            mask_area = mask.astype(bool)  # Convert to a boolean mask
            roi_color = im0[mask_area]  # Extract the color region of interest

            # Perform perspective transformation on roi_color as needed
            # ...

    # Display the annotated frame
    cv2.imshow("YOLOv8 Segmentation and Tracking", annotator.result())

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

In this example, roi_color will contain the color region of interest extracted from the original frame im0 using the boolean mask mask_area. You can then apply your perspective transformation to roi_color.

Remember to adjust the mask extraction and perspective transformation parts to fit your specific requirements. If you need further assistance, feel free to reach out! 🌟

[vasuprint]

from collections import defaultdict

import cv2
import numpy as np

from ultralytics import YOLO

Load the YOLOv8 model

model = YOLO('yolov8n.pt')

Open the video file

Open the video file

video_path = "/workspace/Data/sort_clip_test/Video_2min.mp4"
cap = cv2.VideoCapture(video_path)
video_width = int(cap.get(3))
video_height = int(cap.get(4))
frame_size = (video_width, video_height)
out = cv2.VideoWriter(
"output_video2.avi", cv2.VideoWriter_fourcc(*"XVID"), 30, frame_size
)

Store the track history

track_history = defaultdict(lambda: [])

Loop through the video frames

while cap.isOpened():
# Read a frame from the video
success, frame = cap.read()

if success:
    # Run YOLOv8 tracking on the frame, persisting tracks between frames
    results = model.track(frame, persist=True)

    # Get the boxes and track IDs
    boxes = results[0].boxes.xywh.cpu()
    track_ids = results[0].boxes.id.int().cpu().tolist()

    # Visualize the results on the frame
    annotated_frame = results[0].plot()

    # Plot the tracks
    for box, track_id in zip(boxes, track_ids):
        x, y, w, h = box
        track = track_history[track_id]
        track.append((float(x), float(y)))  # x, y center point
        if len(track) > 30:  # retain 90 tracks for 90 frames
            track.pop(0)

        # Draw the tracking lines
        points = np.hstack(track).astype(np.int32).reshape((-1, 1, 2))
        cv2.polylines(annotated_frame, [points], isClosed=False, color=(230, 230, 230), thickness=10)

    # Write the frame to the output video
    out.write(annotated_frame)

Release the video capture object and close the display window

out.release()
cap.release()

I got the error:

AttributeError Traceback (most recent call last)
Cell In[3], line 36
34 # Get the boxes and track IDs
35 boxes = results[0].boxes.xywh.cpu()
---> 36 track_ids = results[0].boxes.id.int().cpu().tolist()
38 # Visualize the results on the frame
39 annotated_frame = results[0].plot()

AttributeError: 'NoneType' object has no attribute 'int'

And my output is until error:
0: 384x640 4 persons, 2.2ms
Speed: 0.9ms preprocess, 2.2ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)

0: 384x640 4 persons, 2.2ms
Speed: 0.9ms preprocess, 2.2ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)

0: 384x640 4 persons, 2.1ms
Speed: 0.9ms preprocess, 2.1ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)
...
Speed: 0.9ms preprocess, 2.1ms inference, 0.5ms postprocess per image at shape (1, 3, 384, 640)

0: 384x640 (no detections), 2.1ms
Speed: 1.0ms preprocess, 2.1ms inference, 0.2ms postprocess per image at shape (1, 3, 384, 640)

Could you tell me how to fix it please?
I use try to install ! pip install -U ultralytics Again to new version but it still get error. (I use on docker)

My question2
Object detection + tracking can't use infarence with gpu? Because i check gpu process.

Best regard
Print

[vasuprint]

Sorry I forget deleate comment with '#'
This is my code
from collections import defaultdict

import cv2
import numpy as np

'from ultralytics import YOLO

model = YOLO('yolov8n.pt')

video_path = "/workspace/Data/sort_clip_test/Video_2min.mp4"
cap = cv2.VideoCapture(video_path)
video_width = int(cap.get(3))
video_height = int(cap.get(4))
frame_size = (video_width, video_height)
out = cv2.VideoWriter(
"output_video2.avi", cv2.VideoWriter_fourcc(*"XVID"), 30, frame_size
)

track_history = defaultdict(lambda: [])

while cap.isOpened():

success, frame = cap.read()

if success:

    results = model.track(frame, persist=True)

    boxes = results[0].boxes.xywh.cpu()
    track_ids = results[0].boxes.id.int().cpu().tolist()

    annotated_frame = results[0].plot()

    for box, track_id in zip(boxes, track_ids):
        x, y, w, h = box
        track = track_history[track_id]
        track.append((float(x), float(y)))  # x, y center point
        if len(track) > 30:  # retain 90 tracks for 90 frames
            track.pop(0)


        points = np.hstack(track).astype(np.int32).reshape((-1, 1, 2))
        cv2.polylines(annotated_frame, [points], isClosed=False, color=(230, 230, 230), thickness=10)

    out.write(annotated_frame)

out.release()
cap.release()'

[pderrenger]

@vasuprint it seems like you're encountering an AttributeError because the id attribute of the boxes object is None. This could happen if the tracking IDs are not being generated or returned by the model.track() method. Here are a few steps to troubleshoot and resolve the issue:

1. Check Model Compatibility: Ensure that the model you're using (yolov8n.pt) supports object tracking and is capable of returning track IDs. If the model does not support tracking, you'll need to use a model that does.

2. Update Ultralytics YOLOv8: You mentioned trying to update the Ultralytics package. Make sure you have the latest version by running:

   pip install -U ultralytics

   If you're using Docker, ensure that the Docker image is updated with the latest package.

3. Check for Detections: Before accessing the id attribute, check if there are any detections. If there are frames with no detections, the id attribute will be None. You can add a condition to handle such cases:

   if results[0].boxes.id is not None:
       track_ids = results[0].boxes.id.int().cpu().tolist()
       # Rest of your code...

4. GPU Inference: Object detection and tracking can use GPU for inference. Make sure your Docker container has access to the GPU and that you're specifying the correct device when loading the model. For example:

   model = YOLO('yolov8n.pt', device='cuda')

   Also, ensure that NVIDIA drivers and CUDA are properly installed and accessible within the Docker container.

5. Check GPU Utilization: To check if the GPU is being utilized during inference, you can use tools like nvidia-smi while your script is running.

If you continue to face issues, consider reaching out to the Ultralytics team with detailed information about the error, the model you're using, and the environment setup. You can open an issue on the Ultralytics YOLOv8 GitHub repository for further assistance.

[barutlift]

can i detect text in a video using yolov8 and how?

[barutlift]

im sorry i asked by mistake.

i have a large dataset and the label format for this dataset is different from the yolov format. my goal is to convert bounding boxes to the yolov format and train a model. for example i have bounding box coordinates like [525.83, 3.4, 197.64, 33.94], and the width and height of the image are 840 and 1024 respectively. I normalize the coordinates between 0 and 1 for yolov and visualize them but there is a shift in the coordinates

[pderrenger]

@barutlift to convert your bounding box coordinates to the YOLO format, you need to transform them into the format [class_id x_center y_center width height], where all the values are normalized between 0 and 1 relative to the image width and height.

Given your example bounding box [525.83, 3.4, 197.64, 33.94] and the image dimensions (width=840, height=1024), the conversion process would be as follows:

1. Calculate the center of the bounding box (x_center, y_center).
2. Normalize the center coordinates and the width and height of the bounding box by the image width and height.
3. Ensure you have the class_id for the object class the bounding box belongs to.

Here's how you can do it:

# Your bounding box coordinates
bbox = [525.83, 3.4, 197.64, 33.94]

# Image dimensions
img_width, img_height = 840, 1024

# Calculate the center of the bounding box
x_center = bbox[0] + bbox[2] / 2
y_center = bbox[1] + bbox[3] / 2

# Normalize the center coordinates and width and height
x_center_norm = x_center / img_width
y_center_norm = y_center / img_height
width_norm = bbox[2] / img_width
height_norm = bbox[3] / img_height

# Assuming class_id is 0 for demonstration
class_id = 0

# YOLO format bounding box
yolo_bbox = [class_id, x_center_norm, y_center_norm, width_norm, height_norm]

# Print the YOLO format bounding box
print("YOLO format bounding box:", yolo_bbox)

If you're experiencing a shift in the coordinates when visualizing them, it could be due to incorrect calculations or visualization code. Double-check your calculations and ensure that the visualization tool or library you're using interprets the coordinates correctly. If you're using OpenCV or a similar library, remember that it expects the top-left corner coordinates and the width and height, not the center coordinates.

If you need further assistance, please provide more details or the code snippet you're using for visualization, and I'll be happy to help you debug the issue.

[ChenYefet]

For the following code, can there be multiple Results objects within the results list, or is the results list guaranteed to contain only a single Results object? When would the results list be of a size greater than 1?

frame_was_successfully_read, frame = video_capture.read()

if frame_was_successfully_read:
    results = model.track(frame, persist = True)

[RizwanMunawar]

By default, this isn't achievable, but you can generate a list of results to store the outcomes for each frame and subsequently utilize it for your particular task.

Thanks
Ultralytics Team!

[ChenYefet]

Thank you!

Why does the model output a list of a single Results object instead of just the Results object. Is there any benefit to putting it in a list?

[glenn-jocher]

@ChenYefet the model outputs a list of Results objects to maintain consistency across different use cases. When processing batches of images or video frames, the list can contain multiple Results objects, one for each image or frame. By always returning a list, the API remains consistent whether you're processing a single image or multiple images, simplifying the user experience and the underlying code logic.

In scenarios where you're processing a stream or a batch, having a list allows you to iterate over the results or index into specific frames' results if needed. This design choice provides flexibility and scalability for different types of input without changing the interface.

If you have any more questions or need further clarification, feel free to ask!

[sanvir19]

it is possible to perform object tracking using YOLOv8 with a focus on two specific objects out of the 10 that the model detects. The code can be modified to track only those two predefined objects, providing you with the ability to monitor their movements and positions across frames. If you could provide additional details about the two objects you wish to track, I can assist you in adapting the code accordingly.

[pderrenger]

@sanvir19 absolutely, YOLOv8 supports tracking specific objects by filtering the detection classes. You can modify the classes parameter in the track() method to include only the class indices of the two objects you're interested in. Here's a concise example of how you might adjust the tracking code:

from ultralytics import YOLO

# Load your trained model
model = YOLO('path/to/your/model.pt')

# Define the class indices for the two objects you want to track
# For example, if you want to track 'person' (class index 0) and 'car' (class index 2)
classes_to_track = [0, 2]

# Perform tracking, filtering only the specified classes
results = model.track(source='path/to/video.mp4', classes=classes_to_track, stream=True)

# Now, 'results' will only contain detections and tracks for 'person' and 'car'

Remember to replace 'path/to/your/model.pt' and 'path/to/video.mp4' with the actual paths to your model and video file, and adjust classes_to_track with the correct class indices for your use case. If you need further customization or have any questions, feel free to reach out! 🤝

[shyam192]

how to get the detected label of the frame

[RizwanMunawar]

@shyam192 you can use the mentioned code to display the detected label.

import cv2
from ultralytics import YOLO
from ultralytics.utils.plotting import Annotator, colors

model = YOLO("yolov8n-seg.pt")  # segmentation model
names = model.model.names
cap = cv2.VideoCapture("path/to/video/file.mp4")
w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS))

out = cv2.VideoWriter('instance-segmentation.avi', cv2.VideoWriter_fourcc(*'MJPG'), fps, (w, h))

while True:
    ret, im0 = cap.read()
    if not ret:
        print("Video frame is empty or video processing has been successfully completed.")
        break

    results = model.predict(im0)
    annotator = Annotator(im0, line_width=2)

    if results[0].masks is not None:
        clss = results[0].boxes.cls.cpu().tolist()
        masks = results[0].masks.xy
        for mask, cls in zip(masks, clss):
            annotator.seg_bbox(mask=mask,
                               mask_color=colors(int(cls), True),
                               det_label=names[int(cls)])
            print("Detection Label : ", names[int(cls)]) 
    out.write(im0)
    cv2.imshow("instance-segmentation", im0)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

out.release()
cap.release()
cv2.destroyAllWindows()

Hope this helps! Let me know if you have any additional questions.

Thanks and Regards
Ultralytics Team!

[AdadAlShabab]

Facing problem with Persisting Tracks Loop:

Traceback (most recent call last):
File "e:\Ultralytics\tracking.py", line 29, in
track_ids = results[0].boxes.id.int().cpu().tolist()
^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'int'

[RizwanMunawar]

@AdadAlShabab please upgrade the pip package using the mentioned command, hopefully, it will resolve the issue.

pip install -U ultralytics

Thanks
Ultralytics Team!

[AdadAlShabab]

Hi. I have a question. can I ask, please?

…

On Mon, 29 Jan 2024 at 14:15, Muhammad Rizwan Munawar < ***@***.***> wrote: @AdadAlShabab <https://github.com/AdadAlShabab> please upgrade the pip package using the mentioned command, hopefully, it will resolve the issue. pip install -U pip Thanks Ultralytics Team! — Reply to this email directly, view it on GitHub <#7112 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWV4NVQLRD4CEJTVX44FDRLYQ5LCNAVCNFSM6AAAAABA6L2DRWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DENZYGMZDK> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[RizwanMunawar]

Sure!

[NguyenDucQuan12]

I am processing images with a camera, but it has a very large delay between frames. I want to process in real time with low latency (about 1.2s) or maybe skip a few frames to save time. it always gets the current images. Can you give me some solutions

[RizwanMunawar]

@NguyenDucQuan12 try to use the yolov8n-seg.pt model, and you can also reduce the imgsz parameter for better speed. i.e,

results = model.predict(im0, imgsz=320)

I hope this helps! Let me know if you have any additional questions.

Thanks and Regards
Ultralytics Team!

[NguyenDucQuan12]

@RizwanMunawar I tried but it only reduced the delay a little, is there any other way I can try, in each frame I detect, process OCR and write the text on the frame.
Do you have any suggestions for how I can get the lowest latency, thank you

[AdadAlShabab]

@RizwanMunawar Can I run two models simultaneously in one video?
Ex:
model = YOLO('E:/Ultralytics/best.pt')
model = YOLO('yolov8m-pose.pt')
results = model.track(source="file.mp4", show=True, conf=0.3, save=False)
results = model.track(source="file.mp4", show=True, conf=0.3, save=False)

I want that two models will works simultaneously with cumulative results? Is it possible? Please let me know. Thanks in advance!!

[NguyenDucQuan12]

@AdadAlShabab I think you can't, if you want you can try: model1=.... mode2= .....

[RizwanMunawar]

@AdadAlShabab Yes: https://docs.ultralytics.com/modes/track/#multithreaded-tracking