datasets/pose/

[giscus[bot]]

datasets/pose/

Understand the YOLO pose dataset format and learn to use Ultralytics datasets to train your pose estimation models effectively.

https://docs.ultralytics.com/datasets/pose/

[Xiaon-Junf]

Except for kpt_shape: and flip_idx: , can I use "skeleton: []" in my yaml to tell my model how to connect each key point? Or I should write: annotator = Annotator(im=your_image_array, skeleton=custom_skeleton) in my train.py? Or I can do noting while training, and do key point connect job after train?

[pderrenger]

@Xiaon-Junf in the Ultralytics YOLOv8 framework, the skeleton parameter is not used within the dataset YAML file for defining keypoint connections during training. The YAML file primarily specifies the dataset paths, number of keypoints, dimensions, and class names.

For custom keypoint connections (skeleton), you typically handle this post-training when visualizing the results. The Annotator class can be used to draw the keypoints with connections on the images after the model has made predictions. You would specify the skeleton parameter when initializing the Annotator with your custom connections.

Here's an example of how you might use the Annotator for visualization with a custom skeleton:

from ultralytics import YOLO, Annotator

# Define your custom skeleton as a list of connections. Each connection is a tuple of keypoint indices.
custom_skeleton = [(0, 1), (1, 2), (2, 3), ...]

# Load a pretrained YOLOv8 pose model
model = YOLO('yolov8n-pose.pt')

# Run inference on an image
results = model('path/to/your/image.jpg')

# Annotate image with custom skeleton
annotator = Annotator(im=results.imgs[0], skeleton=custom_skeleton)
annotated_image = annotator.image

During training, the model learns to predict the keypoints based on the labels provided in the dataset, but it does not learn the connections between keypoints. The connections are typically used only for visualization purposes and do not influence the training process directly.

If you want to visualize keypoints with connections during training (for example, on validation images), you would modify the code that handles visualization in your training script, similar to the example above. However, for the actual training process, you do not need to specify the skeleton.

[Xiaon-Junf]

感谢您的回复，但是我还是有以下疑问。
首先，我没办法运行这段代码，Annotator类在plotting.py文件中，我将其引入，没有办法使用annotator = Annotator(im=results.imgs[0], skeleton=custom_skeleton)
其次，我发现yolov8在预测人类的图像时并不需要你的这段代码就可以将骨架默认连接起来，参考：https://docs.ultralytics.com/zh/datasets/pose/coco8-pose/#introduction
这个预测集里不仅输出了人类的各关节点，而且可以将它们按一定的顺序、颜色连接起来。
而tiger pose预测集的输出参考如下：
https://docs.ultralytics.com/zh/datasets/pose/tiger-pose/
只能观察到老虎的姿态点，并没有骨架的连接。
所以我产生了疑问，如果这些连接仅用于可视化目的，为什么人类的骨架连接是在训练前或训练中进行的，而其他东西的骨架连接是在训练后再进行的？
最后我想知道如何做到像yolo-pose模型预测人类那样直接输出骨架，也想知道你发给我的代码应该如何修改才能正确地输出连接后的骨架图。
望您工作顺利，身体健康。

[pderrenger]

@Xiaon-Junf 感谢您的进一步询问！🙌

对于您提到的Annotator类的使用问题，确实需要正确引入才能使用。如果您遇到困难，可能需要检查您的环境配置是否正确导入了Ultralytics YOLO库。通常，您可以这样做：

from ultralytics import Annotator

确保您的环境中已经安装了最新版本的Ultralytics YOLO。

关于骨架连接的问题，您是对的，YOLOv8在预测时会自动根据模型训练时的关键点信息进行连接。这是因为模型已经学习了关键点之间的相对位置关系。对于不同的数据集（如人类和老虎），连接的默认方式可能会有所不同，这取决于训练数据集中关键点的定义和顺序。

如果您想要自定义骨架连接，确实需要在预测后进行处理。这通常涉及到使用Annotator类或其他可视化工具来绘制您想要的连接。例如，如果您想要像人类骨架那样直接输出老虎的骨架，您可以在预测后使用自定义的骨架连接进行绘制。

# 假设results是您的预测结果
annotator = Annotator(im=results.imgs[0])
for kpt in results.keypoints:
    annotator.line(kpt[0], kpt[1], color=(255, 0, 0))  # 假设kpt是关键点坐标的列表
annotated_img = annotator.result()

请注意，上述代码只是一个示例，您可能需要根据您的具体需求调整关键点的索引和颜色等参数。

希望这能帮助您解决问题！如果还有其他疑问，欢迎继续提问。祝您工作顺利，身体健康！🍀

[Dmonzer]

How to get the skeleton values? Do they have to be based on the subject? If I don't have custom skeleton

[glenn-jocher]

Hey there! 👋 To get the skeleton values for pose estimation, you don't necessarily need a custom skeleton. You can use predefined keypoints, like those in the COCO-Pose dataset which has 17 keypoints for human poses.

Here's a quick example of how the label format would look for a pose estimation task with keypoints:

<class-index> <x-center> <y-center> <width> <height> <px1> <py1> <px2> <py2> ... <px17> <py17>

Each <pxn> and <pyn> pair represents the normalized x and y coordinates of a keypoint. If you're using the Ultralytics YOLO format, the dataset and model configuration for training is defined in a YAML file, which you'll need to set up according to your dataset's specifics.

If you're starting out and don't have a custom dataset, you might want to try the COCO8-Pose or Tiger-Pose datasets provided by Ultralytics for practice. They follow the same format and are great for training and testing models. Happy coding! 😊

[KhizarHashmi]

I want to use yolo pose estimation on custom dataset, for person activity detection, for annotations I am using cvat annotation tool for skeleton making for each separate activity like working, using_mobile and discussing on each person. After that I am confused on how to extract the json file for yolo format, as what I am getting for my code, when I plot back the contents of the txt file on the jpg file I can't get the suitable output.

Code for Converting JSON files according to YOLO format:

import json
import os

Load the JSON file

with open('person_keypoints_default.json') as f:
data = json.load(f)

Define a mapping from keypoints to pairs for skeleton

skeleton_pairs = data['categories'][0]['skeleton']

Loop over the annotations and process each

for annotation in data['annotations']:
image_id = annotation['image_id']
category_id = annotation['category_id']
keypoints = annotation['keypoints']

# Find the corresponding image info
image_info = next((item for item in data['images'] if item['id'] == image_id), None)
if image_info:
    # Construct the annotation file path
    annotation_filename = os.path.splitext(image_info['file_name'])[0] + '.txt'
    annotation_filepath = os.path.join('./labels', annotation_filename)
    
    # Get the bounding box details
    bbox = annotation['bbox']
    x_center = (bbox[0] + bbox[2] / 2) / image_info['width']
    y_center = (bbox[1] + bbox[3] / 2) / image_info['height']
    width = bbox[2] / image_info['width']
    height = bbox[3] / image_info['height']
    
    # Prepare the keypoints
    kp_str = ''
    for i in range(0, len(keypoints), 3):
        x = keypoints[i] / image_info['width']
        y = keypoints[i + 1] / image_info['height']
        v = keypoints[i + 2]
        kp_str += f' {x} {y} {v}'

    # Combine class id, bbox and keypoints in YOLO format
    yolo_annotation = f'0 {x_center} {y_center} {width} {height}{kp_str}\n'

    # Write to the corresponding annotation file
    with open(annotation_filepath, 'w') as file:
        file.write(yolo_annotation)

Code for View Plottings after conversions for verification:

import cv2
import matplotlib.pyplot as plt
import os

def draw_keypoints(image, keypoints, pairs, colors):
for i, (x, y, v) in enumerate(keypoints):
if v == 1: # Check if the keypoint is visible
cv2.circle(image, (int(x), int(y)), 3, colors[i % len(colors)], 4)
cv2.putText(image, str(i+1), (int(x), int(y)), cv2.FONT_HERSHEY_SIMPLEX, 0.5, colors[i % len(colors)], 1)

for start, end in pairs:
    if keypoints[start-1][2] == 1 and keypoints[end-1][2] == 1:
        cv2.line(image, (int(keypoints[start-1][0]), int(keypoints[start-1][1])), (int(keypoints[end-1][0]), int(keypoints[end-1][1])), (0, 255, 0), 2)

def plot_image(image_path, label_path):
image = cv2.imread(image_path)
height, width, _ = image.shape

with open(label_path, 'r') as file:
    annotations = file.readlines()

for annotation in annotations:
    parts = annotation.strip().split()
    class_id, x_center, y_center, bbox_width, bbox_height = map(float, parts[:5])
    keypoints = [(float(parts[i]) * width, float(parts[i+1]) * height, int(parts[i+2])) for i in range(5, len(parts), 3)]

    # Define pairs based on the skeleton structure
    pairs = [
        (4,6), (5,7), (9,11), (16,14), (1,3), (15,17), (6,8), (7,13), (12,6), (14,12),
        (2,4), (1,2), (7,9), (7,6), (3,5), (8,10), (2,3), (13,12), (13,15)
    ]

    # Define colors for each keypoint
    colors = [(0, 255, 0), (255, 0, 0), (0, 0, 255), (255, 255, 0), (255, 0, 255), (0, 255, 255)] * 3

    # Draw keypoints and skeleton
    draw_keypoints(image, keypoints, pairs, colors)

    # Draw bounding box
    x1, y1 = int((x_center - bbox_width / 2) * width), int((y_center - bbox_height / 2) * height)
    x2, y2 = int((x_center + bbox_width / 2) * width), int((y_center + bbox_height / 2) * height)
    cv2.rectangle(image, (x1, y1), (x2, y2), (255, 0, 0), 2)

plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
plt.axis('off')
plt.show()

Example usage

image_path = './images/20240212_165900.jpg'
label_path = './labels/20240212_165900.txt'
plot_image(image_path, label_path)

If anyone here could guide me on this, I will be highly obliged.
Thanks

[KhizarHashmi]

Yes!

Here is my label file contents,
frame_0001.txt:
0 0.4744967948717948 0.6004663461538461 0.3664294871794872 0.5516153846153846 0.503849358974359 0.34738461538461535 2 0.5280833333333333 0.3265600961538462 2 0.5198076923076923 0.32465865384615383 1 0.6040320512820513 0.33848317307692305 2 0.6021987179487179 0.33484134615384614 1 0.6541666666666667 0.46188221153846154 2 0.6414102564102564 0.4235841346153846 1 0.5249935897435898 0.6376826923076924 2 0.4833685897435897 0.4776754807692308 2 0.29802884615384617 0.5843725961538461 2 0.3453269230769231 0.47012740384615387 2 0.6577115384615384 0.7064855769230769 2 0.632801282051282 0.6108533653846154 1 0.3461891025641025 0.6778822115384615 2 0.4022724358974359 0.528 2 0.36106730769230766 0.8762740384615385 2 0.29128205128205126 0.6455721153846153 2

Same for all:
"width":3120,"height":4160

[KhizarHashmi]

Kindly see the below json file content for annotation key:
"annotations":[{"id":1,"image_id":1,"category_id":1,"segmentation":[],"area":2623461.5872000004,"bbox":[908.8,1350.58,1143.26,2294.72],"iscrowd":0,"attributes":{"occluded":false,"track_id":0,"keyframe":true},"keypoints":[1572.01,1445.12,2,1647.62,1358.49,2,1621.8,1350.58,1,1884.58,1408.09,2,1878.86,1392.94,1,2041.0,1921.43,2,2001.2,1762.11,1,1637.98,2652.76,2,1508.11,1987.13,2,929.85,2430.99,2,1077.42,1955.73,2,2052.06,2938.98,2,1974.34,2541.15,1,1080.11,2819.99,2,1255.09,2196.48,2,1126.53,3645.3,2,908.8,2685.58,2],"num_keypoints":17

And My code logic below:

import json
import os

with open('/media/khizar_smr/Drive_B/Khizar_data_2/EAD_2/Employee_Activity_Data/annotations/waqar_skeleton_2/annotations/person_keypoints_default.json') as f:
data = json.load(f)

skeleton_pairs = data['categories'][0]['skeleton']

for annotation in data['annotations']:
image_id = annotation['image_id']
category_id = annotation['category_id']
keypoints = annotation['keypoints']

image_info = next((item for item in data['images'] if item['id'] == image_id), None)
if image_info:
    # Construct the annotation file path
    annotation_filename = os.path.splitext(image_info['file_name'])[0] + '.txt'
    annotation_filepath = os.path.join('/media/khizar_smr/Drive_B/Khizar_data_2/EAD_2/dataset/waqar_working', annotation_filename)
    
    bbox = annotation['bbox']
    x_center = (bbox[0] + bbox[2] / 2) / image_info['width']
    y_center = (bbox[1] + bbox[3] / 2) / image_info['height']
    width = bbox[2] / image_info['width']
    height = bbox[3] / image_info['height']
    
    kp_str = ''
    for i in range(0, len(keypoints), 3):
        x = keypoints[i] / image_info['width']
        y = keypoints[i + 1] / image_info['height']
        v = keypoints[i + 2]
        kp_str += f' {x} {y} {v}'

    yolo_annotation = f'0 {x_center} {y_center} {width} {height}{kp_str}\n'

    with open(annotation_filepath, 'w') as file:
        file.write(yolo_annotation)

If I am making my label file fine please tell me how should I edit my plotting code.
Thanks

[glenn-jocher]

Hey there! 👋 It looks like you're doing a great job with your custom dataset for pose estimation. If the keypoints aren't plotting correctly, it might be due to the normalization or the drawing logic. Here's a quick tip: make sure the keypoint coordinates in your .txt files are normalized correctly with respect to the image dimensions (width: 3120, height: 4160). When plotting, scale them back to the original image size.

For the plotting code, ensure that the keypoint pairs match the skeleton structure defined in your dataset. If you're still facing issues, try visualizing just one keypoint to simplify debugging. Once that's correct, move on to plotting the skeleton.

If you need more help, the Ultralytics Pose Estimation Datasets Overview provides great insights into the expected format and usage. Keep up the fantastic work! 😊🚀

[KhizarHashmi]

Hello! I greatly appreciate the support and guidance provided by the Ultralytics team. Their responsiveness and assistance to the community are commendable. Regarding the plotting code, I have successfully written it to meet the requirements. To assist others and facilitate improvements, I am sharing it below:

CODE:

import json
import cv2
import matplotlib.pyplot as plt

json_file = 'person_keypoints_default.json'
image_path = 'frame_001.jpg'
label_path = 'frame_001.txt'

#Load JSON file for skeleton information
with open(json_file, 'r') as f:
data = json.load(f)

#Load label file for bounding box and keypoints
with open(label_path, 'r') as f:
label_data = f.read().strip().split()

#Load image
image_file = image_path
image = cv2.imread(image_file)
height, width, _ = image.shape

#Resize image if it's too large
max_dimension = 640 # Maximum width or height of the image
scale_factor = min(max_dimension / width, max_dimension / height)
if scale_factor < 1:
image = cv2.resize(image, (int(width * scale_factor), int(height * scale_factor)))
height, width, _ = image.shape

#Parse label file data
#Format: [class_id, x_center, y_center, width, height, ...keypoints...]
label_data = [float(x) for x in label_data]
class_id = int(label_data[0])
num_keypoints = (len(label_data) - 5) // 3

#Convert from normalized to pixel coordinates for bounding box
x_center, y_center, bbox_width, bbox_height = [int(coord * dim) for coord, dim in zip(label_data[1:5], [width, height, width, height])]
bbox_x1, bbox_y1 = x_center - bbox_width // 2, y_center - bbox_height // 2
bbox_x2, bbox_y2 = x_center + bbox_width // 2, y_center + bbox_height // 2

#Plot bounding box
cv2.rectangle(image, (bbox_x1, bbox_y1), (bbox_x2, bbox_y2), (0, 0, 255), 2)

#Display class ID on top of the bounding box
cv2.putText(image, f'Class ID: {class_id}', (bbox_x1, bbox_y1 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

#Plot keypoints and visibility
keypoints = label_data[5:]
for i in range(num_keypoints):
x, y, v = keypoints[i3:i3+3]
x, y = int(x * width), int(y * height)
if v == 1: # Check if the keypoint is occluded
cv2.circle(image, (x, y), 5, (255, 255, 255), -1) # White for occluded
elif v == 2: # Check if the keypoint is visible
cv2.circle(image, (x, y), 5, (0, 255, 0), -1) # Red for visible

#Draw lines between keypoints using skeleton info from JSON
skeleton = data['categories'][0]['skeleton']
for pair in skeleton:
pt1_idx, pt2_idx = (pair[0] - 1) * 3, (pair[1] - 1) * 3
if keypoints[pt1_idx + 2] > 0 and keypoints[pt2_idx + 2] > 0: # Check if both keypoints are visible or occluded
pt1 = (int(keypoints[pt1_idx] * width), int(keypoints[pt1_idx + 1] * height))
pt2 = (int(keypoints[pt2_idx] * width), int(keypoints[pt2_idx + 1] * height))
cv2.line(image, pt1, pt2, (255, 0, 0), 2)

#plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
#plt.axis('off')
#plt.show()

#Show image
cv2.imshow('Image with Bounding Box and Keypoints', image)
cv2.waitKey(0)
cv2.destroyAllWindows()

[pderrenger]

Hey there! Thanks for sharing your plotting code with the community. It's always great to see collaboration and knowledge-sharing in action. 🤝

Your code looks well-structured and should help others who are working on similar tasks. If anyone is looking to adapt this for their own use, they might want to adjust the skeleton variable to match their specific dataset's keypoint connections.

For those using different datasets or keypoint structures, remember to update the skeleton pairs in the code to match your annotations. This ensures that the lines connecting keypoints accurately represent the intended pose.

Keep up the fantastic work, and if you have any further questions or need assistance, feel free to reach out! Happy coding! 😊👨‍💻

[mattcattb]

I would like to validate some of these models on the raspberry pi, but find the coco8 dataset to be too small and the coco dataset to be too large to store on a pi. How can I create my own dataset and customize how many images are stored within it?

[pderrenger]

@mattcattb hey there! 👋 For creating your own dataset with a customized number of images, you can follow the Ultralytics YOLO format outlined in the docs. Simply gather your images, then for each image, create a corresponding text file (.txt) with the same name as your image file. Inside this text file, format your annotations as described, including object class indices and normalized bounding box coordinates. Here’s a quick example if you're doing pose estimation with 2 dimensions:

0 0.5 0.5 0.2 0.2 0.25 0.25 0.75 0.75

This represents a single object in the image with the class index 0, centered at (0.5, 0.5) with a width and height of (0.2, 0.2), and keypoint coordinates at (0.25, 0.25) and (0.75, 0.75).

For the dataset setup, create a simple YAML file like this:

path: path/to/dataset 
train: images/train  
val: images/val  
names:
  0: your_class_name

Just make sure your images are placed in the directories specified in the YAML file. You can freely decide on the number of images to include by simply managing the content in these directories. Happy dataset crafting! 🎨

[benicio22]

Hello, I want to take the coordinates of each keypoint and classify them with numbers. For example, keypoint 1 has coordinates x,y; keypoint 2 has coordinates x1,y2 and so on, I want to do this for my entire dataset. However, I'm getting errors.
This is my code.
from ultralytics import YOLO
import cv2

model_path = 'C:/Users/idigi/Box/UIUC_ACADEMIC/PHD_THESIS/PROJECTS/GAIT_CATTLE/RESULTS/POSE_MODEL/POSE_TRAIN_MODEL/RESULTS_V1_20EPOCHS_500IMAGES/weights/best.pt'

image_path = 'C:/Users/idigi/Box/UIUC_ACADEMIC/PHD_THESIS/PROJECTS/GAIT_CATTLE/ANGLE_DATASET/DAY/06_13_19/6.13.19-124F/6.13.19-124F_Color_1560438142923.505_198.jpg'
img = cv2.imread(image_path)

model = YOLO(model_path)

results = model(image_path)[0]

for result in results:
for keypoint_indx, keypoint in enumerate(result.keypoints):
# Assuming keypoint is a different data structure, adjust accordingly
# For example, if keypoint is a list or tuple containing the coordinates:
x, y = keypoint[0], keypoint[1] # Assuming keypoint is a list or tuple
cv2.putText(img, str(keypoint_indx), (int(x), int(y)),
cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

cv2.imshow('img', img)
cv2.waitKey(0)

And this is the error:
Traceback (most recent call last):
File "C:\Users\idigi\Documents\EMBRAPA_PROJECT_2023\GAIT_CATTLE_2024\EXTRACT_ANGLES_GAIT.py", line 19, in
x, y = keypoint[0], keypoint[1] # Assuming keypoint is a list or tuple
~~~~~~~~^^^
File "C:\Users\idigi\Documents\EMBRAPA_PROJECT_2023\venv\Lib\site-packages\ultralytics\engine\results.py", line 63, in getitem
return self.class(self.data[idx], self.orig_shape)
~~~~~~~~~^^^^^
IndexError: index 1 is out of bounds for dimension 0 with size 1

Could you help, please? Thank you

[pderrenger]

Hey there! 😊 It looks like there's an issue with how the keypoints are unpacked in your loop. Your model's keypoints attribute returns a Keypoints object, which seems to be structured differently than expected. Let's adjust the code to correctly access the x and y coordinates of each keypoint.

First, ensure you're accessing the keypoints correctly:

keypoints = res.keypoints.xy  # This should give a 2D tensor of shape [num_keypoints, 2]

However, the error hints that the structure might be more complex or not a direct list of tuples (x, y). Let's try accessing individual keypoints with a slight modification:

keypoints = res.keypoints.xy  # Assuming this gives us the keypoints directly
for idx, kpt in enumerate(keypoints):
    x, y = kpt[0].item(), kpt[1].item() # Convert tensor to a regular Python scalar with .item()
    cv2.putText(img, str(idx), (int(x), int(y)), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

Your print statement shows a rich object structure, and we need to ensure we correctly navigate it. Make sure the res.keypoints.xy actually points to the keypoints data directly. If your keypoints have a different structure than expected, you might need to adjust the unpacking to match that structure.

If keypoints are nested or packed differently, you may need to debug further by printing the structure of res.keypoints.xy directly and adjust the access pattern accordingly. Keep in mind YOLOv8's pose estimation keypoints are expected to be in a format accessible as shown, based on the documentation and examples provided.

Let me know if this helps or if you're seeing a different structure! 🚀

[benicio22]

Hi, thanks for the helping. I did what you said. And had this error:
File "C:\Users\idigi\Documents\EMBRAPA_PROJECT_2023\GAIT_CATTLE_2024\EXTRACT_ANGLES_GAIT.py", line 20, in
x, y = kpt[0].item(), kpt[1].item() # Convert tensor to a regular Python scalar with .item()
^^^^^^^^^^^^^
RuntimeError: a Tensor with 2 elements cannot be converted to Scalar

Here is my code:
from ultralytics import YOLO
import cv2

model_path = 'C:/Users/idigi/Box/UIUC_ACADEMIC/PHD_THESIS/PROJECTS/GAIT_CATTLE/RESULTS/POSE_MODEL/POSE_TRAIN_MODEL/RESULTS_V1_20EPOCHS_500IMAGES/weights/best.pt'

image_path = 'C:/Users/idigi/Box/UIUC_ACADEMIC/PHD_THESIS/PROJECTS/GAIT_CATTLE/ANGLE_DATASET/DAY/06_13_19/6.13.19-124F/6.13.19-124F_Color_1560438142923.505_198.jpg'

model = YOLO(model_path)

results = model.predict(image_path)

img=cv2.imread(image_path)
for res in results:
keypoints = res.keypoints.xy # This should give a 2D tensor of shape [num_keypoints, 2]
print(keypoints)
keypoints = res.keypoints.xy # Assuming this gives us the keypoints directly
for idx, kpt in enumerate(keypoints):
x, y = kpt[0].item(), kpt[1].item() # Convert tensor to a regular Python scalar with .item()
cv2.putText(img, str(idx), (int(x), int(y)), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

And I also printed the keypoint:tensor([[[ 979.6750, 357.6494],
[1039.8950, 302.0255],
[1145.4583, 288.6907],
[1245.9303, 286.9016],
[1232.6051, 381.2330],
[1233.4971, 407.7163],
[1207.6041, 459.5955],
[1236.7977, 377.1603],
[1261.2654, 398.0333],
[1222.8828, 474.4590],
[1077.8118, 400.9701],
[1077.3250, 435.8143],
[1080.9961, 456.6836],
[1094.6233, 395.8533],
[1093.1389, 433.5782],
[1102.5928, 469.3304]]])
Could you help me to fix this, please? Thank you

[pderrenger]

Hey there!

It looks like the error you're encountering stems from trying to unpack a PyTorch tensor directly, which mismatches your expectation for simple (x, y) values. Based on the structure you printed, it seems your keypoints tensor is multi-dimensional, and you're handling it as a 2D list. Let's adjust the way you access the keypoints:

Given your keypoints tensor is:

tensor([[[ 979.6750, 357.6494],
         [1039.8950, 302.0255],
         ...
         [1102.5928, 469.3304]]])

It's nested within one additional list. If this structure is consistent, you should index the first list to get to the keypoints.

Try this adjustment in your loop:

keypoints = res.keypoints.xy[0]  # Access the first item to get the list of keypoints
for idx, kpt in enumerate(keypoints):
    x, y = kpt[0].item(), kpt[1].item()  # Now kpt directly refers to each [x, y]
    cv2.putText(img, str(idx), (int(x), int(y)), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 0), 2)

This change should help avoid the RuntimeError and correctly unpack the X, Y coordinates from your keypoints tensor.

Hope this fix gets your script running smoothly! Let me know if there's anything else I can help with. 😊

[benicio22]

Hi, Thank you for the help. Now it is working. Thank you so much.

[pderrenger]

Hey there! 🎉 So glad to hear everything's working for you now! If you have any more questions or need further assistance down the road, don't hesitate to reach out. Happy coding! 😄👨‍💻

[andyrey]

Hi there, I want to label some special point on the image as a key point. It is not for pose detection, usually only one or sometimes two equivalent points per image. What initial weight should I use for training? Thanks!

[pderrenger]

Hey there! For your specific use case, labeling key points on images, I recommend starting your training with one of our pretrained models as your initial weights, particularly yolov8n-pose.pt if available or yolov8n.pt for a more general approach. These models can provide a solid foundation and potentially speed up the learning process for your task. 😄

You would label your key points following the Ultralytics YOLO format for pose estimation tasks, where each point's coordinates are normalized and listed in the text file corresponding to your image. Given you're focusing on one or two key points per image, you could adjust the label format to suit this.

Remember, even if it's not a full pose estimation task, treating each key point as an object with a bounding box (that can be very tightly wrapped around the point) might work. Here's a simplified example for creating your label files:

<class-index> <x> <y> <width> <height> <px1> <py1> <px2> <py2>

For only one point per image, you'll use <px1> <py1>.

Training your model could go something like this:

yolo detect train data=your_dataset.yaml model=yolov8n-pose.pt epochs=100 imgsz=640

Adjust your_dataset.yaml to describe your dataset paths, class names, and keypoints configuration.

Hope this helps, and best of luck with your project! 🚀

[andyrey]

Hi Paula, thank you for above answer, but I still unaware what size of bounding box (BB) can be in my case. These are woodcuts, and I want to detect core center, here I depicted them with red dots, their positions are often excentric and sometimes may be not single. Should I take tightly wrapped BB around? But visually I find these cores inside a family of concentric annual contours or radial scratches, and this info can be lost if I take tightly wrapped BB. What is your opinion?
And keypoints normalization- relative general picture size or BB size?

[pderrenger]

Hey there! For woodcut core center detection where the position is often eccentric and might not be single, considering the visual context of concentric annual contours or radial scratches is crucial. 🌲✨

Given the context you described, tightly wrapping BBs around the red dots (core centers) might indeed miss out on important radial and concentric information. Instead, you could opt for a slightly larger bounding box that encompasses not just the core center but a bit of the surrounding patterns. This way, the model can learn from both the core center and its context, potentially improving detection accuracy.

For keypoints normalization, it's typically relative to the general picture size rather than the BB size. This normalization allows the model to understand the keypoints' positions on a consistent scale across different image sizes.

Here's a rough example of how you might adjust your bounding box:

# Tight BB (may miss contextual info):
<class-index> <x-center> <y-center> <width> <height> <px1> <py1>

# Slightly larger BB (includes some context):
<class-index> <x-center> <y-center> <width+delta> <height+delta> <px1> <py1>

Just ensure that <width+delta> and <height+delta> sizes capture the essential context without adding too much background noise. Adjust delta based on what visually seems to include enough of those annual contours or radial scratches around the core center(s).

Hope this helps fine-tune your woodcut core center detection! Happy coding! 👩‍💻🔍

[1-zym]

Hello, when I trained yolov8-pose, some indicators were not displayed, only various loss were displayed, but these indicators were included in the result.csv file, such as map50, MAP50-90

[1-zym]

现在我遇到一个问题，我用yolov8m-pose使用相同的训练参数、环境，只是GPU不同，训练300个epoch，不使用预训练，其他参数都是默认，我的精度比他的低了10个点，此外我在自己的卡上使用100个epoch训练yolov8n-pose比使用300个epoch
精度也高了7,8个点，你能为我解答这个问题？

[1-zym]

task: pose
mode: train
model: ultralytics/cfg/models/v8/yolov8-pose.yaml
data: ultralytics/cfg/datasets/coco-pose.yaml
epochs: 300
time: null
patience: 100
batch: 64
imgsz: 800
save: true
save_period: -1
cache: true
device: '1'
workers: 8
project: null
name: train35
exist_ok: false
pretrained: false
optimizer: SGD
verbose: true
seed: 0
deterministic: true
single_cls: false
rect: false
cos_lr: false
close_mosaic: 10
resume: false
amp: true
fraction: 1.0
profile: false
freeze: null
multi_scale: false
overlap_mask: true
mask_ratio: 4
dropout: 0.0
val: true
split: val
save_json: false
save_hybrid: false
conf: null
iou: 0.7
max_det: 300
half: false
dnn: false
plots: true
source: null
vid_stride: 1
stream_buffer: false
visualize: false
augment: false
agnostic_nms: false
classes: null
retina_masks: false
embed: null
show: false
save_frames: false
save_txt: false
save_conf: false
save_crop: false
show_labels: true
show_conf: true
show_boxes: true
line_width: null
format: torchscript
keras: false
optimize: false
int8: false
dynamic: false
simplify: false
opset: null
workspace: 4
nms: false
lr0: 0.01
lrf: 0.01
momentum: 0.937
weight_decay: 0.0005
warmup_epochs: 3.0
warmup_momentum: 0.8
warmup_bias_lr: 0.1
box: 7.5
cls: 0.5
dfl: 1.5
pose: 12.0
kobj: 1.0
label_smoothing: 0.0
nbs: 64
hsv_h: 0.015
hsv_s: 0.7
hsv_v: 0.4
degrees: 0.0
translate: 0.1
scale: 0.5
shear: 0.0
perspective: 0.0
flipud: 0.0
fliplr: 0.5
bgr: 0.0
mosaic: 1.0
mixup: 0.0
copy_paste: 0.0
auto_augment: randaugment
erasing: 0.4
crop_fraction: 1.0
cfg: null
tracker: botsort.yaml
save_dir: /opt/storage/zhaoyiming/ultralytics/runs/pose/train35
这是参数文件

[glenn-jocher]

It sounds like you're experiencing significant variations in performance despite using similar training parameters. Differences in GPU architecture can sometimes lead to variations in training results due to factors like hardware-specific optimizations and floating-point precision. Additionally, training stability and convergence can be influenced by random initialization and data augmentation. To address this, ensure you're using the latest version of the Ultralytics package and try setting a fixed random seed for reproducibility. If the issue persists, consider experimenting with different learning rates, batch sizes, and optimizers to find the optimal configuration for your specific setup.

[1-zym]

为什么yolov8n-pose训练100个epoch会比300个epoch要好呢？lr0=0.01 lrf=0.01 ，其他的超参数都一样，都是默认，我不是很明白它的学习率是怎么衰减的。

[glenn-jocher]

The difference in performance between training for 100 epochs and 300 epochs could be due to overfitting. With a learning rate of 0.01 and default parameters, the model might be learning too quickly and overfitting to the training data. To address this, consider using a learning rate scheduler or early stopping. Additionally, ensure that your dataset is well-augmented and that the training process is consistent across different GPUs. For more details on learning rate schedules and training strategies, you can refer to the Ultralytics documentation.

[PaulCashman73]

I've used YoloV8 successfully for instance segmentation of community cats. Now I'm trying to use it for pose estimation and getting nowhere. I have a dataset of 1130 images (output from the instance segmentation) with 20 labeled keypoints. I used the following code to train a pre-trained model:

model = YOLO('yolov8m-pose.pt') # Use a pre-trained model
project = '/content/data/results'
name = '100-epochs-'

results = model.train(data='/content/data/data.yaml', epochs=100, project=project, name=name, patience=0, batch=4, imgsz=640)

My YAML file is pretty vanilla:

train: /content/train/images
val: /content/valid/images
test: /content/test/images

kpt_shape: [20, 3]
flip_idx: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]

nc: 1
names: ['cat']

roboflow:
workspace: feralcatposeestimation
project: feral-cat-pose-detection-6
version: 1
license: CC BY 4.0
url: https://universe.roboflow.com/feralcatposeestimation/feral-cat-pose-detection-6/dataset/1

My results are dismal. Literally, there are no detections found on any of my test images. The precision and recall for the pose is nearly zero. I feel I must be doing something really wrong to achieve these results.

I looked at one of the label files. Reformatted and with extra blanks to show the keypoints it is as follows:

0 # One class
0.33046875. 0.715625 # Are these the x-center and y-center, or lower-left x and y?
0.33203125 0.365625 # Width, height
0.7729089743589743 0.2127192307692308 2 # I assume 2 is visible; it's not documented
0.36486538461538454 0.16201346153846155 2
0.6729679487179487 0.3529711538461539 2
0.3790602564102564 0.32252403846153843 2
0.512726923076923 0.4760394230769231 2
0.4950102564102563 0.5223086538461539 2
0.3149961538461538 0.7037836538461539 0
0.4352820512820513 0.7231221153846155 0
0.19383205128205125 0.7414942307692307 0
0.192075641025641 0.8033788461538462 0
0.5024294871794871 0.954883653846154 2
0.2842653846153846 0.7559990384615386 0
0.279875641025641 0.8072471153846154 0
0.3159115384615384 0.9471951923076924 2
0.36943205128205125 0.7559990384615386 0
0.367675641025641 0.8120817307692307 0
0.367675641025641 0.8662307692307692 0
0.46513461538461537 0.7618 0
0.46601282051282045 0.8140153846153847 0
0.46513461538461537 0.8817019230769232 0

There are 8 visible keypoints (if 2 means visible), which corresponds to the image. But I don't know how to interpret the x,y coordinates in the second line. If they are x-center and y-center of the bounding box, then some of the keypoints are outside the box. If they are the lower left of the bounding box, then other keypoints are outside the bounding box.

I;m not sure where to begin to unravel what is happening. Any advice would be appreciated.

[glenn-jocher]

@PaulCashman73 it looks like you're on the right track with your dataset and YAML configuration. The x and y coordinates in the second line of your label file should indeed be the center of the bounding box, normalized between 0 and 1. If keypoints appear outside the bounding box, it might indicate an issue with the normalization or annotation process. Ensure that all keypoints are correctly normalized and fall within the bounding box coordinates. Additionally, verify that your dataset follows the Ultralytics YOLO format precisely. For further details, you can refer to Ultralytics Pose Estimation Datasets. If the issue persists, try updating to the latest version of the package to see if it resolves any underlying bugs.

[PaulCashman73]

Thanks! This dataset was generated by Roboflow, so I will raise an issue in their support forums to suggest that perhaps the data is not being generated properly.

[glenn-jocher]

Thanks for the update. It's a good idea to check with Roboflow regarding the dataset generation. Meanwhile, ensure that the x and y coordinates in your label files are indeed the center of the bounding box, normalized between 0 and 1. If the keypoints are outside the bounding box, it might indicate an issue with the normalization process. Also, make sure your dataset follows the Ultralytics YOLO format precisely. If the problem persists, try updating to the latest version of the Ultralytics package.

[benicio22]

Hi, I would like to know if the [email protected] for the key point detection using Ultralytics YOLO v8 is the key point the similarity used by the coco dataset. Also, what does this 0.50 mean? It is not the IoU right?

[benicio22]

Do you guys have a function that calculates the metrics per key point, not overall? How can we do that?

[glenn-jocher]

@benicio22 currently, Ultralytics YOLOv8 does not provide a built-in function to calculate metrics per key point. However, you can achieve this by extracting the keypoint predictions and ground truth from the results and then computing the metrics manually. For more details on the dataset format and keypoint handling, you can refer to our documentation at https://docs.ultralytics.com/datasets/pose/.

[LiJiang97]

My training set label file is

0 0.47899 0.45495 0.20011 0.29458 0.38743 0.31863 2 0.54499 0.31326 2 0.57668 0.51331 2 0.38480 0.53105 2 0.38988 0.38655 2 0.54744 0.37768 2 0.56915 0.57890 2 0.39076 0.59874 2

and I set kpt_shape=[8,3] in the yaml file to detect 8 key points.

Why does it prompt

"train: WARNING ⚠️ C:\applications_data\pycharm\yolov8\datasets\object_260_dataset\images\train\image_37.jpg: ignoring corrupt image/label: labels require 14 columns each"

and

" File "C:\applications\miniconda3\envs\yolov8\Lib\site-packages\ultralytics\data\dataset.py", line 161, in get_labels
len_cls, len_boxes, len_segments = (sum(x) for x in zip(*lengths))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 3, got 0)"?

thank you

[glenn-jocher]

@LiJiang97 it seems your label file format might be incorrect. Ensure each row has 14 columns, matching your kpt_shape=[8,3] setting, which requires 8 keypoints with visibility. For more details, please refer to the pose dataset format documentation.

[AdiGaikwadNewpage]

Hello ultralytics team,
I want to use this pose estimation for human action detection like push, walking, falling, fighting, sitting, etc.
Can you help me to achieve this? How do I need to prepare dataset for this?

[glenn-jocher]

@AdiGaikwadNewpage to achieve human action detection using pose estimation, prepare your dataset in the Ultralytics YOLO format, ensuring each image has a corresponding text file with object and keypoint details. For more guidance on dataset preparation, refer to the Pose Estimation Datasets Overview.

[AdiGaikwadNewpage]

Hello @glenn-jocher, Thank you for your help in data preparation.
Can you tell me now that what I need to do to detect action after training a model?

[glenn-jocher]

To detect actions like walking or sitting, you'll need to train a model to recognize pose sequences associated with each action. After training, use the model to predict keypoints and analyze their movement patterns over time to classify actions.