Colcon build fails with parsing errors in Humble but successful in Foxy

[KTHARIKRISHNA]

Bug report

Required Info:

• Operating System:
  • Linux Debian 11, Ubuntu 22.04, Windows 10
• Installation type:
  • From binaries (ROS2 Humble)
• DDS implementation:
  • Default shipped with ROS 2 Humble

Steps to reproduce issue

In ROS 2 Humble, do colcon build for ROS 2 custom message with CaspEvent.msg as shown below:

CaspEvent.msg

#--------------------------------------------------------------------------------------
#
# File : CaspEvent.msg
#
#--------------------------------------------------------------------------------------

# (1) Enum /* extensions for Casp Event types */
uint8 EVTYPE_NO_EVENT = 0                # EvType_NoEvent = 0, /**< no Event */
uint8 EVTYPE_RIGHT_CURVE = 1            # EvType_RightCurve = 1, /**< Right Curve */
uint8 EVTYPE_SPEED_LIMIT = 2            # EvType_SpeedLimit = 2, /**< Speed Limit */
uint8 EVTYPE_ROAD_EXIT = 3              # EvType_RoadExit = 3, /**< Road Exit Right */
uint8 EVTYPE_INTERSECTION = 4           # EvType_Intersection = 4, /**< intersection (also called Xing) */
uint8 EVTYPE_ROUND_ABOUT = 5            # EvType_Roundabout = 5, /**< roundabout */
uint8 EVTYPE_STOP = 6                   # EvType_Stop = 6, /**< stop */
uint8 EVTYPE_TOLL_BOOTH = 7             # EvType_TollBooth = 7, /**< tollbooth */
uint8 EVTYPE_YIELD = 8                  # EvType_Yield = 8, /**< yield */
uint8 EVTYPE_TRAFFIC_LIGHT = 9          # EvType_TrafficLight = 9, /**< traffic light */
uint8 EVTYPE_TRAFFIC_JAM = 10           # EvType_TrafficJam = 10, /**< traffic jam */
uint8 EVTYPE_ROAD_WORK = 11             # EvType_RoadWork = 11, /**< road work */
uint8 EVTYPE_LEFT_CURVE = 12            # EvType_LeftCurve = 12, /**< left curve */
uint8 EVTYPE_MULTICURVE_RL = 13         # EvType_MulticurveRL = 13, /**< S-curve : Right-Left */
uint8 EVTYPE_MULTICURVE_LR = 14         # EvType_MulticurveLR = 14, /**< S-curve : Left-Right */
uint8 EVTYPE_BUMP = 15                  # EvType_Bump = 15, /**< Bump */
uint8 EVTYPE_TSHAPE = 16                # EvType_Tshape = 16, /**< T-Shape intersection */
uint8 EVTYPE_ROAD_EXIT_LEFT = 17        # EvType_RoadExitLeft = 17, /**< Road Exit Left */
                                        # --> /* extensions for other types */ <--
uint8 EVTYPE_MISC = 18                  # EvType_Misc = 18, /**< Miscellaneous : other traffic signs or everything not encoded above (V2x, etc) */
uint8 EVTYPE_SLOPE = 19                 # EvType_Slope = 19, /**< Slope */
uint8 EVTYPE_AVG_SPEED = 20             # EvType_AvgSpd = 20, /**< Average Speed */
uint8 EVTYPE_VEH_STOPPED = 21           # EvType_VehStopped = 21, /**< HMI extension for ConnectedAD pictogram */
uint8 EVTYPE_SLIPPERY_ROAD = 22         # EvType_SlipperyRoad = 22, /**< HMI extension for ConnectedAD pictogram */
uint8 EVTYPE_VEH_APPROACHING = 23       # EvType_VehApproaching = 23 /**< HMI extension for ConnectedAD pictogram */

# (2) Enum Casp Acc Advice
uint8 ACC_NO_ADVICE = 0                 # ACC_NoAdvice           = 0, /**< no Advice to ACC */
uint8 ACC_MAINTAIN_POWER = 1            # ACC_MaintainPower      = 1, /**< noReleasePedalAdviced : advice is given to NOT release the pedal to ACC */
uint8 ACC_COASTING_BRAKING_MIX = 2      # ACC_CoastingBrakingMix = 2, /**< coasting+braking :  natural decel excess above 'rationalization threshold' [20..80%] of a scaled threshold */
uint8 ACC_COASTING_ONLY = 4             # ACC_CoastingOnly       = 4, /**< coasting only :     natural decel excess is > 80 % of a scaled decel */
uint8 ACC_BRAKING_ONLY = 5              # ACC_BrakingOnly        = 5  /**< braking only :      natural decel excess is < 20 % of a scaled decel */

# (3) Enum Glosa-V2X Traffic Light Color
uint8 TL_COLOR_NOLIGHT           = 0 # /**< \n */
uint8 TL_COLOR_GREEN             = 1 #/**< compatible with AV2 protocol coding */
uint8 TL_COLOR_RED               = 2 #/**< compatible with AV2 protocol coding */
uint8 TL_COLOR_ORANGE            = 3 #/**< compatible with AV2 protocol coding */
uint8 TL_COLOR_BLINKING_ORANGE   = 4 #/**< unfortunately incompatible with AV2 protocol coding */

# message
uint8 event_type                        # event type see Enum (1)
int16 distance2event                    # unit m: Distance to event
int16 braking_distance                  # unit m:
int16 advice_distance                   # unit m:
float32 zone_speed                      # unit mps
uint8 advice_status                     # Advice status see Enum (2)

uint8 traffic_light_current_color       # only for TL event Carto and Glosa-V2X (see ASQT-236)
uint8 traffic_light_next_color          # only for TL event Carto and Glosa-V2X (see ASQT-236) For 5x Debug context NextColor == 0 --> TL carto otherwise [1..4] for TL Glosa-V2x: get from: V_x_CoEAdviceStatus
uint8 traffic_light_predictive_color    # only for TL event Carto and Glosa-V2X --> predictiveColour == 0 --> TL carto otherwise [1..4] for TL Glosa-V2x get from: V_x_ContextTypeExtended
uint8 traffic_light_direction           # only for TL event Carto and Glosa-V2X
uint8 traffic_light_time_to_change      # time in second time remaining to next phase.

uint8 adasis_id  # only for V2X static evt

Expected behavior

Build should be successful

Actual behavior

Build failed with the following error:

No terminal defined for '"' at line 97 col 62

sa-V2X Traffic Light Color" "\n"        "/**< 
                                        ^

Expecting: {'PLUS', 'PERCENT', 'STAR', 'MINUS', 'SLASH', 'VBAR', 'COMMA', 'ESCAPED_STRING', 'AMPERSAND', '__ANON_10', 'CIRCUMFLEX', 'RPAR', '__ANON_9'}
 /home/user/dev_ros2_ws/build/swl_visualization_msgs/rosidl_adapter/swl_visualization_msgs/msg/CaspEvent.idl
Error processing idl file: /home/user/dev_ros2_ws/build/swl_visualization_msgs/rosidl_adapter/swl_visualization_msgs/msg/CaspEvent.idl
Traceback (most recent call last):
  File "/opt/ros/humble/share/rosidl_generator_cpp/cmake/../../../lib/rosidl_generator_cpp/rosidl_generator_cpp", line 38, in <module>
    sys.exit(main())
  File "/opt/ros/humble/share/rosidl_generator_cpp/cmake/../../../lib/rosidl_generator_cpp/rosidl_generator_cpp", line 34, in main
    generate_cpp(args.generator_arguments_file)
  File "/opt/ros/humble/lib/python3.8/site-packages/rosidl_generator_cpp/__init__.py", line 38, in generate_cpp
    return generate_files(
  File "/opt/ros/humble/lib/python3.8/site-packages/rosidl_cmake/__init__.py", line 96, in generate_files
    raise(e)
  File "/opt/ros/humble/lib/python3.8/site-packages/rosidl_cmake/__init__.py", line 74, in generate_files
    idl_file = parse_idl_file(locator)
  File "/opt/ros/humble/lib/python3.8/site-packages/rosidl_parser/parser.py", line 64, in parse_idl_file
    content = parse_idl_string(string, png_file=png_file)
  File "/opt/ros/humble/lib/python3.8/site-packages/rosidl_parser/parser.py", line 72, in parse_idl_string
    tree = get_ast_from_idl_string(idl_string)
  File "/opt/ros/humble/lib/python3.8/site-packages/rosidl_parser/parser.py", line 89, in get_ast_from_idl_string
    return _parser.parse(idl_string)
  File "/usr/lib/python3/dist-packages/lark/lark.py", line 293, in parse
    return self.parser.parse(text, start=start)
  File "/usr/lib/python3/dist-packages/lark/parser_frontends.py", line 193, in parse
    return self._parse(text, start)
  File "/usr/lib/python3/dist-packages/lark/parser_frontends.py", line 54, in _parse
    return self.parser.parse(input, start, *args)
  File "/usr/lib/python3/dist-packages/lark/parsers/earley.py", line 292, in parse
    to_scan = self._parse(stream, columns, to_scan, start_symbol)
  File "/usr/lib/python3/dist-packages/lark/parsers/xearley.py", line 137, in _parse
    to_scan = scan(i, to_scan)
  File "/usr/lib/python3/dist-packages/lark/parsers/xearley.py", line 114, in scan
    raise UnexpectedCharacters(stream, i, text_line, text_column, {item.expect.name for item in to_scan}, set(to_scan))
lark.exceptions.UnexpectedCharacters: No terminal defined for '"' at line 97 col 62

sa-V2X Traffic Light Color" "\n"        "/**< 
                                        ^

Expecting: {'PLUS', 'PERCENT', 'STAR', 'MINUS', 'SLASH', 'VBAR', 'COMMA', 'ESCAPED_STRING', 'AMPERSAND', '__ANON_10', 'CIRCUMFLEX', 'RPAR', '__ANON_9'}

make[2]: *** [CMakeFiles/swl_visualization_msgs__cpp.dir/build.make:140: rosidl_generator_cpp/swl_visualization_msgs/msg/casp_event.hpp] Error 1
make[1]: *** [CMakeFiles/Makefile2:242: CMakeFiles/swl_visualization_msgs__cpp.dir/all] Error 2
make: *** [Makefile:141: all] Error 2

Additional information

Build was successful in ROS 2 Foxy but it is broken in Humble.

This is the complete build log:
stdout_stderr.log

[clalancette]

uint8 TL_COLOR_NOLIGHT = 0 # /**< \n */

So I bisected it down to (mostly) this line. My best guess is that the newline here is confusing the parser and making it so that it things the */ is the start of a newline. Removing that \n makes it get by this particular error.

However, even if this is removed, the parser also doesn't like the /* within the comments. That's because we take the message comments and embed them inside of comments in the generated code, so you end up with something like:

/**
  * /**< compatible with AV2 protocol coding */
 */

And that confused the C++ parser.

The best advice I can give to you for now would be to remove the \n and the /** .. */ parts from the comments. There is probably a bug to be fixed for the \n problem, so I'll leave this open on the backlog to look into that.

[KTHARIKRISHNA]

Thanks for the explanation. Currently I am using this msg file without any comments in it right now.
But May I know what is the change that is causing this issue in Humble. As I said, this is working correctly in Foxy. What is the upgrade that happened related to this parsing from Foxy to Humble?

[clalancette]

But May I know what is the change that is causing this issue in Humble.

I'm not entirely sure. It may be the case that in Foxy, we didn't insert the comments from the .msg file into the .hpp files, so the issue just didn't arise.