to_repr and rich breaks logging

[Eldeeqq]

Structlog version: 24.4.0

⭐ First of all - love this project, huge fan, good job.

Hello,

Maybe this is just bad setup on our end, but it was tricky to figure out so I decided to raise it.

We use structlog for logging data from our service in json format, and then uploading it into DataDog. After updating structlog to version 24.4.0 our logging started to break each time we logged an exception ( structlog.get_logger().exception(...))

I was able to trace the root cause to this PR: #627, where the implementation for to_repr changed to:

def to_repr(
    obj: Any,
    max_length: int | None = None,
    max_string: int | None = None,
    use_rich: bool = True, # source of the problem
) -> str:

and I was able to hotfix the the problem on our end by creating our own dict_tracebacks and setting use_rich explicitly.

custom_dict_tracebacks =  ExceptionRenderer(ExceptionDictTransformer(use_rich=False))

Stack Trace

File "/usr/local/lib/python3.12/site-packages/structlog/_base.py", line 165, in _process_event

event_dict = proc(self._logger, method_name, event_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/structlog/processors.py", line 412, in __call__

event_dict["exception"] = self.format_exception(
^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/structlog/tracebacks.py", line 415, in __call__

trace = extract(
^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/structlog/tracebacks.py", line 276, in extract

key: to_repr(
^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/structlog/tracebacks.py", line 147, in to_repr

obj_repr = rich.pretty.traverse(
^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/rich/pretty.py", line 853, in traverse

node = _traverse(_object, root=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/rich/pretty.py", line 767, in _traverse

child_node = _traverse(getattr(obj, field.name), depth=depth + 1)
^^^^^^^^^^^^^^^^^^^^^^^^

AttributeError: 'MemoryObjectItemReceiver' object has no attribute 'item'

Since we don't install rich on the server, I'm not really sure how it was able to import it in the first place (my guess is transitive dependency), or why it failed.

But would it make sense to somehow configure the use_rich flag globally?

[hackermandh]

Do you install bandit? Because that has rich as a dependency (just because they have a one-liner where they wanted to show a loading bar 🥲)

[Eldeeqq]

I checked which modules are installed on the server and neither bandit or rich are being installed. Might've been different when I created this issue 🙁

[hynek]

⭐ First of all - love this project, huge fan, good job.

Thank you! 💛

We use structlog for logging data from our service in json format, and then uploading it into DataDog. After updating structlog to version 24.4.0 our logging started to break each time we logged an exception ( structlog.get_logger().exception(...))

What you mean here, is that you expect single-line JSON lines and Rich formating breaks them into multiple ones? (is this expected @sscherfke?)

I was able to trace the root cause to this PR: #627, where the implementation for to_repr changed to:

def to_repr(
    obj: Any,
    max_length: int | None = None,
    max_string: int | None = None,
    use_rich: bool = True, # source of the problem
) -> str:

and I was able to hotfix the the problem on our end by creating our own dict_tracebacks and setting use_rich explicitly.

custom_dict_tracebacks =  ExceptionRenderer(ExceptionDictTransformer(use_rich=False))

I don't think that's a hot fix – I think you're using configuration as it's intended? 😅

Since we don't install rich on the server, I'm not really sure how it was able to import it in the first place (my guess is transitive dependency), or why it failed.

Not to be lecturing, but you should use recursive lock files to make this deterministic. 🤓

But would it make sense to somehow configure the use_rich flag globally?

We try to avoid having anything global except the configuration and you were able to work around the problem using regular config, so I'm gonna say everything is working as intended?

Depending on what Stefan says, we can have a look at multi-line log lines or whatever is happening there, but adding a global variable to avoid configuration is not our MO. :)