DOCS-9550 - Evaluations with BYOK #26416
Conversation
Preview links (active after the
|
| 1. Go to [**LLM Observability > Applications**][5]. | ||
| 1. Select the application you want to add topics for. | ||
| 1. At the bottom of the left sidebar, select **Configuration**. | ||
| 1. Add topics in the pop-up modal. |
There was a problem hiding this comment.
Hi @barieom, can you confirm that this is the correct location to enter topics?
There was a problem hiding this comment.
Appears to be correct in app.datadoghq!
| @@ -21,28 +21,55 @@ Topics | |||
| : Helps identify irrelevant input for the `topic relevancy` out-of-the-box evaluation, ensuring your LLM application stays focused on its intended purpose. | |||
|
|
|||
| Evaluations | |||
There was a problem hiding this comment.
could we swap the order of topic relevancy and evaluations here?
|
|
||
| After you click **Save**, LLM Observability invokes a `GPT-4o mini` model using the OpenAI API key you provided. | ||
|
|
||
| You can monitor the usage of this API key by querying for the metrics `ml_obs.span.llm.input.tokens`, `ml_obs.span.llm.output.tokens`, and `ml_obs.span.llm.total.tokens`. Filter by the `evaluation:default` tag. |
There was a problem hiding this comment.
@cswatt - could we hold off on this sentence for this release? we'll release this metric after reinvent, as there are some cross product impact of releasing this metric that we want to address
There was a problem hiding this comment.
I'll remove the line!
|
|
||
| ## Select and enable evaluations | ||
|
|
||
| Navigate to [**LLM Observability > Settings > Evaluations**][3]. |
There was a problem hiding this comment.
This should be a step in the list
|
|
||
| Select an LLM application set up with LLM Observability to start customizing its topics and evaluations. | ||
| {{< img src="llm_observability/configuration/settings.png" alt="The Evaluations tab, featuring a list of existing evaluations." style="width:100%;" >}} |
There was a problem hiding this comment.
This screenshot feels unnecessary to me
Co-authored-by: Sandra (neko) <[email protected]>
|
/merge |
Devflow running:
|
dd-devflow
bot
added
mergequeue-status: done
and removed
mergequeue-status: in_progress
labels
What does this PR do? What is the motivation?
Merge instructions
Merge queue is enabled in this repo. To have it automatically merged after it receives the required reviews, create the PR (from a branch that follows the
<yourname>/descriptionnaming convention) and then add the following PR comment:Additional notes
Can someone confirm where in the app you enter topics? From the figma it looks like it'll be
https://app.datadoghq.com/llm/settings/applications