rancher-ai-ui PR 346, round two: token usage after torchiaf's fixes
Our Request changes review (2026-11-16, at 79968cc) → torchiaf replied on all four threads and pushed a76d922 "Support no-usage/partial data conditions" and ca32860 "Add token-usage aria-label". Two passes again: Pass A with Cody's my-pr-review-new-commits (ai-skills c04c11a), Pass B with our process in a fresh agent.
The preview talks to 13.62, which changed since round one: its agent is the released v1.1.1 image (no usage route) and its llm-config has no active LLM, so the panel opens on "active LLM not specified" and never connects. The no-endpoint state and the spinner were seen for real; the other states were driven by stubs in the browser only (LLM config read patched, chat WebSocket and usage response faked). Nothing was written to his environment.
Where it stands
Two of the four threads are done and two are design calls he handed to UX, but the failure case from thread 3 is only half fixed: an agent without the endpoint now shows "Token usage not available", and a 500 on a refresh still drops a shown 1.8K to "Tokens used: 0".
The new states bring two real problems, the partial-data warning reaching mouse users only and a spinner that never stops when the chat can't start, plus a logging bug in the new error path. Nothing new is a blocker. Suggested as a Comment review with one thread reply and three inline comments (two optional nits on top). Your round-one Request changes stays as the PR's review decision until you approve or dismiss it.
CI: the token-usage spec is still red, now for a different reason. Agent PR 507 merged at 13:53, but CI installs the agent chart with its pinned image v1.2.0-alpha.7, which predates 507, so the footer stays on "Token usage not available". It turns green once an agent image with 507 is tagged and the chart points at it. That's the backend dependency you set aside, so it's not in the review.
Your four comments
Your comment
torchiaf's answer
Checked at ca32860
State
By Source shows ui-tools/agent, By Agent shows supervisor (TokenUsage.vue:59)
"We don't have Human readable names defined for those kind of agent… keep those technical agent names", cc oboc-sts
Code unchanged; the popover still shows the ids. sourceLabel still looks up aiConfig.form.section.usage.sources.*, which only his settings-page branch defines, so By Source picks up names when that lands.
Answered: UX's call
The button's accessible name was only "8.5K in total" (TokenUsage.vue:101)
"fixed by adding a specific aria-label"
AX tree: button "Tokens used: 1.8K in total". The label goes through TextLabel's new ariaLabel prop to RcButton.
Addressed
A failed usage request shows "Tokens used: 0", even over a shown 1.8K (useChatMessageComposable.ts:912)
"We now show 'Token usage not available' when the endpoint is not exposed by the Agent."
No endpoint: the real v1.1.1 agent answers 404 {"detail":"Not Found"} and the footer says "Token usage not available" with "The AI Agent does not provide usage data." on hover. Any other failure: fetchChatUsage returns {}, and a 500 on the refresh after an answer turns "1.8K in total" into "Tokens used: 0", disabled (stub).
Half: the 500 case is still open
"Tokens used: 8.5K" vs the design's "Tokens spent: 17,964" (en-us.yaml:117)
"I changed this to help users reading the data – the expanded number is always available in the panel", cc oboc-sts
Unchanged on purpose.
Answered: UX's call
Thread 3, fixed half: the real v1.1.1 agent (no usage route).Thread 3, open half: 1.8K shown (stub)……then a 500 on the refresh after an answer: "Tokens used: 0", disabled.
How the two passes compare
Pass A (Cody's new-commits process)
Pass B (our process)
Lands as
Thread states
Answered · Addressed · Half · Answered
Same
One reply, on thread 3
500 still reads as 0
Yes, stubbed 500
Yes, stubbed 500 and 503
Thread 3 reply
Partial warning is mouse-only
Yes: AX tree, Tab order, axe clean
Yes, plus the spinner has no text either; cites Preview.vue
Inline 1
Spinner never stops when the chat can't start
Yes, real 13.62 (borderline: a state he put to UX)
Yes, real 13.62, as a suggestion
Inline 2
Body read twice, wrong error logged
Noted under thread 3
Own suggestion, with a non-JSON 503 too
Inline 3
aria-label joins two translated strings
—
Optional nit, repo uses one key with a parameter
Optional nit
Unit test asserts a test id nothing renders
Seen, not filed (no-nit rule)
Optional nit
Optional nit
CI token spec red: image alpha.7 has no 507
Yes, not filed
Yes, as a question
Note to you, not posted
"Chat not found" 404 shows "Tokens used: 0"
Checked: accurate for a chat with nothing spent
Mentioned only for the wrong log line
Dropped
Look-only (cursor, icon, copy)
—
Four items
Designer list below
Cost
In session, about 45 min including setup and the 13.62 detour
The icon-alert-alt carries the whole message in a hover tooltip. It has no accessible name and can't take focus: the AX tree has an empty text node for it, Tab from the count skips it, and the button is announced "Tokens used: 6K in total" with nothing about the count being incomplete. axe stays quiet because an icon without a role isn't flagged.
The "Token usage not available" label does the same with its reason, but that follows the footer's existing model label (LlmModelLabel.vue), so it isn't filed. The partial case is different: there is no visible text at all.
Usage is null until a chat id exists, and the new template shows null as loading. On 13.62 today the panel says "active LLM not specified" and "Connection closed", and the footer still shows "Tokens used:" with the spinner after 30 seconds. Before this round it read "Tokens used: 0".
The PR description lists the spinner for "chat not ready / endpoint call ongoing"; a chat that errored never gets ready. It's a state question he partly put to UX, so it's your call whether it goes in.
The body is read with data.json() before the status check, so the error branch's data.text() always throws "body stream already read". A 500 logs that TypeError instead of the agent's message, and a non-JSON 503 from the service proxy throws a SyntaxError even earlier. Seen in the console with the stubbed 500.
Other dynamic aria-labels in the repo use one key with a parameter (ui-tools-config/index.vue:456, Preview.vue:47), which lets a translation order the words.
Nothing renders content-partial, and the test title above says "in the label and in the popover" though the popover has no warning.
For the designer, not in the review
torchiaf already asked oboc-sts to approve the new states, so these stay off GitHub:
Both causes share the label "Token usage not available"; the reason (agent vs LLM) is hover-only and neither says what to do.
The label has cursor: pointer but doesn't click.
The warning uses a grey icon-alert-alt; the repo's other warnings use icon-warning.
Copy: "LLM model" is redundant; "partial or incomplete due to the LLM limited reporting" doubles up and misses a possessive.
From round one, still open on his side: the compact "8.5K" vs the design's exact "17,964", and the technical source and agent ids.
Checked, and not checked
Preview rebuilt at ca32860 (200). Real 13.62: agent v1.1.1 usage route → 404 {"detail":"Not Found"}; panel config error + spinner.
Stubbed in the browser: usage known (count, popover, button name), 500 after a count, 503 non-JSON (Pass B), partial, none reported, "Chat not found".
"None reported" shows "Token usage not available" + "The LLM model does not provide usage data." Fine.
"Chat not found" (agent 507, a chat with no messages and no usage) shows "Tokens used: 0": accurate.
icon-alert-alt exists in @rancher/icons. Unit tests pass (CI and Pass B locally, 13/13).
CI at ca32860: token-usage spec 3/3 red on "Token usage not available" (alpha.7 image); chat.spec scroll handling red, same as main; hooks and message-resource-button passed on retry.
Not checked:
Partial and none with a real LLM: 13.62 can't chat today and its agent has no usage route.
Cypress locally (policy). Narrow panel and dark theme for the new states.
What I'd post
A Comment review: one reply on thread 3, three inline comments, two optional nits. No replies on threads 1 and 4 (UX's call). Thread 2 you can resolve.
Review body
Thanks, the missing endpoint and the button name work now. A few points on the new states inline.
The missing endpoint works now. A failed request still reads as zero, though: after 1.8K, a 500 on the next refresh shows "Tokens used: 0" with the button disabled, because the catch returns an empty object and the footer renders it as a real count.
[before / after screenshots]
This warning only reaches mouse users: the icon has no accessible name and can't take focus, so a screen reader hears "Tokens used: 6K in total" with nothing about the count being incomplete.
When the chat can't start, this spinner never stops. With "active LLM not specified" and "Connection closed", the footer still shows "Tokens used:" spinning after 30 seconds, so a chat that failed looks like a count on its way.
[screenshot]
The body is read here before the status check, so the `data.text()` below always throws "body stream already read": a failed request logs that TypeError instead of the agent's error.