Skip to content

fix(langchain): Stop double-counting usage in multi-candidate responses - #7064

Open
alexander-alderman-webb wants to merge 6 commits into
masterfrom
webb/langchain/stop-double-counting-candidate-tokens
Open

fix(langchain): Stop double-counting usage in multi-candidate responses#7064
alexander-alderman-webb wants to merge 6 commits into
masterfrom
webb/langchain/stop-double-counting-candidate-tokens

Conversation

@alexander-alderman-webb

@alexander-alderman-webb alexander-alderman-webb commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Description

Take token usage from the first candidate object in each generation, instead of summing token usage from each candidate.

The inner list in LLMResult.generations contains candidate generations (multiple output choices). Candidates do not have independent token usage, but langchain hooks duplicate the token usage on each candidate.

Add a test using a google-genai response, as ChatOpenAI models do not enter the else: branch below:

def _record_token_usage(span: "Union[Span, StreamedSpan]", response: "Any") -> None:
token_usage = _get_token_usage(response)
if token_usage:
input_tokens, output_tokens, total_tokens = _extract_tokens(token_usage)
else:
input_tokens, output_tokens, total_tokens = _extract_tokens_from_generations(

Issues

Reminders

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Codecov Results 📊

99182 passed | ⏭️ 6527 skipped | Total: 105709 | Pass Rate: 93.83% | Execution Time: 357m 15s

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +72
Passed Tests 📈 +25
Failed Tests 📉 -1
Skipped Tests 📈 +48

➖ Removed Tests (1)

View removed tests
  • test_continuous_profiler_auto_start_and_stop_sampled_span_streaming[non-experiment-thread]
    • File: tests.profiler.test_continuous_profiler

All tests are passing successfully.

✅ Patch coverage is 88.89%. Project has 2496 uncovered lines.
✅ Project coverage is 89.99%. Comparing base (base) to head (head).

Files with missing lines (1)
File Patch % Lines
sentry_sdk/integrations/langchain.py 88.89% ⚠️ 1 Missing and 1 partials
Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    89.99%    89.99%        —%
==========================================
  Files          193       193         —
  Lines        24946     24947        +1
  Branches      9000      9000         —
==========================================
+ Hits         22451     22451         —
- Misses        2495      2496        +1
- Partials      1435      1435         —

Generated by Codecov Action

@alexander-alderman-webb
alexander-alderman-webb marked this pull request as ready for review August 6, 2026 13:38
@alexander-alderman-webb
alexander-alderman-webb requested a review from a team as a code owner August 6, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant