Cost vs Accuracy
↖ Top-left is bestCompare total inference cost and accuracy across the complete test set. Left = cheaper. Up = more accurate. Each model has a distinct color; lines connect adjacent reasoning levels.
DL compute rate
Cloud cost scenarioDL prices apply the selected hourly rate to recorded DL timing. They are estimates, not cloud invoices. Changing the rate changes DL cost only; recorded LLM and decision API charges stay fixed.
● Low◆ Medium■ High▲ DL⬡ Decisionmodel = color
Mean Confidence vs Accuracy
Selected-label probabilityEach point compares one complete configuration’s mean selected-label confidence with its accuracy across the same complete test set. The full view uses 0–100% on both axes. LLMs report their own estimate; DL and decision models supply class probabilities. On the dashed diagonal, the mean matches accuracy. Bottom-right = overconfident. Top-left = underconfident.
● Low◆ Medium■ High▲ DL⬡ Decisionmodel = color
These aggregate means describe confidence and accuracy. The decision API’s native confidence is a separate metric, shown in the table and tooltips. These means do not establish formal probability calibration or guarantee that an individual answer is correct.
Every run, in detail
Coverage and excluded runs
Only complete runs matching the test data are eligible for the graphs. Missing, failed, partial, or incompatible runs are shown here for review.
HOW TO READ THIS COMPARISON
Measured results. Visible assumptions.
One test set
Accuracy is the fraction of correct sentiment labels across the complete matching test set. Partial runs and runs with failed examples are excluded from the plots, so the comparison keeps the same denominator.
Two kinds of cost
LLM and decision cost sums the retained prediction attempts. Earlier failed attempts stay in API logs and may have unknown charges. DL cost estimates use measured runtime and your chosen hourly compute rate. The timing basis and provider rate are explicit controls.
Original files
The explorer reads saved files in results/. It does not send prompts or make model API requests. The sources below identify the inputs and pricing references. Grouping keeps each configuration separate and never averages or combines scores.