Claude Opus5.5 was invoked on EC2 through the active AWS Bedrock global inference profile. Only MedlyQ3, ScratchMathQ4 and DrawEduMathQ4 were assessed:25inputs and75candidate judgments per dataset,225criterion decisions. Each request supplied one criterion, the unchanged original generation task, its original images and three anonymous candidate responses. No prior ratings, disagreement flags, model identities, kappa target or required Pass/Fail mix were supplied.
Coder13 ScratchMathQ4 was separately rechecked for every required function, assistance scope, material clarity and appropriateness. That main-agent audit was frozen before viewing the new Opus results; prior coder13 labels were known, so it is not a new blind independent human assessment. It found no supported new failures:75Pass. The export preserves the normal fields, all300responses, prior history and unchanged other labels.
| Dataset / criterion | Pass | Fail | Cannot determine | Changed from original Opus |
| Medly / Q3 | 57 | 18 | 0 | 18 |
| ScratchMath / Q4 | 75 | 0 | 0 | 0 |
| DrawEduMath / Q4 | 69 | 6 | 0 | 3 |
ScratchMathQ4 remains allPass. Under the existing criterion, minor repetition and mathematical mistakes alone do not create a separate material completeness/scope failure. No new word-count rule or requirement for artificial variation was introduced. DrawEduMath’s additional failures concern vague, unusable diagnoses rather than arbitrary length penalties.
The comparison retains all300responses and1200ratings. Only the specified Opus cells change; the other975Opus ratings stay fixed. These are assisted post-review comparisons, not fresh independent-human reliability.
| Dataset | Q1 κ / agreement | Q2 κ / agreement | Q3 κ / agreement | Q4 κ / agreement |
| Medly | 0.841 / 94.7% | 0.379 / 58.7% | 0.621 / 88.0% | 0.000 / 98.7% |
| ScratchMath | 0.548 / 81.3% | 0.302 / 65.3% | 0.359 / 70.7% | Undefined / 100.0% |
| MMTutorBench | 0.672 / 93.3% | 0.516 / 88.0% | 0.819 / 97.3% | 0.613 / 90.7% |
| DrawEduMath | 0.730 / 89.3% | 0.661 / 86.7% | 0.722 / 92.0% | 0.648 / 96.0% |
| Overall | 0.698 / 89.7% | 0.458 / 74.7% | 0.580 / 87.0% | 0.626 / 96.3% |
All75requiredresponsesreceived across76physicalattempts. OneServiceUnavailableException forDrawEduMath had no judgment/usage and was retried once with byte-identical payload; that earlier unknownbillingattempt retains a conservative$4.16hold. Known-token list-price estimate: $3.696360; this is not an AWS invoice. No successful judgment or content was replayed. Three nonliteral/abbreviated supporting quotations triggered a strict quotation check; complete bound JSON judgments were finalized from their original cached responses without any paid repair, verdict change or raw-output alteration.
Original question/image/model-response bytes and fixed order are unchanged. Original Desktop reviewer files, prior JSON versions, and original Netlify/Cloudflare study sites remain unchanged. This review uses a new browser-storage namespace; an earlier review draft can be exported separately. Both ordinary JSON downloads remain import-compatible.
The separate focused-disagreement site freezes the prior explicit MedlyQ2/ScratchMathQ2/Q3 differences:79ratings across67responses and29inputs. Those criteria were not rejudged by this Opus task.
[Open focused disagreement review](https://mathematical-response-focused-review.pages.dev/)