← Back to Benchmark Results

openrouter/moonshotai/kimi-k2.5

64.3%
Pass Rate
36/56
Tasks Passed
3
Runs
56.0%
pass@1
64.3%
pass@3
82.1%
Consistency
0.1
Temperature
-
Thinking
892,753
Tokens
$7.96
Cost
1st: 742nd: 20Failed: 2036/56 passed

Known Shortcomings (3)

Sorted by occurrence count (most frequent first)

# Concept AL Concept Count Affected Tasks
1 event-subscriber-parameter-syntax event-subscriber-definition 1 CG-AL-E010

Description: The model generated an event subscriber for the Item table's OnAfterInsertEvent but used incorrect parameter syntax. The AL0282 error 'The member referenced by event subscriber parameter is not found' indicates the model likely used an incorrect parameter type or reference in the EventSubscriber attribute or procedure signature. The correct pattern requires subscribing to Database::Item with the OnAfterInsertEvent and using a 'var Rec: Record Item' parameter. The model likely wrote something like [EventSubscriber(ObjectType::Table, Database::"Item", 'OnAfterInsert', '', false, false)] with an incorrect event name or parameter reference, when it should use 'OnAfterInsertEvent' as the event name.

Correct Pattern: 
codeunit 70001 "Item Event Subscriber"
{
    Access = Internal;

    [EventSubscriber(ObjectType::Table, Database::Item, OnAfterInsertEvent, '', false, false)]
    local procedure ItemOnAfterInsert(var Rec: Record Item; RunTrigger: Boolean)
    begin
        Message('New item created: %1', Rec."No.");
    end;
}
Incorrect Pattern: 
// Generated code not available but error indicates incorrect event subscriber parameter reference on line 8

Error Codes: AL0282
2 page-extension-cardpageid-override page-extension-properties 1 CG-AL-E053

Description: The model failed to generate valid AL code for the page extension. The compilation errors at line 9 (syntax error, identifier expected) and line 14 (integer literal expected) along with other syntax errors suggest the model produced malformed AL code. The task requires using the BC 2025 Wave 1 feature to override the CardPageId property on a list page extension, which is a newer feature the model likely doesn't know about. The model appears to have generated syntactically invalid code, possibly using incorrect syntax for the CardPageId property override and/or the action definition with RunObject/RunPageLink properties.

Correct Pattern: 
pageextension 70053 "CG Item List Extension" extends "Item List"
{
    CardPageId = "Item Card";

    actions
    {
        addlast(Processing)
        {
            action("Show Item Details")
            {
                ApplicationArea = All;
                Caption = 'Show Item Details';
                Image = Card;
                RunObject = page "Item Card";
                RunPageLink = "No." = field("No.");
            }
        }
    }
}
Incorrect Pattern: 
Generated code not found - but errors at line 9 and 14 indicate malformed page extension structure with incorrect property/action syntax

Error Codes: AL0107
3 parse-failure unknown 1 CG-AL-E052

Description: Failed to parse LLM analysis response: Looking at the failure details: 1. The generated code compiled successfully and was published 2. Most tests pass, but `TestDateToText_ValidDate` and `TestDateToText_FirstDayOfYear` fail 3. The test f

Correct Pattern:
Incorrect Pattern: