User Interface
Pages, page extensions, reports, XMLports, and API pages
Report generated: February 14, 2026 at 8:13 PM
Benchmark data: Feb 8, 2026 – Feb 13, 2026
12
Models
7
Tasks
65.5%
Pass Rate
Model Rankings
Model Performance
openai/gpt-5.2-2025-12-11@thinking=high
Runs:3
pass@1:76.2%
pass@3:85.7%
Consistency:85.7%
1st: 152nd: 1Failed: 16/7 passed
Temperature:0.1
Thinking:high
Tokens/run:16,274
Cost/run:$0.17
Known Shortcomings (10)
- interface-definition-syntax 2x
- table-field-caption-property 1x
- query-object-syntax 1x
- query-crossjoin-syntax 1x
- parse-failure 1x
+5 more View all 10
openrouter/x-ai/grok-code-fast-1
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 132nd: 2Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:21,125
Cost/run:$0.19
Known Shortcomings (12)
- query-object-syntax 2x
- multiline-string-literals 1x
- page-extension-cardpageid-override 1x
- json-api-methods 1x
- recordref-fieldref-dynamic-manipulation 1x
+7 more View all 12
anthropic/claude-opus-4-6
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 15Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:40,155
Cost/run:$0.76
Known Shortcomings (6)
- reserved-keyword-as-parameter-name 1x
- cross-join-dataitem-link 1x
- incomplete-procedure-body 1x
- flowfield-calcfields-requirement 1x
- parse-failure 1x
+1 more View all 6
anthropic/claude-opus-4-5-20251101@thinking=50000
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 15Failed: 25/7 passed
Temperature:0.1
Thinking:50,000
Tokens/run:37,635
Cost/run:$0.69
Known Shortcomings (8)
- page-extension-with-table-extension 1x
- reserved-keyword-as-parameter-name 1x
- dictionary-iteration-syntax 1x
- empty-or-malformed-code-generation 1x
- temporary-table-parameter-handling 1x
+3 more View all 8
anthropic/claude-sonnet-4-5-20250929
Runs:3
pass@1:71.4%
pass@3:71.4%
Consistency:100.0%
1st: 132nd: 2Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:17,413
Cost/run:$0.19
Known Shortcomings (8)
- multiline-string-literals 1x
- query-filter-element-syntax 1x
- jsonobject-get-method-signature 1x
- cross-join-dataitem-link-constraints 1x
- reserved-keyword-as-variable-name 1x
+3 more View all 8
openrouter/qwen/qwen3-max-thinking
Runs:3
pass@1:66.7%
pass@3:71.4%
Consistency:85.7%
1st: 112nd: 3Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:13,233
Cost/run:$0.09
Known Shortcomings (12)
- option-field-optionmembers-required 2x
- enum-frominteger-syntax 1x
- list-iteration-pattern 1x
- variant-type-argument-and-interface-definition 1x
- json-object-api-methods 1x
+7 more View all 12
openrouter/moonshotai/kimi-k2.5
Runs:3
pass@1:61.9%
pass@3:71.4%
Consistency:71.4%
1st: 112nd: 2Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:34,490
Cost/run:$0.31
Known Shortcomings (3)
- event-subscriber-parameter-syntax 1x
- page-extension-cardpageid-override 1x
- parse-failure 1x
gemini/gemini-3-pro-preview
Runs:3
pass@1:57.1%
pass@3:57.1%
Consistency:100.0%
1st: 12Failed: 34/7 passed
Temperature:0.1
Thinking:-
Tokens/run:95,468
Cost/run:$0.13
Known Shortcomings (9)
- multiline-string-literals 1x
- inherent-permissions-syntax 1x
- query-crossjoin-column-datasource 1x
- complete-codeunit-generation 1x
- yaml-parsing-string-manipulation 1x
+4 more View all 9
openrouter/z-ai/glm-5
Runs:3
pass@1:52.4%
pass@3:57.1%
Consistency:85.7%
1st: 82nd: 3Failed: 34/7 passed
Temperature:0.1
Thinking:-
Tokens/run:28,933
Cost/run:$0.27
Known Shortcomings (17)
- list-dictionary-of-interface-clear-method 1x
- event-subscriber-event-name 1x
- al-string-literal-escaping 1x
- query-object-syntax 1x
- fluent-api-return-self-codeunit 1x
+12 more View all 17
openrouter/qwen/qwen3-coder-next
Runs:3
pass@1:42.9%
pass@3:42.9%
Consistency:100.0%
1st: 62nd: 3Failed: 43/7 passed
Temperature:0.1
Thinking:-
Tokens/run:16,341
Cost/run:$0.14
Known Shortcomings (19)
- codeunit-generation-empty-output 5x
- interface-definition-syntax 3x
- query-object-syntax 2x
- initvalue-vs-defaultvalue 1x
- text-trim-method-unavailable 1x
+14 more View all 19
openrouter/minimax/minimax-m2.5
Runs:3
pass@1:38.1%
pass@3:71.4%
Consistency:42.9%
1st: 42nd: 4Failed: 25/7 passed
Temperature:0.1
Thinking:-
Tokens/run:26,076
Cost/run:$0.19
Known Shortcomings (18)
- interface-definition-syntax 2x
- text-char-conversion-copystr 1x
- page-object-definition 1x
- event-subscriber-attribute-syntax 1x
- page-extension-and-table-extension-generation 1x
+13 more View all 18
openrouter/deepseek/deepseek-v3.2
Runs:3
pass@1:38.1%
pass@3:42.9%
Consistency:85.7%
1st: 8Failed: 43/7 passed
Temperature:0.1
Thinking:-
Tokens/run:17,948
Cost/run:$0.17
Known Shortcomings (18)
- dictionary-clear-method 1x
- application-area-in-page-extension-field 1x
- multiline-string-literals 1x
- page-extension-cardpageid-override 1x
- errorinfo-custom-dimensions-api 1x
+13 more View all 18
Task Results Matrix
N/M = passed N of M runs (hover for details)
| Task | Description | GPT-5.2 | Grok Code Fast 1 | Claude Opus 4.6 | Claude Opus 4.5 (50K) | Claude Sonnet 4.5 | Qwen3 Max Thinking | Kimi K2.5 | Gemini 3 Pro | Glm 5 | Qwen3 Coder Next | Minimax M2.5 | Deepseek V3.2 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CG-AL-E002 | Create a simple AL card page called "Product Category Card" with ID 70001. The page should be based on the existing "Product Category" table (ID 69001) and display the following fields: - Code - Description - Active - Created Date | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 3/3 | 1/3 | 3/3 |
| CG-AL-E006 | Create a page extension called "Customer Card Extension" with ID 70000 that extends the Customer Card page (page 21). Add the following controls to the General group: - Preferred Contact Method (Option field with values: Email, Phone, Mail, SMS) - Customer Notes (Text field for internal notes) - VIP Customer (Boolean field to mark important customers) | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 2/3 | 3/3 | 3/3 | 0/3 | 1/3 | 0/3 |
| CG-AL-E007 | Create a simple AL report called "Customer List Report" with ID 70000. The report should use the Customer table as its data source and display: - Customer No. (column name: No_Customer, source: "No.") - Customer Name (column name: Name_Customer, source: Name) - City (column name: City_Customer, source: City) - Phone No. (column name: PhoneNo_Customer, source: "Phone No.") | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 |
| CG-AL-E053 | Create a page extension named "CG Item List Extension" with ID 70053 that extends the "Item List" page (Page 31). | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 3/3 | 3/3 | 3/3 | 1/3 | 0/3 |
| CG-AL-M001 | Create a comprehensive API page called "Product API" with ID 70100 that provides full CRUD operations for a Product table. The API should: - Support GET, POST, PATCH, DELETE operations - Include proper error handling and validation - Have fields: id (SystemId), productCode, description, unitPrice, stockQuantity, categoryId - Include proper OData annotations and API versioning (v1.0) - Add business logic validation (price must be positive, stock must be non-negative) - Include proper captions, publisher info (mycompany), and API group (products) | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 |
| CG-AL-M004 | Create an interactive page called "Sales Order Workspace" with ID 70101 that provides comprehensive order management. The page should be PageType Card, based on Sales Header table (filtered to Document Type::Order), and include: | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 0/3 | 0/3 | 0/3 | 2/3 | 2/3 |
| CG-AL-M007 | Create a complex report called "Sales Performance Analysis" with ID 70001 that performs advanced data processing. | 1/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 | 0/3 |