AI / FlutterBench / Tasks / flutter-manage-state-with-bloc / flutter-manage-state-with-bloc__t62-deepseek-coder-v2
Trial Evaluation

flutter-manage-state-with-bloc__t62-deepseek-coder-v2

Task: google/flutter-manage-state-with-bloc
PARTIAL
59 % Composite Reward
Model DeepSeek Coder V2 DeepSeek
Agent Harness deepseek-cli Standard Baseline
Tokens 77.7k 74.7k in / 3.1k out
Cost $0.0113 USD estimated
Execution Phase Durations
Environment Setup10.0s
Agent Setup10.0s
Agent Execution1.6m
Verifier30.0s

Scoring Rubric Breakdown

Composite Reward = 0.60×Outcome + 0.30×Quality + 0.10×DX

Detailed criteria evaluation produced by automated test graders and LLM rubrics:

OUTCOMEWeight 60%
62%
flutter_build:bundle wt: 20% 1.00

flutter build bundle succeeded.

bloc_unit_tests wt: 40% 0.62

Bloc unit tests for state transition validation.

outcome_heuristics/bloc_provider_present wt: 40% 1.00

BlocProvider wrapping widget tree in main.dart.

QUALITYWeight 30%
53%
StaticAnalysisGrader wt: 50% 0.53

5 linter warnings detected.

structural_validation wt: 30% 0.60

Clean separation of concerns and architectural layers.

idiomatic_review wt: 20% 0.53

Dart 3 idioms (pattern matching, records, sealed classes) adherence.

DXWeight 10%
64%
tool_usage wt: 40% 0.45

Standard command-line execution without language server telemetry.

trajectory wt: 30% 0.64

Multiple retry cycles before settling on solution.

error_recovery wt: 30% 0.64

Fast turnaround from initial compile warnings to clean build.

Agent Execution Trajectory

Sequential step timeline captured during autonomous agent execution:

  1. 1
    step0ms
  2. 2
    step0ms
  3. 3
    step0ms
  4. 4
    step0ms
  5. 5
    step0ms

Generated Code Artifacts

Files modified or generated by the agent during this trial:

📄 artifacts/workspace/lib/bloc/counter_bloc.dart Generated 
// Generated implementation for flutter-manage-state-with-bloc
// Model: deepseek-coder-v2
// Verification score: 0.59 (partial)
📄 artifacts/workspace/lib/bloc/counter_event.dart Generated 
// Generated implementation for flutter-manage-state-with-bloc
// Model: deepseek-coder-v2
// Verification score: 0.59 (partial)
📄 artifacts/workspace/lib/bloc/counter_state.dart Generated 
// Generated implementation for flutter-manage-state-with-bloc
// Model: deepseek-coder-v2
// Verification score: 0.59 (partial)
📄 artifacts/workspace/lib/main.dart Generated 
// Generated implementation for flutter-manage-state-with-bloc
// Model: deepseek-coder-v2
// Verification score: 0.59 (partial)
📄 artifacts/workspace/test/counter_bloc_test.dart Generated 
// Generated implementation for flutter-manage-state-with-bloc
// Model: deepseek-coder-v2
// Verification score: 0.59 (partial)

Execution & Verifier Logs

Verifier Standard Output (test-stdout.txt) 
00:00 +0: loading tests/graders.dart
00:01 +1: Environment setup verification passed.
00:02 +2: Task codebase compilation check.
00:03 +3: 3/5 Manage State with BLoC tests passed.
00:04 +3 -1: 2 edge-case assertions failed.
00:05 +4: Static analysis completed with minor hints.
Overall result: PARTIAL (Score: 0.59)
← Back to flutter-manage-state-with-bloc Task ← Back to Tasks Matrix ← Back to Leaderboard