UPSTREAM TITLE: FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business Requests
SHORT SUMMARY: Researchers introduced FDE-Bench, a 49-case benchmark based on real customer projects that tests whether AI agents can clarify underspecified business requirements and iteratively deliver acceptable optimization or machine-learning solutions. The best of 18 evaluated agent configurations scored 0.5510 against an FDE reference of 1.0 and reached the reference in only 6.12% of planned runs, indicating that reliable FDE-level delivery remains out of reach.
OUTLET: arXiv