Coinbase Outperforms Frontier Models with AI-Powered Fraud Prevention

Introduction

Coinbase has been working on a project to improve its fraud detection capabilities using AI. The company has fine-tuned a relatively small AI model, the Qwen3.5-9B model, to catch fraudsters and has achieved impressive results.

Benchmark Showdown

To compare the performance of the Qwen model with frontier systems, Coinbase built a proprietary Onramp fraud benchmark. The benchmark contains 16,140 transactions, of which 813 were confirmed fraud cases. The Qwen model was fine-tuned using reinforcement learning and went head-to-head with frontier systems, including Anthropic's Opus 4.5.

Results

The Qwen model outperformed Opus 4.5 across four metrics: precision, recall, F1 score, and dollar-weighted recall. The margins ranged from 7.5 to 35.4 percentage points. The model also posted a median latency of 0.683 seconds, compared with 1.515 seconds for Opus 4.5, resulting in a 55% reduction.

Cost and Strategy

Training the Qwen model cost under $100 on a single NVIDIA RTX PRO 6000 96GB GPU. Coinbase's findings suggest that the company is leaning towards post-trained models tailored to narrow problems rather than relying on general-purpose models that top the leaderboard.

Implications

A 30% drop in fraudulent transactions means fewer stolen cards slipping through and fewer chargebacks. A 22% cut in fraud value means the losses that do get through are smaller. However, the results should be viewed with caution, as the benchmark is proprietary and the findings apply to a specific task on a specific product.