Arun ArumugamAgent Engineering

Feed

OpenAI measures research acceleration; Anthropic goes public

OpenAI reports agents now complete nearly half of multi-hour research tasks and calls for mandatory reporting of self-improvement indicators. Anthropic priced its IPO and starts trading on Nasdaq. Spotify explains when offline evals can stand in for A/B tests.

Show
  1. OpenAINewsResearchOpenAI: agents now finish nearly half of 2-4 hour research tasks on their ownAutonomous success on 2-4h research tasks rose from 23% to 46% since March, and median researcher agent spend hit $2,400/day. OpenAI argues labs should be required to publicly report self-improvement indicators.
  2. AnthropicNewsLabsAnthropic prices its IPO at $83 a share; trading on Nasdaq as ANT starts todayAnthropic sells about 956M new shares, roughly $79B in fresh capital, and the maker of Claude becomes a public company with the quarterly disclosure duties that come with it.
  3. Spotify EngineeringBlogEvalsSpotify: when you can ship an AI change on offline evals aloneA careful look at skipping A/B tests for LLM features: it only works under strong, drifting assumptions, and judge noise can make a 60% offline-online link look like 24%.