OpenAI's GPT-6 Astra reportedly runs recurrent-depth transformers — Alibaba-affiliated research already diagnosed the catch.

OpenAI's GPT-6 Astra reportedly runs recurrent-depth transformers — Alibaba-affiliated research already diagnosed the catch.

The Information reported that GPT-6 Astra uses "recurrent depth," running the same set of transformer layers multiple times to add computational depth without adding proportional parameters. The disclosure revived interest in an architecture also suspected (never confirmed) behind Claude Mythos's strong GraphWalks BFS scores — Mythos Preview hit 80.0% versus GPT-5.4's 21.4% and Anthropic's own Opus 4.6 at 38.7% — and drew criticism from AI safety researchers concerned that reasoning looping inside hidden states may not surface as a legible, human-readable chain of thought; OpenAI chief scientist Jakub Pachocki publicly responded and said a fuller writeup is coming. Separately, a research team from Alibaba and partner universities had already published two peer-reviewed papers addressing why looped transformers often underperform standard ones at equal compute. MeSH (ICLR 2026, first posted October 2025) diagnoses "computation homogenization" and "information overload" across loop iterations and fixes it with a memory buffer plus learned read/write routers, cutting non-embedding parameters roughly 33% while raising zero-shot accuracy from 49.50% to 50.56% at Pythia-1.4B scale. SpiralFormer (EMNLP 2026) instead varies the sequence resolution processed each loop, coarse-to-fine, cutting FLOPs from 14.08T to 13.13T while lifting five-shot accuracy from 51.93% to 54.37%.

The pattern matters because it's a genuine third scaling lever alongside parameter/data scaling and inference-time chain-of-thought — reusing the same weights for more computational depth without growing storage. But the redundancy problem the papers document (later loop iterations contributing shrinking hidden-state updates) was flagged in the literature nearly a year before Astra shipped, meaning a frontier lab is now productizing an architecture academic work already found needs explicit routing and scheduling machinery to avoid wasted compute. It also sharpens the monitorability debate already trailing Astra's rollout: reasoning that unfolds inside recurring hidden states is harder for the kind of real-time monitoring OpenAI has leaned on to justify the model's risk posture.

Teams evaluating efficiency claims that trade parameters for repeated computation should treat naive looping as a red flag — the credible gains here came from added routing and resolution-scheduling overhead (MeSH's routers add roughly 0.005% extra parameters and 0.014% extra compute), not the loop itself. Investors and safety researchers should expect scrutiny of hidden-state reasoning to intensify as more labs adopt this route, since it removes another point where model reasoning would otherwise show up as inspectable text.

#OpenAI #GPT6Astra #Alibaba #TransformerArchitecture #AIScaling #ModelInterpretability