2025-03-04

ARC-AGI Without Pretraining

A novel approach demonstrates that lossless information compression during inference time can produce intelligent behavior, achieving 34.75% accuracy on ARC-AGI training set without pretraining or extensive datasets. The method, CompressARC, processes each puzzle in 20 minutes using only compression objectives and efficient inference-time computation, challenging conventional reliance on extensive pretraining and data.

Original archive.is archive.ph web.archive.org

Log in to get one-click access to archived versions of this article.

read comments on news aggregators:

Related articles

Writing an LLM from scratch, part 8 -- trainable self-attention

A detailed explanation of implementing trainable self-attention in LLMs, focusing on scaled dot product attention and matrix projections. The article breaks down how attention scores are calculated through query, key, and value matrices, demonstrating how five matrix multiplications can efficiently process token relationships.

Crossing the uncanny valley of conversational voice

Sesame introduces Conversational Speech Model (CSM), advancing voice AI beyond traditional text-to-speech limitations by incorporating contextual awareness and emotional intelligence. The model operates as a single-stage system using transformers to produce more natural and coherent speech, achieving near-human performance in audio quality while still working to improve conversational dynamics.

OpenAI, in deep trouble

OpenAI's GPT-4.5 release has received harsh criticism from industry experts, signaling a potential decline in the company's market leadership. The company faces significant challenges including high operational costs, diminishing competitive advantage, and the departure of key personnel. Despite previous ambitious claims about AGI development, OpenAI appears to be struggling with technical advancement and financial sustainability.

The FFT Strikes Back: An Efficient Alternative to Self-Attention

FFTNet introduces a novel approach to sequence processing using Fast Fourier Transform, achieving O(n log n) complexity compared to traditional self-attention's quadratic complexity. The framework employs spectral filtering and modReLU activation to efficiently capture long-range dependencies, demonstrating superior performance on Long Range Arena and ImageNet benchmarks.

Introducing DeepSearcher: A Local Open Source Deep Research

DeepSearcher is an open-source research agent that builds upon previous work by adding features like conditional execution flow, query routing, and improved interfaces. The system leverages SambaNova's custom hardware for faster inference with the DeepSeek-R1 model, demonstrating advanced concepts in AI research automation through a four-step process of question definition, research, analysis, and synthesis.

Home | Substack

The progression of AI capabilities should be measured by the ratio of useful output per unit of human input, rather than through AGI timelines. Drawing parallels between self-driving cars and language models, the focus should shift to measuring how long AI systems can operate effectively without human intervention. While AI systems are becoming increasingly productive, they may never achieve complete autonomy without human guidance.

Please Commit More Blatant Academic Fraud

A critical analysis of academic fraud in AI research argues that explicit fraud could paradoxically improve scientific standards by forcing greater scrutiny and skepticism. The author suggests that prevalent subtle fraud has become normalized in academia, leading to widespread publication of papers without scientific merit. The piece advocates for intentional academic misconduct as a way to expose and ultimately reform the field's compromised research practices.

GitHub - deepseek-ai/open-infra-index

Deepseek-ai announces plans to open-source five repositories over five consecutive days, sharing production-tested code from their AGI development efforts. The initiative aims to contribute transparently to collective progress in AI development while fostering community-driven innovation.