Aggregators
Epoch Capabilities Index (ECI)
- 1Claude Fable 5.2Anthropic173.4
- 2Gemini 4 CarbonGoogle DeepMind169.8
- 3GPT-6 AstraOpenAI169.0
- 4Claude Opus 5.5Anthropic168.5
- 5GPT-6.1 SolOpenAI168.5
Benchmarks · updated daily
A daily read of the benchmarks that still separate frontier AI models: who leads where, what changed, and links to every source.
Daily link check: 76 of 82 reference pages reachable, 5 block automated checks, 34 changed since the last check.
Top 5 per board, taken only from the named source and date. Compare models within one board, never across boards: harness and split change scores.
Aggregators
Aggregators
General intelligence & reasoning
General intelligence & reasoning
Math
Coding / software engineering
Agentic, computer use & long-horizon
Agentic, computer use & long-horizon
Multilingual (Swedish/Nordic)
Coding / software engineering
Coding / software engineering
Coding / software engineering
Coding / software engineering
Cybersecurity
Design, UI & multimodal
Knowledge, factuality & hallucination
Economic & real-world work value
Economic & real-world work value
Economic & real-world work value
Active / newNear-saturatedSaturatedFrozen / static / other
Every benchmark in the catalogue, grouped by category. Links open the official page; third-party marks pages not run by the benchmark authors.
Crowd-voted human-preference Elo
Combined score from 10 evals (agents 30%, coding 20%, sci. reasoning 20%, general 30%)
Capability scale built with item response theory (IRT) from about 61 benchmarks; hub for independent runs
Monthly-refreshed, contamination-limited general suite
Private expert-graded evals (several categories)
Stanford holistic suite
benchleader.com/benchmarks/helm_meanthird-party
Interactive novel-environment learning, fluid intelligence
2,500 expert questions at the edge of academic knowledge
Unpublished research-level physics
Graduate "Google-proof" science MCQ
Strategic reasoning through chess, poker and Werewolf
Harder multi-task knowledge MCQ
Research-level math, private problems
epoch.ai/frontiermath/tiers-1-4 epoch.ai/benchmarks/frontiermath-tier-4-v2
Olympiad proof writing, graded 0–7
github.com/EnvCommons/IMO-Bench matharena.ai/no_final_answer
Final-answer competition math
Unsolved research math (records first solutions)
Long-horizon, multi-file, multi-language real repo issues
Agentic terminal tasks: software engineering, sysadmin, ML, security
Human preference for generated web apps
Contest programming (Elo), post-cutoff problems
225 Exercism edits in 6 languages
Scientific-computing code
500 Python GitHub issues / visual issues
Paid freelance software tasks
Scientific workflows / compact agent index
Long-horizon real computer-use workflows (about 1.6 h median per task)
Deep-research agent on a fixed verified corpus
Length of task (in human time) an agent completes at 50%
GUI grounding on professional high-res screens
Year-long autonomous business operation (final bank balance)
Tool-agent-user dialogue, incl. banking knowledge and voice
taubench.com/leaderboardunreachable at last check
Multi-app SaaS workflow automation
Hard-to-find web facts
General assistant tool-use questions
personalagentbench.com/benchmarks/gaiathird-party
Kaggle-style ML engineering
Replicating ML research papers
benchlm.ai/benchmarks/paperbenchthird-party
Reproducible web-agent tasks
40 professional CTF tasks
Reproducing real vulnerabilities with working proofs of concept (PoCs)
Multi-step network/ICS attack simulations (32-step and 7-step)
Detect, exploit and patch on real bug bounties (dollar value)
Threat-intelligence knowledge, CVE→CWE, CVSS, ATT&CK
Exploiting critical web-app CVEs
Cyber risk and defensive SOC/patching
CVE proof-of-concept generation and patching ("SWE-bench for security")
Hazardous cyber, bio and chem knowledge proxy; unlearning
Crowd-voted human-preference Elo
Human preference for generated web apps
Crowd-voted web, UI, 3D and graphic design
Expert multimodal (image) reasoning
benchmarklist.com/benchmarks/mmmu_prothird-party
GUI grounding on professional high-res screens
Text-to-image human preference
arena.ai/leaderboard/text-to-image?rankBy= artificialanalysis.ai/image/leaderboard/text-to-image
Video understanding
video-mme.github.io/home%5Fpage.html alphaxiv.org/abs/2604.05015
Knowledge plus hallucination penalty (abstaining beats guessing)
Short-form factuality; grounding, parametric, search, multimodal
kaggle.com/benchmarks/deepmind/simpleqa-verified kaggle.com/benchmarks/google/facts
Long-context retrieval and reasoning (about 100k+ tokens)
huggingface.co/datasets/openai/mrcr huggingface.co/datasets/ArtificialAnalysis/AA-LCR
Long-context reasoning
benchlm.ai/benchmarks/longbench-v2third-party
Generalising to 58 unseen verifiable constraints
Verifiable formatting / multi-turn instruction retention
llmboard.ai/benchmarks/ifeval-270third-party llmboard.ai/benchmarks/multichallenge-428third-party
Harmful agent compliance / safety suite
benchmarklist.com/benchmarks/agentharm benchmarklist.com/benchmarks/helm_safety
Backdoor attacks and defences on LLMs (data, weight, CoT poisoning)
Chat safety across 12 hazard categories
Targeted task-level poisoning of instruction-tuned LLMs
Hazardous cyber, bio and chem knowledge proxy; unlearning
Expert-judged deliverables across 44 occupations
openai.com/index/gdpval artificialanalysis.ai/evaluations/gdpval-aa?models=
Banking, consulting and legal cross-app agent work
Share of real freelance projects completed to client standard
Year-long autonomous business operation (final bank balance)
Paid freelance software tasks
Nordic-language NLU/NLG/knowledge
euroeval.com/leaderboards/swedish euroeval.com/leaderboards/scandinavian
Multilingual reasoning (29 / 42 / 100+ languages)
mmluprox.github.io aclanthology.org/2025.acl-long.919.pdf arxiv.org/html/2510.24081v2
No benchmarks match these filters.
Catalogue: Hieg research, ranked by relevance (9 Oct 2026). Numbers are reproduced only from sources that publish them openly, with attribution and date; nothing is estimated. Raw data (JSON).
Stockholm · Europe/Stockholm
Hieg is a consulting practice for organisations that need sharp judgment on artificial intelligence, cybersecurity, and local sovereign infrastructure — without the slide-deck theatre.
Focus
Five overlapping domains. Engagements often cut across more than one.
Model selection, evaluation, and integration into real workflows. Grounded advice on what is ready, what is noisy, and what will actually ship.
Threat-aware design, hardening, and review for systems that handle sensitive data — including AI pipelines and edge deployments.
Short, high-signal engagements: architecture reviews, build-vs-buy decisions, and executive briefings that survive contact with engineering.
On-prem and private-cloud models when data residency, latency, or policy demand control. From hardware sizing to operational runbooks.
Practical bridging of perception, control, and AI — for teams shipping robots into the physical world, not demos that only work on stage.
Approach
Hieg is run by Gustavo A from Stockholm. The practice exists for teams that already know the generic playbook and need someone who can argue trade-offs, challenge vendors, and stay accountable when the system goes live.
Work is scoped tightly. Deliverables are written for the people who build and operate — not for a marketing archive.
Prefer architectures that keep critical inference, logs, and keys under your control when the risk profile demands it.
Threat models early. Least privilege, measurable surfaces, and no “we’ll harden it later” as a roadmap item.
Benchmarks, failure modes, and operational cost beat trend reports. If it cannot be measured, it is not a recommendation.
Writing
AI agents wired into Ghidra and IDA can now reconstruct how binaries, firmware, apps and legacy systems work; this article covers documented use cases, legal and security limits, and a company playbook.
Tesla and SpaceX/xAI say Digital Optimus will run computers from real-time video on edge hardware, not cloud screenshot loops — what is documented, what is open, and what it means for security, procurement and exit planning.
Model backdoors and poisoned weights are now demonstrated, cheap and invisible to benchmarks — what is proven, what is not, and how to decide who carries the risk.
A practical frame for deciding when local and sovereign AI infrastructure earns its keep.
Where prompts, embeddings, and tools actually expand your attack surface — and what to cut first.
Contact
Describe the problem, constraints, and timeline. Serious enquiries get a reply; vague mass outreach does not.