AI Model Efficiency Milestones

The efficiency of AI models has rapidly improved, with inference costs for large language models (LLMs) falling dramatically, while algorithmic advancements allow for more performance with less compute.

The top 3

  1. Inference Cost Declines: LLM inference costs have fallen by 9x to 900x per year depending on the task, with some fixed-performance LLM inference costs halving every two months.
  2. Algorithmic Efficiency Gains: Researchers have made underlying AI algorithms more efficient, allowing the same performance to be achieved with three times less compute each year.
  3. Hardware Performance/Cost: AI chip performance per dollar has improved by 37% per year, effectively doubling every 2.2 years, while hardware costs decline by 30% annually.

Sources

Open the full topic