Dedicated to Ilya Sutskever.
https://situational-awareness.ai/
Текст большой, но любопытно.
Table of Contents
Each essay is meant to stand on its own, though I’d strongly encourage reading the series as a whole. For a pdf version of the full essay series, click here.
Introduction
History is live in San Francisco.
I. From GPT-4 to AGI: Counting the OOMs
AGI by 2027 is strikingly plausible. GPT-2 to GPT-4 took us from ~preschooler to ~smart high-schooler abilities in 4 years. Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year), algorithmic efficiencies (~0.5 OOMs/year), and “unhobbling” gains (from chatbot to agent), we should expect another preschooler-to-high-schooler-sized qualitative jump by 2027.
II. From AGI to Superintelligence: the Intelligence Explosion
AI progress won’t stop at human-level. Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into ≤1 year. We would rapidly go from human-level to vastly superhuman AI systems. The power—and the peril—of superintelligence would be dramatic.
III. The Challenges
IIIa. Racing to the Trillion-Dollar Cluster
The most extraordinary techno-capital acceleration has been set in motion. As AI revenue grows rapidly, many trillions of dollars will go into GPU, datacenter, and power buildout before the end of the decade. The industrial mobilization, including growing US electricity production by 10s of percent, will be intense.
IIIb. Lock Down the Labs: Security for AGI
The nation’s leading AI labs treat security as an afterthought. Currently, they’re basically handing the key secrets for AGI to the CCP on a silver platter. Securing the AGI secrets and weights against the state-actor threat will be an immense effort, and we’re not on track.
IIIc. Superalignment
Reliably controlling AI systems much smarter than we are is an unsolved technical problem. And while it is a solvable problem, things could easily go off the rails during a rapid intelligence explosion. Managing this will be extremely tense; failure could easily be catastrophic.
IIId. The Free World Must Prevail
Superintelligence will give a decisive economic and military advantage. China isn’t at all out of the game yet. In the race to AGI, the free world’s very survival will be at stake. Can we maintain our preeminence over the authoritarian powers? And will we manage to avoid self-destruction along the way?
IV. The Project
As the race to AGI intensifies, the national security state will get involved. The USG will wake from its slumber, and by 27/28 we’ll get some form of government AGI project. No startup can handle superintelligence. Somewhere in a SCIF, the endgame will be on.
V. Parting Thoughts
What if we’re right?
https://situational-awareness.ai/
Текст большой, но любопытно.
Table of Contents
Each essay is meant to stand on its own, though I’d strongly encourage reading the series as a whole. For a pdf version of the full essay series, click here.
Introduction
History is live in San Francisco.
I. From GPT-4 to AGI: Counting the OOMs
AGI by 2027 is strikingly plausible. GPT-2 to GPT-4 took us from ~preschooler to ~smart high-schooler abilities in 4 years. Tracing trendlines in compute (~0.5 orders of magnitude or OOMs/year), algorithmic efficiencies (~0.5 OOMs/year), and “unhobbling” gains (from chatbot to agent), we should expect another preschooler-to-high-schooler-sized qualitative jump by 2027.
II. From AGI to Superintelligence: the Intelligence Explosion
AI progress won’t stop at human-level. Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into ≤1 year. We would rapidly go from human-level to vastly superhuman AI systems. The power—and the peril—of superintelligence would be dramatic.
III. The Challenges
IIIa. Racing to the Trillion-Dollar Cluster
The most extraordinary techno-capital acceleration has been set in motion. As AI revenue grows rapidly, many trillions of dollars will go into GPU, datacenter, and power buildout before the end of the decade. The industrial mobilization, including growing US electricity production by 10s of percent, will be intense.
IIIb. Lock Down the Labs: Security for AGI
The nation’s leading AI labs treat security as an afterthought. Currently, they’re basically handing the key secrets for AGI to the CCP on a silver platter. Securing the AGI secrets and weights against the state-actor threat will be an immense effort, and we’re not on track.
IIIc. Superalignment
Reliably controlling AI systems much smarter than we are is an unsolved technical problem. And while it is a solvable problem, things could easily go off the rails during a rapid intelligence explosion. Managing this will be extremely tense; failure could easily be catastrophic.
IIId. The Free World Must Prevail
Superintelligence will give a decisive economic and military advantage. China isn’t at all out of the game yet. In the race to AGI, the free world’s very survival will be at stake. Can we maintain our preeminence over the authoritarian powers? And will we manage to avoid self-destruction along the way?
IV. The Project
As the race to AGI intensifies, the national security state will get involved. The USG will wake from its slumber, and by 27/28 we’ll get some form of government AGI project. No startup can handle superintelligence. Somewhere in a SCIF, the endgame will be on.
V. Parting Thoughts
What if we’re right?
SITUATIONAL AWARENESS - The Decade Ahead
Introduction - SITUATIONAL AWARENESS: The Decade Ahead
Leopold Aschenbrenner, June 2024 You can see the future first in San Francisco. Over the past year, the talk of the town has shifted from $10 billion compute clusters to $100 billion clusters to trillion-dollar clusters. Every six months another zero is added…
gonzo-обзоры ML статей
Dedicated to Ilya Sutskever. https://situational-awareness.ai/ Текст большой, но любопытно. Table of Contents Each essay is meant to stand on its own, though I’d strongly encourage reading the series as a whole. For a pdf version of the full essay series…
Если кто любит больше подкасты (на 4.5 часа!), то вот:
https://www.dwarkeshpatel.com/p/leopold-aschenbrenner
https://www.dwarkeshpatel.com/p/leopold-aschenbrenner
Dwarkeshpatel
Leopold Aschenbrenner - China/US Super Intelligence Race, 2027 AGI, & The Return of History
The trillion dollar cluster...
Кстати, кому не интересно 150 страниц про AGI в ближайшие годы, как альтернатива есть 510 страниц про темную материю:
Dark Matter
https://arxiv.org/abs/2406.01705
We review observational, experimental and theoretical results related to Dark Matter.
Dark Matter
https://arxiv.org/abs/2406.01705
We review observational, experimental and theoretical results related to Dark Matter.
arXiv.org
Dark Matter
We review observational, experimental and theoretical results related to Dark Matter.
Картинок вам в ленту.
https://posts.voronoiapp.com/technology/The-Training-Costs-of-AI-Models-Over-Time-1334
Странно, правда, что BERT всего 3k. Мне кажется по старым ценам должно было быть сильно больше
https://posts.voronoiapp.com/technology/The-Training-Costs-of-AI-Models-Over-Time-1334
Странно, правда, что BERT всего 3k. Мне кажется по старым ценам должно было быть сильно больше
gonzo-обзоры ML статей
Кстати, кому не интересно 150 страниц про AGI в ближайшие годы, как альтернатива есть 510 страниц про темную материю: Dark Matter https://arxiv.org/abs/2406.01705 We review observational, experimental and theoretical results related to Dark Matter.
А вот вам ещё прекрасного про физику, но теперь на 3 страницы вместо 510.
The forest as a neutrino detector
https://arxiv.org/abs/2401.14454
The primary challenge in detecting ultrahigh energy (UHE) neutrinos with energies exceeding 10^16 eV is to instrument a large enough volume to detect the extremely low flux, which falls as ∼E^−2. We explore in this article the feasibility of using the forest as a detector. Trees have been shown to be efficient broadband antennas, and may, without damage to the tree, be instrumented with a minimum of apparatus. A large scale array of such trees may be the key to achieving the requisite target volumes for UHE neutrino astronomy.
The forest as a neutrino detector
https://arxiv.org/abs/2401.14454
The primary challenge in detecting ultrahigh energy (UHE) neutrinos with energies exceeding 10^16 eV is to instrument a large enough volume to detect the extremely low flux, which falls as ∼E^−2. We explore in this article the feasibility of using the forest as a detector. Trees have been shown to be efficient broadband antennas, and may, without damage to the tree, be instrumented with a minimum of apparatus. A large scale array of such trees may be the key to achieving the requisite target volumes for UHE neutrino astronomy.
arXiv.org
The forest as a neutrino detector
The primary challenge in detecting ultrahigh energy (UHE) neutrinos with energies exceeding $10^{16}$ eV is to instrument a large enough volume to detect the extremely low flux, which falls as...
Ну и давненько про новые LLM не писали, а сегодня как раз вышла Qwen2 от Alibaba Cloud
https://qwenlm.github.io/blog/qwen2/
5 моделей: Qwen2-0.5B, Qwen2-1.5B, Qwen2-7B, Qwen2-57B-A14B, Qwen2-72B, обычные и instruction-tuned. Контекст до 128k. На бенчмарках выглядит красиво и бьёт Llama 3.
В дополнение к английскому и китайскому поддерживает ещё 27 языков.
Лицензия Apache 2.0 для всех кроме самой большой, у той прежняя Qianwen License.
В будущем обещают мультимодальность с видео и аудио.
https://qwenlm.github.io/blog/qwen2/
5 моделей: Qwen2-0.5B, Qwen2-1.5B, Qwen2-7B, Qwen2-57B-A14B, Qwen2-72B, обычные и instruction-tuned. Контекст до 128k. На бенчмарках выглядит красиво и бьёт Llama 3.
В дополнение к английскому и китайскому поддерживает ещё 27 языков.
Лицензия Apache 2.0 для всех кроме самой большой, у той прежняя Qianwen License.
В будущем обещают мультимодальность с видео и аудио.
Qwen
Hello Qwen2
GITHUB HUGGING FACE MODELSCOPE DEMO DISCORD
Introduction After months of efforts, we are pleased to announce the evolution from Qwen1.5 to Qwen2. This time, we bring to you:
Pretrained and instruction-tuned models of 5 sizes, including Qwen2-0.5B, Qwen2-1.5B…
Introduction After months of efforts, we are pleased to announce the evolution from Qwen1.5 to Qwen2. This time, we bring to you:
Pretrained and instruction-tuned models of 5 sizes, including Qwen2-0.5B, Qwen2-1.5B…
Сергей Марков дописал свою большую двухтомную книгу про ИИ «Охота на электроовец: большая книга искусственного интеллекта»! Поздравляем!
Я читал ранние версии и знаю, что Сергей проделал огромную работу по сбору исторического материала. Абсолютно разумно и без хайпа, с технически достоверными обзорами архитектур. Рекомендую.
Полные электронные версии книги (epub, docx, pdf) можно скачать с сайта бесплатно:
https://markoff.science
Я читал ранние версии и знаю, что Сергей проделал огромную работу по сбору исторического материала. Абсолютно разумно и без хайпа, с технически достоверными обзорами архитектур. Рекомендую.
Полные электронные версии книги (epub, docx, pdf) можно скачать с сайта бесплатно:
https://markoff.science
Ну и чтобы продолжить с книгами и ростом количества страниц, вот ещё прекрасное
Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Machine Learning
https://www.cis.upenn.edu/~jean/math-deep.pdf
На 2196 страниц.
Приятных выходных!
Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Machine Learning
https://www.cis.upenn.edu/~jean/math-deep.pdf
На 2196 страниц.
Приятных выходных!