Locator: 50287AI.
I absolutely cannot believe I ended up talking with Russ Bodnyk for about an hour today. Had I known, I would have spent three hours with him, but my wife needed to "move on."
Note: this is for my personal edification and to some members of the extended family. This is not for the general reader of the blog.
I had the highly unexpected opportunity to spend an hour with Russ Bodnyk today. Holy mackerel.
For the date-time stamp.
Russ Bodnyk. Interview. Intellibus. What's important about this video for AI novices: the jargon. The interview lasts almost three hours. Parameters. See also this post. See also this post.
Billions of parameters:
- 7 -- generally all he needs (llama 1);
- 13 -- occasionally uses;
- 70 -- can access if necessary;
- 120 --
- 170 - 500:
- 150 trillion -- about what the human mind is estimated to have.
The current LLMs are roughly equivalent to the mouse brain.
If current LLMs have upwards of 150 billion parameters, and are about equivalent to the mouse brain (synapses), when is it estimated that LLMs will come close to matching the human brain?
Bodnyk: however, thinks it's more than just counting synapses and parameters.
Genetics: I brought this up. The DNA - RNA dichotomy. DNA is boring; nothing there, in the big scheme of things, compared with RNA. RNA is where all the work is done. DNA is simply storage (DRAM).
LLMs doubling time: 16 months (notice it beats Moore's Law of 24 months.
Tag: Yann LeCun AMI Kurzweil
Ilya Sutskever: co-founder of OpenAI, the company behind ChatGPT.
Geoffrey Hinton: University of Toronot; Sutskever's
Phrase of note: convolutional neural networks (CNNs). Wiki link.
Yann André Le Cun, born 8 July 1960, is a French–American computer scientist working in the fields of artificial intelligence, machine learning, computer vision, robotics and image compression.
He is the Jacob T. Schwartz Professor of Computer Science at the Courant Institute of Mathematical Sciences at New York University. He served as Chief AI Scientist at Meta Platforms before leaving to work on his own startup company.
AI prompt: Russ Bodnyk often states that AI will reach its first major milestone fourteen years from now. Why would he say fourteen years as opposed to fifteen years or ten years or twenty years, but 14 years? What's so "special" about 14 years from now?
Reply:
Book: Why Machines Learn: The Elegant Math Behind Modern AI, Anil Ananthaswamy, c. 2024/2025.
