Open Threads

Okay, so we have a plan for the series on Artificial Intelligence that was proposed in the post earlier this week.

Artificial Intelligence: A Question for Balloon Juice Peeps

This will be a 7-part series.  It’s the same material as the outline that showed 5 parts, just broken into 7 posts instead of 5 parts.

So if you’re interested, mark your calendars for Wednesdays at 7:30 pm.  (blog time)

Part 1:  Oct 29
Part 2:  Nov 5
Part 3:  Nov 12
Part 4:  Nov 19

skip Thanksgiving week

Part 5:  Dec 3
Part 6:  Dec 10
Part 7:  Dec 17

Revised Outline

Post 1: What is AI, And How Did We Get Here

• Who am I, what do I do, and why do I think I have anything useful to say about AI?
• Overview of series
• What is AI?
  • Modern “AI” is all deep learning, which is all machine learning, which is all statistical learning. There is no reasoning, or anything else analogizable to human cognition, beyond “learning”. For the purposes of these posts, we will stick to the term “machine learning,” and avoid the obnoxious term AI (except in scare quotes) because that term only brings business/funding/marketing distractions into a discussion that I would like to keep as technically sound as I can.
  • That said, machine learning represents a very powerful set of techniques for assimilating masses of data, and structuring reasonable decisions based on that data.
• A brief history of the deep learning revolution.
  • The state of play in applied mathematics and statistics before the revolution
  • The computer scientists see an opportunity seize the day
  • Incredible achievements, blinding speed of progress
  • Costs and benefits of progress.
    • Benefits:
      • Solutions to problems previously regarded as insoluble (image classification, natural language processing, voice processing, high-dimensional modeling, many others)
      • Solutions to hard scientific problems (protein folding, experiment design and control, literature search, climate/weather modeling, many others)
    • Costs:
      • The abandonment of scientific epistemology
      • The embrace of heuristics over algorithms
      • The analogy to financial mathematics
      • Goodbye to Model Correctness, Model Verification, Model Validation.

Post 2: “AI” State of Play

• Different types of data, different approaches, different levels of success
• Two outlooks on machine learning: Model-Centric (the consensus view) and “Data-Centric” (the rebel alliance :-) ). Description of the differences in these outlooks, and in some of the research agenda consequences.
• The data-centric view informs the remainder of the discussion. So this is going to sound like a very strange story to anyone who has been following developments in “AI”, irrespective of their level of technical literacy.
• Where are we today in “AI”?
  • The current research/investment landscape focuses on language models (tokenized data)
    • The shock of OpenAI’s GPT models, ChatGPT
    • The embarrasment of “hallucinations”
    • Hyperscaling
    • The research/business consensus: hyperscaling will remove hallucinations, reach “Artificial General Intelligence” (“AGI”)
• Does this story make sense?
  • The Underwear Gnomes: …Profit!
  • The problem of science&technology groupthink
  • The corruption of scientific discourse by Tech company business interests. The analogies to climate science, and to the Tobacco Institute.

Post 3: There is no AGI Down This Road

• At least, there is no scientifically credible case that computer reasoning that can even theoretically emerge from machine learning. And reasoning is a necessary condition for AGI.
• The circular argument for LLM “reasoning”
• The sheer implausibility of the “Learning to AGI” hypothesis
  • On inverse problems
  • The inverse problem from language to cognition
  • The absurdity of transformers as tools for solving the required inverse problem, and as mediums for embodying the solution
• Claims for “General Intelligence” in chatbot output are pure, ascientific bullshit, as in “not even wrong.” Those claims are nonetheless taken very seriously by the “AI” research community.

Post 4: If There Were AGI, What Would It Look Like

• What is “learning”. Bayesian updating, in words rather than with equations
• What could reasoning actually mean in this context?
  • The discontinuity of “Aha!”
  • Machine learning is a continuous process that doesn’t do “Aha!”
• How might one go about getting to “Artificial Aha!” (AA!)
  • Reasoning as Model Reform
  • Reasoning as Evidence Reform
• AGI may be possible by other means, but that isn’t where we are going currently. “AI” is machine learning technology, and reasoning essentially transcends learning.

Post 5: Hallucinations

• What are “AI Hallucinations”
  • Some fun examples. Legal briefs, homework, airline bookings…
• Why are hallucinations a problem?
  • If reasoning really “emerges” from learning, why is it so crippled, and how could it be cured of mental illness?
• Natural Language Processing (NLP). History of the transformer architecture, the attention mechanism.
• Approximating the distribution of text.
• The sampling dilemma: most probable words vs. most probable sentences vs. most probable paragraphs, etc.
• Why is the approximation to the distribution so bad? Token embedding. Euclidean corruption of token sequences. Examples.
• Hallucinations are never going away: they are a structural feature of all current approaches to NLP. You can’t train your way out of them with more data, and you cant bejigger the code to fix them.

Post 6: The Pathology At The Heart Of Hyperscaling

• The hyperscaling consensus.
  • Insane investments. Compute, power, human effort.
  • Buy-in to the consensus beyond the Tech Industry (academia and govvernment)
  • The costs:
    • Academia is shut out of “AI” research, due to costs.
    • Government very nearly is as well
    • Investors, fund managers, capital, committed to an outlook wherein $300-400B/year for the next 5-10 years required to develop and train models. 60GW of additional electrical generation expected for the US alone, just to keep the GPUs running.
• The data inefficiency of deep learning/neural network methods
  • Overparametrization has costs and benefits:
    • Easy to learn and exploit vector/image, natural science data.
    • Unaffordably hard to learn/exploit natural language data
• What have we learned from transformers?
  • Natural language can in fact be modeled.
  • If we keep doing it this way, the cost in power, effort, and money is unaffordable.
• Why are we not looking for alternatives?
  • Riff on groupthink
  • What might an alternative approach look like?

Post 7: AI Winter III in 5, 4, 3,…

• The historical cycles of the previous 2 “AI Winters”.
  • 1. Progress, (2) excitement, (3) investment, (4) hype, (5) disillusionment, (6) disinvestment+wringing of hands
  • We appear to be at the beginnings of stage (5). See Ed Zitron’s blog.
• The basic story that justifies the crazy investments in AI is that hyperscaling and more data will abate hallucinations and achieve AGI. AGI will cause markets to arise that amortize those investments, and reward the players that made them (and punish those that di not)
• This argument is wrong, for technical reasons.
  • There is no AGI on this technological path
  • Hallucinations are a structural feature of all current “AI”, and cannot be trained or finessed away
  • Hyperscaling, which is driven by the pathologically inefficient use of data made by transformers, only means that it will take an infinite amount of time, compute, money, energy, and effort to get to the end ofthis rainbow and discover that there is no pot of gold labelled “AGI, Profit!” there.
• Ultimately, however, historical evidence indicates that the dispositive argument will be financial rather than technical. Investors are going to lose confidence in an industry that cannot figure out how any player can be profitable (again, see Zitron). The reasons for their failure are ultimately technical, but the agents who close down the show will be financiers and shareholders who don’t like how their money is being spent.
  • My guess is that the reckoning, and AI Winter III, are coming in 2026. Don’t make any investment decisions based on what I write, though.
• Then what?
  • Getting research out from under the corrupting influence of the tech industry
  • Babies and bathwater.

Mark Your Calendars for the Wednesday Series on Artificial IntelligencePost + Comments (21)