Stephen Wolfram on ChatGPT

Stephen Wolfram on ChatGPT

• Am still reading stephen wolfram's book on how neural network and chatGPT works.
• still have not finished it. I read in detail, including reading most embedded links.

• Been reading it since Feb.
• And have been reading a ton on machine learning and neural network, on Wikipedia and scikit, YouTube 3blue1brown, academic papers etc.

• Wolfram's piece, is the best, i'd say.
• It isn't a narrow tech tutorial for just programers, but a broad scientific look at the issue, with respect to the whole science field and math.
• It is also the most lucid, capturing the tech detail of what happens, and why things work.

• https://x.com/xah_lee/status/1708571489417781469

• 〔What Is ChatGPT Doing … and Why Does It Work? 2023-02-14 By Stephen Wolfram. At https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/〕

• Neural network design and their effects.
• here are examples of best neural net of given design, with small number of nodes and layers.
• They are trying to learn a square function. Input is just a real number.

• Neural net can never do simple calculations, such as calculate Pi to a thousand places.
• A deep philosophical damning conclusion.
• Poignantly true and insightful.

• Wolfram is saying that AI cannot compute. (read his blog for his exact words)
• in other words, if we have artificial general intelligence like Mr Data in Star Trek, it won't be able to multiply 2 big numbers like a pocket calculator can.

• another way to put this is that, if one day we have sentient AI, it won't be able to predict weather.
• we still need our old super computers for that.

• Need to think over this. lots to think about.

• https://x.com/xah_lee/status/1707297829226737676
• https://x.com/xah_lee/status/1707307145015026100

• Artificial intelligence. Typical neural network layers.
• Typically start with a 2D matrix (or n-D), each layer usually reduce shape or dimension, and eventually become 1D (just a vector. aka list of numbers) down to a single number.

• Neural net training is currently mostly sequential.

• chatGPT, represent every words by numbers.
• https://x.com/xah_lee/status/1708534434981638570