AI: How ChatGPT Works

By Xah Lee. Date: . Last updated: .

first, you need to learn how neural network works. see Neural Network Tutorial

How ChatGPT Works by Andrej Karpathy

Let's build GPT: from scratch, in code, spelled out.
Andrej Karpathy
Jan 17, 2023

We build a Generatively Pretrained Transformer (GPT), following the paper “Attention is All You Need” and OpenAI's GPT-2 / GPT-3. We talk about connections to ChatGPT, which has taken the world by storm. We watch GitHub Copilot, itself a GPT, help us write a GPT (meta :D!) . I recommend people watch the earlier makemore videos to get comfortable with the autoregressive language modeling framework and basics of tensors and PyTorch nn, which we take for granted in this video.

who is Andrej Karpathy

Andrej Karpathy (born 23 October 1986[1]) is a Slovak-Canadian computer scientist who served as the director of artificial intelligence and Autopilot Vision at Tesla. Karpathy currently works for OpenAI.[2][3][4] He specializes in deep learning and computer vision.

2023-03-11 from Wikipedia

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs. It is a rewrite of minGPT that prioritizes teeth over education. Still under active development, but currently the file train.py reproduces GPT-2 (124M) on OpenWebText, running on a single 8XA100 40GB node in about 4 days of training. The code itself is plain and readable: train.py is a ~300-line boilerplate training loop and model.py a ~300-line GPT model definition, which can optionally load the GPT-2 weights from OpenAI. That's it.

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.

Stephen Wolfram on ChatGPT

chatgpt jailbreak

since 2023-01, there is heavy censorship. but many have found clever ways to bypass it.

chatgpt jailbreak 2023-03-11
chatgpt jailbreak 2023-03-11
chatgpt jailbreak 2023-03-04
chatgpt jailbreak 2023-03-04

unsorted

The Illustrated Transformer By Jay Alammar. At https://jalammar.github.io/illustrated-transformer/

when artificial intelligence becomes a man

Geoffrey Hinton on Impact and Potential of AI

Full interview: "Godfather of artificial intelligence" talks impact and potential of AI
CBS Mornings
Mar 25, 2023

Geoffrey Hinton

Geoffrey Everest Hinton CC FRS FRSC[12] (born 6 December 1947) is a British-Canadian cognitive psychologist and computer scientist, most noted for his work on artificial neural networks. Since 2013, he has divided his time working for Google (Google Brain) and the University of Toronto. In 2017, he co-founded and became the Chief Scientific Advisor of the Vector Institute in Toronto.[13][14]

With David Rumelhart and Ronald J. Williams, Hinton was co-author of a highly cited paper published in 1986 that popularised the backpropagation algorithm for training multi-layer neural networks,[15] although they were not the first to propose the approach.[16] Hinton is viewed as a leading figure in the deep learning community.[17][18][19][20][21] The dramatic image-recognition milestone of the AlexNet designed in collaboration with his students Alex Krizhevsky[22] and Ilya Sutskever for the ImageNet challenge 2012[23] was a breakthrough in the field of computer vision.[24]

Hinton received the 2018 Turing Award, together with Yoshua Bengio and Yann LeCun, for their work on deep learning.[25] They are sometimes referred to as the “Godfathers of AI” and “Godfathers of Deep Learning”,[26][27] and have continued to give public talks together.[28][29]

from Wikipedia Geoffrey Hinton

Artificial Intelligence