Large Language Model (LLM)

Word Embedding

Because Networks work with data and numbers, cannot read and understand the semantic meaning of the text or image (thats why we use ImageNET for example). There are different models where the input is text and it gets converted into a vector. You create or relate vectors together, placing it into the space, close to each other if similar meanings and some other further away.

Word2Vec

Uses Word2Vec. projector.tensorflow.org. Gets the relations between words in space.

Lets say we have words and by passing it through word2Vec we get colors, per word with an array one dimensional of data, numbers. Each word would have a different array and they get related.

Sentiment

You give a sentence. It returns a score. If close to 0, very negative. If close to 1, very positive. Another way to turn words to numbers or data. Is trained in imDB reviews. Is a great data set.

Understanding Large Language Models (LLMs)

large models that were born from text and stays in text.

GPT = Generative Pre-trained Transformers. Generate more text for you. Trained in a lot of text. It was pretraining (as much as possible text on pre training stage). Even ind video models pre training woud be the same and the post train would be fine tuning stage. Transformers: main architecture that llms are using. The main difference and new technology. Complete question. Complete paper, attention is all you need from 2017. Before, you would add text and just receive more text. Most of the time would not even make sense.

Input → Transformers, word and next word in probability→ output Softmax

Tokenization

sometimes one word is.a token or two workds is a token.

Attention

Mechanism that enables the model to dynamically focus on different parts of the input sequence when predicting each part of the output sequence, much like how human attention works when focusing on different aspects.