Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 0467c82

Browse files
rasbtrasbt
authored
GPT to Llama (rasbt#368)
* GPT to Llama * fix urls
1 parent dc1b1a0 commit 0467c82

File tree

7 files changed

+1644
-0
lines changed

7 files changed

+1644
-0
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ ch05/01_main-chapter-code/model.pth
3434
ch05/01_main-chapter-code/model_and_optimizer.pth
3535
ch05/03_bonus_pretraining_on_gutenberg/model_checkpoints
3636
ch05/06_user_interface/gpt2
37+
ch05/07_gpt_to_llama/models--meta-llama--Llama-2-7b
38+
ch05/07_gpt_to_llama/models--meta-llama--Llama-2-7b-chat
3739

3840
ch06/01_main-chapter-code/gpt2
3941
ch06/02_bonus_additional-experiments/gpt2

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,7 @@ Several folders contain optional materials as a bonus for interested readers:
116116
- [Adding Bells and Whistles to the Training Loop](ch05/04_learning_rate_schedulers)
117117
- [Optimizing Hyperparameters for Pretraining](ch05/05_bonus_hparam_tuning)
118118
- [Building a User Interface to Interact With the Pretrained LLM](ch05/06_user_interface)
119+
- [Converting GPT to Llama](ch05/07_gpt_to_llama)
119120
- **Chapter 6:**
120121
- [Additional experiments finetuning different layers and using larger models](ch06/02_bonus_additional-experiments)
121122
- [Finetuning different models on 50k IMDB movie review dataset](ch06/03_bonus_imdb-classification)

‎ch05/07_gpt_to_llama/README.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# Converting GPT to Llama
2+
3+
4+
5+
This folder contains code for converting the GPT implementation from chapter 4 and 5 to Meta AI's Llama architecture:
6+
7+
- [converting-gpt-to-llama2.ipynb](converting-gpt-to-llama2.ipynb): contains code to convert GPT to Llama 2 7B step by step and loads pretrained weights from Meta AI

0 commit comments

Comments
 (0)