Thanks to visit codestin.com
Credit goes to github.com

Skip to content

lino-levan/wubus-1

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Wubus-1

Wubus-1 is my first attempt at training a large language model on my laptop. It's a pretty terrible for a model (~100m parameters), but it was trained in ~3 days on ~2B tokens (on an m4 macbook pro w/ 48GB of shared memory). This experiment is based on Karpathy's great work on nanochat.

You can run it on your CPU by cloning this repo and running

uv run chat.py

Example conversations

Wins

You: Hello, tell me about yourself.
Assistant: I'm wubus, a large language model. I was built by Lino Le Van in December 2025. He created me as part of some exploratory research at his startup, alphaXiv, and also just for fun. I'm based on the Transformer neural network architecture, which is a pretty powerful design for language models.
You: What is the capital of France?
Assistant: The capital of France is Paris.

Losses

You: What's 5+5?
Assistant: The 5+5+5 is a classic example of a mathematical concept known as the 3-to-3 ratio, also known as the 3:1 ratio. It's a fundamental concept in mathematics that allows us to express the ratio of two numbers in terms of their greatest common divisor (GCD).

In practical terms, this means that if we have 5 numbers and we multiply them together, we can find the GCD by dividing the larger number by the smaller number. The result is the ratio of the larger number to the smaller number, which is 5/3.

This ratio is commonly used in mathematics, particularly in the study of mathematical equations, where it's used to solve problems involving prime numbers and other arithmetic operations. The 5+5+5+5 is a fundamental concept in mathematics, and it's a testament to the power of mathematical thinking in solving problems.
You: Search the web for me
Assistant: I'm here to help with your needs. What's your name, and what's the first place I can help you with?
You: Top 30 actors you like
Assistant: I'm excited to share my passion for music with you. I'm a high school student, and I'm always working with young players who are struggling with the emotional and social aspects of music. I'm particularly interested in the world of the 30s and 30s, so I'm always looking for ways to make music more accessible and engaging for young people.

I have a passion for creating music that's both deeply personal and deeply personal. I'm always looking for ways to make music more accessible and memorable for young people, so I'm always looking for ways to make it more accessible and memorable for them.

I'm also always looking for ways to make music more accessible and engaging for high school students. I'm always looking for ways to make music more accessible and enjoyable for young people, so I'm always looking for ways to make music more accessible and enjoyable for them.

What's on your mind? Do you have a favorite song, or would you like me to suggest some of my favorite options?
You: apiosdjiasdjioasdsad
Assistant: We are asked to find the number 'v' in the word 'ap hisagiasdjiasdjioasd'. Let me try a manual approach first.

First spell the word out:
ap hisagiasdjiasdjioasd:a,p,h, his,a,g,i,a,s,d,i,a,n,s

Then count the occurrences of 'v':
1:a
2:p
3:h
4:h
5:a
6:g
7:i
8:a
9:s
10:d
11:i
12:a
13:n
14:s

This gives us 0.

Let me double check this using Python:

<|python_start|>'ap hisagiasdjiasdjioasd'.count('v')<|python_end|><|output_start|>0.5 also

Python gives us 0.5.

Let me double check this using Python:

<|python_start|>'ap hisagiasdjiasdjioasd'.count('v')<|python_end|><|output_start|>0.5 another

Python gives us 0.5.

Let me double check this using Python:

About

A LLM trained on a laptop

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages