lm.c

lm.c

lm.c

Lightweight CPU Inference Engine for Large Language Models using a single file

lm.c is a lightweight, portable, and efficient inference engine designed for running large language models (LLMs) on CPU systems. Built entirely in pure C99, it supports GGUF file format models with all quantization types while maintaining minimal dependencies.

Features

✅ Pure C implementation - Zero external dependencies beyond standard library
⚡ CPU-optimized - Designed for maximum performance on standard processors
📦 Full GGUF support - All quantization formats (Q4_0 to IQ1_M) and metadata types
🌐 Portable - Runs on any platform with a C99 compiler (Linux, macOS, Windows)
🔍 Detailed inspection - View complete model structure and metadata
🧠 Memory efficient - Zero-copy tensor access with memory mapping

What has been DONE so far

🗂️ GGUF Parser
- Handles all metadata types and quantization formats.
🏭 Model Structure
- Represents transformer architecture with separate Q/K/V weights and proper layer organization.

Supported Models

lm.c supports all models in GGUF format through its architecture-agnostic design, including:

Model Family	Supported Features	Quantization Support
LLaMA	RoPE, RMSNorm, SwiGLU	All 30+ GGML types
Mistral	Sliding Window Attention	Mixed precision inference
Mixtral	Sparse MoE Experts	Per-expert quantization
Phi	Parallel Residual Blocks	Block-wise dequantization
Gemma	GeGLU, RMSNorm	Multi-format tensors
Qwen	GQA, RMSNorm	Hybrid quantization
Custom Models	Flexible architecture adaptation	Full quantization range

And any other model with GGUF support

Supported Quantization Types

All GGML quantization formats are supported:

GGML_TYPE_F32, GGML_TYPE_F16, GGML_TYPE_Q4_0, GGML_TYPE_Q4_1,
GGML_TYPE_Q5_0, GGML_TYPE_Q5_1, GGML_TYPE_Q8_0, GGML_TYPE_Q8_1,
GGML_TYPE_Q2_K, GGML_TYPE_Q3_K, GGML_TYPE_Q4_K, GGML_TYPE_Q5_K,
GGML_TYPE_Q6_K, GGML_TYPE_Q8_K, GGML_TYPE_IQ2_XXS, GGML_TYPE_IQ2_XS,
GGML_TYPE_IQ3_XXS, GGML_TYPE_IQ1_S, GGML_TYPE_IQ4_NL, GGML_TYPE_IQ3_S,
GGML_TYPE_IQ2_S, GGML_TYPE_IQ4_XS, GGML_TYPE_I8, GGML_TYPE_I16,
GGML_TYPE_I32, GGML_TYPE_I64, GGML_TYPE_F64, GGML_TYPE_IQ1_M

GGUF in detail

Architecture Overview

System Workflow

graph TD
    A[GGUF File] --> B[Header Parsing]
    B --> C[Metadata Extraction]
    B --> D[Tensor Info Loading]
    C --> E[Model Configuration]
    D --> F[Tensor Data Mapping]
    E --> G[Inference Engine]
    F --> G
    G --> H[Tokenization]
    H --> I[Transformer Layers]
    I --> J[Sampling]
    J --> K[Output Generation]

Core Components

Component	Description
🗂️ GGUF Parser	Handles all metadata types and quantization formats
🏭 Model Structure	Represents transformer architecture with separate Q/K/V weights and proper layer organization
🧠 Quantization Engine	Supports 30+ GGML quantization formats
⚡ CPU Inference	Optimized transformer execution
🌐 Portable Runtime	Single-file C99 implementation

GGUF File Structure

graph TD
    A[Magic Header] --> B[Version]
    B --> C[Tensor Count]
    C --> D[Metadata KVs]
    D --> E[Tensor Names]
    E --> F[Dimensions]
    F --> G[Quantization Types]
    G --> H[Tensor Data]

struct gguf_header_t {
    uint32_t magic;          // "GGUF"
    uint32_t version;        // Format version
    uint64_t tensor_count;   // Number of tensors
    uint64_t metadata_kv_count;
    gguf_metadata_kv_t metadata_kv[]; // Key-value pairs
};

Transformer Architecture

graph TB
    A[Input] --> B[Token Embeddings]
    B --> C[RMS Normalization]
    C --> D[Multi-Head Attention]
    D -->|Q/K/V| E[Linear Projections]
    C --> F[RMS Normalization]
    F --> G[Feed Forward Network]
    G -->|SwiGLU| H[Gate/Up/Down]
    D --> I[Residual Add]
    G --> I
    I --> J[Output Projection]
    J --> K[Sampling]
    K --> L[Output Text]

Memory Efficient Design

Zero-copy tensor access: Memory-mapped weights
On-the-fly dequantization: Process quantized weights directly
Reusable buffers: Single allocation for activations
Optimized KV cache: Compressed storage format
SIMD registers: Vectorized operations
Thread pools: Parallel execution

graph TB

Inference Workflow

flowchart TD
    A[Input Text] --> B[Tokenization]
    B --> C[Embedding Lookup]
    C --> D[Layer Norm]
    D --> E[Attention]
    E --> F[FFN]
    F --> G[Residual Add]
    G --> H[Output Projection]
    H --> I[Sampling]
    I --> J[Generated Text]

Performance Optimizations

Quantization-aware ops: Direct processing of compressed weights
Block processing: Optimized cache utilization
Memory mapping: Zero-copy weight access
Thread parallelism: Layer-wise execution
SIMD vectorization: AVX2/NEON acceleration
Mixed precision: FP16/FP32 hybrid compute

Getting Started

Prerequisites

C compiler (GCC, Clang, or MSVC)

Building & Running

Clone repository

git clone https://github.com/oderoi/lm.c.git
cd lm.c

download any gguf-model from Huggingface

Example

wget "https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q4_K_M.gguf?download=true"

Build with GCC

gcc lm.c -o lm

build with Clang

clang lm.c -o lm

Usage

Inspect any GGUF model

Syntax : ./lm path/to/model.gguf

./lm  "mistral-7b-instruct-v0.2.Q4_K_M.gguf"

Running Inference is comming soon

Output

Loading GGUF file: mistral-7b-instruct-v0.2.Q4_K_M.gguf
========================================
==== Loading GGUF Header ====
Magic: 0x46554747
Version: 3
Tensor Count: 291
Metadata KV Count: 24



==== Loading Metadata Key-Value Store ====
Metadata[0]: Key='general.architecture', Type=8
Metadata[1]: Key='general.name', Type=8
Metadata[2]: Key='llama.context_length', Type=4
Metadata[3]: Key='llama.embedding_length', Type=4
Metadata[4]: Key='llama.block_count', Type=4
Metadata[5]: Key='llama.feed_forward_length', Type=4
Metadata[6]: Key='llama.rope.dimension_count', Type=4
Metadata[7]: Key='llama.attention.head_count', Type=4
Metadata[8]: Key='llama.attention.head_count_kv', Type=4
Metadata[9]: Key='llama.attention.layer_norm_rms_epsilon', Type=6
Metadata[10]: Key='llama.rope.freq_base', Type=6
Metadata[11]: Key='general.file_type', Type=4
Metadata[12]: Key='tokenizer.ggml.model', Type=8
Metadata[13]: Key='tokenizer.ggml.tokens', Type=9
Metadata[14]: Key='tokenizer.ggml.scores', Type=9
Metadata[15]: Key='tokenizer.ggml.token_type', Type=9
Metadata[16]: Key='tokenizer.ggml.bos_token_id', Type=4
Metadata[17]: Key='tokenizer.ggml.eos_token_id', Type=4
Metadata[18]: Key='tokenizer.ggml.unknown_token_id', Type=4
Metadata[19]: Key='tokenizer.ggml.padding_token_id', Type=4
Metadata[20]: Key='tokenizer.ggml.add_bos_token', Type=7
Metadata[21]: Key='tokenizer.ggml.add_eos_token', Type=7
Metadata[22]: Key='tokenizer.chat_template', Type=8
Metadata[23]: Key='general.quantization_version', Type=4



==== Loading Tensor Info ====
Tensor[0]: Name='token_embd.weight', Dims=2, Type=Q4_K, Offset=0
  Dimensions: 4096x32000
Tensor[1]: Name='blk.0.attn_q.weight', Dims=2, Type=Q4_K, Offset=73728000
  Dimensions: 4096x4096
Tensor[2]: Name='blk.0.attn_k.weight', Dims=2, Type=Q4_K, Offset=83165184
  Dimensions: 4096x1024
Tensor[3]: Name='blk.0.attn_v.weight', Dims=2, Type=Q6_K, Offset=85524480
  Dimensions: 4096x1024
Tensor[4]: Name='blk.0.attn_output.weight', Dims=2, Type=Q4_K, Offset=88965120
  Dimensions: 4096x4096
Tensor[5]: Name='blk.0.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=98402304
  Dimensions: 4096x14336
Tensor[6]: Name='blk.0.ffn_up.weight', Dims=2, Type=Q4_K, Offset=131432448
  Dimensions: 4096x14336
Tensor[7]: Name='blk.0.ffn_down.weight', Dims=2, Type=Q6_K, Offset=164462592
  Dimensions: 14336x4096
Tensor[8]: Name='blk.0.attn_norm.weight', Dims=1, Type=F32, Offset=212631552
  Dimensions: 4096
Tensor[9]: Name='blk.0.ffn_norm.weight', Dims=1, Type=F32, Offset=212647936
  Dimensions: 4096
Tensor[10]: Name='blk.1.attn_q.weight', Dims=2, Type=Q4_K, Offset=212664320
  Dimensions: 4096x4096
Tensor[11]: Name='blk.1.attn_k.weight', Dims=2, Type=Q4_K, Offset=222101504
  Dimensions: 4096x1024
Tensor[12]: Name='blk.1.attn_v.weight', Dims=2, Type=Q6_K, Offset=224460800
  Dimensions: 4096x1024
Tensor[13]: Name='blk.1.attn_output.weight', Dims=2, Type=Q4_K, Offset=227901440
  Dimensions: 4096x4096
Tensor[14]: Name='blk.1.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=237338624
  Dimensions: 4096x14336
Tensor[15]: Name='blk.1.ffn_up.weight', Dims=2, Type=Q4_K, Offset=270368768
  Dimensions: 4096x14336
Tensor[16]: Name='blk.1.ffn_down.weight', Dims=2, Type=Q6_K, Offset=303398912
  Dimensions: 14336x4096
Tensor[17]: Name='blk.1.attn_norm.weight', Dims=1, Type=F32, Offset=351567872
  Dimensions: 4096
Tensor[18]: Name='blk.1.ffn_norm.weight', Dims=1, Type=F32, Offset=351584256
  Dimensions: 4096
Tensor[19]: Name='blk.2.attn_q.weight', Dims=2, Type=Q4_K, Offset=351600640
  Dimensions: 4096x4096
Tensor[20]: Name='blk.2.attn_k.weight', Dims=2, Type=Q4_K, Offset=361037824
  Dimensions: 4096x1024
Tensor[21]: Name='blk.2.attn_v.weight', Dims=2, Type=Q6_K, Offset=363397120
  Dimensions: 4096x1024
Tensor[22]: Name='blk.2.attn_output.weight', Dims=2, Type=Q4_K, Offset=366837760
  Dimensions: 4096x4096
Tensor[23]: Name='blk.2.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=376274944
  Dimensions: 4096x14336
Tensor[24]: Name='blk.2.ffn_up.weight', Dims=2, Type=Q4_K, Offset=409305088
  Dimensions: 4096x14336
Tensor[25]: Name='blk.2.ffn_down.weight', Dims=2, Type=Q6_K, Offset=442335232
  Dimensions: 14336x4096
Tensor[26]: Name='blk.2.attn_norm.weight', Dims=1, Type=F32, Offset=490504192
  Dimensions: 4096
Tensor[27]: Name='blk.2.ffn_norm.weight', Dims=1, Type=F32, Offset=490520576
  Dimensions: 4096
Tensor[28]: Name='blk.3.attn_q.weight', Dims=2, Type=Q4_K, Offset=490536960
  Dimensions: 4096x4096
Tensor[29]: Name='blk.3.attn_k.weight', Dims=2, Type=Q4_K, Offset=499974144
  Dimensions: 4096x1024
Tensor[30]: Name='blk.3.attn_v.weight', Dims=2, Type=Q6_K, Offset=502333440
  Dimensions: 4096x1024
Tensor[31]: Name='blk.3.attn_output.weight', Dims=2, Type=Q4_K, Offset=505774080
  Dimensions: 4096x4096
Tensor[32]: Name='blk.3.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=515211264
  Dimensions: 4096x14336
Tensor[33]: Name='blk.3.ffn_up.weight', Dims=2, Type=Q4_K, Offset=548241408
  Dimensions: 4096x14336
Tensor[34]: Name='blk.3.ffn_down.weight', Dims=2, Type=Q6_K, Offset=581271552
  Dimensions: 14336x4096
Tensor[35]: Name='blk.3.attn_norm.weight', Dims=1, Type=F32, Offset=629440512
  Dimensions: 4096
Tensor[36]: Name='blk.3.ffn_norm.weight', Dims=1, Type=F32, Offset=629456896
  Dimensions: 4096
Tensor[37]: Name='blk.4.attn_q.weight', Dims=2, Type=Q4_K, Offset=629473280
  Dimensions: 4096x4096
Tensor[38]: Name='blk.4.attn_k.weight', Dims=2, Type=Q4_K, Offset=638910464
  Dimensions: 4096x1024
Tensor[39]: Name='blk.4.attn_v.weight', Dims=2, Type=Q4_K, Offset=641269760
  Dimensions: 4096x1024
Tensor[40]: Name='blk.4.attn_output.weight', Dims=2, Type=Q4_K, Offset=643629056
  Dimensions: 4096x4096
Tensor[41]: Name='blk.4.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=653066240
  Dimensions: 4096x14336
Tensor[42]: Name='blk.4.ffn_up.weight', Dims=2, Type=Q4_K, Offset=686096384
  Dimensions: 4096x14336
Tensor[43]: Name='blk.4.ffn_down.weight', Dims=2, Type=Q4_K, Offset=719126528
  Dimensions: 14336x4096
Tensor[44]: Name='blk.4.attn_norm.weight', Dims=1, Type=F32, Offset=752156672
  Dimensions: 4096
Tensor[45]: Name='blk.4.ffn_norm.weight', Dims=1, Type=F32, Offset=752173056
  Dimensions: 4096
Tensor[46]: Name='blk.5.attn_q.weight', Dims=2, Type=Q4_K, Offset=752189440
  Dimensions: 4096x4096
Tensor[47]: Name='blk.5.attn_k.weight', Dims=2, Type=Q4_K, Offset=761626624
  Dimensions: 4096x1024
Tensor[48]: Name='blk.5.attn_v.weight', Dims=2, Type=Q4_K, Offset=763985920
  Dimensions: 4096x1024
Tensor[49]: Name='blk.5.attn_output.weight', Dims=2, Type=Q4_K, Offset=766345216
  Dimensions: 4096x4096
Tensor[50]: Name='blk.5.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=775782400
  Dimensions: 4096x14336
Tensor[51]: Name='blk.5.ffn_up.weight', Dims=2, Type=Q4_K, Offset=808812544
  Dimensions: 4096x14336
Tensor[52]: Name='blk.5.ffn_down.weight', Dims=2, Type=Q4_K, Offset=841842688
  Dimensions: 14336x4096
Tensor[53]: Name='blk.5.attn_norm.weight', Dims=1, Type=F32, Offset=874872832
  Dimensions: 4096
Tensor[54]: Name='blk.5.ffn_norm.weight', Dims=1, Type=F32, Offset=874889216
  Dimensions: 4096
Tensor[55]: Name='blk.6.attn_q.weight', Dims=2, Type=Q4_K, Offset=874905600
  Dimensions: 4096x4096
Tensor[56]: Name='blk.6.attn_k.weight', Dims=2, Type=Q4_K, Offset=884342784
  Dimensions: 4096x1024
Tensor[57]: Name='blk.6.attn_v.weight', Dims=2, Type=Q6_K, Offset=886702080
  Dimensions: 4096x1024
Tensor[58]: Name='blk.6.attn_output.weight', Dims=2, Type=Q4_K, Offset=890142720
  Dimensions: 4096x4096
Tensor[59]: Name='blk.6.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=899579904
  Dimensions: 4096x14336
Tensor[60]: Name='blk.6.ffn_up.weight', Dims=2, Type=Q4_K, Offset=932610048
  Dimensions: 4096x14336
Tensor[61]: Name='blk.6.ffn_down.weight', Dims=2, Type=Q6_K, Offset=965640192
  Dimensions: 14336x4096
Tensor[62]: Name='blk.6.attn_norm.weight', Dims=1, Type=F32, Offset=1013809152
  Dimensions: 4096
Tensor[63]: Name='blk.6.ffn_norm.weight', Dims=1, Type=F32, Offset=1013825536
  Dimensions: 4096
Tensor[64]: Name='blk.7.attn_q.weight', Dims=2, Type=Q4_K, Offset=1013841920
  Dimensions: 4096x4096
Tensor[65]: Name='blk.7.attn_k.weight', Dims=2, Type=Q4_K, Offset=1023279104
  Dimensions: 4096x1024
Tensor[66]: Name='blk.7.attn_v.weight', Dims=2, Type=Q4_K, Offset=1025638400
  Dimensions: 4096x1024
Tensor[67]: Name='blk.7.attn_output.weight', Dims=2, Type=Q4_K, Offset=1027997696
  Dimensions: 4096x4096
Tensor[68]: Name='blk.7.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=1037434880
  Dimensions: 4096x14336
Tensor[69]: Name='blk.7.ffn_up.weight', Dims=2, Type=Q4_K, Offset=1070465024
  Dimensions: 4096x14336
Tensor[70]: Name='blk.7.ffn_down.weight', Dims=2, Type=Q4_K, Offset=1103495168
  Dimensions: 14336x4096
Tensor[71]: Name='blk.7.attn_norm.weight', Dims=1, Type=F32, Offset=1136525312
  Dimensions: 4096
Tensor[72]: Name='blk.7.ffn_norm.weight', Dims=1, Type=F32, Offset=1136541696
  Dimensions: 4096
Tensor[73]: Name='blk.8.attn_q.weight', Dims=2, Type=Q4_K, Offset=1136558080
  Dimensions: 4096x4096
Tensor[74]: Name='blk.8.attn_k.weight', Dims=2, Type=Q4_K, Offset=1145995264
  Dimensions: 4096x1024
Tensor[75]: Name='blk.8.attn_v.weight', Dims=2, Type=Q4_K, Offset=1148354560
  Dimensions: 4096x1024
Tensor[76]: Name='blk.8.attn_output.weight', Dims=2, Type=Q4_K, Offset=1150713856
  Dimensions: 4096x4096
Tensor[77]: Name='blk.8.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=1160151040
  Dimensions: 4096x14336
Tensor[78]: Name='blk.8.ffn_up.weight', Dims=2, Type=Q4_K, Offset=1193181184
  Dimensions: 4096x14336
Tensor[79]: Name='blk.8.ffn_down.weight', Dims=2, Type=Q4_K, Offset=1226211328
  Dimensions: 14336x4096
Tensor[80]: Name='blk.8.attn_norm.weight', Dims=1, Type=F32, Offset=1259241472
  Dimensions: 4096
Tensor[81]: Name='blk.8.ffn_norm.weight', Dims=1, Type=F32, Offset=1259257856
  Dimensions: 4096
Tensor[82]: Name='blk.9.attn_q.weight', Dims=2, Type=Q4_K, Offset=1259274240
  Dimensions: 4096x4096
Tensor[83]: Name='blk.9.attn_k.weight', Dims=2, Type=Q4_K, Offset=1268711424
  Dimensions: 4096x1024
Tensor[84]: Name='blk.9.attn_v.weight', Dims=2, Type=Q6_K, Offset=1271070720
  Dimensions: 4096x1024
Tensor[85]: Name='blk.9.attn_output.weight', Dims=2, Type=Q4_K, Offset=1274511360
  Dimensions: 4096x4096
Tensor[86]: Name='blk.9.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=1283948544
  Dimensions: 4096x14336
Tensor[87]: Name='blk.9.ffn_up.weight', Dims=2, Type=Q4_K, Offset=1316978688
  Dimensions: 4096x14336
Tensor[88]: Name='blk.9.ffn_down.weight', Dims=2, Type=Q6_K, Offset=1350008832
  Dimensions: 14336x4096
Tensor[89]: Name='blk.9.attn_norm.weight', Dims=1, Type=F32, Offset=1398177792
  Dimensions: 4096
Tensor[90]: Name='blk.9.ffn_norm.weight', Dims=1, Type=F32, Offset=1398194176
  Dimensions: 4096
Tensor[91]: Name='blk.10.attn_q.weight', Dims=2, Type=Q4_K, Offset=1398210560
  Dimensions: 4096x4096
Tensor[92]: Name='blk.10.attn_k.weight', Dims=2, Type=Q4_K, Offset=1407647744
  Dimensions: 4096x1024
Tensor[93]: Name='blk.10.attn_v.weight', Dims=2, Type=Q4_K, Offset=1410007040
  Dimensions: 4096x1024
Tensor[94]: Name='blk.10.attn_output.weight', Dims=2, Type=Q4_K, Offset=1412366336
  Dimensions: 4096x4096
Tensor[95]: Name='blk.10.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=1421803520
  Dimensions: 4096x14336
Tensor[96]: Name='blk.10.ffn_up.weight', Dims=2, Type=Q4_K, Offset=1454833664
  Dimensions: 4096x14336
Tensor[97]: Name='blk.10.ffn_down.weight', Dims=2, Type=Q4_K, Offset=1487863808
  Dimensions: 14336x4096
Tensor[98]: Name='blk.10.attn_norm.weight', Dims=1, Type=F32, Offset=1520893952
  Dimensions: 4096
Tensor[99]: Name='blk.10.ffn_norm.weight', Dims=1, Type=F32, Offset=1520910336
  Dimensions: 4096
Tensor[100]: Name='blk.11.attn_q.weight', Dims=2, Type=Q4_K, Offset=1520926720
  Dimensions: 4096x4096
Tensor[101]: Name='blk.11.attn_k.weight', Dims=2, Type=Q4_K, Offset=1530363904
  Dimensions: 4096x1024
Tensor[102]: Name='blk.11.attn_v.weight', Dims=2, Type=Q4_K, Offset=1532723200
  Dimensions: 4096x1024
Tensor[103]: Name='blk.11.attn_output.weight', Dims=2, Type=Q4_K, Offset=1535082496
  Dimensions: 4096x4096
Tensor[104]: Name='blk.11.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=1544519680
  Dimensions: 4096x14336
Tensor[105]: Name='blk.11.ffn_up.weight', Dims=2, Type=Q4_K, Offset=1577549824
  Dimensions: 4096x14336
Tensor[106]: Name='blk.11.ffn_down.weight', Dims=2, Type=Q4_K, Offset=1610579968
  Dimensions: 14336x4096
Tensor[107]: Name='blk.11.attn_norm.weight', Dims=1, Type=F32, Offset=1643610112
  Dimensions: 4096
Tensor[108]: Name='blk.11.ffn_norm.weight', Dims=1, Type=F32, Offset=1643626496
  Dimensions: 4096
Tensor[109]: Name='blk.12.attn_q.weight', Dims=2, Type=Q4_K, Offset=1643642880
  Dimensions: 4096x4096
Tensor[110]: Name='blk.12.attn_k.weight', Dims=2, Type=Q4_K, Offset=1653080064
  Dimensions: 4096x1024
Tensor[111]: Name='blk.12.attn_v.weight', Dims=2, Type=Q6_K, Offset=1655439360
  Dimensions: 4096x1024
Tensor[112]: Name='blk.12.attn_output.weight', Dims=2, Type=Q4_K, Offset=1658880000
  Dimensions: 4096x4096
Tensor[113]: Name='blk.12.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=1668317184
  Dimensions: 4096x14336
Tensor[114]: Name='blk.12.ffn_up.weight', Dims=2, Type=Q4_K, Offset=1701347328
  Dimensions: 4096x14336
Tensor[115]: Name='blk.12.ffn_down.weight', Dims=2, Type=Q6_K, Offset=1734377472
  Dimensions: 14336x4096
Tensor[116]: Name='blk.12.attn_norm.weight', Dims=1, Type=F32, Offset=1782546432
  Dimensions: 4096
Tensor[117]: Name='blk.12.ffn_norm.weight', Dims=1, Type=F32, Offset=1782562816
  Dimensions: 4096
Tensor[118]: Name='blk.13.attn_q.weight', Dims=2, Type=Q4_K, Offset=1782579200
  Dimensions: 4096x4096
Tensor[119]: Name='blk.13.attn_k.weight', Dims=2, Type=Q4_K, Offset=1792016384
  Dimensions: 4096x1024
Tensor[120]: Name='blk.13.attn_v.weight', Dims=2, Type=Q4_K, Offset=1794375680
  Dimensions: 4096x1024
Tensor[121]: Name='blk.13.attn_output.weight', Dims=2, Type=Q4_K, Offset=1796734976
  Dimensions: 4096x4096
Tensor[122]: Name='blk.13.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=1806172160
  Dimensions: 4096x14336
Tensor[123]: Name='blk.13.ffn_up.weight', Dims=2, Type=Q4_K, Offset=1839202304
  Dimensions: 4096x14336
Tensor[124]: Name='blk.13.ffn_down.weight', Dims=2, Type=Q4_K, Offset=1872232448
  Dimensions: 14336x4096
Tensor[125]: Name='blk.13.attn_norm.weight', Dims=1, Type=F32, Offset=1905262592
  Dimensions: 4096
Tensor[126]: Name='blk.13.ffn_norm.weight', Dims=1, Type=F32, Offset=1905278976
  Dimensions: 4096
Tensor[127]: Name='blk.14.attn_q.weight', Dims=2, Type=Q4_K, Offset=1905295360
  Dimensions: 4096x4096
Tensor[128]: Name='blk.14.attn_k.weight', Dims=2, Type=Q4_K, Offset=1914732544
  Dimensions: 4096x1024
Tensor[129]: Name='blk.14.attn_v.weight', Dims=2, Type=Q4_K, Offset=1917091840
  Dimensions: 4096x1024
Tensor[130]: Name='blk.14.attn_output.weight', Dims=2, Type=Q4_K, Offset=1919451136
  Dimensions: 4096x4096
Tensor[131]: Name='blk.14.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=1928888320
  Dimensions: 4096x14336
Tensor[132]: Name='blk.14.ffn_up.weight', Dims=2, Type=Q4_K, Offset=1961918464
  Dimensions: 4096x14336
Tensor[133]: Name='blk.14.ffn_down.weight', Dims=2, Type=Q4_K, Offset=1994948608
  Dimensions: 14336x4096
Tensor[134]: Name='blk.14.attn_norm.weight', Dims=1, Type=F32, Offset=2027978752
  Dimensions: 4096
Tensor[135]: Name='blk.14.ffn_norm.weight', Dims=1, Type=F32, Offset=2027995136
  Dimensions: 4096
Tensor[136]: Name='blk.15.attn_q.weight', Dims=2, Type=Q4_K, Offset=2028011520
  Dimensions: 4096x4096
Tensor[137]: Name='blk.15.attn_k.weight', Dims=2, Type=Q4_K, Offset=2037448704
  Dimensions: 4096x1024
Tensor[138]: Name='blk.15.attn_v.weight', Dims=2, Type=Q6_K, Offset=2039808000
  Dimensions: 4096x1024
Tensor[139]: Name='blk.15.attn_output.weight', Dims=2, Type=Q4_K, Offset=2043248640
  Dimensions: 4096x4096
Tensor[140]: Name='blk.15.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=2052685824
  Dimensions: 4096x14336
Tensor[141]: Name='blk.15.ffn_up.weight', Dims=2, Type=Q4_K, Offset=2085715968
  Dimensions: 4096x14336
Tensor[142]: Name='blk.15.ffn_down.weight', Dims=2, Type=Q6_K, Offset=2118746112
  Dimensions: 14336x4096
Tensor[143]: Name='blk.15.attn_norm.weight', Dims=1, Type=F32, Offset=2166915072
  Dimensions: 4096
Tensor[144]: Name='blk.15.ffn_norm.weight', Dims=1, Type=F32, Offset=2166931456
  Dimensions: 4096
Tensor[145]: Name='blk.16.attn_q.weight', Dims=2, Type=Q4_K, Offset=2166947840
  Dimensions: 4096x4096
Tensor[146]: Name='blk.16.attn_k.weight', Dims=2, Type=Q4_K, Offset=2176385024
  Dimensions: 4096x1024
Tensor[147]: Name='blk.16.attn_v.weight', Dims=2, Type=Q4_K, Offset=2178744320
  Dimensions: 4096x1024
Tensor[148]: Name='blk.16.attn_output.weight', Dims=2, Type=Q4_K, Offset=2181103616
  Dimensions: 4096x4096
Tensor[149]: Name='blk.16.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=2190540800
  Dimensions: 4096x14336
Tensor[150]: Name='blk.16.ffn_up.weight', Dims=2, Type=Q4_K, Offset=2223570944
  Dimensions: 4096x14336
Tensor[151]: Name='blk.16.ffn_down.weight', Dims=2, Type=Q4_K, Offset=2256601088
  Dimensions: 14336x4096
Tensor[152]: Name='blk.16.attn_norm.weight', Dims=1, Type=F32, Offset=2289631232
  Dimensions: 4096
Tensor[153]: Name='blk.16.ffn_norm.weight', Dims=1, Type=F32, Offset=2289647616
  Dimensions: 4096
Tensor[154]: Name='blk.17.attn_q.weight', Dims=2, Type=Q4_K, Offset=2289664000
  Dimensions: 4096x4096
Tensor[155]: Name='blk.17.attn_k.weight', Dims=2, Type=Q4_K, Offset=2299101184
  Dimensions: 4096x1024
Tensor[156]: Name='blk.17.attn_v.weight', Dims=2, Type=Q4_K, Offset=2301460480
  Dimensions: 4096x1024
Tensor[157]: Name='blk.17.attn_output.weight', Dims=2, Type=Q4_K, Offset=2303819776
  Dimensions: 4096x4096
Tensor[158]: Name='blk.17.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=2313256960
  Dimensions: 4096x14336
Tensor[159]: Name='blk.17.ffn_up.weight', Dims=2, Type=Q4_K, Offset=2346287104
  Dimensions: 4096x14336
Tensor[160]: Name='blk.17.ffn_down.weight', Dims=2, Type=Q4_K, Offset=2379317248
  Dimensions: 14336x4096
Tensor[161]: Name='blk.17.attn_norm.weight', Dims=1, Type=F32, Offset=2412347392
  Dimensions: 4096
Tensor[162]: Name='blk.17.ffn_norm.weight', Dims=1, Type=F32, Offset=2412363776
  Dimensions: 4096
Tensor[163]: Name='blk.18.attn_q.weight', Dims=2, Type=Q4_K, Offset=2412380160
  Dimensions: 4096x4096
Tensor[164]: Name='blk.18.attn_k.weight', Dims=2, Type=Q4_K, Offset=2421817344
  Dimensions: 4096x1024
Tensor[165]: Name='blk.18.attn_v.weight', Dims=2, Type=Q6_K, Offset=2424176640
  Dimensions: 4096x1024
Tensor[166]: Name='blk.18.attn_output.weight', Dims=2, Type=Q4_K, Offset=2427617280
  Dimensions: 4096x4096
Tensor[167]: Name='blk.18.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=2437054464
  Dimensions: 4096x14336
Tensor[168]: Name='blk.18.ffn_up.weight', Dims=2, Type=Q4_K, Offset=2470084608
  Dimensions: 4096x14336
Tensor[169]: Name='blk.18.ffn_down.weight', Dims=2, Type=Q6_K, Offset=2503114752
  Dimensions: 14336x4096
Tensor[170]: Name='blk.18.attn_norm.weight', Dims=1, Type=F32, Offset=2551283712
  Dimensions: 4096
Tensor[171]: Name='blk.18.ffn_norm.weight', Dims=1, Type=F32, Offset=2551300096
  Dimensions: 4096
Tensor[172]: Name='blk.19.attn_q.weight', Dims=2, Type=Q4_K, Offset=2551316480
  Dimensions: 4096x4096
Tensor[173]: Name='blk.19.attn_k.weight', Dims=2, Type=Q4_K, Offset=2560753664
  Dimensions: 4096x1024
Tensor[174]: Name='blk.19.attn_v.weight', Dims=2, Type=Q4_K, Offset=2563112960
  Dimensions: 4096x1024
Tensor[175]: Name='blk.19.attn_output.weight', Dims=2, Type=Q4_K, Offset=2565472256
  Dimensions: 4096x4096
Tensor[176]: Name='blk.19.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=2574909440
  Dimensions: 4096x14336
Tensor[177]: Name='blk.19.ffn_up.weight', Dims=2, Type=Q4_K, Offset=2607939584
  Dimensions: 4096x14336
Tensor[178]: Name='blk.19.ffn_down.weight', Dims=2, Type=Q4_K, Offset=2640969728
  Dimensions: 14336x4096
Tensor[179]: Name='blk.19.attn_norm.weight', Dims=1, Type=F32, Offset=2673999872
  Dimensions: 4096
Tensor[180]: Name='blk.19.ffn_norm.weight', Dims=1, Type=F32, Offset=2674016256
  Dimensions: 4096
Tensor[181]: Name='blk.20.attn_q.weight', Dims=2, Type=Q4_K, Offset=2674032640
  Dimensions: 4096x4096
Tensor[182]: Name='blk.20.attn_k.weight', Dims=2, Type=Q4_K, Offset=2683469824
  Dimensions: 4096x1024
Tensor[183]: Name='blk.20.attn_v.weight', Dims=2, Type=Q4_K, Offset=2685829120
  Dimensions: 4096x1024
Tensor[184]: Name='blk.20.attn_output.weight', Dims=2, Type=Q4_K, Offset=2688188416
  Dimensions: 4096x4096
Tensor[185]: Name='blk.20.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=2697625600
  Dimensions: 4096x14336
Tensor[186]: Name='blk.20.ffn_up.weight', Dims=2, Type=Q4_K, Offset=2730655744
  Dimensions: 4096x14336
Tensor[187]: Name='blk.20.ffn_down.weight', Dims=2, Type=Q4_K, Offset=2763685888
  Dimensions: 14336x4096
Tensor[188]: Name='blk.20.attn_norm.weight', Dims=1, Type=F32, Offset=2796716032
  Dimensions: 4096
Tensor[189]: Name='blk.20.ffn_norm.weight', Dims=1, Type=F32, Offset=2796732416
  Dimensions: 4096
Tensor[190]: Name='blk.21.attn_q.weight', Dims=2, Type=Q4_K, Offset=2796748800
  Dimensions: 4096x4096
Tensor[191]: Name='blk.21.attn_k.weight', Dims=2, Type=Q4_K, Offset=2806185984
  Dimensions: 4096x1024
Tensor[192]: Name='blk.21.attn_v.weight', Dims=2, Type=Q6_K, Offset=2808545280
  Dimensions: 4096x1024
Tensor[193]: Name='blk.21.attn_output.weight', Dims=2, Type=Q4_K, Offset=2811985920
  Dimensions: 4096x4096
Tensor[194]: Name='blk.21.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=2821423104
  Dimensions: 4096x14336
Tensor[195]: Name='blk.21.ffn_up.weight', Dims=2, Type=Q4_K, Offset=2854453248
  Dimensions: 4096x14336
Tensor[196]: Name='blk.21.ffn_down.weight', Dims=2, Type=Q6_K, Offset=2887483392
  Dimensions: 14336x4096
Tensor[197]: Name='blk.21.attn_norm.weight', Dims=1, Type=F32, Offset=2935652352
  Dimensions: 4096
Tensor[198]: Name='blk.21.ffn_norm.weight', Dims=1, Type=F32, Offset=2935668736
  Dimensions: 4096
Tensor[199]: Name='blk.22.attn_q.weight', Dims=2, Type=Q4_K, Offset=2935685120
  Dimensions: 4096x4096
Tensor[200]: Name='blk.22.attn_k.weight', Dims=2, Type=Q4_K, Offset=2945122304
  Dimensions: 4096x1024
Tensor[201]: Name='blk.22.attn_v.weight', Dims=2, Type=Q4_K, Offset=2947481600
  Dimensions: 4096x1024
Tensor[202]: Name='blk.22.attn_output.weight', Dims=2, Type=Q4_K, Offset=2949840896
  Dimensions: 4096x4096
Tensor[203]: Name='blk.22.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=2959278080
  Dimensions: 4096x14336
Tensor[204]: Name='blk.22.ffn_up.weight', Dims=2, Type=Q4_K, Offset=2992308224
  Dimensions: 4096x14336
Tensor[205]: Name='blk.22.ffn_down.weight', Dims=2, Type=Q4_K, Offset=3025338368
  Dimensions: 14336x4096
Tensor[206]: Name='blk.22.attn_norm.weight', Dims=1, Type=F32, Offset=3058368512
  Dimensions: 4096
Tensor[207]: Name='blk.22.ffn_norm.weight', Dims=1, Type=F32, Offset=3058384896
  Dimensions: 4096
Tensor[208]: Name='blk.23.attn_q.weight', Dims=2, Type=Q4_K, Offset=3058401280
  Dimensions: 4096x4096
Tensor[209]: Name='blk.23.attn_k.weight', Dims=2, Type=Q4_K, Offset=3067838464
  Dimensions: 4096x1024
Tensor[210]: Name='blk.23.attn_v.weight', Dims=2, Type=Q4_K, Offset=3070197760
  Dimensions: 4096x1024
Tensor[211]: Name='blk.23.attn_output.weight', Dims=2, Type=Q4_K, Offset=3072557056
  Dimensions: 4096x4096
Tensor[212]: Name='blk.23.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=3081994240
  Dimensions: 4096x14336
Tensor[213]: Name='blk.23.ffn_up.weight', Dims=2, Type=Q4_K, Offset=3115024384
  Dimensions: 4096x14336
Tensor[214]: Name='blk.23.ffn_down.weight', Dims=2, Type=Q4_K, Offset=3148054528
  Dimensions: 14336x4096
Tensor[215]: Name='blk.23.attn_norm.weight', Dims=1, Type=F32, Offset=3181084672
  Dimensions: 4096
Tensor[216]: Name='blk.23.ffn_norm.weight', Dims=1, Type=F32, Offset=3181101056
  Dimensions: 4096
Tensor[217]: Name='blk.24.attn_q.weight', Dims=2, Type=Q4_K, Offset=3181117440
  Dimensions: 4096x4096
Tensor[218]: Name='blk.24.attn_k.weight', Dims=2, Type=Q4_K, Offset=3190554624
  Dimensions: 4096x1024
Tensor[219]: Name='blk.24.attn_v.weight', Dims=2, Type=Q6_K, Offset=3192913920
  Dimensions: 4096x1024
Tensor[220]: Name='blk.24.attn_output.weight', Dims=2, Type=Q4_K, Offset=3196354560
  Dimensions: 4096x4096
Tensor[221]: Name='blk.24.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=3205791744
  Dimensions: 4096x14336
Tensor[222]: Name='blk.24.ffn_up.weight', Dims=2, Type=Q4_K, Offset=3238821888
  Dimensions: 4096x14336
Tensor[223]: Name='blk.24.ffn_down.weight', Dims=2, Type=Q6_K, Offset=3271852032
  Dimensions: 14336x4096
Tensor[224]: Name='blk.24.attn_norm.weight', Dims=1, Type=F32, Offset=3320020992
  Dimensions: 4096
Tensor[225]: Name='blk.24.ffn_norm.weight', Dims=1, Type=F32, Offset=3320037376
  Dimensions: 4096
Tensor[226]: Name='blk.25.attn_q.weight', Dims=2, Type=Q4_K, Offset=3320053760
  Dimensions: 4096x4096
Tensor[227]: Name='blk.25.attn_k.weight', Dims=2, Type=Q4_K, Offset=3329490944
  Dimensions: 4096x1024
Tensor[228]: Name='blk.25.attn_v.weight', Dims=2, Type=Q4_K, Offset=3331850240
  Dimensions: 4096x1024
Tensor[229]: Name='blk.25.attn_output.weight', Dims=2, Type=Q4_K, Offset=3334209536
  Dimensions: 4096x4096
Tensor[230]: Name='blk.25.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=3343646720
  Dimensions: 4096x14336
Tensor[231]: Name='blk.25.ffn_up.weight', Dims=2, Type=Q4_K, Offset=3376676864
  Dimensions: 4096x14336
Tensor[232]: Name='blk.25.ffn_down.weight', Dims=2, Type=Q4_K, Offset=3409707008
  Dimensions: 14336x4096
Tensor[233]: Name='blk.25.attn_norm.weight', Dims=1, Type=F32, Offset=3442737152
  Dimensions: 4096
Tensor[234]: Name='blk.25.ffn_norm.weight', Dims=1, Type=F32, Offset=3442753536
  Dimensions: 4096
Tensor[235]: Name='blk.26.attn_q.weight', Dims=2, Type=Q4_K, Offset=3442769920
  Dimensions: 4096x4096
Tensor[236]: Name='blk.26.attn_k.weight', Dims=2, Type=Q4_K, Offset=3452207104
  Dimensions: 4096x1024
Tensor[237]: Name='blk.26.attn_v.weight', Dims=2, Type=Q4_K, Offset=3454566400
  Dimensions: 4096x1024
Tensor[238]: Name='blk.26.attn_output.weight', Dims=2, Type=Q4_K, Offset=3456925696
  Dimensions: 4096x4096
Tensor[239]: Name='blk.26.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=3466362880
  Dimensions: 4096x14336
Tensor[240]: Name='blk.26.ffn_up.weight', Dims=2, Type=Q4_K, Offset=3499393024
  Dimensions: 4096x14336
Tensor[241]: Name='blk.26.ffn_down.weight', Dims=2, Type=Q4_K, Offset=3532423168
  Dimensions: 14336x4096
Tensor[242]: Name='blk.26.attn_norm.weight', Dims=1, Type=F32, Offset=3565453312
  Dimensions: 4096
Tensor[243]: Name='blk.26.ffn_norm.weight', Dims=1, Type=F32, Offset=3565469696
  Dimensions: 4096
Tensor[244]: Name='blk.27.attn_q.weight', Dims=2, Type=Q4_K, Offset=3565486080
  Dimensions: 4096x4096
Tensor[245]: Name='blk.27.attn_k.weight', Dims=2, Type=Q4_K, Offset=3574923264
  Dimensions: 4096x1024
Tensor[246]: Name='blk.27.attn_v.weight', Dims=2, Type=Q6_K, Offset=3577282560
  Dimensions: 4096x1024
Tensor[247]: Name='blk.27.attn_output.weight', Dims=2, Type=Q4_K, Offset=3580723200
  Dimensions: 4096x4096
Tensor[248]: Name='blk.27.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=3590160384
  Dimensions: 4096x14336
Tensor[249]: Name='blk.27.ffn_up.weight', Dims=2, Type=Q4_K, Offset=3623190528
  Dimensions: 4096x14336
Tensor[250]: Name='blk.27.ffn_down.weight', Dims=2, Type=Q6_K, Offset=3656220672
  Dimensions: 14336x4096
Tensor[251]: Name='blk.27.attn_norm.weight', Dims=1, Type=F32, Offset=3704389632
  Dimensions: 4096
Tensor[252]: Name='blk.27.ffn_norm.weight', Dims=1, Type=F32, Offset=3704406016
  Dimensions: 4096
Tensor[253]: Name='blk.28.attn_q.weight', Dims=2, Type=Q4_K, Offset=3704422400
  Dimensions: 4096x4096
Tensor[254]: Name='blk.28.attn_k.weight', Dims=2, Type=Q4_K, Offset=3713859584
  Dimensions: 4096x1024
Tensor[255]: Name='blk.28.attn_v.weight', Dims=2, Type=Q6_K, Offset=3716218880
  Dimensions: 4096x1024
Tensor[256]: Name='blk.28.attn_output.weight', Dims=2, Type=Q4_K, Offset=3719659520
  Dimensions: 4096x4096
Tensor[257]: Name='blk.28.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=3729096704
  Dimensions: 4096x14336
Tensor[258]: Name='blk.28.ffn_up.weight', Dims=2, Type=Q4_K, Offset=3762126848
  Dimensions: 4096x14336
Tensor[259]: Name='blk.28.ffn_down.weight', Dims=2, Type=Q6_K, Offset=3795156992
  Dimensions: 14336x4096
Tensor[260]: Name='blk.28.attn_norm.weight', Dims=1, Type=F32, Offset=3843325952
  Dimensions: 4096
Tensor[261]: Name='blk.28.ffn_norm.weight', Dims=1, Type=F32, Offset=3843342336
  Dimensions: 4096
Tensor[262]: Name='blk.29.attn_q.weight', Dims=2, Type=Q4_K, Offset=3843358720
  Dimensions: 4096x4096
Tensor[263]: Name='blk.29.attn_k.weight', Dims=2, Type=Q4_K, Offset=3852795904
  Dimensions: 4096x1024
Tensor[264]: Name='blk.29.attn_v.weight', Dims=2, Type=Q6_K, Offset=3855155200
  Dimensions: 4096x1024
Tensor[265]: Name='blk.29.attn_output.weight', Dims=2, Type=Q4_K, Offset=3858595840
  Dimensions: 4096x4096
Tensor[266]: Name='blk.29.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=3868033024
  Dimensions: 4096x14336
Tensor[267]: Name='blk.29.ffn_up.weight', Dims=2, Type=Q4_K, Offset=3901063168
  Dimensions: 4096x14336
Tensor[268]: Name='blk.29.ffn_down.weight', Dims=2, Type=Q6_K, Offset=3934093312
  Dimensions: 14336x4096
Tensor[269]: Name='blk.29.attn_norm.weight', Dims=1, Type=F32, Offset=3982262272
  Dimensions: 4096
Tensor[270]: Name='blk.29.ffn_norm.weight', Dims=1, Type=F32, Offset=3982278656
  Dimensions: 4096
Tensor[271]: Name='blk.30.attn_q.weight', Dims=2, Type=Q4_K, Offset=3982295040
  Dimensions: 4096x4096
Tensor[272]: Name='blk.30.attn_k.weight', Dims=2, Type=Q4_K, Offset=3991732224
  Dimensions: 4096x1024
Tensor[273]: Name='blk.30.attn_v.weight', Dims=2, Type=Q6_K, Offset=3994091520
  Dimensions: 4096x1024
Tensor[274]: Name='blk.30.attn_output.weight', Dims=2, Type=Q4_K, Offset=3997532160
  Dimensions: 4096x4096
Tensor[275]: Name='blk.30.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=4006969344
  Dimensions: 4096x14336
Tensor[276]: Name='blk.30.ffn_up.weight', Dims=2, Type=Q4_K, Offset=4039999488
  Dimensions: 4096x14336
Tensor[277]: Name='blk.30.ffn_down.weight', Dims=2, Type=Q6_K, Offset=4073029632
  Dimensions: 14336x4096
Tensor[278]: Name='blk.30.attn_norm.weight', Dims=1, Type=F32, Offset=4121198592
  Dimensions: 4096
Tensor[279]: Name='blk.30.ffn_norm.weight', Dims=1, Type=F32, Offset=4121214976
  Dimensions: 4096
Tensor[280]: Name='blk.31.attn_q.weight', Dims=2, Type=Q4_K, Offset=4121231360
  Dimensions: 4096x4096
Tensor[281]: Name='blk.31.attn_k.weight', Dims=2, Type=Q4_K, Offset=4130668544
  Dimensions: 4096x1024
Tensor[282]: Name='blk.31.attn_v.weight', Dims=2, Type=Q6_K, Offset=4133027840
  Dimensions: 4096x1024
Tensor[283]: Name='blk.31.attn_output.weight', Dims=2, Type=Q4_K, Offset=4136468480
  Dimensions: 4096x4096
Tensor[284]: Name='blk.31.ffn_gate.weight', Dims=2, Type=Q4_K, Offset=4145905664
  Dimensions: 4096x14336
Tensor[285]: Name='blk.31.ffn_up.weight', Dims=2, Type=Q4_K, Offset=4178935808
  Dimensions: 4096x14336
Tensor[286]: Name='blk.31.ffn_down.weight', Dims=2, Type=Q6_K, Offset=4211965952
  Dimensions: 14336x4096
Tensor[287]: Name='blk.31.attn_norm.weight', Dims=1, Type=F32, Offset=4260134912
  Dimensions: 4096
Tensor[288]: Name='blk.31.ffn_norm.weight', Dims=1, Type=F32, Offset=4260151296
  Dimensions: 4096
Tensor[289]: Name='output_norm.weight', Dims=1, Type=F32, Offset=4260167680
  Dimensions: 4096
Tensor[290]: Name='output.weight', Dims=2, Type=Q6_K, Offset=4260184064
  Dimensions: 4096x32000



==== Loading Tensor Data ====
Tensor data loaded: 4367704064 bytes
Tensor data starts at file offset: 735520



==== GGUF File Loaded Successfully ====
File: mistral-7b-instruct-v0.2.Q4_K_M.gguf
Total file size: 4368439584 bytes
Tensors: 291
Metadata entries: 24



==== Building Transformer Model Structure ====
Architecture: llama
Model Name: mistralai_mistral-7b-instruct-v0.2
Hidden Size: 4096
Intermediate Size: 14336
Layers: 32
Attention Heads: 32
KV Heads: 8
Head Dimension: 128
Max Sequence Length: 32768
Vocabulary Size: 32000
RoPE Theta: 10000.0
RMS Norm Epsilon: 1.000000e-05
Layer 0: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 1: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 2: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 3: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 4: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 5: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 6: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 7: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 8: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 9: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 10: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 11: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 12: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 13: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 14: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 15: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 16: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 17: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 18: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 19: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 20: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 21: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 22: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 23: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 24: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 25: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 26: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 27: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 28: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 29: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 30: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]
Layer 31: Attn[Q:✓ K:✓ V:✓ O:✓] FFN[W1:✓ W2:✓ W3:✓]



Model Statistics:
Total Parameters: 7241732096 (7.24B)
Estimated Model Size: 27625.02 MB
Actual File Size: 4166.07 MB



==== Model Architecture Details ====
Architecture: llama
Model Name: mistralai_mistral-7b-instruct-v0.2



Model Dimensions:
  Hidden Size: 4096
  Intermediate Size: 14336
  Vocabulary Size: 32000
  Max Sequence Length: 32768



Attention Configuration:
  Number of Heads: 32
  Number of KV Heads: 8
  Head Dimension: 128
  Total Attention Dim: 4096
  Using Grouped Query Attention (GQA)
  Query Groups: 4



Layer Configuration:
  Number of Layers: 32
  RoPE Theta: 10000.0
  RMS Norm Epsilon: 1.000000e-05



Embedding Layers:
  Token Embeddings: ✓
  Position Embeddings: ✗



Output Layers:
  Output Norm: ✓
  Output Projection: ✓



Layer-by-Layer Breakdown:
  Layer  0: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer  1: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer  2: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer  3: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer  4: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer  5: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer  6: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer  7: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer  8: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer  9: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 10: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 11: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 12: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 13: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 14: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 15: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 16: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 17: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 18: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 19: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 20: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 21: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 22: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 23: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 24: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 25: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 26: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 27: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 28: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 29: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 30: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]
  Layer 31: Attn[Norm:✓ Q:✓ K:✓ V:✓ O:✓] FFN[Norm:✓ W1:✓ W2:✓ W3:✓]



Model Statistics:
  Total Parameters: 7241732096 (7.24B)
  Estimated Size: 27625.02 MB
  File Size: 4166.07 MB
  Compression Ratio: 6.63x



First layer query weights data loaded successfully at address: 0x797651d79010
Tensor: blk.0.attn_q.weight
Type: Q4_K
Dimensions: 4096x4096
Offset: 73728000 bytes


==== Model successfully loaded and structured ====
The transformer model is now ready for inference!

Roadmap

Why lm.c?

Universal Compatibility: Runs any GGUF model.
Minimal Footprint - Single C file < 100KB.
Educational - Clean, readable and documented codebase.
Portable - Runs on x86, ARM, RISC-V.
Efficient - Beats frameworks in CPU perf.
Future-proof - Full GGUF spec coverage.

Contributing

Contributions are welcome! See CONTRIBUTING.md for guide lines.

License

MIT License - see LICENSE.

Acknowledgments

llama.cpp
GGUF
Huggingface
Open-source LLM community.

lm.c - Bringing large language models to every CPU, everywhere.

Name		Name	Last commit message	Last commit date
Latest commit History 25 Commits
img		img
.gitattributes		.gitattributes
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
gguf-file-format.md		gguf-file-format.md
lm.c		lm.c
lm.c_architecture.md		lm.c_architecture.md

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

lm.c

Features

What has been DONE so far

Supported Models

Supported Quantization Types

Architecture Overview

System Workflow

Core Components

GGUF File Structure

Transformer Architecture

Memory Efficient Design

Inference Workflow

Performance Optimizations

Getting Started

Prerequisites

Building & Running

Usage

Output

Roadmap

Why lm.c?

Contributing

License

Acknowledgments

About

Uh oh!

Releases

Packages

Languages

License

oderoi/lm.c

Folders and files

Latest commit

History

Repository files navigation

lm.c

Features

What has been DONE so far

Supported Models

Supported Quantization Types

Architecture Overview

System Workflow

Core Components

GGUF File Structure

Transformer Architecture

Memory Efficient Design

Inference Workflow

Performance Optimizations

Getting Started

Prerequisites

Building & Running

Usage

Output

Roadmap

Why lm.c?

Contributing

License

Acknowledgments

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages