Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Simple repo that compiles and runs llama2.c on the Web

License

Notifications You must be signed in to change notification settings

andreaTP/llama2.c-web

 
 

Repository files navigation

Minimal llama2.c to WebAssembly port

Demo

  • No build system. Just a bash script.
  • No Emscripten.
  • Stripped down to the bare minimum. Every line of code has a purpose.

This repo is Mac only for now.

This builds on a minimal C++/C + libc to WASM template that I recommend to look at if you want to learn how the bare WASM stack works without the additional the complexity. Additional considerations to build llama2.c:

  • mmap is not part of the WASI standard but there's an emulation available on the wasi-sdk by passing the D_WASI_EMULATED_MMAN flag to the compiler and linking against wasi-emulated-mman library. For this we also need to link against clang_rt.builtins-wasm32 that the wasi-sdk distributes separatetly.

Setup

The command below downloads a wasi-sdk release bundle that contains the WASI headers, libraries, compiler (clang) and linker (wasm-ld)

./setup.sh

Usage

Compiles and links C++ code to WASM

./build.sh

Starts a local Web server so you can run the code. Open in your browser http://localhost:8080

./run.sh

Prior work

https://github.com/michaelfranzl/clang-wasm-browser-starterpack/tree/dev/examples/11

https://medium.com/@michaelyuan_88928/running-llama2-c-in-wasmedge-15291795c470

https://stackoverflow.com/a/29694977/717508

https://github.com/taybenlor/runno

Notes

/vendor/wasi.js is built from the runno wasi js runtime by running mpm run build. There are two small modifications on 9b9dc1f3142c that I might submit upstream:

  1. The ability to pass a WebAssembly.Memory object to the runtime.
  2. The ability to pass an object with JS-defined functions that can be invoked from native code.

I included a wasi.js.original file as a references for the differences (git diff wasi.js wasi.js.original)

https://github.com/taybenlor/runno/commit/9b9dc1f3142c

About

Simple repo that compiles and runs llama2.c on the Web

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C 95.4%
  • WebAssembly 3.3%
  • Python 1.1%
  • Jupyter Notebook 0.1%
  • JavaScript 0.1%
  • Shell 0.0%