This repository is part of the code release for the Siggraph Asia 2023 paper "From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans".

[paper] [project page]

• 2025/08/15: The installation was simplified and the code and dependancies were updated to work on python3.12.

This repo contains the pytorch SKEL loader and the code to align it to SMPL sequences.

SKEL is a parametric body shape and skeleton model. Its shape parameter θ lets you change the body shape and its pose parameter q lets you pose the skeleton and skin in an anatomically plausible way. Given shape and pose parameters, SKEL returns joint locations, a body mesh, and a skeleton mesh. SKEL is differentiable and can be fit to various data like motion capture or SMPL sequences.

For more details, please check our Siggraph 2023 paper: From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans.

• Quickstart
• Installation
  • Set up the environment
  • Downloading SKEL
  • Aitviewer
  • SMPL and Mesh package
• Demos
  • Pose parameters
  • Shape space
  • Rigging
  • SKEL sequence
  • Aligning SKEL to SMPL mesh
  • Aligning SKEL to SMPL sequences
• Acknoledgments
• Citation
• License
• Contact

For a minimal use, you will have to follow the install instructions in Set up the environment and Downloading SKEL.

To test the SKEL model, you can then run:

python quickstart.py 

This runs the forward pass of SKEL and saves the output as separated body and skeleton meshes.

You can then try to change the SKEL parameters knowing the following. SKEL is controled by two parameters:

• betas controls the shape of the body (tall, small, skinny, etc...). It is a vector of size 10. The values should remain between -2 and 2. The first value of the vector roughly controls the height, the second value the weight.

• pose controls the body pose. It is a vector of size 46, most values are angles expressed in radian, and can vary between -3.14 and 3.14. You can see which of the 46 parameters controls which body articulation here .

SKEL(beta, pose) yields:

• A skeleton 3D mesh
• A body mesh
• Anatomical joint locations

The rest of this readme will let you setup our SKEL vizualizer and run some demo code. We hope you enjoy our work!

Clone this repository

git clone https://github.com/MarilynKeller/SKEL
cd SKEL

Create a virtual environment and install the SKEL package

bash install.sh 

Source the virtual environment

source skel_venv/bin/activate

Create an account on https://skel.is.tue.mpg.de/ and https://smpl.is.tue.mpg.de/. The following script will ask for your login and password for each of these website to download the data.

Then run:

bash fetch_skel.sh

If this fails, you can also download the data manually from the SKEL download page and the SMPL download page and place them in the data folder of this repository. The expected structure is:

SKEL/ # Root of the repository
└── data/
    └── skel/
        ├── skel_female.pkl
        ├── skel_male.pkl
        └── ...

For running the demos other than the quickstart you will need the following.

If you want to run an alignment to a SMPL body or sequence, you need to download the SMPL model.

First, create an account on https://smpl.is.tue.mpg.de/. Then run the following script to download the SMPL model:

bash fetch_smpl.sh

SMPLH is a version of SMPL with the parametric hands from MANO. The AMASS dataset is in this format. To view the sample AMASS sequence in aitviewer, you need to download the SMPLH model.

First, create an account on https://mano.is.tue.mpg.de/. Then run the following script to download the SMPLH model:

bash fetch_smplh.sh

If it fails, you can also download the data manually from the download pages of https://smpl.is.tue.mpg.de/ and https://mano.is.tue.mpg.de/, and place them in the data folder of this repository. Note that SMPLH files require pre-processing, done by the fetch_smplh.sh script.

The final expected structure for these models is:

SKEL/ # Root of the repository
└── data/
    ├── skel/
    │   ├── skel_female.pkl
    │   ├── skel_male.pkl
    │   └── ...
    └── smpl/
        ├── SMPL_FEMALE.pkl
        ├── SMPL_MALE.pkl
        └── SMPL_NEUTRAL.pkl
    └── smplx/
        ├── SMPLH_FEMALE.pkl
        ├── SMPLH_MALE.pkl
        └── SMPLH_NEUTRAL.pkl

For visualizing the fitting process you need the MPI mesh package, you can install it with the following line:

! Note that the MPI mesh package is only supported on Linux systems.

pip install git+https://github.com/MarilynKeller/mesh

Vizualize the shape space:

python examples/skel_betas.py --gender female 

Visualize the effects of the pose parameters of SKEL:

python examples/skel_poses.py --gender male

Visualize the skinning weights of the skin and bones to the SKEL kinematic tree:

python examples/skel_rigging.py --gender female  

Visualize the kinematic tree and joint locations of SKEL:

python examples/skel_kintree.py --gender female

You can see a visual of the joint ids here and their names and the list of degrees of freedom here

Visualize a SKEL sequence. You can find a sample SKEL motion in data/skel/sample_motion/ and the corresponding SMPL motion.

python examples/skel_sequence.py data/skel/sample_motion/01_01_poses_skel.pkl -z 

To visualize the SMPL sequence alongside :

python examples/skel_sequence.py /path/to/skel_models_v1.0/sample_motion/01_01_poses_skel.pkl -z --smpl_seq /path/to/skel_models_v1.0/sample_motion/01_01_poses.npz

SKEL can be aligned to SMPL meshes:

python examples/align_to_SMPL_frame.py --smpl_data 'examples/samples/img_fit/emily-sea-coiWR0gT8Cw-unsplash_0.npz' 

SKEL can be aligned to SMPL sequences, here is an example:

python examples/align_to_SMPL_seq.py examples/samples/amass_seq/CMU_01_01.npz -D 

You can visualize the result with:

python examples/skel_sequence.py output/CMU_01_01/CMU_01_01_skel.pkl -z 

You can download more SMPL sequences of this kind from the AMASS Download page, and selecting the SMPL+H G sequences.

This research was done partly at The Movement Lab in Stanford and Perceiving Systems at the Max Planck Institute for Intelligent Systems.

We thank Neelay Shah for generating all the necessary SMPL fits, A. A. Osman for his expertise on parametric body models, Shashank Tripathi for his help with MOYO, Giorgio Becherini for his expertise on AMASS and mocap fit evaluation, Peter Kultis, Yao Feng and Yuliang Xu for feedbacks on the paper. We also thank the TML lab and the NMBL lab at Stanford for the fruitful discussions, especially Jennifer Maier for her expertise on the shoulder.

If you use this software, please cite the following work and software:

@inproceedings{keller2023skel,
  title = {From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans},
  author = {Keller, Marilyn and Werling, Keenon and Shin, Soyong and Delp, Scott and 
            Pujades, Sergi and Liu, C. Karen and Black, Michael J.},
  booktitle = {ACM ToG, Proc.~SIGGRAPH Asia},
  volume = {42},
  number = {6},
  month = dec,
  year = {2023},
}

This code and model are available for non-commercial scientific research purposes as defined in the LICENSE.txt file.

For any questions about SKEL loading, please contact [email protected].

For commercial licensing, please contact [email protected]