YAI Conference compatition: YAICON Oct 2025 - Dec 2025 Team member Deogyong Kim (13th), Inhee Cho (13th), Yoonju Jung (14th), Hyunbo Simm (15th), Jeongju Lee (16th)

Existing AI models often struggle to form sustainable relationships with humans due to issues like persona drift. To address this, our project aims to develop a human-like AI agent capable of maintaining a consistent persona and establishing lasting emotional bonds with users.

To achieve a more natural and robust persona compared to conventional AIs, we designed the persona using vector steering and prompting techniques. This approach prevents persona drift even during real-time interactions. Furthermore, to provide genuine emotional support, we integrated a highly natural TTS model that mimics human speech patterns.

This project utilizes the frontend from Open-LLM-VTuber.

The model used is available at BOOTH. The copyright of the model belongs to miHoYo.

The downloaded model should be added to the live2d-models folder under the name 'huohuo_v2'.

We discovered an issue with the connection between the LLM's emotion parsing and the model's expression mapping in this project, and we have implemented a fix for it.

We implemented an LLM agent's temperament-based persona using vector steering techniques and established interaction with the agent.

We utilized EasySteer to extract persona vectors and apply steering to the model served via vLLM. We would like to express our sincere gratitude to the original authors for open-sourcing this excellent project.

We classified innate human temperament into the following five axes:

1. Emotional Reactivity

2. Behavioral Inhibition / Boldness

3. Activity & Arousal

4. Attentional Control

5. Sensory Sensitivity

Specifically, we extracted persona vectors corresponding to emotional reactivity and sensory sensitivity. Furthermore, to enhance the relationship between the LLM agent and the user, we also extracted vectors representing romantic and friendship dynamics. By steering these four distinct persona vectors into the model, we constructed the LLM agent's unique temperamental persona.

The core scripts for extracting persona vectors using EasySteer are make_persona_vector.py and pca_make_persona_vector.py.

Mechanism: make_persona_vector.py extracts a persona vector by feeding two contrasting datasets (positive vs. negative) to the model. It captures the internal activations for each dataset and computes the vector by subtracting the negative activations from the positive ones.

Data Preparation: To extract a vector for a specific trait, you need a dataset of sentences aligning with that trait (positive) and a dataset of opposing sentences (negative).

How to Run: Do not run the core logic scripts directly. Instead, use the following execution scripts:

run_make_persona_vector.py

pca_run_make_persona_vector.py

You can extract the vectors by configuring parameters such as MODEL_NAME, POSITIVE_DATA_FILE, and NEGATIVE_DATA_FILE within these scripts before execution.

After generating the persona vectors, save the configuration details, such as the specific vectors to be used for steering and their corresponding steering intensities, into default_vectors.json

[
  {
    "path": "vectors/emotional_reactivity_exaone32_diffmean.gguf", 
    "scale": 0.2, 
    "target_layers": [31, 36, 41, 46],  
    "algorithm": "direct", 
    "normalize": true 
  },
  {
    "path": "vectors/sensory_sensitivity_exaone32_diffmean.gguf",
    "scale": 0.2,   
    "target_layers": [33, 38, 42, 47],
    "algorithm": "direct",
    "normalize": true
  }
]

After configuring default_vectors.json, you can launch the OpenAI-compatible server using the following command:

DOWNLOAD_DIR=./download_model \ // 
VLLM_USE_V1=1 python openai_steer_server.py \
--model LGAI-EXAONE/EXAONE-3.5-32B-Instruct-AWQ \
--quantization "awq" \
--gpu-memory-utilization 0.9 \
--max-model-len 8094 \
--port 8000 \
--gpu 0 \
--default-steer-vectors "default_vectors.json" \
--trust-remote-code 

We utilize CosyVoice for TTS. We have defined 5 samples corresponding to specific emotions: Joy, Sadness, Anger, Disgust, and Fear. The system selects the appropriate TTS sample based on the LLM's emotion choice. However, due to copyright restrictions, we are unable to release the voice samples.

This project was made possible by the excellent open-source projects listed below. We would like to express our sincere gratitude to the original authors.

Open-LLM-Vtuber

EasySteer