DeepLog LSTM

"DeepLog LSTM: Anomaly Detection using LSTM and AI based root cause analysis

Getting Started

1. Build environment

   git clone <repo>
   cd DeepLog-LSTM
   python -m venv venv
   . venv/bin/activate
   pip install -r requirements.txt

2. Run local example

   We use open data OpenStack from logpai's loghub

   2.1. Preprocess

   cd example/
   python preprocess.py

   2.2. Train

   num-class is count of event_id_map, where event_id_map is generated by preprocess.py. num-candidates is self-define, here we define num-candidates is num-class*0.1

   python train.py --num-class 1143 --num-candidates 114 --epochs 35 --window-size 3 --local True

   2.3. Predict

   python predict.py --threshold 25

   2.4. UI

   streamlit run frontend.py

   Note: add your openai key to .streamlit/secrets.toml in project folder OPENAI_API_KEY=""

3. Result

   Accuracy	0.9525801
   Precision	0.9235754
   Recall	0.6523478
   F1	0.7646214

Folder Structure

.
├── README.md
├── deeplog
│   ├── __init__.py
│   └── deeplog.py
├── example
│   ├── data
│   │   └── OpenStack
│   │       ├── anomaly_labels.txt
│   │       ├── openstack_abnormal.log
│   │       ├── openstack_normal1.log
│   │       └── openstack_normal2.log
│   ├── predict.py
│   ├── preprocess.py
│   ├── train.py
│   └── frontend.py
└── requirements.txt