PyHook is an MIT licensed software, written in Python and C++, for access and modification of ReShade back buffer.

PyHook consists of two elements:

• Python code that finds process with ReShade loaded, injects addon into it

  and allows to process frames in code via dynamic pipelines.
• C++ addon DLL written using ReShade API that exposes back buffer via shared memory

  and allows to configure dynamic pipelines in settings UI.

• Automatic ReShade detection and DLL validation
• Automatic addon DLL injection
• Shared memory as storage
• Dynamic pipelines ordering and settings in ReShade UI via addon
• Pipeline lifecycle support (load/process/unload)
• Pipeline settings callbacks (before and after change)
• Pipeline multipass - process frame multiple times in single pipeline
• Frame processing in Python via numpy array
• JSON file with pipelines settings
• Local Python environment usage in pipeline code
• Automatic pipeline files download

Do note that multisampling is not supported by PyHook at all with any API.

Pipeline	GTA V	Crysis 3	Trek to Yomi
None
DNN Super Resolution Scale: 2 Model: FSRCNN
Cartoon-GAN Scale: 1.0
Colorization Scale: 1.0
Cartoon-GAN and Colorization Scale: 1.0
Style Transfer Scale: 1.0 Model: Mosaic
Multi Style Transfer Scale: 1.0 Style: Pencil
Object Detection YOLO Model: Medium
Semantic Segmentation Scale: 1.0 PIDNet model: Cityscape(Large)
Depth Estimation Scale: 1.0

Benchmark setup:

• UNIGINE Superposition 64-bit DX11
• 1280x720, windowed, lowest preset
• Intel Core i9 9900KS
• RTX 2080 Super 8GB
• 32GB DDR4 RAM

Benchmark command:

$ .\superposition.exe -preset 0 -video_app direct3d11 -shaders_quality 0 -textures_quality 0 ^
-dof 0 -motion_blur 0 -video_vsync 0 -video_mode -1 ^
-console_command "world_load superposition/superposition && render_manager_create_textures 1" ^
-project_name Superposition -video_fullscreen 0 -video_width 1280 -video_height 720 ^
-extern_plugin GPUMonitor -mode 0 -sound 0 -tooltips 1

Results:

DNN super-resolution is crucial for fast AI pipeline processing. It allows to process multiple AI effects much faster due to smaller input frame.

As shown in the flowchart super-resolution consists of following steps:

• Scale base image down by some factor.
• Process small frame through AI pipelines to achieve much better performance.
• Scale processed frame back using DNN super-resolution.

PyHook uses ReShade ImGui UI to display list of available pipelines and their respective settings.

To display pipeline list, open ReShade UI and go to Add-ons tab:

Settings for enabled pipelines are displayed below mentioned list:

Supported UI widgets (read more in pipeline template):

• Checkbox
• Slider (integer value)
• Slider (float value)
• Combo box (single value select)

• ReShade >= 5.0.0

• Python == 3.10.6 for 64-bit | 3.10.4 for 32-bit (for pipelines only)

• CUDA == 11.3* (optional for AI pipelines only)

• cuDNN == 8.4.1* (optional for AI pipelines only)

• Only for specific pipelines: Any libraries that are required by pipeline code.

  Do note that AI pipelines that requires PyTorch or TensorFlow will not work on 32-bit system because libraries does not support it.

* CUDA and cuDNN version should be last supported by your GPU and pipeline modules.

• Same as runtime, but for ReShade addon only included headers are needed
• Boost == 1.80.0 (used for Boost.Interprocess shared memory)
• Dear ImGui == 1.86
• NumPy == 1.23.2
• psutil == 5.9.2
• Pyinjector == 1.1.0
• Requests == 2.28.1

• Same as build
• PyInstaller == 5.3
• Python Standard Library List == 0.8.0

You can download selected PyHook archives from Releases.

1. Download and unpack zip catalog with PyHook executable, addon and pipelines.

2. Prepare Python local environment (read more in Virtual environment) and download pipelines files if needed.

   Pipelines has own directories with download.txt file that has list of files to download.

3. Start game with ReShade installed.

4. Start PyHook.exe.

For custom pipelines (e.g. AI ones) install requirements and setup ENV variables that points to Python3 binary in required version.

Available ENV variables:

• LOCAL_PYTHON_32 (path to 32-bit Python)
• LOCAL_PYTHON_64 (path to 64-bit Python)
• LOCAL_PYTHON (fallback path if none of above is set)

Models for pipelines can be downloaded by links from download.txt that are supplied in their respective directory.

If antyvirus detects PyHook as dangerous software add exception for it because it is due to DLL injection capabilities.

To build PyHook simply run build.bat in Anaconda Prompt.

If any Python package is missing try to update your conda environment and add conda-forge channel:

$ conda config --add channels conda-forge

To build PyHook addon download Boost and place header files in Addon/include. Then open *.sln project and build given release.

PyHook allows to freely use virtual environment from Anaconda.

To create virtual env (64-bit) u can use following commands in Anaconda Prompt:

$ conda create -n pyhook64env python=3.10.6 -y
$ conda activate pyhook64env
$ conda install pip -y
$ pip install -r any_pipeline.requirements.txt
$ conda deactivate

For 32-bit different Python version have to be used (no new version at the time of writing):

$ set CONDA_FORCE_32BIT=1                         // Only for 64-bit system
$ conda create -n pyhook32env python=3.10.4 -y
$ conda activate pyhook32env
$ conda install pip -y
$ pip install -r any_pipeline.requirements.txt
$ conda deactivate
$ set CONDA_FORCE_32BIT=                          // Only for 64-bit system

When virtual environment is ready to be used, copy it's Python executable path and set system environment variables described in Installation.

• Anaconda for virual environment
• CUDA == 11.3 (or last supported by your GPU and pipeline modules)
• cuDNN == 8.4.1 (or last supported by your CUDA version)
• Visual Studio >= 16 with C++ support
• git for version control
• CMake for source build

After installation make sure that following environment variables are set:

• CUDA_PATH (e.g. "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3")

• PATH with paths to CUDA + cuDNN and CMake, e.g.:

  "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\bin"

  "C:\Program Files\CMake\bin"

When requirements are set, run Anaconda Prompt and follow code from file: build_opencv_cuda.bat

After build new environment variables have to be set:

• OpenCV_DIR (e.g. "C:\OpenCV\OpenCV-4.6.0")
• PATH, add path to OpenCV built binaries (e.g. "C:\OpenCV\OpenCV-4.6.0\x64\vc16\bin")
• OPENCV_LOG_LEVEL "ERROR", to suppress warning messages

>>> import cv2
>>> print(cv2.cuda.getCudaEnabledDeviceCount())
>>> print(cv2.getBuildInformation())

>>> import sys
>>> print(sys.executable)
'C:\\Users\\xyz\\anaconda3\\envs\\opencv_build\\python.exe'

DNN Super Resolution pipeline supports both CPU and GPU OpenCV versions and will be used as benchmark.

Benchmark setup:

• Game @ 1280x720 resolution, 60 FPS
• DNN Super Resolution pipeline with FSRCNN model
• Intel Core i9 9900KS
• RTX 2080 Super 8GB
• 32GB DDR4 RAM

Results:

NOTE: Values in GPU Mem MB and RAM MB contains memory loaded by pipeline only (game not included).

Conclusion:

GPU support allows to achieve over 4x better performance for best quality (2x) DNN super resolution and almost 2x for worst (4x).

• Added PyHook settings file.
• Fixed ImGui beeing affected for ReShade version up to 5.4.2.
• Added AI depth estimation pipeline example using https://github.com/wolverinn/Depth-Estimation-PyTorch
• Added AI semantic segmantation pipeline example using https://github.com/XuJiacong/PIDNet
• Fixed float inaccuracy in pipeline settings.
• Added AI object detection pipeline example using https://github.com/ultralytics/yolov5
• Added AI style transfer pipeline example using https://github.com/zhanghang1989/PyTorch-Multi-Style-Transfer
• Added automatic pipeline files download on startup.

• Added support for DirectX 12 and Vulkan with fallback for older ReShade version.
• Added support for Vulkan DLL names.
• Added AI super resolution example using OpenCV DNN super resolution.
• Added multistage (multiple passes per frame) pipelines support.
• Improved error handling in ReShade addon.
• Added error notification on settings save.
• Improved synchronization between PyHook and addon.
• Added OpenGL support.
• Added multiple texture formats support.
• Added logs removal from DLL loading.
• Added JSON settings for pipelines.
• Added combo box selection in settings UI.
• Added AI colorization pipeline example using https://github.com/richzhang/colorization
• Added AI Cartoon-GAN pipeline example using https://github.com/FilipAndersson245/cartoon-gan
• Added dynamic modules load from local Python environment.
• Added fallback to manual PID supply.
• Updated pipeline template.
• Added new callbacks for settings changes (before and after change).
• Added ReShade UI for pipeline settings in ImGui.
• Added pipeline utils to faster pipeline creation.
• Added dynamic pipeline variables parsing.
• Added shared memory segment for pipeline settings.
• Added AI style transfer pipeline example using https://github.com/mmalotin/pytorch-fast-neural-style-mobilenetV2
• Added pipeline lifecycle support (load/process/unload).
• Added pipeline ordering and selection GUI in ReShade addon UI.
• Added shared memory for configuration.
• Added multisampling error in PyHook.
• Added pipeline processing for dynamic effects loading.
• Added shared data refresh on in-game settings changes.
• Disabled multisampling on swapchain creation.
• Fixed error display on app exit.

• Initial version.