Syncs your VNyan camera over to LIV mixed reality
Allows VNyan to be your VTuber renderer allowing for high quality models and full redeem support - no more VRMs!

Please read these instructions carefully before trying to set this up!

It is important to understand how this plugin works. Normally LIV will split the VR scene into effectively three image layers, background, glow and foreground. It will then build up a composite image: Background, VRM avatar, glow layer and foreground. This setup works by replacing the VRM avatar layer with a capture from VNyan and manually doing the compositing of the layers in OBS, instead of letting LIV do it. In order to make VNyan fit correctly there are a pair of plugins, one for VNyan that sends its camera position, over to the LIV plugin which sets the camera plugin to match VNyan.

• VNyan
  • SteamVR tracking configured and working
  • Spout2 output configured and working
  • (Recommended) a .VSFAvatar model ¹
  • (Optional) My Extras Plugin to prevent accidentally resizing VNyan while in VR
• OBS
  • Spout2 plugin
  • Source Clone plugin
  • Advanced Mask plugin
• LIV VR
  • Mixed Reality Avatar mode configured and working
  • (Optional) A VRM file with the same rigging as your VSFAvatar
• General
  • A sufficiently powerful PC
  • Understanding of OBS filters
  • Understanding of the Windows filesystem, DLL files, JSON files etc.

• Download the latest version from the Releases Page
• Enable plugins in VNyan settings, if not already enabled
• Copy VNyan-LIV.dll into the VNyan\Items\Assemblies folder
• Copy LIV_VNyan.dll into C:\Users\<you>\Documents\LIV\Plugins\CameraBehaviours

• Start Virtual Cameras and Avatars
• Set capture to "Auto"
  • Target resolution: Your OBS canvas size, or maybe smaller if you stream with large borders
• Set capture to "Manual"
  • Target: your game
  • Effect: Dump + Composite
    
• Set LIV output to double your target resolution (e.g. 1920x1080 = 3840x2160)
• Put your VR headset on, open LIV from the circle on the floor and enable avatars
• Go to: Camera 1 -> Plugin -> LIVnyan to enable the plugin
• Do one of the following:
  • (Better performance) Disable rendering all parts of your avatar within LIV and disable tracking, or
  • (Better visiblity) Load a VRM version of your avatar, and enable whatever trackers you actually have
  • (Best option, but paid) Disable rendering & tracking in LIV, pop out the OBS preview to a separate window and then use XSOverlay (or similar) to display that preview within VR

Note: The numbers in this section assume your target resolution is 1920x1080. If it is not then subsitue your actual target resolution as appropriate

• Capture LIV Output using Game Capture, name it "LIV Quadrants"

  • Using the transform editor on "LIV Quadrants", crop right: 1920 top: 1080 from your LIV Output. Do not use filters for this!

• Source clone "LIV Quadrants", name it "LIV Alpha Mask"

  • Add a crop filter, crop left: 1920 bottom: 1080. Do not use a transform for this!
  • Update: If this filter is giving you a solid white square, try changing the LIV effect from "Dump+Composite" to just "Dump", set bottom crop to 0 and top crop to 1080
  • Place this behind the "LIV Quadrants" source so it is completely hidden

• Capture VNyan Spout2 output in the usual way

  • place this in front of "LIV Quadrants"

• Source clone "LIV Quadrants" again, name it "LIV Glow".

  • Add a crop filter, crop right 1920 bottom: 1080.
  • Right click it -> Blending Mode -> Add
  • place this in front of your VNyan Spout capture

• Source clone "LIV Quadrants" again, name it "LIV Front Layer".

  • Add a crop filter, crop right 1920 bottom: 1080.
  • Place this in front of "LIV Glow"
  • Add an "Advanced Mask" filter
    • Source: "LIV Alpha Mask"
    • Filter type: "greyscale"

  The end result should look something like this:
  

• Ensure that VNyan's window size is the same size as all your LIV sources
• Configure SteamVR and calibrate your trackers in the usual way
• Ensure the plugin is active by one of the following methods
  • Click the button in the VNyan plugins screen to toggle enable/disable
  • Call the _lum_liv_enable trigger
  • Edit LIVnyan.cfg to set ActiveOnStart = true and restart VNyan
• If activated successfully, your model should appear slightly more zoomed out than normal. This is expected behaviour. When deactivated it will revert to default.

Many VRoid models have their arms too short. The closer your model matches your IRL height and proportions the less likely you are to have weapons fly away from your hand when your arms are outstretched. Elbow IK should also work better.
For games that don't show objects/weapons in your hand, these calibration issues will be less noticible.
The zoom out effect is because VNyan uses a "physical camera" setting that emulates the distortion curved lens of an IRL camera to give a more natural look. LIV does not do this and cannot easily be changed. Instead we simply disable the physical camera options in VNyan while LIVnyan sync is active

Clicking the plugin button toggles camera sync

_lum_liv_enable - enable camera sync
_lum_liv_disable - disable camera sync

Settings are stored in LIVnyan.cfg inside your VNyan profile directory (default %APPDATA%..\LocalLow\Suvidriel\VNyan). This file will appear the first time you start VNyan with LIVnyan installed. Any changes will only take effect after restarting VNyan
ActiveOnStart - Camera sync will start as soon as VNyan loads
LogEnabled - The VNyan plugin will log to player.log in the main VNyan profile directory. The LIV plugin will log to %USERPROFILE%\Documents\LIV\Plugins\CameraBehaviours\LIVnyan.log
LogSpam - Both plugins will log sent/recieved camera position, rotation & FOV, plus settings info every single frame.
These logs will get very big very quickly. Only enable this for troubleshooting!
CursedCamera - Adds a delay to all camera movements in VNyan, while still updating LIV in realtime. Fixes the issue during fast pans where your model moves ahead of the VR world. See the Advanced Timing section before changing this option

• LIV manual target and effect set correctly (LIV does not remember these settings)
• Check your OBS scene and ensure that your model appears in the correct place
• Drag the camera around in the VNyan window, and check that LIV's output follows it
• Are foreground objects and glowing objects correctly passing in front of your model
• Are your feet being chopped off (See the troubleshooting section for this)
• Have any physics on your model glitched while you were doing all this? (reload avatar if so)

Because this setup is merging two separate video feeds from two different sources, there is going to be some manual sync needed to get the best results.
I recommend using a game where you are always holding an object, such as Beat Saber, and recording these tests

First move your hands around and observe that your saber moves ahead of your hands.
Using a video player that allows you to step through frame by frame (I use AVIDemux), make a note of how many frames it takes for your hand to reach the saber. Multiply this by 16.667 and set LIV's camera latency to the resulting value.
Test and record again, adjust camera latency as necessary, repeat until you are 100% happy with the result. Do not go on to the next step until then.

If you will only be using a static camera this step is not strictly necessary.
Create a temporary set of nodes that switch between camera positions with a transition time of around 1000ms, start recording and fire these nodes If your model moves ahead of the VR world then you will need to use the "Cursed Camera" feature. This feature intercepts VNyan camera movement, sends it to LIV and then forces a camera movement delay in VNyan. An unavoidable side effect of enabling this feature is that manual camera movements will feel laggy As before, count how many frames it takes for LIV's camera to catch up to VNyan's. The result will likely be the same (or very close) to the LIV camera delay you set in the previous step. call _lum_liv_enable passing this result into value1 and move the camera again. Hopefully it now lines up, but if necessary re-record and adjust until it's perfect Once you have the final value, close VNyan, edit LIVnyan.cfg and set Cursed camera there to make the change permanent Remember that if you change the LIV camera latency, you will also need to adjust the Cursed Camera latency

These are completely optional, but are how I do things

• Have an "EnableVR" websocket trigger that enables SteamVR and LipSync, disables ARKit & LeapMotion. Also have a "DisableVR" websocket that does the opposite. Put these on a toggle button on your stream deck.
• If you have physics, especially skirts, and will be taking stream breaks: have a scene change button for your VR scene that waits for ~4 seconds, then reloads your avatar, waits another second and then switches scene. This will minimise any issues with skirt clipping caused by pretzeling, giving you time to get to a neutral position in your VR space before forcing a physics reset
• If you use VoiceMeeter, have both your regular microphone and your VR headset microphone routed to the same virtual audio cable. Connect OBS, discord etc. to this virtual cable, then have a stream deck button that mutes one and unmutes the other, and vice versa
• Also in VoiceMeeter, have a virtual audio cable for discord comms and/or redeems, have this routed to both your regular headphones and your VR headset
• VoiceMeeter again: Have a "Rescan audio" button that runs Voicemeeter.exe -r Since the Index headset is generally not connected until you launch SteamVR, VoiceMeeter will not pick it up as a valid sound target on launch, but forcing a rescan like this fixes that without having to manually kill and re-open it
• If your are playing a rhythm game and are routing your audio through VoiceMeeter, carefully check your audio calibration. For me the lag created by voicemeeter is 120ms, which is quite significant
• Configure VNyan to switch to 1920x1080 on launch, then use my Extras Plugin to lock VNyan's window size. Preventing accidental resizing errors when you're trying to do an emergency recalibrate
• Use XSOverlay or similar to show your OBS preview window within VR, and check it frequently, especially if you have physics on your model
• Take full advantage of VNyan's features. Go wild with redeems, Poiyomi shaders, Magica cloth. You put all the effort in to set this up, so make use of it!
• Follow LumKitty on https://twitch.tv/LumKitty :3

Adjust back/forward offset in LIV's camera advanced settings

If your monitor resolution is too small, LIV will crop the output window to your actual size. If you are streaming at 1080p and do not have a 4K monitor this will affect you.
Use Virtual Display Driver to create a fake 3840x2160 display. As an added bonus this hides the LIV output window from your actual monitor which makes it significantly less annoying (although having a fake monitor is annoying in different ways, so remember to disable it outside of VR streaming) WARNING: If you are using Virtual Display Driver. Ensure that you enable the display before you start SteamVR. Otherwise you will hit a bug SteamVR when trying to interact with your desktop that causes the mouse pointer to be massively offset from where you point your controller. Only fix at this point is to restart SteamVR. Guaranteed this will bite you in the arse mid-stream, so be careful!

Another LIV feature. It's suppossed to tell you where your camera is currently. Apparently the only way to disable it is to set camera 1 to selfie and then you have the option to disable it, then you can set it back to plugin. This setting does not persist between sessions. Unfortunately this is not something I can fix.

Increase the Camera latency in LIV's camera calibration settings until it lines up

Enable the "Cursed Camera" delay feature in LIVnyan.cfg or by passing a value to int1 when calling _lum_liv_enable that approximately matches the camera latency setting in LIV

Use Sjatar's Stylistic Screen Light plugin
Add a "Sprout Sender" filter to "LIV Front Layer" after crop/pad but before Advanced Mask. Name the source "screen"

Either open an issue here (preferred), or ask in the LIVnyan thread on Suvi's Discord

If you find this plugin useful, please consider sending a follow or a raid my way. If you somehow make millions using it, consider sharing some of that with me :3