Viseron is a self-hosted, local only NVR implemented in Python. The goal is ease of use while also leveraging hardware acceleration for minimal system load.

• Records videos on detected objects
• Lookback, buffers frames to record before the event actually happened
• Multiplatform, should support any x86-64 machine running Linux, aswell as RPi3.
  Builds are tested and verified on the following platforms:
  • Ubuntu 18.04 with Nvidia GPU
  • Ubuntu 18.04 running on an Intel NUC
  • RaspberryPi 3B+
• Supports multiple different object detectors:
  • Yolo Darknet using OpenCV
  • Tensorflow via Google Coral EdgeTPU
• Motion detection
• Native support for RTSP and MJPEG
• Supports hardware acceleration on different platforms
  • CUDA for systems with a supported GPU
  • OpenCL
  • OpenMax and MMAL on the RaspberryPi 3B+
• Zones to limit detection to a particular area to reduce false positives
• Masks to limit where motion detection occurs
• Stop/start cameras on-demand over MQTT
• Home Assistant integration via MQTT

Choose the appropriate docker container for your machine. Builds are published to Docker Hub

docker run --rm \
--privileged \
-v <recordings path>:/recordings \
-v <config path>:/config \
-v /etc/localtime:/etc/localtime:ro \
-v /dev/bus/usb:/dev/bus/usb \
-v /opt/vc/lib:/opt/vc/lib \
--name viseron \
--device /dev/vchiq:/dev/vchiq --device /dev/vcsm:/dev/vcsm \
roflcoopter/viseron-rpi:latest

version: "2.4"
services:
  viseron:
    image: roflcoopter/viseron-rpi:latest
    container_name: viseron
    volumes:
      - <recordings path>:/recordings
      - <config path>:/config
      - /etc/localtime:/etc/localtime:ro
      - /dev/bus/usb:/dev/bus/usb
      - /opt/vc/lib:/opt/vc/lib
    devices:
      - /dev/vchiq:/dev/vchiq
      - /dev/vcsm:/dev/vcsm
    privileged: true

docker run --rm \
-v <recordings path>:/recordings \
-v <config path>:/config \
-v /etc/localtime:/etc/localtime:ro \
--name viseron \
roflcoopter/viseron:latest

version: "2.4"

services:
  viseron:
    image: roflcoopter/viseron:latest
    container_name: viseron
    volumes:
      - <recordings path>:/recordings
      - <config path>:/config
      - /etc/localtime:/etc/localtime:ro

docker run --rm \
-v <recordings path>:/recordings \
-v <config path>:/config \
-v /etc/localtime:/etc/localtime:ro \
--name viseron \
--device /dev/dri \
roflcoopter/viseron-vaapi:latest

version: "2.4"

services:
  viseron:
    image: roflcoopter/viseron-vaapi:latest
    container_name: viseron
    volumes:
      - <recordings path>:/recordings
      - <config path>:/config
      - /etc/localtime:/etc/localtime:ro
    devices:
      - /dev/dri

docker run --rm \
-v <recordings path>:/recordings \
-v <config path>:/config \
-v /etc/localtime:/etc/localtime:ro \
--name viseron \
--runtime=nvidia \
roflcoopter/viseron-cuda:latest

version: "2.4"

services:
  viseron:
    image: roflcoopter/viseron-cuda:latest
    container_name: viseron
    volumes:
      - <recordings path>:/recordings
      - <config path>:/config
      - /etc/localtime:/etc/localtime:ro
    runtime: nvidia

VAAPI support is built into every container. To utilize it you need to add --device /dev/dri to your docker command.
EdgeTPU support is also included in all containers. To use it, add -v /dev/bus/usb:/dev/bus/usb --privileged to your docker command.

The config.yaml has to be mounted to the folder /config.
If no config is present, a default minimal one will be created.
Here you need to fill in atleast your cameras and you should be good to go.

cameras:
  - name: Front door
    mqtt_name: viseron_front_door
    host: 192.168.30.2
    port: 554
    username: user
    password: pass
    path: /Streaming/Channels/101/
    width: 1920
    height: 1080
    fps: 6 
    motion_detection:
      interval: 1
      trigger_detector: false
    object_detection:
      interval: 1
      labels:
        - label: person
          confidence: 0.9
        - label: pottedplant
          confidence: 0.9

Used to build the FFMPEG command to decode camera stream.
The command is built like this:
"ffmpeg" + global_args + input_args + hwaccel_args + codec + "-rtsp_transport tcp -i " + (stream url) + filter_args + output_args

A default ffmpeg decoder command is generated, which varies a bit depending on the Docker container you use,

ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -c:v h264_cuvid -rtsp_transport tcp -i rtsp://<username>:<password>@<host>:<port><path> -f rawvideo -pix_fmt nv12 pipe:1

ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -rtsp_transport tcp -i rtsp://<username>:<password>@<host>:<port><path> -f rawvideo -pix_fmt nv12 pipe:1

ffmpeg -hide_banner -loglevel panic -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -c:v h264_mmal -rtsp_transport tcp -i rtsp://<username>:<password>@<host>:<port><path> -f rawvideo -pix_fmt nv12 pipe:1

This means that you do not have to set hwaccel_args unless you have a specific need to change the default command (say you need to change h264_cuvid to hevc_cuvid)

cameras:
  - name: name
    host: ip
    port: port
    path: /Streaming/Channels/101/
    motion_detection:
      area: 0.07
      mask:
        - points:
            - x: 0
              y: 0
            - x: 250
              y: 0
            - x: 250
              y: 250
            - x: 0
              y: 250
        - points:
            - x: 500
              y: 500
            - x: 1000
              y: 500
            - x: 1000
              y: 750
            - x: 300
              y: 750

Masks are used to exclude certain areas in the image from triggering motion.
Say you have a camera which is filming some trees. When the wind is blowing, motion will probably be detected.
Draw a mask over these trees and they will no longer trigger said motion.

cameras:
  - name: name
    host: ip
    port: port
    path: /Streaming/Channels/101/
    zones:
      - name: zone1
        points:
          - x: 0
            y: 500
          - x: 1920
            y: 500
          - x: 1920
            y: 1080
          - x: 0
            y: 1080
        labels:
          - label: person
            confidence: 0.9
      - name: zone2
        points:
          - x: 0
            y: 0
          - x: 500
            y: 0
          - x: 500
            y: 500
          - x: 0
            y: 500
        labels:
          - label: cat
            confidence: 0.5

Points are used to form a polygon.

points:
  - x: 522
    y: 11
  - x: 729
    y: 275
  - x: 333
    y: 603
  - x: 171
    y: 97

object_detection:
  type: darknet
  interval: 6
  labels:
    - label: person
      confidence: 0.9
      height_min: 0.1481
      height_max: 0.7
      width_min: 0.0598
      width_max: 0.36
    - label: truck
      confidence: 0.8

Labels are used to tell Viseron what objects to look for and keep recordings of.
The available labels depends on what detection model you are using.
For the built in models you can check the label_path file to see which labels that are available, see commands below.

The max/min width/height is used to filter out any unreasonably large/small objects to reduce false positives.

motion_detection:
  interval: 1
  trigger_detector: true
  timeout: true
  max_timeout: 30
  width: 300
  height: 300
  area: 0.1
  frames: 3

TODO Future releases will make the motion detection easier to fine tune. Right now its a guessing game

recorder:
  lookback: 10
  timeout: 10
  retain: 7
  folder: /recordings

A default ffmpeg encoder command is generated, which varies a bit depending on the Docker container you use,

ffmpeg -hide_banner -loglevel panic -f rawvideo -pix_fmt nv12 -s:v <width>x<height> -r <fps> -i pipe:0 -y -c:v h264_nvenc <file>

ffmpeg -hide_banner -loglevel panic -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -f rawvideo -pix_fmt nv12 -s:v <width>x<height> -r <fps> -i pipe:0 -y -c:v h264_vaapi -vf "format=nv12|vaapi,hwupload" <file>

ffmpeg -hide_banner -loglevel panic -f rawvideo -pix_fmt nv12 -s:v <width>x<height> -r <fps> -i pipe:0 -y -c:v h264_omx <file>

This means that you do not have to set hwaccel_args unless you have a specific need to change the default command (say you need to change h264_nvenc to hevc_nvenc)

mqtt:
  broker: mqtt_broker.lan
  port: 1883
  username: user
  password: pass

logging:
  level: debug

Any value in config.yaml can be substituted with secrets stored in secrets.yaml.
This can be used to remove any private information from your config.yaml to make it easier to share your config.yaml with others.

The secrets.yaml is expected to be in the same folder as config.yaml.
The full path needs to be /config/secrets.yaml.

Here is a simple usage example.
Contents of /config/secrets.yaml:

camera_ip: 192.168.1.2
username: coolusername
password: supersecretpassword

Contents of /config/config.yaml:

cameras:
  - name: Front Door
    host: !secret camera_ip
    username: !secret username
    password: !secret password

Here I will show you the system load on a few different machines/configs.
All examples are with one camera running 1920x1080 at 6 FPS.
Motion and object detection running at a 1 second interval.

Intel i3-9350K CPU @ 4.00GHz 4 cores with Nvidia GTX1660 Ti

Intel NUC NUC7i5BNH (Intel i5-7260U CPU @ 2.20GHz 2 cores) using VAAPI and OpenCL

Intel NUC NUC7i5BNH (Intel i5-7260U CPU @ 2.20GHz 2 cores) without VAAPI or OpenCL

Viseron integrates into Home Assistant using MQTT discovery and is enabled by default if you configure MQTT.
Viseron will create a number of entities depending on your configuration.

Camera entity
A camera entity will be created for each camera.
Default state topic: homeassistant/camera/{mqtt_name from camera config}/image
Images will be published to this topic with drawn objects and zones if publish_image: true is set in the config.
Objects that are discarded by a filter will have blue bounding boxes, while objects who pass the filter will be green.
Zones are drawn in red. If an object passes its filter and is inside the zone, it will turn green.
Motion contours that are smaller than configured area are drawn in dark purple, while bigger contours are drawn in pink.
Masks are drawn with an orange border and black background with 70% opacity.

Binary Sensors
A variable amount of binary sensors will be created based on your configuration.

1. A binary sensor showing if any tracked object is in view.
   Default state topic: homeassistant/binary_sensor/{mqtt_name from camera config}/object_detected/state
2. A binary sensor for each tracked object showing if the label is in view.
   Default state topic: homeassistant/binary_sensor/{mqtt_name from camera config}/{label}/state
3. A binary sensor for each zone showing if any tracked object is in the zone.
   Default state topic: homeassistant/binary_sensor/{mqtt_name from camera config}/{zone}/state
4. A binary sensor for each tracked object in a zone showing if the label is in the zone.
   Default state topic: homeassistant/binary_sensor/{mqtt_name from camera config}/{zone}_{label}/state
5. A binary sensor showing if motion is detected.
   Default state topic: homeassistant/binary_sensor/{mqtt_name from camera config}/motion_detected/state

Switch
A switch entity will be created for each camera.
The switch is used to arm/disarm a camera. When disarmed, no system resources are used for the camera.
Default state topic: homeassistant/switch/{mqtt_name from camera config}/state
Default command topic: homeassistant/switch/{mqtt_name from camera config}/set\

• If you are experiencing issues with a camera, I suggest you add debug logging to it and examine the logs

• UI

  • Create a UI for configuration and viewing of recordings

• Detectors

  • Pause detection via MQTT
  • Move detectors to specific folder
  • Allow specified confidence to override height/width thresholds
  • Dynamic detection interval, speed up interval when detection happens for all types of detectors
  • Implement an object tracker for detected objects
  • Make it easier to implement custom detectors

• Watchdog Build a watchdog for the camera process

• Recorder

  • Weaving, If detection is triggered close to previous detection, send silent alarm and "weave" the videos together.
  • Dynamic lookback based on motion

• Properties: All public vars should be exposed by property

• Docker

  • Try to reduce container footprint

https://devblogs.nvidia.com/object-detection-pipeline-gpus/

Donations are very appreciated and will go directly into more hardware for Viseron to support.