NRVCORP CIS_DVS_NPU Host Application Documentation

This document describes the CIS_DVS_NPU host application by NRVCORP. It is a prototype that detects moving objects in a static environment. The DVS class detects Regions of Interest (ROI) from the DVS sensor output using NRV's custom algorithm. The CIS class then crops frames based on this ROI data, focusing on areas with potential motion, which improves the detection of small or low-light objects. The cropped frames are processed by NRV's NPU using the YoloV3Tiny model, and the object detection results are displayed in an OpenCV window.

Actions

DVS Class

(link)

  • Receives data from the DVS sensor.

  • An internal member function analyzes it to obtain ROI bounding boxes, for detecting moving objects in a static environment.

CIS Class

(link)

  • Receives data from the CIS sensor.

  • An internal member function receives ROI bounding box coordinates from the DVS class, and translates it onto the CIS point of view.

NPU Class

(link)
The NPU class pipelines the next 3 steps :

  • Crops, preprocesses CIS frames and sends it to the NPU board through PCIe.

  • The NPU board runs YoloV3Tiny on the input image.

  • Receives and postprocesses model outputs on a OpenCV window.

Execution Steps

1) If you don't have the embeddedsw repository on your host PC, clone it (link).

2) Open the terminal in root permission by running sudo -i.

3) Navigate to the 4.CIS_DVS_NPU/host directory.

4) Run the following:

make all
bash tiny-yolov3-voc.sh