How to create your own Detector Plugin

What is a Detector Plugin?

Detector Plugins are image analysis engines. Usually Object Detection or Facial Recognition.

See a list of included Detector Plugins.

Plugin Pieces

pluginBase.js	Used for Plugin connected over Websocket.
pluginWorkerBase.js	Used for Plugin connected as Worker process.
shinobi-**************.js	The detection function set for the plugin. The stars in the filename represent a variable name with a shinobi- prefix.
INSTALL.sh	Plugins generally come with an install script to quickly start using the plugin.
Dockerfile	Some Plugins come with a Dockerfile that allows it to be containerized with Docker. This may require additional for the user configuration if used for installation.
package.json	Contains the list of Node.js modules (NPM) that are required for this plugin to operate. Generally they have the same things for pluginBase.js to operate.

The Important Piece

To explain this we will look at the Tensorflow plugin.

In this Repository you will find the shinobi-tensorflow.js file. This is the detection function set for the TensorFlow plugin.

Below is the key portion of this file. You'll see the declaration of s.detectObject. This is what makes the plugin do the detection and send back the response to the main process.

You'll notice there is a slight amount of post-processing after the detection result is offered for the given frame. This is to conform the drawn boxes to the format Shinobi recognizes. This way it can draw the boxes on the live stream and all display them over the recordings.

const ObjectDetectors = require('./ObjectDetectors.js')(config);

s.detectObject = function(buffer,d,tx,frameLocation,callback){
    new ObjectDetectors(buffer).process().then((resp)=>{
        var results = resp.data
        if(results[0]){
            var mats = []
            results.forEach(function(v){
                mats.push({
                    x: v.bbox[0],
                    y: v.bbox[1],
                    width: v.bbox[2],
                    height: v.bbox[3],
                    tag: v.class,
                    confidence: v.score,
                })
            })
            var isObjectDetectionSeparate = d.mon.detector_pam === '1' && d.mon.detector_use_detect_object === '1'
            var width = parseFloat(isObjectDetectionSeparate  && d.mon.detector_scale_y_object ? d.mon.detector_scale_y_object : d.mon.detector_scale_y)
            var height = parseFloat(isObjectDetectionSeparate  && d.mon.detector_scale_x_object ? d.mon.detector_scale_x_object : d.mon.detector_scale_x)
            tx({
                f:'trigger',
                id:d.id,
                ke:d.ke,
                details:{
                    plug:config.plug,
                    name:'Tensorflow',
                    reason:'object',
                    matrices:mats,
                    imgHeight:width,
                    imgWidth:height,
                    time: resp.time
                },
                frame: buffer
            })
        }
        callback()
    })
}

Making your own Detector Plugin

First you need to have the following to do this.

• Know how to setup a plugin (Try setting up the Tensorflow plugin)
• Knowledge of how to use Node.js (JavaScript)
• Have a Detection Engine that operates in Node.js or another language, like Python.

Let's Begin.

1. Navigate to your Shinobi folder. Default is /home/Shinobi.

cd /home/Shinobi

1. We're going to copy the tensorflow plugin and rename it to being a new plugin. For this example we will be making a Gun Detector Plugin.

cp plugins/tensorflow plugins/gundetector

2. Open the newly copied folder.

cd plugins/gundetector

3. Now we need to edit some files to make them designated as "gundetector".

conf.sample.json	Replace all instances of "Tensorflow" with "GunDetector".
conf.json	Replace all instances of "Tensorflow" with "GunDetector".
Dockerfile	Replace all instances of "tensorflow" with "gundetector". You will also need to strip all installer portions only relevant to tensorflow and replace them with your detection engine's required dependencies.
INSTALL.sh	Replace all instances of "Tensorflow" with "GunDetector". Also replace all instances of "tensorflow" with "gundetector". You will also need to strip all installer portions only relevant to tensorflow and replace them with your detection engine's required dependencies.
package.json	Replace all instances of "tensorflow" with "gundetector". You will also need to strip all installer portions only relevant to tensorflow and replace them with your detection engine's required dependencies.
shinobi-tensorflow.js	Rename the file to shinobi-gundetector.js. As mentioned in the above section, this is the most important portion of the entire plugin. This is where your analyzation code will be placed. Modify the s.detectObject function to use your detection engine's frame analyzation function. s.detectObject is run against each frame sent to the detector plugin.
test.js	Create a simple script that downloads an image and does analyzation against it, as if it were sent from Shinobi. This is added to help users quickly test the detection engine without actually attaching it to Shinobi.

4. Now that you have your plugin setup. You can try it out by running the following :

node shinobi-gundetector.js

You can daemonize it alongside Shinobi like this

pm2 start shinobi-gundetector.js

Be sure you are the OS user that installed Shinobi, so if you installed while as root then become root before starting the plugin.

5. Great, it's now running! The Object Detection section should say  GunDetect Connected appearing in the section header. You can now follow any of the Object Detection setup guides to use your new plugin!