TensorFlow.js

This version of the TensorFlow Plugin has been officially tested with the following GPUs.

• NVIDIA RTX 3070 (~300 fps continuous detection possible)
• NVIDIA GTX 1660

The GPUs were running with NVIDIA Drivers v515, CUDNN v8, CUDA Toolkit 11.2 on Ubuntu 20.04 and Rocky 9

tfjs 4.1.0 requires a minimum compute capability of 6.0 for NVIDIA GPUs.

CPU only Object Detection use is not recommended unless your CPU is powerful.

Required NVIDIA Drivers for GPU Support

The "Run Installer" command does not offer Driver Installation. You must select Driver Installation manually from the Commands drop down for this plugin.

NVIDIA Drivers v515, CUDNN v8, CUDA Toolkit 11.2 Installers are offered for Ubuntu 20.04 and Rocky 9

Installing and Downloading the Plugin

1. Login to your Shinobi's Superuser panel.
   • https://YOUR_SHINOBI/super
2. Open the "Plugin Manager" tab.
3. In the listing select the latest "Object Detection (tfjs 4.1.0)" plugin to Download.
4. Skip this for CPU Only or if you already did it before. Select "Install NVIDIA Drivers v515, CUDNN v8, CUDA Toolkit 11.2 (Ubuntu 20.04 and Rocky 9)"
5. Hit Run Installer for the newly downloaded plugin.
6. Once Installed click "Enable" and restart Shinobi.
7. Reopen the Plugin Manager and run the Test Object Detector command for either CPU or GPU.
8. Now go ahead and login to the main Dashboard to setup a Monitor with Object Detection!
   • https://YOUR_SHINOBI/
   • Watch this video for a visual guide with some extra tips

Your test should look similar to this.

############################################
@tensorflow/tfjs-node(-gpu) module test for Object Detection
GPU Test for Tensorflow Module
############################################
Loading https://cdn.shinobi.video/images/test/car.jpg
Detecting upon https://cdn.shinobi.video/images/test/car.jpg
{ data: { data: [] }, type: undefined, time: 59 }
No Matrices...
Done https://cdn.shinobi.video/images/test/car.jpg
Loading https://cdn.shinobi.video/images/test/bear.jpg
Detecting upon https://cdn.shinobi.video/images/test/bear.jpg
{
  data: [ { bbox: [Array], class: 'bear', score: 0.9899230599403381 } ],
  type: undefined,
  time: 39
}
Detected Objects!
{
  x: 84.47683596611023,
  y: 137.2664852142334,
  width: 341.1593613624573,
  height: 189.80571269989014,
  tag: 'bear',
  confidence: 0.9899230599403381
}
Done https://cdn.shinobi.video/images/test/bear.jpg
Loading https://cdn.shinobi.video/images/test/people.jpg
Detecting upon https://cdn.shinobi.video/images/test/people.jpg
{
  data: [
    { bbox: [Array], class: 'person', score: 0.7666133642196655 },
    { bbox: [Array], class: 'person', score: 0.6917366981506348 },
    { bbox: [Array], class: 'person', score: 0.5978450775146484 },
    { bbox: [Array], class: 'person', score: 0.5886031985282898 },
    { bbox: [Array], class: 'person', score: 0.5719519257545471 }
  ],
  type: undefined,
  time: 40
}
Detected Objects!
{
  x: 75.10916063189507,
  y: 1.8466887474060059,
  width: 123.42189314961433,
  height: 249.71912240982056,
  tag: 'person',
  confidence: 0.7666133642196655
}
{
  x: 2.2964478135108948,
  y: 92.02121257781982,
  width: 165.2727364897728,
  height: 312.9169178009033,
  tag: 'person',
  confidence: 0.6917366981506348
}
{
  x: 295.71580016613007,
  y: -0.2599790096282959,
  width: 118.83705461025238,
  height: 216.8119761943817,
  tag: 'person',
  confidence: 0.5978450775146484
}
{
  x: 129.25813829898834,
  y: 131.48706221580505,
  width: 185.93936240673065,
  height: 274.5884048938751,
  tag: 'person',
  confidence: 0.5886031985282898
}
{
  x: 409.1404616832733,
  y: 39.31091523170471,
  width: 200.14932918548584,
  height: 365.9385917186737,
  tag: 'person',
  confidence: 0.5719519257545471
}
Done https://cdn.shinobi.video/images/test/people.jpg
#END_PROCESS

Learn more about Downloading and Installing Plugins here