How to Cluster multiple Shinobi (Child Nodes)

What are Child Nodes? What is Clustering?

Child Nodes are the machines that are connected to the main Shinobi engine to provide added CPU and GPU power to the workload.

Clustering is when you have multiple machines working together.

The childNodes object in conf.json

There are a few option to set. Options are only required if you choose to use the childNodes object.

Option	Required	Type	Description	Default
enabled	no	bool	Whether or not to enable the childNode system. For Master and Child.	true
mode	yes	string	Type of module in the childNode system.  For Master and Child.	master
port	no	integer	The port running on the Master Node in which Child Nodes connect on.  For Master.	8288
key	yes	string	This string is sent back and forth to authenticate each request.  For Master and Child.	none
host	yes	string	The connection address of the Master Node including the port.  For Child.	none
masterDoWorkToo	no	bool	This flag decides if Master Node should be used for Monitor Processes. For Master.	false
maxCpuPercent	no	integer	This decides at what CPU percentage a Child Node cannot be used. For Master.	75

How to setup a Master Node

1. Open your conf.json and add the following options to activate the Child Node connection port. The default connection port is 8288 .

Remember to set the key value to something random, anything other than what is provided by default.

  "childNodes": {
      "enabled" : true,
      "mode" : "master",
      "port" : 8288,
      "key" : "aMatchingKey1234",
      "masterDoWorkToo": false,
      "maxCpuPercent": 75
  }

2. Then restart Shinobi to activated the changes.

pm2 restart camera

3. Now you can attach Child Nodes on port 8288.

How to setup a Child Node

Warning : You must never set a Master Node to a child unless it is a clean Shinobi install with no videos remaining in the file system.

1. As an example let's assume that our Master Node has the IP address 192.168.1.48 and is listening for children on port 8288.  You will need to setup your configuration as follows. Remember to match the key value or it will not connect.

  "childNodes": {
      "enabled" : true,
      "mode" : "child",
      "host" : "192.168.1.48:8288",
      "key" : "aMatchingKey1234"
  }

2. Now you must restart Shinobi to turn it into a Child Node.

pm2 restart camera

3. Now you when you start a monitor (or restart) it will use the first available child node with less than 80% CPU use. The automatic method for choosing a Child Node may base off other details instead of just the CPU in the future.

Where is the repository for the childNode code?

It is all bult into camera.js so you just need to download another copy of Shinobi on another machine and set the childNodes options to Child Node mode. You just need the following installed on the machine before attempting to run Shinobi in Child Node Mode.

• Node.js 12.x (12.22.7 Stable)
• FFmpeg 3.2.4 to 4.1

Database engine is not required. The Child Node will proxy all database requests through the Master Node.

What about the database?

You do not need to install any database engine on the Child Nodes. No MariaDB, No MySQL, No SQLite, etc.

All children will use the Master Node to execute database queries. Only the Master Node needs to have connection details.

You can leave the db  as default for all child nodes because they will proxy all database requests through the Websocket to the master.

What about the Detectors?

The Built-In Motion Detector by Kevin Godell  will operate the same. The motion detection will occur on the Node the monitor is running on. If a monitor is running on a Child Node then motion detection will occur on that same Child Node.

The Plugin based detectors will be a little different. Each Node, Child and Master, will have to be connected to an instance of a plugin.

You can run multiple instances of the same plugin on one machine or each machine run a version of the plugin and have them connect accordingly.