Home Assistant Person Detection (Deepstack AI Tutorial)

Smart home artificial intelligence may sound like something from the future, even in today’s modern times.

However the future is here!

Person, object and facial detection are easily accessible from the Home Assistant platform!

In this tutorial we will walk through the process of setting up the Deepstack artificial intelligence platform, in order to detect the number of people in a room.

If you are looking for next level presence detection, keep reading…!

Table of Contents

1. Prerequisite
2. Deepstack topologies
   1. Deepstack & Home Assistant together
   2. Deepstack running on a computer
   3. Deepstack running on a seperate server
3. Configure Deepstack
   1. Install Docker
      1. Mac OS X
      2. Windows
      3. Linux
   2. Run Docker container
   3. Activating the Deepstack API
4. Configure Home Assistant
   1. Adding camera local image
   2. Add image processing integration
   3. Add Deepstack to user interface
   4. Create an automation
5. Conclusion

Prerequisite

You should have Home Assistant installed and running and a basic understanding of YAML including how to edit configuration.yaml. You should also understand how to use a service and entity within Home Assistant.

If you are new to Home Assistant or you are not yet familiar with editing YAML files then you should definitely check out my beginners tutorial on YAML first. I would also recommend taking a look at my other tutorials on automation and scripts.

You will also need an x86 based machine with Docker. It is possible to run Deepstack on Raspberry Pi but due to the limited processing power, you need to use an Intel Neural Compute Stick.

If you are already running your Home Assistant server on an x86 machine (NUC, recycled laptop, full-blown server etc..) then you can just use the same machine.

If you are running Home Assistant on Raspberry Pi then you can run Deepstack on your Linux or Windows 10 computer. This is a great way to try out the system and if you like it enough, perhaps you can decide to upgrade to an NUC system.

Please note that the Deepstack Docker container is optimised for Linux and Windows 10. You may experience issues trying to run it under other versions of Windows or on Mac OS X.

Deepstack topologies

There are several topologies that could be adopted in order to run Deepstack, depending on your hardware setup. Take a look at the diagrams below and choose the one that is most suitable for your setup.

Deepstack & Home Assistant together

The easiest way to integrate Deepstack and Home Assistant is just to run Deepstack on the same computer as Home Assistant.

If you run Deepstack on the same machine as Home Assistant, you can simply use the internal local IP address 127.0.0.1 with the Facebox port 5000 when referencing Deepstack in Home Assistant.

Deepstack running on a computer

If you are running your Home Assistant on Raspberry Pi and you don’t have a spare computer to use for Deepstack, you can run it on your normal desktop machine or laptop.

When referencing Deepstack in the Home Assistant configuration, simply use the local IP address from your computer with the Deepstack port 5000. You will still be able to access your Home Assistant user interface using the designated port, 8123 by default.

Deepstack running on a seperate server

Perhaps you have Home Assistant running on a Raspberry Pi and already have another x86 server on your network for other applications. It is possible to have both the Home Assistant server and Deepstack server running on seperate machines on the network.

Just reference Deepstack in Home Assistant using the local IP address for the machine you choose to use for Deepstack, with the port 5000. You can connect to the Home Assistant server with any other machine as usual.

Configure Deepstack

Once you have chosen and suitable topology the next thing to do is install Docker. If you have chosen to run Deepstack on the same machine as Home Assistant and Docker is already installed, feel free to skip ahead to running the Docker container.

Install Docker

The easiest way to use Docker on Mac or Windows is to download and install Docker Desktop. Linux users should follow the instructions for their chosen flavour of Linux.

Mac OS X

You can download Docker Desktop for Mac from the Docker website. Once downloaded just mount the .dmg file and then drag Docker to the applications folder.

Windows

You can download Docker Desktop for Windows from the Docker website. Once downloaded just follow the installation wizard.

Linux

The installation procedure is well documented on the Docker website. Choose from one of the following flavours of Linux to open the installation instructions in a new window:

• CentOS
• Debian
• Fedora
• Ubuntu

Run Docker container

Once you have Docker installed on your host system, we need to pull the latest Docker image.

sudo docker pull deepquestai/deepstack

Note that if you have an NVIDIA GPU, it is possible to boost the speed considerably by installing the NVIDIA Deepstack container instead.

Activating the Deepstack API

First we need to head over to the Deepstack website in order to create a free account and get an authorization key. The basic free account gives unlimited access for a single installation.

Once we have the authorization key we can run Deepstack. We must run it without any recognition first in order to enter the authorization key.

sudo docker run -v localstorage:/datastore -p 5000:5000 deepquestai/deepstack

Once Deepstack is running, point your browser to the following address and replace <your-ip> with the IP address of your Deepstack server.

http://<your-ip>:5000

You will be presented with the Deepstack activation key section so go ahead and enter the activation key from your Deepstack account.

Now that Deepstack has been activated we must stop the Deepstack container and restart it with vision detection activated.

First we need to find the container ID of the Deepstack Docker container. Go ahead and enter the following command.

docker ps -l

You should see the image “deepquestai/deepstack” and corresponding container ID. In my case the container ID is e42bfa13306c.

We can stop the container with the following command, replacing <container-id> with your own container ID.

docker stop <container-id>

Now we can restart the Docker container with vision detection. Go ahead and enter the following command, replacing <auth-key> with your own Deepstack authorization key.

docker run -e VISION-DETECTION=True -e API-KEY="<auth-key>" -v localstorage:/datastore -p 5000:5000 --name deepstack -d deepquestai/deepstack:noavx

Deepstack should now be running in the background of your host system so we can move on to the configuration of Home Assistant. Awesome!

Configure Home Assistant

At the time of writing (HA version 0.106.6) the Deepstack component is not yet officially part of Home Assistant, therefore we must manually add it.

If you do not already have the custom_components folder in your config directory, go ahead and add it.

/config/custom_components

Next head over to the Hass-Deepstack Github repository and download the .zip file and extract it. Go ahead and move the custom_components/deepstack_object folder into your Home Assistant config/custom_components folder.

Now you should have the following folder in your Home Assistant directory.

/config/custom_components/deepstack_object

Once you have copied the folder, go ahead and restart Home Assistant.

Adding camera local image

In order to test Deepstack we will add a test image as camera entity in Home Assistant. This will allow us to present the image to Facebox via the API to check that the connection is working.

You can use this image or try another if you prefer. Go ahead and save it as /config/www/image.jpg (using Samba). If you don’t have the www folder in your config directory, you can just create it.

Next add we need to add the local_file platform to the camera integration. This will create a camera entity called camera.local_file.

camera:
  - platform: local_file
    file_path: /config/www/image.jpg

Add image processing integration

Go ahead and open up the configuration.yaml file. First we need to add the Deepstack platform to the image_processing integration.

image_processing:
  - platform: deepstack_object

Next we need to specify the IP address and port of the machine running Deepstack, as well as the authorization key.

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.83
    port: 5000
    api_key: d3f*****-****-****-****-************

Next we will specify a scan interval. This is the number of seconds between each refresh, bearing in mind that lower numbers may dramatically consume system resources.

Note that you do not need to specify a scan interval if you only wish to scan manually, such as when a proximity sensor is triggered.

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.83
    port: 5000
    api_key: d3f*****-****-****-****-************
    scan_interval: 5

Next we will specify a list of targets we wish to detect. For this example we will include person.

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.83
    port: 5000
    api_key: d3f*****-****-****-****-************
    scan_interval: 5
    target:
      - person

Finally we will select a camera entity to monitor. For this example we will use the camera.local_file that we created earlier.

image_processing:
  - platform: deepstack_object
    ip_address: 192.168.1.83
    port: 5000
    api_key: d3f*****-****-****-****-************
    scan_interval: 5
    target:
      - person
    source:
      - entity_id: camera.local_file
        name: deepstack_person_detector

Once all the code is added to configuration.yaml, go ahead and restart Home Assistant. A full list of configuration variables can be found on the Github page.

Add Deepstack to user interface

Now that we have our test image and integration configured, we can add them to the user interface and see if Deepstack is working. From the overview page, click on the three dots and then configure UI in the top right-hand corner of the page.

This will switch on the UI editor mode. I will create a new tab called Deepstack for the cards but you can put them anywhere you like.

Once on the desired tab, click the orange circle with the plus towards the bottom left of the screen and then choose the picture glance card.

Enter a name for the card, something like Deepstack Test. Choose camera.local_file as the camera entity. Click save once done and you should now see the test image in the user interface.

Next we need to add an entities card for the image processing elements. Enter a name for the card, something like Image Processing. Add the image_processing.deepstack_person_detector entity to the entities list and click save.

Now you should be able to see the test image and also the number of people detected. Deepstack should recognise a total of 6 faces if you are using this image.

Create an automation

So now that we are able to perform object detection, how can we use it?

It is possible to create an automation that is fired when a person is recognised by using specifying the event type as deepstack.object_detected within the event platform.

The following automation will trigger when a person is detected.

- alias: Si recognised
  trigger:
    platform: event
    event_type: image_processing.detect_face
    event_data:
      object: person
  action:
   - service: light.turn_on
     entity_id: light.treelamp

There are also additional parameters that can be specified under the event_data.

• Confidence: the confidence score between 0 and 1.
• Box: the bounding box of the object defined as (y_min, X_min, y_max, x_max).
• centroid: the center of the bounding box defined by coordinates (x,y).

Both the bounding box and centroid can be used to determine whether a specified object is detected within a certain ROI (region of interest).

Conclusion

Machine learning is a fascinating topic and as we have discovered in this tutorial, even the most basic setup can add a exciting potential functionality to a smart home.

You should now have the basic Deepstack system integrated into your Home Assistant setup so you can start to develop some creative uses for this new found AI functionality. If you have a spare Pi lying around, why not check out this tutorial so you can learn how to turn it into a remote IP camera for Deepstack!

Alternatively why not check out some of my other awesome Home Assistant tutorials!