Control your home lights with Augmented Reality

Recently i saw a video from the company called Leap Motion. The video was of them using an Augmented Reality headset , Project Northstar, to control lights. It was awesome. I haven’t got one yet so i figured, let’s try to do it with the things that i do have. Philips hue and a smartphone. I’m also using a raspberry pi but you can use this code on your PC. It probably wont be active 24/7 but for most cases that’s fine.

Data flow

The image above basically shows the data flow. On my raspberry pi i have a while loop that checks the database for changes on a specific document. Once that change happens i send a API request to my philips hue gateway with the updated value. And on my Unity application i just update the document with the wanted status of the light.

For you to follow this “semi-complete” guide you have to start by following two getting started guides.

Start by following this tutorial. It guides you through everything needed to know about setting up Augmented Reality on your android / ios device. It would take a whole blog post from me to guide you set it up. So, just follow that and you’ll be fine. Second, get started with firebase, specifically firestore. Go through their videos abit to understand the noSQL database structure. If you’ve ever worked with databases before you’ve probably used Mysql. This is way different , and in my opinion, way more fun to work with 🙂

Here is a guide to help you get started with firestore It also goes through how to implement it into your unity project.

Now. When all of that is done. Lets start off by creating our Python scripts to handle the database document reads and philips hue api requests.

In python, I created a folder called Hue, with two py classes. One called and the other Inside my i imported 4 modules.

Install requests and/or firebase_admin by using pip install <module name>.
For firestore to work with your Python scripts you need a service key aswell. You can go into your firebase console and download it through project settings -> service accounts -> generate new private key. Download that and put it into your project hierarchy. You can also use the code provided by firebase in their “tutorial” window where you downloaded it from. Next you can go into your database in the firebase console and create a new collection. I named mine rpi and created a document called test with a single string value.



Then back into the python scripts you have to create a reference to the document in the database and make python listen for changes on that document.

If you were to go into your firebase console and manually change the value you would see the value change. If you had a print(s) in the on_snapshot method that is. Then we need to be able to use this change to update our lights. For this we need to understand how the philips hue api works.
I recommend reading this.

I’ve got multiple lights so inside i made a class called Light. This means i can just make multiple objects of that class and send updates to individual lights if i ever want to make more functionality. This is how i did it :

So if you’ve followed that philips hue api you can understand what this code does. The only thing you need to replace is the XXXXX in the self.url inside __init__ method to your “password / access key” or what they called it. So back inside our this is how it looks like when I’ve completed it.

Now if you run this script, go into the firebase console. Go to your database and change the value you will see that the light in your room changes. Amazing! 🙂 If you’ve set this up on your raspberry pi you can change your light from anywhere in the world aswell. Really simple.

Now in Unity. If you are trying to replicate this, I’m expecting you to have followed that video tutorial on how to get started with AR foundation and AR core. And to have some experience in Unity since before. So after that video. What i did was make two booleans inside the script from the video tutorial. And when I pressed the screen once i flagged a bool called “hasPlaced” to true. and “shouldUpdatePose” to false. Which was true as default. Then in update only running “updatePlacementPose” and “updatePlacementIndicator” if shouldUpdatePose is true.

then made a small variation to another method:

So once the user has pressed to place an object. I turn a bool to true so it can’t be placed again and call another script to check if I’ve “pressed” the object (in my case a button i 3D modeled in blender).

So then you can create a new Script and call it whatever you want, i called it BrightnessController.
And it’s a really simple script. I just shoot a ray from the AR camera and check if i’ve hit the object. If i have a call another script that i made called “WebRequests”. We’ll come to that soon. But this is how my BrightnessController looks like:


PS. make sure to set the layer on your target object to match the variable layer in your script. Also make sure the object has a collider.

Inside the newly created script called WebRequests. I first made a method which i called CreateNewUser (you can follow the guide on firestore getting started to understand authentication. It’s very simple 🙂  )

Then i manually called that method to get a verified account on my firebase database. Next, i also created a method called Login which i call from my start method.

When that is done you should be able to login successfully and connect.

Then all we need to do first make a method called HandleQueries which our previous script can call. And then make that script call either the method to turn on our lights or turn off. By sending either 255 or 0 to our stored variable in our document on the database.

And.. That should be it. If you’ve got any problem don’t hesitate to contact me and I’ll do my best to help. I also didn’t want to explain everything thoroughly because from my perspective, It’ll do you some good to troubleshoot and figure things out by yourself.

Leave a Reply