Description
This post is the last of three and describes the software needed to build a web controlled home automation center. The previous two describes how to add a serial interface to a router and how to build a microprocessor lab board.
Features
- Software for the router (OpenWrt)
- Software for the web server (html and CGI)
- Software for the lab board (PIC16F628 assembler)
- Protocol for serial communication between router and lab board
- Protocol for 433MHz radio (and html generation tools)
Development
My first project for the web access lab board is an Internet controlled home automation system. This allows me to control the lights in my apartment with my Smartphone or iPad (accessing a web page). I already have the infrastructure – custom built control panels and hacked low cost receivers (I will try to post them later).
In a way using a separate microprocessor for the radio communication is overkill. You could bit bang the radio data directly from the router using the power LED, but the microprocessor adds the ability to have more features (like IR communication) and for me it was a quicker hack since I had most of the code ready.
This is how it works:
- A mini router is running OpenWrt with a tiny web server
- The web server shows a plan of my apartment with icons for all controllable lights.
- The icons have links that starts CGI scripts
- The CGI scripts are sending control data to the serial port of the router
- A PIC microprocessor decodes the data and transmits it as 433MHz radio signal
- An RF switch or dimmer receives the data wirelessly and changes the attached light accordingly.
Installing the router
It was very easy to install and set up OpenWrt on the router (by following the steps in this link http://embeddedtimes.blogspot.com/2011/09/tp-link-tl-wr703n-tiny-linux-capable.html). The biggest problem was to change the baud rate on the serial port. This is done with stty, but the problem is that stty isn’t included in the busybox that comes with the default OpenWrt package (I tried picoterm, microcosm and setserial without success). You can either install this custom firmware or just replace busybox with this version. Try to avoid deleting all symbolic links to busybox, or you will be busy for a while (trust me, I know…).
Make sure the web server starts on boot and that the baud rate is set to 9600:
stty -F /dev/ttyATH0 raw speed 9600 -crtscts cs8 -parenb -cstopb
When everything is ready you should have web access to the web server on the router and a serial port running @ 9600 8n1
Setting up the web interface
The user interface showing three different mood buttons, a floor plan for the apartment (the magenta box turns off the complete room) and the individual controls for the lights.
I wanted to have a visual control over all the lights in my apartment. I did a graphic floor plan and added light control icons for all the controllable lights and saved it as a png. Then I created a webpage that displayed the image and used image map to link the icons to different CGI scripts. In the future I will encode the data in the html file to a generic CGI script, but as a quick fix I created different CGI scripts for each light, one for light on, one for light off and others for light intensity (see radio protocol further down for more details). I then uploaded the files to the router and set the right permissions to the CGI files. As a final step I connected my logic analyzer to the serial port of the router and verified that everything worked as it should.
Open Source serial protocol
Instead of inventing something new I thought it was a good idea to use a protocol that already exists. I found this open source protocol that is used in a product called tellstick, sold in Sweden. It does the job and it allows support for many different standards of radio recievers. Here is a brief description of the protocol:
The first byte defines type of command; if it is an “extended send command” the following four bytes declare four different timing lengths in 10us values. Then one byte with package length of the data followed by the real data and ending with a “+”. The data is encoded so that the length of each high or low flank is matched to one of the four different timing lengths (written as two bits).
The lab board
The microprocessor receives the serial protocol (using the USART), decodes it and sends it as radio, by bit banging the radio data, to a hacked radio module from a remote control. All the code is written in assembler (1075 lines); you can find the compiled .hex file and the source at the end of this post. I used a PICkit2 programmer attached to the ICSP port of the lab board to upload the microprocessor code. The lab board is designed to be used with a router for web control, but it also works great together with an ftdi cable for USB control from a PC.
Radio protocol
Since the serial protocol is generic and supports several radio protocol standards I could choose the one that I think is the best. For me it is important that each remote receiver can respond to several different transmissions and not just a single id. This allows me to arrange different lights into groups so that a single command can set the light on several devices at once. And by arranging all lights in a room to one command, I can turn off the complete room with one click. It is important that the devices have unique addresses so that my neighbors won’t control my lights by accident. All this is possible with the NEXA (learning code) protocol which allows 67 million different codes. To generate different codes for my devices I wrote this set of JavaScripts that generates the raw data that is transmitted to the lab board to control the lights.
Version tracker – microprocessor lab board files (PIC16F628 assembler)
0.1 Hard coded radio commands
0.2 Telstick protocol
0.3 Added serial interface
0.4 Adjusted timing
Future IR transmission (to control stereo, TV etc)
IR reception (to control the lights from an IR remote control)
Version tracker – JavaScript (to generate nexa commands)
0.1 First version (only using hex code)
0.2 Hex code generation
0.3 Added light intensity support
0.4 Added different hex outputs
Future No updates planned
Documents
Web server package (html and CGI files)
Lab board compiled file (.hex)
Lab board source code (PIC 16F628 assembler)
Radio protocol generator (html with JavaScript)
See also
Licensing
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
WOW!
Very impressive project! I love the simplicity of your solution (in hardware and in software interface).
I’ve just discovered your blog and I’ll bookmark it right now.
It would be great some details (maybe a guide) on how you create a webserver on your router and how do you pass the serial data (from and to) the micro!
I’m actually on Arduino platform but I think it will be quite similar :)
Anyway, good job!
Thank you for the positive feedback!
Double check these links and let me know if there is anything you are missing?
Install openwrt with webserver:
http://embeddedtimes.blogspot.com/2011/09/tp-link-tl-wr703n-tiny-linux-capable.html
Download the webserver package and go thru the .cgi files to see how the data is being copied to the serial port.
http://vonkonow.com/wordpress/wp-content/uploads/2012/01/webserver.zip
And read thru this link regarding the serial protocol:
http://developer.telldus.se/doxygen/TellStick.html
It would be great to see an Arduino version of the serial to RF conversion. One tip could be to connect the Arduino directly to the USB port of the router. Then you don’t need to solder anything to access the serial port. You will need drivers to the usb2serial adapter on the Arduino board. Google OpenWrt and Arduino and I’m sure you will find how to install the drivers (might be different if you are using old board with FTDI or new board with Atmel chip…)
Good luck and let me know how it goes!
Thanks for very inspiring tutorial.
I connected Arduino to Wr703N and to read out sensors and contorl 433 MHz power outlets.
http://www.instructables.com/id/Wireless-power-outlets-for-home-automation-using-A/
Next will try 433 MHz thermometer.
May be we could collaborate and do some projects togather.
Hi Janis
Nice project!
Did you have any problem to get the Arduino USB2serial driver to work with the router? I think I read that the Uno could be a bit tricky…
It would be great to do something together; do you have anything special in mind?
/j
Hi!
Great write-up. I am going to buy one of these and have a shot at web enabling my home automation system now. My plan is to try and get one of these to talk serial to my home alarm system, to control its relay outputs. My only concern is that I have ZERO Linux or website/CGI etc. experience. Am I hoping for too much??
Is it difficult to creat the scripts to send serial commands out when pressing the website buttons? I was hoping I could get it to receive serial as well so the website can tell me if alarm panel inputs are in alarm etc.
Thanks for sharing your creation
To be honest I’m far from being a Linux expert myself. I think you can make it work, even if you have no scripting experience; it will just take some more time searching the web for similar scripts ;)
How can i convert the output of you’re protocol generator to the actual “tdtool –raw” input like described here?
I’m not familiar with what kind of data tdtool –raw generates, but the good thing with JavaScript is that it is easy to tweak. Just open the nexa_v1.html in your favorite text editor and alter it until it generates the data you need…
/j
The problem is that i don’t understand the protocol for generating my own code, so i hoped you could implement it in you’re script.
http://developer.telldus.se/doxygen/TellStick.html
Hi
I’m not sure I understand exactly what you need (or even what the raw format in tdtool is).
Please explain what you are trying to do…
Hi, I have some home automation experience and also have few wr703 routers, few more than I care to admit ;)
I would love to make this project and also make it secure, I’m far from security expert but I’m aware of security issues with any wireless solution that can be easily sniffed and replayed.
I have an idea, it would be nice to have some kind of dynamic password in header of the packet that client sends so that server accepts only commands over wireless that signed with dynamic password. Both client and server would have some kind of key that would seed dynamic passwords.
Hi
I’m glad that you liked the project. I must also admit that I have a few more 703 routers then I really need ;)
I think that adding crypto is a great idea! It could be done both at the web interface and for the radio. I guess that it should be quite easy to switch to https for the transmission and check for a certificate that allows you to transmit data.
For the radio, you could add a simple crypto that can be implemented in the receiver. I have already modified my receivers with a custom cpu, so it is definitely possible to implement. It would be a lot better than “security by obscurity”…
Hi! Sorry for delay. Haven’t visited your site for some months.
“Did you have any problem to get the Arduino USB2serial driver to work with the router? I think I read that the Uno could be a bit tricky…”
Arduino Uno works fine:
http://lectroleevin.wordpress.com/2011/10/26/arduino-openwrt-usb-connection/
opkg update
opkg list | grep -i acm
kmod-usb-acm – 2.6.32.16-1 – Kernel support for USB ACM devices (modems/isdn controllers)
opkg install kmod-usb-serial
opkg install coreutils-stty
opkg install kmod-usb-acm
Nice, Thnx for the info!
Will try it someday! It’s a great way to connect your Arduino to the web and you don’t need to access (open and solder) the serial connection in the router.
Very nice project…I really like home automation stuff!
Hi, what a project! GREAT!
May i ask you a few questions about it?
I’d like to build a kind of home security system using the wr703n and a 433MHz receiver (additional also transmitter) to deal with the standard wireless door/window contacts (e.g. http://www.ebay.at/itm/Wireless-Door-Window-Entry-Detector-Sensor-Contact-315-433-Mhz-/180779447655?pt=UK_CCTV&hash=item2a174b7567)
I made a similar project with an arduino uno/ethernet shield with the transmitter hooked on and using the rc-switch lib to decode the transmission.
.) Is there a way to do this on the wr7033n too?
.) does an similar lib (rc-switch) exist?
MANY thansk for any reply/help
Your’s with best wishes from austria
Peter
Hi
Sounds like an interesting project!
I’m sure it can be done, in theory I guess you could hook a RF receiver directly to rx and write a custom driver to handle the data. I’m not a Linux expert though, so I would probably add a small CPU (like the PIC board in this project) handling the radio communication and then forward it to the wr703. I have added a project called “the one” http://vonkonow.com/wordpress/2012/04/d1/ that is specifically designed to decode this kind of data. It is written in PIC assembler and should be quite easy to implement on this board. But use the environment that you are most familiar with… If you have working code on an Arduino, just hook it up to the wr703 and you are good to go! The easiest way would be to use the USB port on the router and let Linux handle the serial interface via USB.
Please note that 433MHz radio transmission is quite easy to disturb with a jammer. I would not recommend it for any critical alarm system…
/johan
Very interesting to read about your project. Best of luck!!
Great project!
Maybe you know if it will work with light switches like this one:
http://www.aliexpress.com/item//629004768.html
Some guy with Arduino had problems with it:
http://forum.ardumote.com/viewtopic.php?f=2&t=76
Thanks!
Hi would you mind sharing which blog platform you’re working with? I’m planning to
start my own blog soon but I’m having a tough time selecting between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your layout seems different then most blogs and I’m looking for something unique.
P.S Sorry for being off-topic but I had to ask!
NP
Im using WordPress with a slightly modified Twenty Eleven theme. Good luck!
/j
Thanks foг οne’s marvelous posting! I definitely enjoyed reading it, you will be a great author.I will be sure to bookmark your blog and will often come back at some point. I want to encourage continue your great writing, have a nice day!
Hi,
nice stuff there!
I’m on a project for IoT and smart home (www.souliss.net) from more than a year and I would like to include some webhooks to it, your solution looks a nice option.
May you would like to have a chat and evaluate for a collaboration.
Thanks,
Dario.
i recive this “The CGI process did not produce any response”
i need help, thanks for sharing
Strange I have not encountered that problem…
Have you checked the file permissions for the scripts (chmod)?
/j
Hey I was wondering can we also recieve with the /dev/ttyATH0, if you have any idea the help would be much appreciated, when I use cat /dev/ttyATH0 >> a.txt it just hangs and ^C and nothing shows up :(
Hi
I have not tried it, but I guess it should work (depending on what you have transmitting on the other end). Since the protocol is asynchronous maybe it could appear like it freezes, when it actually is waiting for data…
Did you find any solution?
/j
Hi Johan, what are your plans to develop this project further. I think I could use this project, is the hardware available for sale?
Hi
Actually, I don’t have any more plans for this project, its working and that good enough for me ;)
I don’t plan to sell any custom built hardware, but all components are available online if you want to build one yourself.
One day I might redesign the hardware and use the usb port instead, then all you need is a custom usb stick that get you up and running (trinket would be perfect for this).
/j
Very nice project! I don’t see how to interface to the radio module and what kind of radio module you have used… Do you have the schematic?
I used a PIC16F628 that bit banged the radio data directly to the radio transmitter on a radio remote control. I disconnected the chip that reads the keyboard and generates the radio data and connected port A2 to that pin. You also need to power the remote (vcc & gnd). You can also buy ready rf modules, just make sure they use the same frequency and modulation.
Sry, I don’t have any schematics…
Check this link for more info of the pic:
http://vonkonow.com/wordpress/2012/01/web-access-lab-board/
Thank you for the fast reply.
I don’t have a CNC mill so I think that I for the first step I want to make the circuit on a breadboard, then on a PCB with normal components (no SMD), but I need the schematic!
I’ve made the schematic studying your PCB:
http://www.calitri.org/lab-board.jpg
– Can you tell me the function of A3 and A1 pins?
– The IR led and module and the switches are for future use, right?
– The not connected pins are right?
– The XTAL is necessary with your code, isn’t it?
THANKS FOR YOUR WORK!
Hi
Good work!
I think you nailed the layout! As you noticed I designed the board to be future proof, adding pads for all unused pins. I think I planned to use the switches to send commands like all on /off . I also added IR in and out and RF in (maybe R3).
The only thing you are missing in the schematic is the decoupling caps between power and ground, a power led and the icsp connector!
It’s up to you if you want to use the xtal or use internal osc. The internal works great for most cases, I added the xtal to eliminate any doubts about transmitting with the wrong timing (at different temps).
– For the oscillator I think (hope) that I can config the fuse in the programming options without change anything in the code…
– I don’t understand how to learn the code to give to the 433MHz module. How can you do that?
Sure, or just change:
_OS equ _XT_OSC ; External oscillator
To:
_OS equ _INTRC_OSC_NOCLKOUT ; Internal oscillator
It depends on what RF devices you use and what equipment you have. If you use the nexa devices the protocol I added is generic, and all you have to do is to change the house and unit codes, using the javascript. The script generates the serial data that is send to the pic (generating the rf signal). The serial protocol is generic and basically defines four timing values (10uS) and the data timing, by alternating between the four timing values (if I remember correct), se this link for more info:
http://developer.telldus.com/doxygen/TellStick.html
If you use another RF protocol you can use a simple oscilloscope or logic analyzer to look at the signal and try to replicate it. The RF signal is so slow that even a soundcard can be used for input…
You need a RF receiver to do this. The easiest way is to open a mains switch (not connected to the mains) and hook up power and ground (usually 5v). You can then find the rf in and probe the signal there…
Good luck!
I’ve just found a great solution for the above.
http://rurandom.org/justintime/index.php?title=Cheapest_ever_433_Mhz_transceiver_for_PCs
Great article!
I’m trying to learn more around HW, and you inspire me.
Would this http://goo.gl/s7ivgN work as your lab board? Looks like I should be able to connect it directly.
No I dont think it will work. Your module uses FSK modulation. Most rf switches uses ASK…
Try to modify a existing remote (see above)
Hi Johan
Could you explain in a nutshell the “hacked low cost receivers” part? I think it is in connection with your “each remote receiver can respond to several different transmissions and not just a single id” statement.
To put it simply, I’m curious whether this multi-signal capability of the receivers is something to easily achieve or not by someone non electric engineer.
Thank you
/j
Sure
I have replaced the microprocessor in off the shelf receivers with a custom 8-pin cpu (pic12f269). The program is generic, but you need to cut off the power, gnd, rf signal in the receiver and then program the microprocessor and solder it inside the receiver.
It’s not rocket science, but since it is high voltage you should have a clue of what you are doing…
/johan
I found this solution more convenient to me
http://www.jubito.org
It seems quite useful if you already have a Windows or Linux server up and running 24/7.
For me a router is more attractive since it is tiny, does not create any noise, has a lower cost and consumes less energy…
/j
I use it on rasp pi, so I think we draw :)
Nice Project :)
I am interestet in great Tutorials and Projects like this.
http://www.einplatinencomputer.com