Heating Older Houses

As some will be aware I am in the process of monitoring my domestic heating system, with the intention of developing a heating controller that offers better performance than the existing timer/thermostat one that is fitted.

The Navitrino has now been running for a few days, logging the outside temperature, the living room temperature and controlling the boiler so as to keep the living room temperature between comfortable limits.

The graphs show the variation in inside and outside temperatures over 31st October and 1st of November. On the 31st we had the woodstove blazing - not a very controllable heatsource - thus the peak of 24C and the slow cooling back down to 18C over a period of 16 hours. During this time outside temperatures were fairly mild - between 6 and 12C.

I believe that the thermal demands are unique to each property, which coupled with widely different lifestyles and levels of occupancy mean that there is no "one size fits all" solution to domestic heating control.

I have already written some control software, to control the heating, and found it convenient to split the day up into 3 time periods - loosely called Day, Evening and Night corresponding to periods of occupation and activity. For example in the evening when the living room is normally occupied, 20C is a comfortable temperature, whilst during the day 19C may be more appropriate. At night time, when the livingroom is unoccupied it may be acceptable to relax the temperature requirement to 17C, which allows more economical use of gas, but allows for a relatively quick warm-up.

The first stage is to monitor the thermal profile of the house, and the boiler behaviour and gas usage, for different external temperatures and generate a simple model of the thermal behaviour of the property.

My house has solid 9" walls and 4" internal brick walls. The living room has a large chimney breast which also retains a fair amount of heat.

What I hope to establish is a series of warm-up times, based on a given starting room temperature and setpoint versus the outside temperature.

Once the thermal mass of the room has been warmed up, the house will take much less heating power to keep it at the comfortable temperature.

For example, yesterday lunchtime between 12 and 1pm, it took 27kWh of gas to raise the room temperature from 17C to 20C. Once warmed up, the 20C room temperature could be maintained for the rest of the day and overnight with just 3kW heat (57kWh energy). During this time, the outside temperature varied between 12C and 9C - so not particularly cold.

The intention is to repeat these measurements for a range of outside temperatures. What is likely is that the 1 hour/ 27kWh "boost" heat will become longer in duration and use more kWh as the outside temperature falls.

I should mention that the heating system at the moment consists of 3 room radiators - with TRV, a bathroom towel radiator and 25m2 of underfloor heating. This heats the living room, kitchen, bathrom, my work room and 1 bedroom on TRV set to low. These rooms we consider to be the core of the house, leaving hall, and unused spare bedroom not directly heated.

Gas Meters Revisited!

In the last couple of days I have run a multicore cable from my work room at the back of the house to the gas meter which is under the stairs.

After two days of frustration caused by noisy optical sensors and false triggering, I decided to use one of my newly acquired Hall effect sensors as used on the Lister spark ignition circuit.

I have used a Hall effect sensor Farnell 178-4735

http://www.farnell.com/datasheets/621734.pdf )

clipped exactly in the same place as the optoreflective switch.

The Hall sensor is mounted so that the face with writing on it, which is triggered by a passing North pole is pointing towards the least significant digit wheel.

This wheel contains the magnet, roughly in the same position as the silver reflective zero digit.

The hall sensor will work from 3V or 5V systems and needs a 10K pull-up resistor because it is open-collector.

As the magnet passes, the open collector output goes cleanly to 0V. Many of my problems with the optical sensor was due to signal bounce and noise pick-up. This inexpensive Hall sensor seems to have cured all that.

To connect up you will need a 3 way cable, or if running a long distance - something like screened microphone cable would be suitable.

I'm still testing to make sure that my pulse bounce and false counting problems have gone away - it's interrupt code on the Arduino, so needs careful debugging to make sure its working correctly.

This gas logging ( and central heating control and monitoring) is all part of the work that I am doing on the Open source Navitrino project.

Navitrino Progress

Introduction.
Navitrino is a modular open source project intended for monitoring and controlling domestic energy usage.

Navitrino is an experimental platform, compatible with low cost Arduino hardware, which can be customised to meet the requirements of the individual's home energy installation. Navitrino can be assembled from off the shelf Arduino hardware, and programmed readily using the Arduino IDE.

Today's domestic energy systems are becoming more sophisticated with the addition of solar water heating, solar pV, heat-pumps and the like. Navitrino has been designed to monitor the key parameters of the system, make decisions based on usage, lifestyle and climate conditions and intelligently co-ordinate the various energy sources.

Navitrino was conceived with the following uses in mind:

1. Solar Water Heating Controller - simple control with circulation pump relay
2. Central Heating / woodburner controller - determines best usage of hot water
3. Electricity Monitor - whole house electricity consumption
4. Gas consumption monitor - pulse counter on optical or magnetic sensor on gas meter
5. Battery management system pV / battery charge controller/datalogger for wind or solar pV
6. Gasifier controller
7. CHP controller - engine start/stop, rpm, voltage monitor etc
8. General purpose control tasks
9. Temperature, climate and weather monitoring and datalogging
10. General purpose energy monitor for performing home energy surveys

Navitrino as a concept was intended to be as flexible as possible, so if a new bit of equipment or technique comes available, Navitrino can be extended and updated to cater for the upgraded system.

Navitrino is based on a nework of interconnected nodes. Whilst one device might be controlling solar and domestic water heating, another device may be used to monitor and log the household electricity and gas usage. Devices can share data with one another using a simple serial data format which is compatible with both wired and wireless networks.

Navitrino uses low cost, off the shelf, open source Arduino hardware as the basis of the project.
The Freeduino and accompanying sensor shield from NuElectronics, which has real time clock and SDcard interface provides the hardware.

Alternatively, if the final intention is to incorporate a Zigbee wireless network the "Stalker" board from Seeeduino Labs in Hong Kong also makes a ready to run platform.

Central Heating Controller.

During the summer months I have been slowly getting together all the necessary elements of the Navitrino project; sourcing the hardware, building up the sensors and writing some rudimentary code routines to exercise the hardware.

As we have now officially entered the UK heating season, one of the first projects for Navitrino is to monitor and control my central heating system.

I have mentioned this in an earlier post about using a DIY controller to monitor and control the home central heating system. Now is the time to put these ideas into action.

The task is fairly simple, the controller will monitor the living room temperature using a thermistor temperature sensor, and turn the central heating boiler on and off in order to maintain the living room temperature at a level as set by a rotary potentiometer.

In addition to the room temerature probe and the set-point control, the Navitrino also monitors outside temperature, the hot water tank temperature (2 places) and a light sensor placed in the living room. The idea of the light sensor is that it can be used to determine the living room occupancy - when the lights are switched off at bedtime, the heating will either shut down or revert to a lower temperature setting.

The controller also incorporates an interrupt driven pulse counter, which allows it to count pulse data from a gas or electricity meter - so that home energy consumption can be monitored. I am particularly interested in gas consumption and being able to relate it to central heating usage and outside temperatures.

Over the last few days, the prototype Navitrino has been controlling the central heating, datalogging temperatures and counting pulses from the gas meter.

Originally, I used an opto reflective sensor to detect the "silvered zero" passing by on the least significant digit wheel of the gas meter index. Whilst this initially showed promise, I found that the rising and falling edges of the pulses were prone to "bounce" which caused false triggering of the pulse counter. I have subsequently replaced the opto-sensor with a Hall Effect magnetic sensor - because the same reflective zero also contains a small magnet, intended to trigger Hall sensors or reed relays.

Arduinoids - The Rise of the Machines

Back in July I was contacted by Nikki of fizzPop Hackspace in Birmingham about making very low cost Arduino clones on stripboard. I hadn't really given it much thought until I realised how easy it would be to make an Arduino compatible device on stripboard (or breadboard) for something less than a fiver!

I've tried to standardise the design, to make use of the ATmega328 pin-out - which lends itself for an efficient layout.
Here's a couple of recent boards - one is a controller for a spark ignition system, and the other is a general purpose layout - just the Arduino in the corner of a vast expanse of prototyping board.

Below is the prototype spark ignition controller. It is based on the Atmel ATmega328 microcontroller and uses the standard FTDI USB to serial cable as a means of programming it.

The schematic is here: Lister_spark_IC.pdf

The following I/O pins were used

Pin 2 Serial Rx Input from PC

Pin 3 Serial Tx Output to PC

Pin 4 Digital Input from Hall Effect sensor (Arduino Digital Input 2)

Pin 14 Digital Output to trigger power transistor (Arduino Digital Output 8).

Pin 19 Digital OUtput to Drive LED (Arduino Digital Output 13).

The board is clocked with a 16MHz crystal to retain compatability with Arduino. The firmware is developed as normal using the Arduino IDE.

The board has a 5V regulator, a rotary pot to delay the spark and a blue LED to flash in time with the firing of the power transistor. The TIP122 transistor is mounted on a small heatsink. Connections to +12V battery, ground and the low tension to the ignition coil are provided on 1/4" automotive spade terminals. The 6 way cable exiting to the top left is an FTDI USB to serial cable. This provides +5V for testing and the serial interface for programming the board and getting rpm and ignition delay data back to the laptop.

The 3 pin Hall sensor plugs into the pin-socket just below the FTDI cable header - it has the same pin order as the Honeywell sensor +5V, 0V and signal out. For testing the sensor device can be plugged straight into this connector, and later extended with a 3 way cable.

The board measures 3" x 4.5" and there is space below the heatsink to take additional circuitry. You will need bit of stripboard at least 28 holes by 32 tracks to copy this layout.

Only 4 of the I/O pins are being used in this application, leaving the remainder for other future use.

One possibility is to add a further pair of TIP122 transistors, so that this board could form the basis of the spark controller for the 3 cylinder Kubota engine. If additional TIP122 transistors are added, they need to be isolated from each other if used on a common heatsink.

From the top the spade connectors are +12V, battery ground and coil output.

As a general point, the pin-out of the Arduino (see below) and ATmega328 maps very easily onto breadboard or stripboard. This means that small Arduino compatible projects like this can be made on stripboard or even breadboard at a fraction of the cost of buying the real Arduino and building a board onto it.

Buildtime was approximately 5 hours - but that included a lot of thinking time to get the component positioning sensible. To copy this board would take a couple of hours.

The second board is built along similar lines. At first it's just a microcontroller in the top corner of the board.

Here's the Partslist for the simple Arduino clone

1 ATmega328 microcontroller
1 28 0.3" pin DIL socket
1 7805 5V voltage regulator
1 16MHz HC49-4 crystal
2 10K resistors
2 330R resistors
1 1K resistor
2 22pF ceramic capacitors
4 100nF ceramic capacitors
2 22uF 16V electrolytc capacitors
1 1N4001 diode
1 LED - to show that life exists!
1 Stripboard at least 28 holes x 20 strips
1 6 pin 0.1" header
1 optional reset switch
Additional 5 and 6 pin 0.1" SIL sockets for connecting external I/O
single core hook-up wire - 4 colours used

1 FTDI cable for programming (optional) (ATmega328 can be programmed on Arduino board and swapped across).

The following additional parts were used on the spark ignition controller project.

1 10K rotary pot or trimmer pot
1 Hall sensor (Honeywell SS441R)
1 neodymium disc magnet
1 M3 (or equiv) bolt and nut for fixing heatsink
1 3 pin 0.1" socket
3 0.25" pcb spade connectors
1 TIP122 darlington power transistor
1 Heatsink

Datalogging & file storage with OpenLog

OpenLog is a an open source datalogger developed by Nathan Seidle of Spark Fun Electronics.

It consists of an ATmega328 microcontroller interfaced to a SDcard or microSD card. It retains a high degree of compatability with Arduino hardware - and indeed, early prototypes were developed by adding a shield with an SD card socket to an Arduino.

OpenLog is a serial datalogger. It does exactly what it says on the tin, and records all serial data that it sees on it's serial Rx pin. It is compatible with FAT16 and latterly FAT32, and so can work with SD cards up to 2Gbyte and 16Gbyte respectively.

It has a fairly simple set of commands which allow you to read, write and append the files on the card, plus traditional DOS-like commands for changing directory, deleting files etc.

As part of the Arduino Slave Network project, I thought it would be neat to port OpenLog to a suitable serial slave, and have it act as a central fileserver for the rest of the network. As the slaves are somewhat limited in their RAM resource, and cannot hold much data locally, having a central storage facility will be an asset to the system.

In this way, slaves with sensors connected could record perhaps a hour's worth of data and then write it out to the communications network, where it could be loaded into the RAM buffer on the Master or Hub unit - but more importantly stored as a CSV file on the OpenLog slave.

With this in mind the first task was to make an OpenLog and get it working. I have a NuElectronics "Sensor" shield, which has an SDcard socket and a Dallas RTC on board. Most of the I/O lines on the NuElectronics board were directly compatible with those in the OpenLog design, apart for the chip select line, which is on Digital 5, rather than Digital 10 on OpenLog - which used D 5 for running a status LED.

The quick solution was to modify the OpenLog code to move the status LED down to Digital 2, and then set the pin mode of Digital 5 to be an input. I then hooked a wire jumper from Digital 10 to Digital 5 - so that D10 still selected the SD card, and D5 remained a high impedance input.
To test OpenLog, you need a terminal emulator programme. Microsoft no longer include a terminal with their bundled software, so you have to find one that works. I eventually chose Tera Term 4.67 which seems to work well. I had no success with Termite or the Arduino Terminal window - neither of which appear to send CRTL z correctly.

OpenLog Storage Control.

To access data from the Slave RAM, there will have to be a command from the Master which requests the slave to write a block of data. The Master can also be used to create the filenames for the slave data to be stored and put the OpenLog into append mode at the right time to receive the slave data.

Suppose we want to create a file called temperature1.csv to accept temperature data from Slave 1.

The Master will have created this filename using the new command:

new temperature1.txt

Then before the request to the slave for data, it should issue an append command:

append temperature1.txt

Followed by the command to slave 1 to send its RAM contents eg. write out the first 1024 16bit integers from RAM:

w,1,0,1023

The slave will then write these as comma separated integer variables finishing with a ? to inform the Master that it has finished the command process. The Master then needs to send the escape sequence to the OpenLog so that the logging to temperature1.csv is closed.

The escape sequence could always be incorporated into the end of the data issued by the slave - this would then need no intervention from the Master to close the appended file.

Applying the Slaves

The Arduino Slave is designed to be simple to make and very low cost. Just an ATmega328 microcontroller, a 16MHz crystal and a reset circuit. In addition a 74AHC125 tristate buffer is used to control access to the serial network and ack as a buffer and line driver. One digital output line is needed to control the tristate buffer - leaving all of the remaining I/O lines for the Slave application.

The primary function of the slaves is for remote distributed monitoring and control tasks. Slaves are used to monitor changes in physical quantities - such as temperature and then perform some related control task. If the role of the slave is merely for monitoring a changing quantity - such as gas meter pulse counting, then the Slave acts as a small part of a multi-channel datalogger.

The Slaves are intended to run semi-autonomously. For example if a Slave is running a room temperature monitoring and control application, it is expected to do this without regular updates from the hub. It will continue to maintain the temperature set point and control the radiator valve accordingly until it receives an update to the set point form the Hub.

RAM resources on the '328 microcontroller are limited to 2K, of which about 1.5K will be allocated as a buffer to allow the slave to record up to 1 hour of data, and then feed this forward to the hub in the form of a CSV file sent across the network. Central to this store and feedforward mechanism working is the implementation of the OpenLog opensource datalogger on the Hub. A slave could take temperature readings every 6 seconds for an hour, package them up into a CSV file and then output them to the network in about 5 seconds at 9600 baud - in the time between taking consecutive temperature readings.

The datastreams will be organised as channels. A channel is a series of sampled readings taken from a time variant physical variable - for example "living room temperature", "electricity instantaneous power", "solar panel power output". A channel can be named, logged and then passed up to the Hub or laptop for subsequent further processing, graphing or display.

Towards the Networked Home

In the last few weeks I have been putting together the various tools I will need to make a distributed Home Monitoring Network. Previous posts have discussed most of the basics of this project, here I thought I'd summarise the project and what we are going to do with it over the next few months.

In essence, the network consists of several Arduino Slaves, all connected together on a simple wired serial network and communicating with a Master "Hub" device.

I'm doing this in conjunction with my friends at OpenEnergy with the intention of providing a demonstration system of what can be achieved with low cost hardware and a simple wired network. We are making the most use of open source hardware and software, and by keeping the system simple, we hope that it will be of interest to others. Later on, we might extend it to include some wireless nodes, but for now we want to keep the costs and complexity down.

Central to every node on our network is a small microcontroller board, based on the ATmega328 microcontroller, and firmware produced using the Arduino IDE. Keeping things compatible with Arduino hardware and firmware will ensure that others can try out our methods with minimum of fuss. At a bare minimum, our network could be used to connect an Arduino board to a laptop, so that it can be individually controlled over a few metres of cable by simple serial commands typed into a serial terminal emulator.

To extend the system beyond 1 board, some simple additional hardware is needed to ensure that only one Arduino Slave can access the network at a time. This involves fitting a 74AHC125 quad tristate buffer to each slave, such that only one will drive the network at a time. This device costs about 20p and is easy to add to an existing Arduino. The 74AHC125 and a 4 way screw terminal connector could be fitted to a protoshield board.

I've been programming my bootloader into new ATmega328 chips using another Arduino as a programmer, and in this way it keeps the cost of the Slave down. The basic Slave can be built for less than £5 which means that having several slaves on a network is still inexpensive.

Slaves send and receive data at 9600baud as simple comma separated variable (CSV) commands. The command set is compact, relatively easy to remember and can be typed from a serial terminal programme. The use of a CSV command means that commands can be readily interpreted using simple command interpreter code, and a low overall overhead to the application sketch. Additional commands are easy to write and can be added as the application needs them. Further details of the command set and application sketches will appear in a later post.

So with the basics of a wired network in place, it's time to start applying it to a real household application. In Snowdonia we worked on a monitoring and control system for a small solar pV installation, but at a planning meeting at the start of the weekend we discussed all the applications which could easily be handled with such a monitoring and control network. Here's a few suggestions in a non-exhaustive list for domestic and renewable energy applications:

1. Room by room, central heating controller
2. Gas consumption monitor
3. Whole house electricity monitor
4. Solar water heating controller
5. Solar Photovoltaic controller
6. Wind turbine controller
7. Battery Bank Management System
8. Generator controller
9. CHP controller
10. Greenhouse monitor/plant irrigation system

Bootloading Arduinos

What I really love about the Arduino is that it is so accessible and comes with all the utilities to make it work. For example, it takes just minutes for a Newcomer to connect up an Arduino and flash a LED. I think this accessability and ease of use across all the main platforms is one of it's key attractions - and something that other vendors of microcontroller kit have failed to appeciate or capitalise upon.

Once you gain more experience of programming, you can choose to move away from the "safety net" offered by the Arduino IDE, and use the GCC compiler and AVRdude tools directly. However for most simple applications there is no problem just using the Arduino flavour of the C language - provided you don't want to do anything too fast or clever.

One of the nice features is the ability to use an Arduino to program the bootloader into a new IC. Whilst pre-loaded chips are available from a number of suppliers - they charge a premium for these - and if you are making a lot of Arduino based devices - it's much cheaper to program your own - using the easy to use "Burn Bootloader" tool that comes as standard with the Arduino IDE. We used this for the first time on the Snowdonia Mega-Hack Session and found it to be quick and reliable.

Using this utility, I programmed up 10 ATmega328 ICs with the Arduino bootloader in under 30 minutes, and for a cost of about £3.30 per chip (includes VAT and postage from CoolComponents). This saved me approximately £2.90 per chip! CoolComponents are located near Clapham Junction Station and have keen prices on ATmega328s and have a very quick turnaround - 24hrs.

To make use of this Arduino ISP facility, you either have to have 2 Arduino boards, or 1 Arduino and a small breadboard. I used the breadboard method as described in the Arduino Tutorial: Using the Arduino as an ISP

In order to make this device, you need to make up a clock crystal and reset circuit for a ATmega328 on a small breadboard and connect up the Vcc and Gnd pins. Then you use the existing Arduino board and connect Digitals 10 - 13 across to your target '328. This brings across MOSI, MISO, SCK and Digital 10 which is used to reset the target processor. You also need a feed of +5V and 0V from the Arduino board to power your target.

David A. Mellis's programming application also uses 3 other port lines for driving LED indicators. Digital 7 flashes to show that programming is underway, Digital 8 will light up an LED if there is an error and Digital 9 is a Heartbeat to show that the ISP application is running. You don't have to fit these LEDs - but if you are doing this for the first time, it's a good confidence booster that all is well.

During programming the Rx and Tx LEDs on the Arduino board flash rapidly as the serial data is sent from the PC. The LED on digital 13 also flashes quite quickly. At the end of the programming session, the target microcontroller is reset and starts to execute a LED flashing routine on digital 13 - which shows up as a faint slow flash on the programming Arduino LED. This confirms that the bootloader has been programmed and that it is running code. This is a neat feature, as when it comes to use these bootloaded ICs in DIY Arduinos, the LED flashing code will still be present to tell you that the processor is running.

The picture above shows my bootloading breadboard - not the tidiest of layouts, but works fine. There are just 6 wires to connect across from the Arduino, or 7 if you want the Error LED. The breadboard has a 16MHz crystal, two 22pF loading capacitors and a 10K resistor on the reset line. If you are actively building DIY Arduino hardware, you will have all these components to hand anyway!

Now that I'm boodloading my own mega328 chips I think that the cost of a DIY Arduino Slave has dropped to about £4.50 a unit.

Solar Stuff

Micro-inverters from Enphase Energy

A small inverter fitted directly to the back of the solar pV panel - one per panel.

Performs Maximum Power Point Tracking (MPPT) on a panel by panel basis, leading to higher overall system efficiency and compensating for panels that might be partially shaded.

Enphase also do a neat little display which shows you solar panel and room thermostat data. It would certainly be a good idea to be able to monitor all your household energy systems from a common display. These devices communicate via Powerline Carrier (PLC).

Perhaps it would be possible to "roll your own" display using the 2.8" colour touch screen shield from NuElectronics.

Autumn Almanac

It's been a short week what with the Bank Holiday Monday, and my new employer's policy of closing the firm at lunchtime on Fridays. With us rapidly moving into the Autumnal month of September, here's a round-up of some of the things I have been inspired with this week.

New Toys.

I've been tinkering with a new Texas Instruments DSP dev-board at work, which will become part of a dc motor control system. I first used TI DSPs back in the late 1980s with the TMS320 series, and wow have they come on in performance since then.

The dev-board is based around their "Delfino" DSP which is a 32bit floating point DSP capable of 150MHz clock speeds. It has 512K of program space, 68K of RAM and 64 general purpose I/O pins - 18 of which can be PWM for driving power stages. It also has 16, 12 bit ADC inputs, which can be run in two banks of 8, for making simultaneous measurements - such as voltage and current, for calculating instantaneous power.

The Delfino is mounted on a small strip of PCB known as a "control card", which brings out 100 pins in a common footprint. This makes it easy to swap control cards for any of the DSPs in the TI range, and ensure functional pin compatability - a neat idea. It removes the user from the difficulties of handling 176 pin ball grid arrays (BGAs), and it means that you can develop your application on one of the larger parts and then move down to a smaller, more cost effective part, later, when the code is finalised.

Putting the processing elements onto a common footprint is a great way of taming what would otherwise be a difficult technology. The raw technology is grouped into a series of manageable ports or interfaces, and the user is presented with a simplified programming model.

On a simpler level, - this is exactly what the Arduino does. It simplifies the microcontroller's interfaces into a series of easily understood pin functions, and presents an extra programming layer into the code which divorces the new user from the complexities of manipulating I/O at a register level. This makes the technology much more accessible to the new user, but at the expense of a speed reduction in I/O operations - compared to direct I/O register manipulation.

When the User becomes more experienced in programming, then the "stabilisers" can be removed. Learning to programme an Arduino in C will hopefully produce a whole new generation of embedded programmers - who have the confidence to get "down and dirty" to the hardware level. This is something that has perhaps been lacking in recent years, as coders generally develop applications in high level languages on sophisticated platforms such as laptops and mobile phones.

Networked Arduinos

A week after the Snowdonia Mega-Hack session, the dust is starting to settle on the networked Arduino project. Trystan has been developing firmware so that he can use the networked slave as part of his solar photovoltaic monitoring system. This was chosen as an example application as something that would illustrate the capabilities of the network slave as a means of monitoring pV panel volts, amps and watts and controlling a battery charging system.

The low cost serially connected slave would make an ideal candidate for a battery monitoring system (BMS). Using optically isolated serial comms, one slave could be applied to each 12V battery, in a larger battery bank or an electric vehicle battery. The low power requirements of each slave would allow it to be powered from the battery it was monitoring - with its I/O referenced to the 0V terminal of each battery.

Data Logging.

Another neat application which has come to light is the OpenLog Data Logger This is an ATmega328 interfaced to a 2Gbyte SD card. It has a simple serial interface, and it just sits an logs all the serial data it sees on its RxI pin. The source code and schematics are all open source, allowing this application to be easily incorporated into some home-made hardware.

One idea would be to use it to record "both sides" of the data on the Arduino serial network. The slaves could be interrogated at regular intervals for stored data, which they transmit to the network, and the OpenLog stores this data away to flash in a large text file. OpenLog comes with a small but neat collection of commands allowing data to be retrieved, files to be appended and erased. This seems like an easy method to get data from the slaves into permanent storage, from where it can be accessed by laptop, sent via ethernet to the web, or whatever.

2Gbytes is a huge amount of storage. Even sending continuous data at 9600baud, it would take more than 25 days to fill the SD card.

Central Storage and Real Time

The Arduino slaves are lacking in vast amounts of local storage. There is 2K of RAM on the ATmega328 microcontroller, and if just over half of that was available for data storage, it could hold about 600, 16 bit readings. Using a store and feed-forward method, the slave could hold say an hour's worth of instantaneous power readings, taken at 6 second intervals and then send the whole file up to the central storage. At 9600baud, this data transfer would take about 4 seconds - easy enough to do once every hour between taking sensor readings.

The problems of limited storage have been with us for much of the last 60 years of computing. Certainly until the late 1980s, memory was an expensive resource, and computer engineers found ways of connecting many users to the one central mainframe and sharing its memory resources between several users.

This is exactly the same situation we have with our network of distributed Arduino slaves - albeit on a very much smaller scale. We have to find a clever way of sharing central resources, such as file storage, realtime clock and ethernet connection between the slaves. By storing the slave data in RAM and then forwarding it to the central SDcard at regular intervals would provide a sensible compromise to storing data, without over-complicating the slaves with their own additional storage. The feed forward model could also be applied to the Master device - collecting data from the slaves storing in SD card and then forwarding this data to a laptop, or web connected server. This would give the Arduino network a degree of autonomy and not reliant on having a PC permanently switched on to service the data.

The NuElectronics Sensor Shield comes with RTC and SDcard for data storage. With a bit of juggling it should be possible to implement the OpenLog firmware onto this shield. The RTC timestamp could be distributed as a CSV packet across the wired network at regular intervals - so that slaves could synchronise their local time to the realtime.


CurrentCost Channels and Pachube Feeds.

One idea, starting to "bubble to the surface" is the concept of having a number of data channels which can be monitored. This is something which the CurrentCost energy monitors have implemented, where originally they had a separate channel for the energy monitored on each of the three phases of an electricity supply. The LCD display would allow basic channel information to be viewed.

The concept was extended to allow data from Individual Applance Monitors (IAMs), to present the energy used by an appliance such as washing machine or fridge to be given their own channel. The CC128 display unit was given a capacity of 10 channels, with the user being able to step between them using a button on the front panel. This approach allows other sensors to be added later to enhance a sensing network - and with the Current Cost DevBoard, enthusiasts can add their own sensors, linking back to the CC128 via their proprietry wireless link. Thanks to @jtonline and @yellowpark for letting me know about the CurrentCost Technical Blog. -in which Chris Dalby, a developer at CurrentCost introduces some of their technology to the Homecamp/Hacker community.

Incidently - for those interested in wireless networks, JeeLabs has introduced a kit for a wireless JeeNode. Its based on the ubiquitous ATmega328 and a wireless transceiver module. It presents 4 identical "ports" to which various sensors or devices can be attached. It's all open source and based on Arduino technology. At 17.50 Euros it seems good value for money, and the wireless module used is the same one which CurrentCost use, which to the experienced coder should open up some possibilites for integrating the two systems.

So if we consider any source of data, such as electricity instantaneous power, or room temperature, or solar panel output could be expressed as a data channel. The CurrentCost display allows basic monitoring of channels using its restricted LCD, but this could be extended to an application running on an O2 Joggler display or a laptop etc. With an application running on a laptop, iPhone or via the web, would give enhanced monitoring options such as historical graphing, comparing channels or using them for further processing or other applications such as Pachube - where several physical channels are combined together as a Pachube feed.

Monitoring your Heating System

A couple of months back, I had some ideas about Smarter Heating Controls - with the weather turning Autumnal, it's time to start thinking about these ideas again.

Here are a few tips and gizmos to help you put together a DIY boiler and heating monitor

Pipe Clip Thermistor Temperature sensors from Rapid Electronics

The Arduino code needed to linearise a thermistor and display it in degerees C

Wireless actuators for thermostatic radiator valves from Conrad Electronics

Low-cost pulse output water flow meter froom SeeedStudio Depot - these are otherwise stupidly expensive in the UK. Easily read using a pulse input on the Arduino.

September - Time to Reflect

July and August have been busy months, what with starting a new job in July and developing a whole bunch of Arduino ideas with the guys from openenergymonitor.org, Pachube and the London Hackspace.

I thought I'd summarise the project so far - and think about where it's all heading.

Low Cost, Arduino Compatible Devices.

At the heart of this project is the popular, open source hardware Arduino microcontroller board. An 8-bit microcontroller with clock circuit and programming interface, which can be programmed in a C like language from an IDE which runs on Windows, Linux and Mac platforms. The Arduino and its derivatives have become very popular amongst developers, hackers and other creative communities. Whilst an Arduino costs about £12.50 from a number of vendors, the hardware is trivial and easily built on stripboard or breadboard for about £5 in components. Such was the attraction of having a number of low cost Arduinos, that in early August I considered the notion of DIY Arduinos having seen some examples produced by the FizzPop hackspace in Birmingham.

The Internet of Things.

At the end of July, I attended a MiniBar event in London discussing the Internet of Things. For some time I had an interest in the idea of a number of low cost smart sensors, all connected to the Internet and able to communicate with applications and each other. The Arduino already had an ethernet shield available, so I wondered just how cheap you could make a home-made, web connected Arduino. This led on to the "ether-duino" project - an Arduino compatible web server for £12. Whilst the hardware was relatively easy to build on a breadboard, the ethernet library to control the ethernet controller IC was a bit out of my depth, so I was delighted to receive assistance from Andrew Lindsay and John Crouchley - both of whom had worked with this controller and had some example sketches available.

Low Cost Web Connectivity.

For some time, I had been thinking on how these network connected microcontrollers would communicate with each other, and also with the internet, so by way of experiment I developed a serial command interpreter which allowed Arduino compatible hardware to be controlled with simple serial commands. It was not much of a step on from this to realise that if these serial commands could be passed across the internet or other network then it would be possible to get these network connected devices to talk to each other. Smart sensors could pass data and commands between similar devices across the wired network and the internet. So I came up with a simple method by which the web-enabled Arduinos could pass simple messages between each other using Pachube as a message broking service. This bit of experimental work came to fruition in the third week of August when I finally got it together and succeeded in getting one web enabled Arduino to control a LED on another similarly web connected device.

A Smart Sensor Network

Having succeeded to make a breadboard version, to prove the concept of the web connected Arduino, I worked on a pcb version, done in conjunction with the London Hackspace. These would be inexpensive boards made from conventional components which anyone with basic soldering skills could put together. They would provide the basic circuit to connect to the internet and allow experimentation with internet connectivity for smart sensors. Whilst making the prototype, I realised that one of the components on the board, a tristate buffer, had two spare gates, and these would make an excellent means to extend the communication network beyond the web, and allow several slave sensor boards to be connected to a network radiating out from the web connected Master unit.

Building the Network.

With these thoughts in mind, I travelled up to Snowdonia, for a weekend build session with the lads from openenergy and it was here that we were to share the ideas and develop the hardware and firmware for the network connected Arduinos. We built up a total of 4 slave sensors, two with displays, and proved that we could send data and commands down 300m of network cable. We used one of the network sensors to monitor and control a solar pV installation and lead acid battery monitor.

The Next Steps.

So in just a few weeks we have developed a system of low cost network sensors which can be accessed through the internet and send commands and data to each other. From the outset this project was to be experimental, and make use of readily available low cost components and be easy to build by anyone interested in the Arduino Project.

The techniques that have been used may not be optimal in terms of performance, but they are easy to follow, by anyone wishing to build something similar. Now that the fundamentals have been proven, it is time to make up a batch of low cost pcbs, which will support both the web connected Master, or the simpler network Slave. With the infusion of low cost hardware into the hacker community, plus simple firmware to control it, then hopefully the project will soon gain momentum.

A Wired Network for Arduinos - Snowdon Net

Over the course of the Bank Holiday Weekend, we took off to the remote hills of Snowdonia with the purpose of developing a simple serial communications network for the Arduino based hardware using the ATMega328.

The emphasis was on simple. It had to be something that any Arduino enthusiast could wire up on a breadboard and get working, and it had to be low cost. I accept that there are better network drivers such as RS485, and a whole host of protocols, but as this was to be partly an educational exercise, as well as a bit of fun, the methods used had to be clear and easy to learn.

In the previous post I introduced the design. Now I share some of the more specific detail of the implementation. A later post will cover the programming and command specification.

Background - the need for a low cost sensor network.

The openenergy project is all about finding cost effective ways to monitor and control renewable and domestic energy systems, such as solar water heating, photovoltaics, electricity and gas monitoring.

We had realised that with very low cost microcontroller hardware, we could afford to use a micro at each sensor node, to make a network of distributed smart sensors. The Arduino project has given a great opportunity to make low cost hardware for as little as £5 per device. The next logical step was to find a cost effective means of connecting such devices together in a network, co-ordinated by a Master unit. Sensors could be interrogated by the Master, and then with a web-enabled Master, the sensor data could be accessed remotely via the internet - using a service such as Pachube.

The system was intended to be cheap, and easy to build for the sort of people who tinker with Arduinos. We could have used a more elaborate system based on RS485, but in order to introduce beginners to networking devices, a very simple signalling method was devised. It uses a single 74AHC125 quad tristate buffer as a line driver, and 5.08mm pitch screw terminals for connecting the network cable - so no fancy connectors or crimping tools are needed to make a connection. It is designed in a way to be compatible with stripboard or built on a low cost protoshield. Indeed, on stripboard, a slave Arduino can be built for under £5 including the ATmega328. The first prototypes were built on breadboards, so that they could be readily modified as new ideas were tried out.

The weekend consisted of building experimental hardware, debugging it and writing firmware to get it running. It was in many ways a learning exercise, as each step of the design we had to work out from scratch. During the course of the weekend, we learned a lot about networking microcontrollers and devising simple control structures in firmware to get it to work as a system.

Day 4 Monday 30th August 2010.

We started with the basic stripped down Arduino consisting of ATmega328 and 16MHz crystal and added a 74AHC125 quad tristate line driver chip. The line driver allows the Arduino to communicate over several tens of metres of 4 core telephone or CAT5 network cable, and also allows an addressing scheme such that only one slave has access to the network at a time - to avoid conflicts.

We devised a simple serial protocol, which can be typed in at a terminal emulator, and using a FTDI cable, access any slave on the network and set up a one to one communications session. We also devised a sketch to run on a standard Arduino, such that it could act as a master and control several slaves. For the purpose of demonstrating the system we built up 3 slaves and connected them at 30m, 35m and 40m respectively away from the master.

The network consists of 4 wires: TX, Rx, +V and 0Vand we used some CAT5 UTP network cable we had available. It could equally work on standard 4 core telephone cable (CW1308 - about £12 for 100m). This wire has a resistance of about 97 ohms per km, and this dictates the voltage drop seen at the power terminals of the remote slaves. A 100m length will drop about a volt if the slave draws 100mA.

We standardised on the following colour scheme, which is loosely based on the ethernet standard but just using the solid colours - rather than the colour/white striped pair - which would be used if we were working with RS485 differential signals:

Brown +5V supply from regulated psu to power all slaves. We used a 2A 5V supply.
Blue 0V system and signal ground return line.
Orange Tx Transmit signal from Master to Slaves
Green Rx Receive signal from Slaves to Master

The use of orange and green for Tx and Rx is the same as ethernet colour coding

The aide-a-memoire for the colour scheme is that Orange is the Outward transmission from the Master.

Our DC power was put on the brown and blue wires. We chose brown for +5V, purely because in the UK we associate brown with live and blue with neutral. This might be in conflict with some of the Power over Ethernet recommendations - but it seemed logical to us at the time.


Above. An Arduino connected to the network using a quad buffer chip and driving a serial LCD.

The 74AHC125 can be thought of as our network access chip. It uses it's tristate buffers to either isolate or connect the Rx and Tx lines of the Slave chip to the network. In order to control the network, and programming operations, we provisionally reserved 3 digital output lines:

Digital 5 /Tx Tristate Enable Setting this low connects the Slave Tx to the network
Digital 6 /Rx Tristate Enable Setting this low connects the Slave Rx to the network
Digital 7 /Autoreset Setting this low causes the Slave to reset itself and allow over network re-programming.

For the simplest network implementation only digital output 5 is needed.

Below: The network chip and the 4 way screw terminal block assembled on a small breadboard to add the network to an Arduino. These parts could be assembled on a protoshield or just a bit of 0.1" stripboard. The basic network uses Rx, Tx and digital output 5 to control the Tx tristate buffer.


We found that connecting digital 5 to an LED connected through a resistor to +5V made a good indicator - lighting up when each Slave had been selected and was communicating with the bus.

If the anode of the LED is connected to the Tx pin rather than +5V, the LED will come on when selected and flicker whe the slave is sending data to the network. A similar LED on the Rx line shows when there is commands or traffic from the master on the Rx bus.

Connecting the 74AHC125

Pin 1 Tx Enable Connect to Digital 5
Pin 2 Tx Connect to Digital 1 (Tx)
Pin 3 Output Connect to green network wire
Pin 4 Rx Enable Connect to Digital 6 (or 0V on simple system)
Pin 5 Input Connect to orange network wire
Pin 6 Rx Connect to Digital 0 (Rx)
Pin 7 Ov Connect to oV ground (blue wire)
Pin 14 +5V Connect to +5V (brown wire)

Pins 8 - 13 of the 74AHC125 remain unconnected, but could be used for other buffering tasks.

If the 74AHC125 is mounted on a small piece of stripboard or protoshield, a 4 way 5.08mm pitch screw terminal block can be used as the connection to the network. It can be inserted next to the 74AHC125 such that it connects directly to pins 3, 5 and 7 leaving only +5V to be connected to the remaining pin.


In the afternoon we decided to prove the distance over which we could send data. We had a drum of 250 metres of CAT5 UTP (unshielded twisted pair) network cable, which we quickly added to our experimental network and hastened together another display slave.

After a quick reprogram of the Master we now had a master and 4 slaves on the network and a total cable distance approaching 300 metres. The Master could successfully transmit and receive data from all of the slaves and display data on a remote LCD connected to the slave that was 300m away from the master.

It would be a relatively simple matter to enhance the network to use differential signalling - such as is used in RS-485. Instead of the 74AHC125 driver chip - a couple of MAX485 (or similar) differential line drivers could be used. Taking this approach would allow the Arduino to be used as a DMX (lighting) controller. However, we chose to remain with simple TTL level serial - as this is the native signalling format used by the Arduino, adequate for our purposes - and it is easy to look at and debug with an FTDI cable to monitor the TX and Rx lines.

The first application for the networked devices was to control a photovoltaic monitoring system. This unit measures the voltage and current produced by a pair of 75W solar pV panels and monitors the charge into a 12V deep cycle lead acid battery. If the battery reaches a full state of charge, a dump load consisting of up to 5, relay selected halogen 12V bulbs could be turned on. Trystan wrote some firmware to obtain voltas and amps readings from the solar controller and send them to the Master device, and a programme running on the Master determined when the dump load relays should be activated.

All in all, a very successful and useful addition to the openenergy.org project.

The Snowdonia Mega Hack Group are (left to right)

Suneil Tagore, Trystan Lea, Ken Boak and Glyn Hudson.

Snowdonia Weekend Hack Session

Friday 27th August - The Mission Begins.

Your Mission (should you choose to accept it) - to develop from scratch, a network of serial slaves controlled from a Master Arduino and to use it to monitor a photovoltaic renewable energy system.

Time Allowed 3 days.

Personnel:

Trystan Lea - Chief Coder and host
Suneil Tagore - Renewables Consultant
Ken Boak - Hardware builder and tea maker

Resources:

10 Arduinos of various types - official, Freeduinos and DIY breadboarded units
2 NuElectronics Ethernet shields
1 Etherduino - a DIY ethernet compatible clone built on breadboard.
Box of useful electronic component bits
2 x 75W solar pV panels
Hall effect sensors for current sensing
110Ah Deep cycle battery
200W 120V Inverter
500W UPS found in skip - cruddy modified sine wave
250m drum of ethernet cable
Lots of tea and firewood

Location: Secret Snowdonian Bothy - outside the Trawsfynydd Nuclear Plant Fall Out Zone

Over the course of the August Bank Holiday I was invited up to North Wales to join Trystan Lea and Suneil Tagore, of openenergymonitor.org for an extended weekend build session with the aim of further developing the openenergy project. Trystan had developed a number of Arduino based control systems and was looking for a way to connect them together as a network.




I had first met Trystan and Suneil back at the April 2009 Homecamp Un-conference, and was impressed with their open source energy monitor based on the Arduino. We had kept in close touch over the last year and the plan was to pull our collective ideas together over the course of the weekend and develop some of the concepts into a working system.

I had recently been working on "Etherduino" a web-enable compatible Arduino built on breadboard, and I had a few ideas on how we could put together a network consisting of a Master and several Slaves - all connected with a 9600 baud serial wired network.

The openenergy devices are a series of Arduino based units that can be applied to electricity monitoring, controlling solar water heating, gas meter monitoring and other purposes related to energy conservation and renewables. Trystan had already developed the electricity monitor and the solar water heating pump controller, but wanted to tie them together with a simple remote display so that various paramenters could be viewed from a central display - for convenience located in the kitchen.

I had donated a couple of solar panels to the project, and we decided a good use of the network was to use it to extend the openenergy system to monitor a small pV system and control an inverter and battery system.

So with all these individual elements working in isolation a plan was devised to develop a low cost, simple means of connecting them together in a network so that they could be monitored and controlled from one central point by a master device.

The Concept Emerges.

The Arduino, and many other microcontroller based devices,generally communicate via a serial interface, running at TTL levels and modest baudrates. On the Arduino, the standard is 9600 baud, and this makes communication with a laptop easy using a FTDI serial to USB cable.

We had decided that because the cost of adding a microcontroller slave to a system was in the order of £5 per node, then it was cheap and easy to compose a system of a series of distributed nodes all communicating through a network.

We agreed that the system of simple slave boards connected by a wired network would be easiest and cheapest to implement, and as the Arduino is fundamentally a device with serial input and output and a serial programming interface we thought that we would stick to a simple text based serial interface running at 9600 baud 8,n,1.

In oder to make the network easy to build, test and modify, it was decided to keep the logic levels as those of the Arduino, and not introduce any further complication such as differential signalling (eg RS485). In essence, one Arduino can communicate with another just by cross-connecting their Rx and Tx pins with a cable and providing a common signal ground between the 0V GND pins.

We realised that for the network to be practical it should allow several devices to be connected to the one network, and there should be some method to prevent them all trying to access the network at the same time - which would lead to very corrupted data and a breakdown in communications.

We also wanted to distribute dc power down a spare wire of the network, so that this could be used to power the microcontroller and sensors at the far end. As the Arduino is fundamentally a 5V part, we wanted to restrict the voltage on the bus to +5V so that any accidents in wiring would not result in damaged ICs.

Finally we needed a method to prevent more than one device accessing the the Tx line at the same time. Thus a 4-wire network was selected with the signals being Tx,Rx 0V and +5V. This made the use of cheap telephone cable or CAT 5 network cable quite practical.

The next question was how far could we reliably send 9600 baud down a telephone extension cable and get uncorrupted data at the far end? After a few beers on Friday night we did some simple tests using a 30m length of 4 core cable connected to and Arduino and found that 9600 baud was quite happy over that distance. With this proven we were ready to specify, design and build the full network.

Day Two - Saturday 28th August

On the Saturday morning I built up a couple of slave Arduinos on breadboards - and because they were fitted with displays, we decided to call them Disciples. The plan was build up identical hardware and document it, so that I could take a unit back to London and one could be kept in Wales - so that both Trystan and I were working with the same platform.

The Disciple consisted of the ATmega328, a 16MHz clock circuit and a header to allow an FTDI cable to be plugged in for programming. In addition, a 74AHC125 quad tristate buffer was fitted which allowed us to drive a cable network, with only one slave transmitting to the bus at a time. It was later decided that we would also use another '125 buffer on the Rx line of the slave, to improve the signal from the network and to allow the slave Rx pin to be isolated from the network for local programming operations.

By early evening on the Saturday we had managed to connect two Disciple slaves to the network and Trystan had written a sketch which accessed them individually form the Master and sent a command to flash an LED. Very soon we had both slaves "ping-ponging" their LEDs on and off under the complete control of the Master Arduino.

Day 3 - Sunday August 29th.

Not such an early start because we were tired from the previous mega-hack session. The aim for the day was to incorporate the slave code into a photovoltaic solar panel and battery monitoring system.

Much of the morning was spent wiring up the pV monitor panel with two Hall-effect sensors - one for pV current and the other for load current. A bank of 5 relays controlled by the Arduino allowed 5, 12V lamps to be selectively turned on, thus putting a variable load on the system. this was a first step to developing a full maximum peak power point tracking system, which Suneil is working on.

By lunchtime the sun was out and we were starting to get a peak of about 10A at 14V, which was about all my 75W panels were capable of.

Trystan then started to merge the slave control sketch with the pV monitoring sketch - so we could get remote pV data - volts, amps and watts from the pV controller and send this data up to the master unit. The 74AHC125 tristate line driver was temporarily added to the pV controller on a separate breadboard, to be tidied up later once we had the system working.

Towards the end of the afternoon we had 3 slaves working on the network, one of which was the pV monitor. Fortunately this co-incided with a sunny spell and the pV panels were in full sunshine and being remotely monitored.



We then set about writing some code to send text to the Disciples (display slaves) such that the command packets send on the network would be displayed on the second line of the LCD, and any response from the Disciple would be echoed to line 4. Below is the display slave displaying a typical net packet. @,28 selects slave 28, and this is confirmed when the green "slave active" LED coming on - bottom right.




Shortly after that we went down the village pub for a well deserved pint (or three) and an evening meal. The weekend had been a great success and we had achieve all of our objectives - and there was still Monday left to tidy up a few loose ends.

Social Networking for Arduinos.

Now that I have got a nice, low cost, web-enabled Arduino (WEA) it’s time for it to get on line and develop a whole new social life.

Messages passed between Arduinos are “Tweet like” in their nature – and so I have coined the name micro-twit (micro tweet having been already taken) or u-twit to describe them.
Building on the Twitter model, an Arduino can follow a feed and can also have others following its feed. In the master slave relationship, where only the master is web enabled, then the slaves automatically become followers of the master (disciples?) and also followers of anything the master happens to be following – as the messages are transparently fed to the local network via the serial port.

In order to define a basic system, we must specify the exact nature and length of the messages. The command interpreter running on the Arduino has to decode the instruction and also have a knowledge of the context of the message. Unfortunately the Arduino would struggle to run a full XML decoder, so a lightweight version is needed to interpret the context of the data contained in the message.

The nature of the commands also has to be specified. Some will be hardware specific, such as setting port pins or PWM values or reading ADC values. Others will be more compound and depend on the application the Arduino is running, for example boiler control or electricity consumption monitoring.

Part of the command, the leading letter, is used to define the action routine associated with it. For human convenience and by way of mnemonic, this is given a letter to specify it. Another field of the command is used to define the destination of the command, in other words the sub address of the microcontroller which will ultimately execute the command. Addressing is also partly handled by the number of the feed to which you subscribe. A device which subscribes to feed 8729 for example is effectively on a given network, whilst a device which subscribes to feed 8730 is on another network. 8729 might handle all messages to do with room temperature control, boiler control and solar water heating, whilst 8730 is concerned with gas and electricity monitoring and logging events such as front door opening or doorbell ringing.

As the message persists on the Pachube feed until the next update, there will need to be some mechanism to inform the subscriber that it is a new message, and not lead to the situation where the subscriber continues to repeat the same action following each time it checks for updates. The message should therefore have a “message number” appended to it so that the subscriber can tell whether it is a new message or just the old one still there.

In order to put this lot together, I need to use my command interpreter to enter the message into the sending Arduino. This creates a comma separated text string which can then be loaded into the message buffer of the Pachube client. This will allow different commands to be entered by the laptop and send them up to the Pachube feed. This is now running and produces commands of a standard format: a leading letter followed by 5 comma separated arguments. If the arguments are left off, they default to zero. For example both b1 and b,1 will produce the output b,1,0,0,0,0

With the command interpreter running on one ethernet enabled arduino, I can type a command at the keyboard, have it echoed to the screen in standard format and then PUT up to my Pachube feed as a CSV string.

The second Arduino is running the GET code, such that it feeds the received text string into the command interpreter and also echoes it out to the serial port for viewing on the terminal or passing on to the local network.

With this all working, a command typed to one Arduino appears in the serial debug window of the second Arduino some seconds later. The reason for this latency is that there is an access limit on the Pachube API, and trying to accesss at anything less than 10 second intervals will cause an error and a warning message from Pachube!

A Pachube feed for this has been created here:

http://www.pachube.com/feeds/9675

Messaging with Pachube - micro twittering for Arduinos

Regulars will realise that I am busy finding ways of getting low cost web enabled Arduino based boards to talk to each other.

The solution that I stumbled upon is to use a Pachube feed as a message depository, and have other ethernet enabled Arduinos subscribe to this feed and take action based on the CSV data returned from the feed.

Suppose we have a command string B,5,100,200,50 which is used to set the brightness levels of a RGB LED to 100,200,and 50 respectively. The preceding letter "B" identifies this as a Brightness setting command and 5 is the sub-address of the Arduino board which we want to execute this command.

So a controlling, web enabled Arduino can publish the B,5,100,200,50 command to a Pachube feed in the form of a CSV string, and other web enabled Arduinos will subscribe to this feed, retrieve the CSV string, decode the command - and Arduino 5 will act on the message, setting its PWM outputs accordingly.

It's a fairly trivial example, but it does illustrate how a simple message can be passed from one controlling device to another using Pachube as a message brokerage service. The advantage of using Pachube is that it provides a certain level of security by way of it's API-key, and the low overhead of getting CSV data to and from Pachube makes the method applicable to the limited resources of a small microcontroller.

It occurred to me that this pub-sub model of messaging between micros is a scaled down version of what Twitter offers and the messages from the web enabled sensors are "Tweet like". Just to be topical, and to give it a name, I'm going to call this "micro twitting" with the message being known as a u-twit. (Homage to YouTube and indirectly my programming abilities). It was going to be called micro-tweet, but someone has already nabbed that name!

The contents of the message can either take the form of instructional and numerical command, or numerical sensor data. The exact nature and context of the message contents can be signified by a message type identifier.

A command interpreter running on the Arduino, identifies the message type, and any data content and acts accordingly. If in the case that the message is not intended for the web-connected master device, then it can be passed on, unchanged to the serial port and on to the local network of slaves.

Any Arduino hardware running a micro-twit interpreter can be connected to the serial bus and decode and act upon the message. It might be possible to use the system of hash-tags and @, as used in Twitter to identify subjects and publishers/users.

The pieces started falling into place with the assistance of John Crouchley's Pachube GET sketch implemented on an Ardunio with NuElectronics ethernet shield. John enthusiastically produced this at the drop of a hat, realising that I was wishing to do something similar.

I've now got John's sketch running on one Arduio/NuElectronics shield, printing out the CSV data sent to my Pachube feed by my ATmega328/ENC28J60 combo board.

Slowly the pieces are coming together :-)

Getting it Together

The various development threads of the web enabled smart sensors are now coming together into a convergent design. In previous posts I have discussed ethernet controller hardware, serial command interpreters and messaging methods all under the banner of smarter energy control. Now is the time to bring it all together into a tangible system.

The plan is to build a network of simple devices for home automation, monitoring and control – applicable to home heating, renewable energy and domestic energy monitoring.
The hardware is based on the ever-popular Arduino which consists of an ATmega328 with a clock circuit and programming interface on a credit card sized pcb. In order to keep system costs down, I have taken the essence of the Arduino, and come up with a very much stripped down version which can be built on a prototyping breadboard or small piece of stripboard. None of this is original work, there are several similar examples on the web, from which I drew some inspiration.

Applying the Boards.

These stripped down boards will be deployed around the house and used in a number of applications. One device will be used to monitor my electricity readings, based on the design by openenergymonitor.org. Some of its spare sensor channels will be used to monitor temperature and humidity in the rooms at the front of my house and monitor the doorbell and front door openings. This choice of functionality is purely because the electricity meter is at the front of the house and close to the front door – a matter of cabling convenience. A digital output will be used to control a power resistor which is used to activate the thermostatic radiator valve – for selective control of the radiator in the front room. Warming the wax bulb causes the wax to expand and close the valve. A standard TRV can be turned off in about 10 minutes with a 1W power resistor fitted t the valve.

Another device will monitor my gas consumption using an opto-reflective pulse counter fitted to the register of the gas meter. Again for geographical convenience, spare sensing channels will monitor the living room temperature, at floor and ceiling levels and operate a circulation fan. Further channels will monitor the flow and return temperatures of my woodburning stove back boiler, and operate the circulation pump and hot water valve for optimal warm up and selective water heating.
A further device will gather temperature readings from the gas boiler and co-ordinate its operation in conjunction with the woodburning stove.
Outside in the shed where my solar water heating panel is fitted to the roof, a fourth device will control the circulation pump and monitor the small solar pV battery charging array also located on the shed roof.

So, rapidly there is the need for a network of about 4 distributed control boards and the means to co-ordinate their operation.

Serial Communications

The controller boards will all make use of a standard asynchronous serial bus working at 9600baud. The ATmega328 as used on the board has a single UART and this is used in the Arduino for programming via a serial bootloader and for communicating with the host PC. A small modification t o the serial bus will allow it to work in a multi-drop mode with several slave boards communicating, one at a time with the master board. Because of bus wiring limitations, direct slave to slave communication will not be possible, but instead routed through the master unit. A tristate buffer on the Tx line of each slave will allow individual boards to access the bus, and connect to the Rx line of the master unit. The serial bus can be implemented in 4 core telephone cable, with spare wires supplying 9V power and ground to each slave. Each slave will have an on-board 5V regulator for local regulated power. 0.5mm telephone cable will carry about 1A of power. Slaves can have battery back-up provided by NiMH or lithium cells.

Master Unit

The master unit is fitted with an Ethernet interface and connects to the broadband router. In its simplest mode it could operate entirely as a transparent serial gateway, and all data received at its Ethernet port is echoed out of its serial port. Conversely any data received on its serial bus (from the slaves) is echoed up to the Ethernet port. This would allow the slaves to be remotely interrogated from a terminal emulator web application running on a remote PC.

The slaves will be addressed using a simple 8-bit sub-addressing scheme. The master unit will have a sub-address of 255, and slaves in the range of 1 to 254. Sub-address 0 will be reserved as a multi broadcast option to act upon all slaves together – for example for a system reset.

Command Interpreter.

This is a bit of code which allows the distributed control boards to be controlled by serial commands either from a laptop PC or another source of serial data. It was devised to be as flexible as possible and be extendible if needed at a later date. The commands take the format of several alpha or numerical parameters separated by commas and terminated by the newline character. This is a format which is easy to type at a terminal emulation program – such as Hyperterminal or Termite, and can easily be stored as a CSV file or datastream. Indeed a control board could be made to follow a script of interpreted serial commands.

To illustrate the use of the serial commands, consider the familiar application of driving an RGB LED from the 3 PWM channels of an Arduino. The PWM requires an 8 bit number, and the 3 PWM channels individually drive the red, green and blue LEDs within the multicolour array allowing any colour from a 24 bit colour space to be selected.

The command interpreter evolved from being able to interpret simple commands such as R100 – meaning set the red PWM channel to a value of 100, to being able to interpret longer strings with more arguments such a B,5,100,200,50 – which would set the Brightness on slave board 5 of the RGB channels respectively to 100,200 and 50.

Once the basic framework of the command interpreter is in place, additional commands and their associated action routines are easily added.

Tying it all together with Pachube.

Once there is a mechanism for controlling the hardware with simple comma separated commands it is a relatively simple step to implement web based control. The idea is to use the CSV feed format supported by Pachube to act as a repository of serial commands, such that they can be issued to any web connected sensor network that has subscription rights to that particular Pachube feed.

The principle of command will follow a publisher-subscriber relartionship using the Pachube feed to act as a message broking intermediary. Using the RGB multicolour LED example above, if the command B,5,100,200,50 is PUT up onto a Pachube feed, then any device which has the necessary API-Key authoriasation, can perform a GET operation on that feed, and receive the B,5,100,200,50 command string. This would be buffered by the ethernet enabled master device and re-transmitted through its serial port, where it would be received by slave number 5 and acted upon.

Note that there is currently no error checking or correction on the local serial bus, and it is assumed that command packets are received without error or corruption. If the network distances are short (about 20m) and there is only one device transmitting at any one time, then the error rate should be fairly low. If better error detection and correction is needed, the serial packets could be further encoded using a protocol such as SNAP by High Tech Horizons.

Ethernet Board

Here's the schematic for the ethernet board - you should be able to open this up in a new window and enlarge it to see the detail. If you want to receive this by email in EagleCAD format please email me at ken dot boak at gmail dot com or get in touch via Twitter @Monsonite.

This is about as simple as it gets, the ATmega328 with 16MHz clock circuit. All of the '328 I/O lines are brought out as per the Arduino. The power supply consists of a dc input feeding a 5V and 3V3 voltage regulator. The ENC28J60 has a 25MHz crystal and connects to the '328 via the SPI bus. A 74AHC125 acts as a level converter for the /INT and MOSI lines from the ANC28J60 to the '328. The ethernet connection is simplified by using a Magjack - an RJ45 connector with all the magnetics integrated into the connector.

The prototype was breadboarded and I used a FTDI cable to program it and supply power. It is currently supplying regular temperature readings up to Pachube

Master and Servant

Not exactly a cheesy homage to Depeche Mode's 1984 electropop single but more of a concept for connecting together distributed physical computing devices - honest!

Now that I have got the low cost Arduino compatible ethernet board running as described in the last post, it's time to get on with the main project and show that this cheap hardware can be used for home automation, energy monitoring and remote control applications.

The aim is to build up a network of interconnected "Arduinoids"* and have them controllable via the internet. A couple of weeks back I wrote my serial command interpreter, so that a laptop can send commands to an Arduino over it's serial interface, now it's time to extend this so that any device can command the Arduino - remotely via the internet.

If we think in terms of the Arduino as being a programmable element or smart node, designed to monitor sensors, switches or perform some processing function such as control a LCD, then by adding these nodes to a communications network we can distribute them around the environment to provide local monitoring and control, all controlled by a master which is connected to the internet.

The command string we will use is nothing more than comma separated variables (CSV). The Arduino receives this in its serial buffer and decodes it into a command plus one or more arguments. The proposed format of the command is as follows:

Command Letter, sub-address, arg1,arg2,arg3...

The command letter is chosen to be memorable to the human operator, for example B to represent the command that sets the brightness of a multi-colour LED array for example.

The sub-address is a way of directing the command to a specific Arduino board connected to the network.

Arg1,Arg2 and Arg3 are numerical arguments - which in this example might refer to the value of the 8-bit PWM signal sent to each of the red, green and blue LEDs in the multicolour array - thus giving a 24-bit control over the colour.

The plan is therefore to use Pachube as a simple message broker or mailbox server. A device wanting to send a control command to an other Arduino will PUT, or publish a short CSV string up to a Pachube feed. Arduinos needing to be controlled will regularly subscribe to this feed, using the GET method of the Pachube API and receive the command string in CSV format.

By decoding the command and the sub-address, the master device can then output the command string onto the serial bus such that the slave whose sub-address is given, can execute the command.

Typing serial commands into a web page application text box would create the corresponding CSV command string, and PUT it up to the Pachube feed. When the web-connected master subscribes to this feed the command string should ultimately control the remote slave hardware, with the master acting as a transparent serial to ethernet gateway. A simple serial command format would allow any serial device to be connected through the master and used for monitoring and control.

If each "slave" unit costs about a tenner, then is perfectly reasonable to have several in the household environment, each dedicated to specific tasks and comunicating back to the master unit. For example one could monitor domestic electricity and gas usage, another monitors individual room temperatures, outside temperture and controls the central heating, another controls the solar water heating - etc, etc.

Whilst low cost wireless is prolific amongst gadgets for data communications, it still does not solve the age-old problem of how do you power a remote device. So for this reason I am going to suggest a backbone network of wired devices, using wireless only where cables are not practical. A wireless shield could always be added - as the board will accept standart Arduino shields Borrowing from telephone extension cable techniques, a 4 wire telephone extension lead can provide power, ground and serial communication lines very cheaply for distances up to 50m. 0.5mm solid core telephone cable will carry about an amp - so plenty of current available for running a few sensor slaves.

The slaves will in fact be "Smart Sensor Nodes", consisting of a microcontroller and associated sensors etc. Using Arduino compatible hardware allows, for example a remote temperature sensing board to monitor between 6 and 8 thermistors and up to 11 digital inputs or outputs, which could be used for driving pump control relays, LCD displays or whatever. The slave hardware can easily be extended - using a 74HCT595 serial to parallel shift register will give another 8 output lines - controlled from the SPI bus. Adding a 4051 analogue switch will give an additional 8 analogue input channels. If many more I/O lines are needed then the Arduino Mega could be used - these are now very cost effective.

Using a common serial command format as discussed in an earlier post, means that we are not tied to Arduino hardware, but any microcontroller that can implement a UART in software or hardware. The advantage of remaining Arduino compatible, however, is that remote slaves could ultimately be programmed over the serial bus, so that updates can be made from a central location. By issuing a "reset" command to a specific slave, you could put it into bootloader mode and send a program update to it.

So the idea evolved of a number of serial slaves all connected to a serial bus. Various "multi-drop" addressable bus schemes are available such as RS485, but it was the two spare 74AHC125 tristate buffers left over on the ethernet board which gave me the idea of using them as tristate-able linedrivers - so that a slave could be polled by the master, and given access to the bus. In this way the slaves transmit pin would remain tristate, allowing the bus to be used by others, until allowed access.


* Arduinoid - something derived from the Arduino, augmented in some way with additional functionality - such as built in ethernet controller etc.