A Wired Network for Arduinos - Snowdon Net

Over the course of the Bank Holiday Weekend, we took off to the remote hills of Snowdonia with the purpose of developing a simple serial communications network for the Arduino based hardware using the ATMega328.

The emphasis was on simple. It had to be something that any Arduino enthusiast could wire up on a breadboard and get working, and it had to be low cost. I accept that there are better network drivers such as RS485, and a whole host of protocols, but as this was to be partly an educational exercise, as well as a bit of fun, the methods used had to be clear and easy to learn.

In the previous post I introduced the design. Now I share some of the more specific detail of the implementation. A later post will cover the programming and command specification.

Background - the need for a low cost sensor network.

The openenergy project is all about finding cost effective ways to monitor and control renewable and domestic energy systems, such as solar water heating, photovoltaics, electricity and gas monitoring.

We had realised that with very low cost microcontroller hardware, we could afford to use a micro at each sensor node, to make a network of distributed smart sensors. The Arduino project has given a great opportunity to make low cost hardware for as little as £5 per device. The next logical step was to find a cost effective means of connecting such devices together in a network, co-ordinated by a Master unit. Sensors could be interrogated by the Master, and then with a web-enabled Master, the sensor data could be accessed remotely via the internet - using a service such as Pachube.

The system was intended to be cheap, and easy to build for the sort of people who tinker with Arduinos. We could have used a more elaborate system based on RS485, but in order to introduce beginners to networking devices, a very simple signalling method was devised. It uses a single 74AHC125 quad tristate buffer as a line driver, and 5.08mm pitch screw terminals for connecting the network cable - so no fancy connectors or crimping tools are needed to make a connection. It is designed in a way to be compatible with stripboard or built on a low cost protoshield. Indeed, on stripboard, a slave Arduino can be built for under £5 including the ATmega328. The first prototypes were built on breadboards, so that they could be readily modified as new ideas were tried out.

The weekend consisted of building experimental hardware, debugging it and writing firmware to get it running. It was in many ways a learning exercise, as each step of the design we had to work out from scratch. During the course of the weekend, we learned a lot about networking microcontrollers and devising simple control structures in firmware to get it to work as a system.

Day 4 Monday 30th August 2010.

We started with the basic stripped down Arduino consisting of ATmega328 and 16MHz crystal and added a 74AHC125 quad tristate line driver chip. The line driver allows the Arduino to communicate over several tens of metres of 4 core telephone or CAT5 network cable, and also allows an addressing scheme such that only one slave has access to the network at a time - to avoid conflicts.

We devised a simple serial protocol, which can be typed in at a terminal emulator, and using a FTDI cable, access any slave on the network and set up a one to one communications session. We also devised a sketch to run on a standard Arduino, such that it could act as a master and control several slaves. For the purpose of demonstrating the system we built up 3 slaves and connected them at 30m, 35m and 40m respectively away from the master.

The network consists of 4 wires: TX, Rx, +V and 0Vand we used some CAT5 UTP network cable we had available. It could equally work on standard 4 core telephone cable (CW1308 - about £12 for 100m). This wire has a resistance of about 97 ohms per km, and this dictates the voltage drop seen at the power terminals of the remote slaves. A 100m length will drop about a volt if the slave draws 100mA.

We standardised on the following colour scheme, which is loosely based on the ethernet standard but just using the solid colours - rather than the colour/white striped pair - which would be used if we were working with RS485 differential signals:

Brown +5V supply from regulated psu to power all slaves. We used a 2A 5V supply.
Blue 0V system and signal ground return line.
Orange Tx Transmit signal from Master to Slaves
Green Rx Receive signal from Slaves to Master

The use of orange and green for Tx and Rx is the same as ethernet colour coding

The aide-a-memoire for the colour scheme is that Orange is the Outward transmission from the Master.

Our DC power was put on the brown and blue wires. We chose brown for +5V, purely because in the UK we associate brown with live and blue with neutral. This might be in conflict with some of the Power over Ethernet recommendations - but it seemed logical to us at the time.


Above. An Arduino connected to the network using a quad buffer chip and driving a serial LCD.

The 74AHC125 can be thought of as our network access chip. It uses it's tristate buffers to either isolate or connect the Rx and Tx lines of the Slave chip to the network. In order to control the network, and programming operations, we provisionally reserved 3 digital output lines:

Digital 5 /Tx Tristate Enable Setting this low connects the Slave Tx to the network
Digital 6 /Rx Tristate Enable Setting this low connects the Slave Rx to the network
Digital 7 /Autoreset Setting this low causes the Slave to reset itself and allow over network re-programming.

For the simplest network implementation only digital output 5 is needed.

Below: The network chip and the 4 way screw terminal block assembled on a small breadboard to add the network to an Arduino. These parts could be assembled on a protoshield or just a bit of 0.1" stripboard. The basic network uses Rx, Tx and digital output 5 to control the Tx tristate buffer.


We found that connecting digital 5 to an LED connected through a resistor to +5V made a good indicator - lighting up when each Slave had been selected and was communicating with the bus.

If the anode of the LED is connected to the Tx pin rather than +5V, the LED will come on when selected and flicker whe the slave is sending data to the network. A similar LED on the Rx line shows when there is commands or traffic from the master on the Rx bus.

Connecting the 74AHC125

Pin 1 Tx Enable Connect to Digital 5
Pin 2 Tx Connect to Digital 1 (Tx)
Pin 3 Output Connect to green network wire
Pin 4 Rx Enable Connect to Digital 6 (or 0V on simple system)
Pin 5 Input Connect to orange network wire
Pin 6 Rx Connect to Digital 0 (Rx)
Pin 7 Ov Connect to oV ground (blue wire)
Pin 14 +5V Connect to +5V (brown wire)

Pins 8 - 13 of the 74AHC125 remain unconnected, but could be used for other buffering tasks.

If the 74AHC125 is mounted on a small piece of stripboard or protoshield, a 4 way 5.08mm pitch screw terminal block can be used as the connection to the network. It can be inserted next to the 74AHC125 such that it connects directly to pins 3, 5 and 7 leaving only +5V to be connected to the remaining pin.


In the afternoon we decided to prove the distance over which we could send data. We had a drum of 250 metres of CAT5 UTP (unshielded twisted pair) network cable, which we quickly added to our experimental network and hastened together another display slave.

After a quick reprogram of the Master we now had a master and 4 slaves on the network and a total cable distance approaching 300 metres. The Master could successfully transmit and receive data from all of the slaves and display data on a remote LCD connected to the slave that was 300m away from the master.

It would be a relatively simple matter to enhance the network to use differential signalling - such as is used in RS-485. Instead of the 74AHC125 driver chip - a couple of MAX485 (or similar) differential line drivers could be used. Taking this approach would allow the Arduino to be used as a DMX (lighting) controller. However, we chose to remain with simple TTL level serial - as this is the native signalling format used by the Arduino, adequate for our purposes - and it is easy to look at and debug with an FTDI cable to monitor the TX and Rx lines.

The first application for the networked devices was to control a photovoltaic monitoring system. This unit measures the voltage and current produced by a pair of 75W solar pV panels and monitors the charge into a 12V deep cycle lead acid battery. If the battery reaches a full state of charge, a dump load consisting of up to 5, relay selected halogen 12V bulbs could be turned on. Trystan wrote some firmware to obtain voltas and amps readings from the solar controller and send them to the Master device, and a programme running on the Master determined when the dump load relays should be activated.

All in all, a very successful and useful addition to the openenergy.org project.

The Snowdonia Mega Hack Group are (left to right)

Suneil Tagore, Trystan Lea, Ken Boak and Glyn Hudson.

Snowdonia Weekend Hack Session

Friday 27th August - The Mission Begins.

Your Mission (should you choose to accept it) - to develop from scratch, a network of serial slaves controlled from a Master Arduino and to use it to monitor a photovoltaic renewable energy system.

Time Allowed 3 days.

Personnel:

Trystan Lea - Chief Coder and host
Suneil Tagore - Renewables Consultant
Ken Boak - Hardware builder and tea maker

Resources:

10 Arduinos of various types - official, Freeduinos and DIY breadboarded units
2 NuElectronics Ethernet shields
1 Etherduino - a DIY ethernet compatible clone built on breadboard.
Box of useful electronic component bits
2 x 75W solar pV panels
Hall effect sensors for current sensing
110Ah Deep cycle battery
200W 120V Inverter
500W UPS found in skip - cruddy modified sine wave
250m drum of ethernet cable
Lots of tea and firewood

Location: Secret Snowdonian Bothy - outside the Trawsfynydd Nuclear Plant Fall Out Zone

Over the course of the August Bank Holiday I was invited up to North Wales to join Trystan Lea and Suneil Tagore, of openenergymonitor.org for an extended weekend build session with the aim of further developing the openenergy project. Trystan had developed a number of Arduino based control systems and was looking for a way to connect them together as a network.




I had first met Trystan and Suneil back at the April 2009 Homecamp Un-conference, and was impressed with their open source energy monitor based on the Arduino. We had kept in close touch over the last year and the plan was to pull our collective ideas together over the course of the weekend and develop some of the concepts into a working system.

I had recently been working on "Etherduino" a web-enable compatible Arduino built on breadboard, and I had a few ideas on how we could put together a network consisting of a Master and several Slaves - all connected with a 9600 baud serial wired network.

The openenergy devices are a series of Arduino based units that can be applied to electricity monitoring, controlling solar water heating, gas meter monitoring and other purposes related to energy conservation and renewables. Trystan had already developed the electricity monitor and the solar water heating pump controller, but wanted to tie them together with a simple remote display so that various paramenters could be viewed from a central display - for convenience located in the kitchen.

I had donated a couple of solar panels to the project, and we decided a good use of the network was to use it to extend the openenergy system to monitor a small pV system and control an inverter and battery system.

So with all these individual elements working in isolation a plan was devised to develop a low cost, simple means of connecting them together in a network so that they could be monitored and controlled from one central point by a master device.

The Concept Emerges.

The Arduino, and many other microcontroller based devices,generally communicate via a serial interface, running at TTL levels and modest baudrates. On the Arduino, the standard is 9600 baud, and this makes communication with a laptop easy using a FTDI serial to USB cable.

We had decided that because the cost of adding a microcontroller slave to a system was in the order of £5 per node, then it was cheap and easy to compose a system of a series of distributed nodes all communicating through a network.

We agreed that the system of simple slave boards connected by a wired network would be easiest and cheapest to implement, and as the Arduino is fundamentally a device with serial input and output and a serial programming interface we thought that we would stick to a simple text based serial interface running at 9600 baud 8,n,1.

In oder to make the network easy to build, test and modify, it was decided to keep the logic levels as those of the Arduino, and not introduce any further complication such as differential signalling (eg RS485). In essence, one Arduino can communicate with another just by cross-connecting their Rx and Tx pins with a cable and providing a common signal ground between the 0V GND pins.

We realised that for the network to be practical it should allow several devices to be connected to the one network, and there should be some method to prevent them all trying to access the network at the same time - which would lead to very corrupted data and a breakdown in communications.

We also wanted to distribute dc power down a spare wire of the network, so that this could be used to power the microcontroller and sensors at the far end. As the Arduino is fundamentally a 5V part, we wanted to restrict the voltage on the bus to +5V so that any accidents in wiring would not result in damaged ICs.

Finally we needed a method to prevent more than one device accessing the the Tx line at the same time. Thus a 4-wire network was selected with the signals being Tx,Rx 0V and +5V. This made the use of cheap telephone cable or CAT 5 network cable quite practical.

The next question was how far could we reliably send 9600 baud down a telephone extension cable and get uncorrupted data at the far end? After a few beers on Friday night we did some simple tests using a 30m length of 4 core cable connected to and Arduino and found that 9600 baud was quite happy over that distance. With this proven we were ready to specify, design and build the full network.

Day Two - Saturday 28th August

On the Saturday morning I built up a couple of slave Arduinos on breadboards - and because they were fitted with displays, we decided to call them Disciples. The plan was build up identical hardware and document it, so that I could take a unit back to London and one could be kept in Wales - so that both Trystan and I were working with the same platform.

The Disciple consisted of the ATmega328, a 16MHz clock circuit and a header to allow an FTDI cable to be plugged in for programming. In addition, a 74AHC125 quad tristate buffer was fitted which allowed us to drive a cable network, with only one slave transmitting to the bus at a time. It was later decided that we would also use another '125 buffer on the Rx line of the slave, to improve the signal from the network and to allow the slave Rx pin to be isolated from the network for local programming operations.

By early evening on the Saturday we had managed to connect two Disciple slaves to the network and Trystan had written a sketch which accessed them individually form the Master and sent a command to flash an LED. Very soon we had both slaves "ping-ponging" their LEDs on and off under the complete control of the Master Arduino.

Day 3 - Sunday August 29th.

Not such an early start because we were tired from the previous mega-hack session. The aim for the day was to incorporate the slave code into a photovoltaic solar panel and battery monitoring system.

Much of the morning was spent wiring up the pV monitor panel with two Hall-effect sensors - one for pV current and the other for load current. A bank of 5 relays controlled by the Arduino allowed 5, 12V lamps to be selectively turned on, thus putting a variable load on the system. this was a first step to developing a full maximum peak power point tracking system, which Suneil is working on.

By lunchtime the sun was out and we were starting to get a peak of about 10A at 14V, which was about all my 75W panels were capable of.

Trystan then started to merge the slave control sketch with the pV monitoring sketch - so we could get remote pV data - volts, amps and watts from the pV controller and send this data up to the master unit. The 74AHC125 tristate line driver was temporarily added to the pV controller on a separate breadboard, to be tidied up later once we had the system working.

Towards the end of the afternoon we had 3 slaves working on the network, one of which was the pV monitor. Fortunately this co-incided with a sunny spell and the pV panels were in full sunshine and being remotely monitored.



We then set about writing some code to send text to the Disciples (display slaves) such that the command packets send on the network would be displayed on the second line of the LCD, and any response from the Disciple would be echoed to line 4. Below is the display slave displaying a typical net packet. @,28 selects slave 28, and this is confirmed when the green "slave active" LED coming on - bottom right.




Shortly after that we went down the village pub for a well deserved pint (or three) and an evening meal. The weekend had been a great success and we had achieve all of our objectives - and there was still Monday left to tidy up a few loose ends.

Social Networking for Arduinos.

Now that I have got a nice, low cost, web-enabled Arduino (WEA) it’s time for it to get on line and develop a whole new social life.

Messages passed between Arduinos are “Tweet like” in their nature – and so I have coined the name micro-twit (micro tweet having been already taken) or u-twit to describe them.
Building on the Twitter model, an Arduino can follow a feed and can also have others following its feed. In the master slave relationship, where only the master is web enabled, then the slaves automatically become followers of the master (disciples?) and also followers of anything the master happens to be following – as the messages are transparently fed to the local network via the serial port.

In order to define a basic system, we must specify the exact nature and length of the messages. The command interpreter running on the Arduino has to decode the instruction and also have a knowledge of the context of the message. Unfortunately the Arduino would struggle to run a full XML decoder, so a lightweight version is needed to interpret the context of the data contained in the message.

The nature of the commands also has to be specified. Some will be hardware specific, such as setting port pins or PWM values or reading ADC values. Others will be more compound and depend on the application the Arduino is running, for example boiler control or electricity consumption monitoring.

Part of the command, the leading letter, is used to define the action routine associated with it. For human convenience and by way of mnemonic, this is given a letter to specify it. Another field of the command is used to define the destination of the command, in other words the sub address of the microcontroller which will ultimately execute the command. Addressing is also partly handled by the number of the feed to which you subscribe. A device which subscribes to feed 8729 for example is effectively on a given network, whilst a device which subscribes to feed 8730 is on another network. 8729 might handle all messages to do with room temperature control, boiler control and solar water heating, whilst 8730 is concerned with gas and electricity monitoring and logging events such as front door opening or doorbell ringing.

As the message persists on the Pachube feed until the next update, there will need to be some mechanism to inform the subscriber that it is a new message, and not lead to the situation where the subscriber continues to repeat the same action following each time it checks for updates. The message should therefore have a “message number” appended to it so that the subscriber can tell whether it is a new message or just the old one still there.

In order to put this lot together, I need to use my command interpreter to enter the message into the sending Arduino. This creates a comma separated text string which can then be loaded into the message buffer of the Pachube client. This will allow different commands to be entered by the laptop and send them up to the Pachube feed. This is now running and produces commands of a standard format: a leading letter followed by 5 comma separated arguments. If the arguments are left off, they default to zero. For example both b1 and b,1 will produce the output b,1,0,0,0,0

With the command interpreter running on one ethernet enabled arduino, I can type a command at the keyboard, have it echoed to the screen in standard format and then PUT up to my Pachube feed as a CSV string.

The second Arduino is running the GET code, such that it feeds the received text string into the command interpreter and also echoes it out to the serial port for viewing on the terminal or passing on to the local network.

With this all working, a command typed to one Arduino appears in the serial debug window of the second Arduino some seconds later. The reason for this latency is that there is an access limit on the Pachube API, and trying to accesss at anything less than 10 second intervals will cause an error and a warning message from Pachube!

A Pachube feed for this has been created here:

http://www.pachube.com/feeds/9675

Messaging with Pachube - micro twittering for Arduinos

Regulars will realise that I am busy finding ways of getting low cost web enabled Arduino based boards to talk to each other.

The solution that I stumbled upon is to use a Pachube feed as a message depository, and have other ethernet enabled Arduinos subscribe to this feed and take action based on the CSV data returned from the feed.

Suppose we have a command string B,5,100,200,50 which is used to set the brightness levels of a RGB LED to 100,200,and 50 respectively. The preceding letter "B" identifies this as a Brightness setting command and 5 is the sub-address of the Arduino board which we want to execute this command.

So a controlling, web enabled Arduino can publish the B,5,100,200,50 command to a Pachube feed in the form of a CSV string, and other web enabled Arduinos will subscribe to this feed, retrieve the CSV string, decode the command - and Arduino 5 will act on the message, setting its PWM outputs accordingly.

It's a fairly trivial example, but it does illustrate how a simple message can be passed from one controlling device to another using Pachube as a message brokerage service. The advantage of using Pachube is that it provides a certain level of security by way of it's API-key, and the low overhead of getting CSV data to and from Pachube makes the method applicable to the limited resources of a small microcontroller.

It occurred to me that this pub-sub model of messaging between micros is a scaled down version of what Twitter offers and the messages from the web enabled sensors are "Tweet like". Just to be topical, and to give it a name, I'm going to call this "micro twitting" with the message being known as a u-twit. (Homage to YouTube and indirectly my programming abilities). It was going to be called micro-tweet, but someone has already nabbed that name!

The contents of the message can either take the form of instructional and numerical command, or numerical sensor data. The exact nature and context of the message contents can be signified by a message type identifier.

A command interpreter running on the Arduino, identifies the message type, and any data content and acts accordingly. If in the case that the message is not intended for the web-connected master device, then it can be passed on, unchanged to the serial port and on to the local network of slaves.

Any Arduino hardware running a micro-twit interpreter can be connected to the serial bus and decode and act upon the message. It might be possible to use the system of hash-tags and @, as used in Twitter to identify subjects and publishers/users.

The pieces started falling into place with the assistance of John Crouchley's Pachube GET sketch implemented on an Ardunio with NuElectronics ethernet shield. John enthusiastically produced this at the drop of a hat, realising that I was wishing to do something similar.

I've now got John's sketch running on one Arduio/NuElectronics shield, printing out the CSV data sent to my Pachube feed by my ATmega328/ENC28J60 combo board.

Slowly the pieces are coming together :-)

Getting it Together

The various development threads of the web enabled smart sensors are now coming together into a convergent design. In previous posts I have discussed ethernet controller hardware, serial command interpreters and messaging methods all under the banner of smarter energy control. Now is the time to bring it all together into a tangible system.

The plan is to build a network of simple devices for home automation, monitoring and control – applicable to home heating, renewable energy and domestic energy monitoring.
The hardware is based on the ever-popular Arduino which consists of an ATmega328 with a clock circuit and programming interface on a credit card sized pcb. In order to keep system costs down, I have taken the essence of the Arduino, and come up with a very much stripped down version which can be built on a prototyping breadboard or small piece of stripboard. None of this is original work, there are several similar examples on the web, from which I drew some inspiration.

Applying the Boards.

These stripped down boards will be deployed around the house and used in a number of applications. One device will be used to monitor my electricity readings, based on the design by openenergymonitor.org. Some of its spare sensor channels will be used to monitor temperature and humidity in the rooms at the front of my house and monitor the doorbell and front door openings. This choice of functionality is purely because the electricity meter is at the front of the house and close to the front door – a matter of cabling convenience. A digital output will be used to control a power resistor which is used to activate the thermostatic radiator valve – for selective control of the radiator in the front room. Warming the wax bulb causes the wax to expand and close the valve. A standard TRV can be turned off in about 10 minutes with a 1W power resistor fitted t the valve.

Another device will monitor my gas consumption using an opto-reflective pulse counter fitted to the register of the gas meter. Again for geographical convenience, spare sensing channels will monitor the living room temperature, at floor and ceiling levels and operate a circulation fan. Further channels will monitor the flow and return temperatures of my woodburning stove back boiler, and operate the circulation pump and hot water valve for optimal warm up and selective water heating.
A further device will gather temperature readings from the gas boiler and co-ordinate its operation in conjunction with the woodburning stove.
Outside in the shed where my solar water heating panel is fitted to the roof, a fourth device will control the circulation pump and monitor the small solar pV battery charging array also located on the shed roof.

So, rapidly there is the need for a network of about 4 distributed control boards and the means to co-ordinate their operation.

Serial Communications

The controller boards will all make use of a standard asynchronous serial bus working at 9600baud. The ATmega328 as used on the board has a single UART and this is used in the Arduino for programming via a serial bootloader and for communicating with the host PC. A small modification t o the serial bus will allow it to work in a multi-drop mode with several slave boards communicating, one at a time with the master board. Because of bus wiring limitations, direct slave to slave communication will not be possible, but instead routed through the master unit. A tristate buffer on the Tx line of each slave will allow individual boards to access the bus, and connect to the Rx line of the master unit. The serial bus can be implemented in 4 core telephone cable, with spare wires supplying 9V power and ground to each slave. Each slave will have an on-board 5V regulator for local regulated power. 0.5mm telephone cable will carry about 1A of power. Slaves can have battery back-up provided by NiMH or lithium cells.

Master Unit

The master unit is fitted with an Ethernet interface and connects to the broadband router. In its simplest mode it could operate entirely as a transparent serial gateway, and all data received at its Ethernet port is echoed out of its serial port. Conversely any data received on its serial bus (from the slaves) is echoed up to the Ethernet port. This would allow the slaves to be remotely interrogated from a terminal emulator web application running on a remote PC.

The slaves will be addressed using a simple 8-bit sub-addressing scheme. The master unit will have a sub-address of 255, and slaves in the range of 1 to 254. Sub-address 0 will be reserved as a multi broadcast option to act upon all slaves together – for example for a system reset.

Command Interpreter.

This is a bit of code which allows the distributed control boards to be controlled by serial commands either from a laptop PC or another source of serial data. It was devised to be as flexible as possible and be extendible if needed at a later date. The commands take the format of several alpha or numerical parameters separated by commas and terminated by the newline character. This is a format which is easy to type at a terminal emulation program – such as Hyperterminal or Termite, and can easily be stored as a CSV file or datastream. Indeed a control board could be made to follow a script of interpreted serial commands.

To illustrate the use of the serial commands, consider the familiar application of driving an RGB LED from the 3 PWM channels of an Arduino. The PWM requires an 8 bit number, and the 3 PWM channels individually drive the red, green and blue LEDs within the multicolour array allowing any colour from a 24 bit colour space to be selected.

The command interpreter evolved from being able to interpret simple commands such as R100 – meaning set the red PWM channel to a value of 100, to being able to interpret longer strings with more arguments such a B,5,100,200,50 – which would set the Brightness on slave board 5 of the RGB channels respectively to 100,200 and 50.

Once the basic framework of the command interpreter is in place, additional commands and their associated action routines are easily added.

Tying it all together with Pachube.

Once there is a mechanism for controlling the hardware with simple comma separated commands it is a relatively simple step to implement web based control. The idea is to use the CSV feed format supported by Pachube to act as a repository of serial commands, such that they can be issued to any web connected sensor network that has subscription rights to that particular Pachube feed.

The principle of command will follow a publisher-subscriber relartionship using the Pachube feed to act as a message broking intermediary. Using the RGB multicolour LED example above, if the command B,5,100,200,50 is PUT up onto a Pachube feed, then any device which has the necessary API-Key authoriasation, can perform a GET operation on that feed, and receive the B,5,100,200,50 command string. This would be buffered by the ethernet enabled master device and re-transmitted through its serial port, where it would be received by slave number 5 and acted upon.

Note that there is currently no error checking or correction on the local serial bus, and it is assumed that command packets are received without error or corruption. If the network distances are short (about 20m) and there is only one device transmitting at any one time, then the error rate should be fairly low. If better error detection and correction is needed, the serial packets could be further encoded using a protocol such as SNAP by High Tech Horizons.

Ethernet Board

Here's the schematic for the ethernet board - you should be able to open this up in a new window and enlarge it to see the detail. If you want to receive this by email in EagleCAD format please email me at ken dot boak at gmail dot com or get in touch via Twitter @Monsonite.

This is about as simple as it gets, the ATmega328 with 16MHz clock circuit. All of the '328 I/O lines are brought out as per the Arduino. The power supply consists of a dc input feeding a 5V and 3V3 voltage regulator. The ENC28J60 has a 25MHz crystal and connects to the '328 via the SPI bus. A 74AHC125 acts as a level converter for the /INT and MOSI lines from the ANC28J60 to the '328. The ethernet connection is simplified by using a Magjack - an RJ45 connector with all the magnetics integrated into the connector.

The prototype was breadboarded and I used a FTDI cable to program it and supply power. It is currently supplying regular temperature readings up to Pachube

Master and Servant

Not exactly a cheesy homage to Depeche Mode's 1984 electropop single but more of a concept for connecting together distributed physical computing devices - honest!

Now that I have got the low cost Arduino compatible ethernet board running as described in the last post, it's time to get on with the main project and show that this cheap hardware can be used for home automation, energy monitoring and remote control applications.

The aim is to build up a network of interconnected "Arduinoids"* and have them controllable via the internet. A couple of weeks back I wrote my serial command interpreter, so that a laptop can send commands to an Arduino over it's serial interface, now it's time to extend this so that any device can command the Arduino - remotely via the internet.

If we think in terms of the Arduino as being a programmable element or smart node, designed to monitor sensors, switches or perform some processing function such as control a LCD, then by adding these nodes to a communications network we can distribute them around the environment to provide local monitoring and control, all controlled by a master which is connected to the internet.

The command string we will use is nothing more than comma separated variables (CSV). The Arduino receives this in its serial buffer and decodes it into a command plus one or more arguments. The proposed format of the command is as follows:

Command Letter, sub-address, arg1,arg2,arg3...

The command letter is chosen to be memorable to the human operator, for example B to represent the command that sets the brightness of a multi-colour LED array for example.

The sub-address is a way of directing the command to a specific Arduino board connected to the network.

Arg1,Arg2 and Arg3 are numerical arguments - which in this example might refer to the value of the 8-bit PWM signal sent to each of the red, green and blue LEDs in the multicolour array - thus giving a 24-bit control over the colour.

The plan is therefore to use Pachube as a simple message broker or mailbox server. A device wanting to send a control command to an other Arduino will PUT, or publish a short CSV string up to a Pachube feed. Arduinos needing to be controlled will regularly subscribe to this feed, using the GET method of the Pachube API and receive the command string in CSV format.

By decoding the command and the sub-address, the master device can then output the command string onto the serial bus such that the slave whose sub-address is given, can execute the command.

Typing serial commands into a web page application text box would create the corresponding CSV command string, and PUT it up to the Pachube feed. When the web-connected master subscribes to this feed the command string should ultimately control the remote slave hardware, with the master acting as a transparent serial to ethernet gateway. A simple serial command format would allow any serial device to be connected through the master and used for monitoring and control.

If each "slave" unit costs about a tenner, then is perfectly reasonable to have several in the household environment, each dedicated to specific tasks and comunicating back to the master unit. For example one could monitor domestic electricity and gas usage, another monitors individual room temperatures, outside temperture and controls the central heating, another controls the solar water heating - etc, etc.

Whilst low cost wireless is prolific amongst gadgets for data communications, it still does not solve the age-old problem of how do you power a remote device. So for this reason I am going to suggest a backbone network of wired devices, using wireless only where cables are not practical. A wireless shield could always be added - as the board will accept standart Arduino shields Borrowing from telephone extension cable techniques, a 4 wire telephone extension lead can provide power, ground and serial communication lines very cheaply for distances up to 50m. 0.5mm solid core telephone cable will carry about an amp - so plenty of current available for running a few sensor slaves.

The slaves will in fact be "Smart Sensor Nodes", consisting of a microcontroller and associated sensors etc. Using Arduino compatible hardware allows, for example a remote temperature sensing board to monitor between 6 and 8 thermistors and up to 11 digital inputs or outputs, which could be used for driving pump control relays, LCD displays or whatever. The slave hardware can easily be extended - using a 74HCT595 serial to parallel shift register will give another 8 output lines - controlled from the SPI bus. Adding a 4051 analogue switch will give an additional 8 analogue input channels. If many more I/O lines are needed then the Arduino Mega could be used - these are now very cost effective.

Using a common serial command format as discussed in an earlier post, means that we are not tied to Arduino hardware, but any microcontroller that can implement a UART in software or hardware. The advantage of remaining Arduino compatible, however, is that remote slaves could ultimately be programmed over the serial bus, so that updates can be made from a central location. By issuing a "reset" command to a specific slave, you could put it into bootloader mode and send a program update to it.

So the idea evolved of a number of serial slaves all connected to a serial bus. Various "multi-drop" addressable bus schemes are available such as RS485, but it was the two spare 74AHC125 tristate buffers left over on the ethernet board which gave me the idea of using them as tristate-able linedrivers - so that a slave could be polled by the master, and given access to the bus. In this way the slaves transmit pin would remain tristate, allowing the bus to be used by others, until allowed access.


* Arduinoid - something derived from the Arduino, augmented in some way with additional functionality - such as built in ethernet controller etc.

An Arduino Compatible Web Server for £12

In the last 2 weeks I have been working towards making a very low cost Arduino compatible web server based on the ATmega328 interfaced to an ENC28J60 ethernet controller.

With a spare afternoon this weekend - I finally got around to finishing the wiring up of the breadboard and getting it running.

It's fairly standard stuff, having been done by Tuxgraphics, NuElectronics and others - but for me the attraction was the simplicity of the stripped down design and making something a lot cheaper than you can buy it - slowly eroding at the myth that there is anything complex or costly about the Arduino.

This combination usually costs about £40 if you buy an Arduino and a low cost ethernet shield from NuElectronics. So I decided to make my own and show that it could be done for considerably less. It is my intention to get down to having a web connected microcontroller for under £10 - but even as a prototype this can be done for about £12.

The board consists of just 3 ICs, the ATmega328, the ENC28J60 and a 74AHC125 quad buffer which acts as a 3.3V to 5V level converter. Additionally there are two crystals, 25MHz and 16MHz, two voltage regulators 3V3 and 5V and about 30 other passive Rs and Cs. The connection to the ethernet is done using the RJ45 "Magjack" breakout board from CoolComponents. This costs just £1.99. The ATmega328 and the ENC28J60 were also sourced from CoolComponents.

The webserver connects to my laptop using a FTDI USB to serial cable. This saves having to put an FTDI chip on the breadboard. The serial connection and the power is at the top right of the picture. The Magjack is at the top left. The 3V3 regulator to power the ENC28J60 is at the extreme lower left.

The next plan is to get these components onto a professionally made double sided pcb. It will be the same size as the Arduino, and share the same connector footprint - but using entirely through hole components, can be built by anyone who can solder.

This project is being done is association with the London Hackspace. With the introduction of a low cost ethernet enabled microcontroller board, many fun projects are possible, and the low cost and easy construction allows many more people to tinker with web connected hardware.

Andrew Lindsay has written code for a Pachube client to run on this combination of chips. With this you can read data from sensors such as temperature, humidity and light sensors and publish this data up on the Pachube site.

My Pachube feed features between 4 and 6 thermistors - measuring various temperatures around my house.

Making it.

With a little forethought, the stripped down version can be built on a single (63 row) breadboard.

Make sure you connect all the 5 Vdd pins and the 4 Vss pins of the ENC28J60 to 3V3 and 0V respectively. Fit a 100nF decoupling capacitor on each of the Vdd pins.

Both the 16MHz and 25MHz crystals can be loaded with 22pF capacitors. Fit the crystal and the capacitors right next to the pins of the ICs.

The Magjack breakout board makes it very easy to connect the ethernet. You need to fit pin headers onto the breakout board - except for the pins marked SH and SH2 (shield) and then it will plug neatly into the breadboard.

Keep the wiring to the ethernet magjack short and neat - we are expecting quite a lot from breadboard at the sort of frequencies used in ethernet. I used 51ohm 1% resistors for the ethernet termination circuitry. Most designs say 49.9 ohm 1%, but as long as you are close to 50 ohm it's not an issue.

The ATmega328 and 74AHC125 run off the 5V rail, but the ENC28J60 runs off 3V3. Make sure your 3V3 regulator is producing 3V3 before you make the final connection to the ENC28J60.

The purpose of the 74AHC125 is to convert the 3V3 levels output from the ENC28J60 up to 5V so that they are compatible with the 5V used by the ATmega328. The two lines that need buffering are /INT and MISO (master in slave out). The buffer used for MISO is enabled with the /CS line - this means that when the ENC28J60 is not selected, the MISO line is tristated by the buffer and available for use by any other SPI device. There are 2 spare buffers in the 74AHC125 package and they could be used for other purposes.

I splashed out and bought a ATmega328 which was already pre-programmed with the Arduino bootloader. Whilst you pay a significant premium for this, I wanted to make sure that my '328 would run Arduino code first, and was not the cause of any failure in the build. Subsequent '328s will be bought unprogrammed and blown with the bootloader code, using this neat AVRISP trick using an Arduino board.

I use an FTDI cable both to program and power this design. Note that the 5V supply form the FTDI cable is limited to around 70mA. The FTDI cable, whilst it offers a programmable output pin in the form of /RTS is not compatible with the Arduino programming which uses DSR to reset the ATmega328. Some changes would be needed to the programming code in the IDE to support this cable. In the meanwhile I just keep pushing the reset button until the IDE recognises the bootloader and begins the programming sequence.


Parts List

1 ATmega328 microcontroller (with preprogrammed Arduino bootloader) - CoolComponents £5.00 or £2.81 without bootloader
1 ENC28J60 ethernet controller - CoolComponents £1.99
1 Magjack breakout board - CoolComponents £1.99

The remainder of the parts came from Farnell Components:

1 74AHC125 quad buffer 174-9617
1 16MHz crystal 161-1761
1 25MHz crystal 161-1783
4 22pF ceramic capacitors 114-1760
10 100nF ceramic capacitors 121-6444
2 10nF ceramic capacitors 121-6435
2 10uF electrolytic capacitors 945-1056
1 7805 5V regulator 156-4483
1 78L33 3V3 regulator 146-7768
1 2K resistor 934-1480
4 51ohm 1% resistors 934-3342
2 270 ohm resistors 933-9353
3 10K resistors 933-9060
1 tact switch 181-3689

You will also need:

1 breadboard
1 FTDI USB-serial cable - becoming a standard bit of kit for talking to microcontrollers.

DIY Arduinos

The stripboard Arduino - a simple design for about £5 in parts!

With a recent flurry of Arduino activity, I have now realised that I can never have too many Arduinos!

I'm now using them for all sorts of tasks, and it's a shame to strip down a project just to borrow the Arduino from it to try something new. So I decided it was time to start to make some cheap DIY Arduinos on stripboard and use them for the various tasks.

The Arduino is really not much more than an Atmel ATmega328, a clock oscillator circuit, a voltage regulator and a serial interface. However it is the Arduino community and the hundreds of thousands of hours (possibly millions of hours) of open source development work that makes it so much more than the sum of its parts.

Such is the demand for Arduino ICs that some suppliers are running short of ATmega328 devices! (Microchip must be reeling that it is an Atmel AVR that has been so widely adopted by the hobbyist community!).

Last night I came across the fizzPop Hackerspace in Birmingham, who are going to run a workshop to build some DIY Arduinos. They believe that a stripboard DIY "Arduino" could be built for about £4.50 in parts.

The simplicity of the design, and the fact that you can use an FTDI USB/serial cable to program it convinced me that my days of buying Freeduinos and Nanos (at £16.50 each) were coming to an end. The Nano is certainly neat with it's microUSB connector and two additional ADC channels, but the standard Arduino with its "blighted" pin-out doesn't make economic sense - especially if you are not intending using standard shields.

The 28pin DIL version of the ATmega328 is clearly the preferred choice for prototyping and breadboarding. Atmel in their wisdom, put the power and ground pins in the middle, close to the clock oscillator, which makes for a very neat and tidy clock crystal and decoupling arrangement on stripboard.

The great thing about the stripboard 'duino, is that with the right stackable header pins, you can make it plug into a standard breadboard, yet only 50mm x 25mm wide.

Programming and SPI Bus Connector Ideas

Another interesting discovery was how to use an existing Arduino programmed up as an AVRISP to burn the bootloader into new ATmega328P chips.

Using the MOSI, MISO, SCLK, power and ground pins connected directly from the "master" to the board being programmed means that copying from one board to the other would be as simple as stacking them such that these pins stack together. However, stacked Arduinos get a bit ungainly, so the thought arose of having a right angle male header on one side of the board, and a right angle female socket on the other side of the board such that two boards could be connected by placing them next to each other and mating up the SPI bus connectors.

Having a dedicated SPI connector would also allow SPI devices - such as the ENC28J60 ethernet controller to be plugged into this bus, and pick up power and ground and a chip select line too. Extending the SPI connector to 8 pins would allow three separate chip select lines for daisy chained SPI devices.

If you use 20 way stackable headers, it allows access to every pin of the ATmega328 and gives you 6 spare pins on each header for connecting a USB to serial FTDI cable and also a unique ISP "jump starter" connection - so that one boarduino could be stacked on top of another for initial copying of the bootloader - or for a complete cloning operation - as describes in this self-replicating sketch

Arduino and Internet

For a couple of years Tuxgraphics have been promoting the use of the ATmega328 with the Microchip ENC28J60 ethernet controller. Now this may be a bit heretical, when the "official"Arduino ethernet shield uses the Wiznet W5100 based shield. Indeed the official version does have better software library support, but costs a lot more. The W5100 is an 80 pin device, with both SPI and conventional address and data buses. It's available from Sparkfun Electronics for about $6 but still leaves you with the problem of putting it onto a board. Sparkfun do have a range of ethernet breakout boards but they are all close to $40 - a price that really does not reflect the value of the components on them.

The chip solution chosen by Tuxgraphics offers the lowest cost and is readily prototyped. They offer a pcb kit for about 18 Euros to which you add your own ATmega and ENC28J60. However if you are prepared to build it yourself on stripboard it offers a means to get a web-enabled Arduino clone for about £10 -£15 - which is a step closer to getting lots of AVR based smart sensors onto the 'net.

So it's clear that the next step is to put together some stripboard clones, and start working on some real applications.

Arduino Command Interpreter

After 10 years of PIC assembly language (and Z80 etc back in the 1980s), I've now got started writing short applications for the Arduino in C.

There are some program routines that you use over and over, and so I thought I would have a go at writing some of them in C, so that I have them available in my programming "toolkit". One of the most useful, is a means of getting serial data in and out of the microcontroller, and more specifically the means for the micro to accept serial commands and data from a terminal programme. This is a fundamental building block of any system and has been implemented on just about every computing device since the year dot.

So I set about writing some code for the Arduino to allow it to interpret and act upon commands sent over the serial link. These serial commands could come from a terminal emulator running on a laptop, from an ethernet to serial gateway, or any other device, such as another Arduino that can generate serial data.

There are many small applications in control and monitoring which require some form of human/machine interface so that the program actions can be modified according to some numerical input. It occurred to me that this functional block is something which could be used from one project to another, so I decided to start from scratch and write something simple which would allow me to achieve basic control.

One example of how it could be used, is for the setting of the current time into a real time clock programme. If you could just enter the hours, minutes and seconds from a terminal emulator, which would set the appropriate variables in the code to those values, then it would be very handy. Or if you had a sketch which counted pulses from a gas or electricity meter, being able to set the initial reading from the meter, so that the pulse counter follows the actual readings on the meter - again a useful function.

As the Arduino can be made to act on serial data, this could come from any source, not just a laptop USB connection. For example it might come from a master Arduino which sends commands over a RS485 bus to a series of slaves, or from an ethernet to serial gateway which receives commands from a remote server or browser.

This week I have been working on an application which reads values from the Arduino ADC channels displays them on the terminal, and requires PWM data to be sent to three output pins either under manual or automatic control.

Realising that this was going to be a useful building block, and my C skills need all the exercise they can get, I set about writing the command interpreter.

Initially I used a single alpha character a-z, followed by a 3 digit number. The alpha character acted as a mnemonic so that you could easily remember the various commands. This was fine for setting the PWM channels - such that f100 would set the PWM for the cooling fan speed, and b123 would set the brightness of some LEDs.

However, following a discussion with some friends it was deemed that this might be a little too simple and that the command format should be extended to allow a more elaborate scheme. At the same time, the command interpreter is an overhead to the rest of the sketch, so it needs to remain fairly compact and not too unwieldly.

So I extended the format to allow the use of sub-addressing. The parameter immediately following the alpha character would be in the range of 0 - 255 and would allow you to direct the command to any one of 255 slave devices, with zero being reserved as the local device.

I then followed the sub-address with a pair of comma separated arguments. These were to be unsigned integers with the range 0 to 65535, with the intention that they could be used for 16-bit addressing of memory devices. More arguments could be added, but for most purposes two were deemed to be sufficient.

The basis of the command interpreter is to hold the serial input in a suitable sized array, and then parse through it to strip off the various paramenters and arguments as dictated by the commas which are used as separators. This string parser is the basis of the command interpreter. Reading in the string of characters is fairly simple, just put them into an array until you come across a carriage return character (13 decimal).

Then you strip of the initial command alpha character and the digits which form the sub address. You keep copying these numerical characters into a smaller sub-address array, until you encounter the first comma, at which point you know you have all the digits.
Then you form arrays for the first and second arguments, knowing that they are also separated by a comma and will end with a carriage return.

Once you have formed separate strings for the sub-address and arguments, you just need to enumerate them using the atoi (ascii to integer) C function.

So the string parser has stripped out the various fields and stored them into arrays, which have been enumerated. At this point you can print them out to the terminal to check that your parser has got them correctly - and not made any errors!

We then need to form a set of action routines, which do the real work when a command is executed. For this we need to decode the alpha character and use it plus the arguments to control functions within the code. For simplicity I chose to use a switch-case statement method to look for certain ascii character values and perform certain code blocks.

The other tasks performed by the Arduino were arranged to be executed once per second. This included reading the various ADC channels, averaging them out and converting to meaningful values - like a thermistor reading in degrees centigrade. As these tasks were fairly quickly performed, it left a lot of spare time for the Arduino to loop around the command interpreter - so that the response to new commands was fairly rapid, and didn't appear to distract from the once per second operations.