Datalogging & file storage with OpenLog

OpenLog is a an open source datalogger developed by Nathan Seidle of Spark Fun Electronics.

It consists of an ATmega328 microcontroller interfaced to a SDcard or microSD card. It retains a high degree of compatability with Arduino hardware - and indeed, early prototypes were developed by adding a shield with an SD card socket to an Arduino.

OpenLog is a serial datalogger. It does exactly what it says on the tin, and records all serial data that it sees on it's serial Rx pin. It is compatible with FAT16 and latterly FAT32, and so can work with SD cards up to 2Gbyte and 16Gbyte respectively.

It has a fairly simple set of commands which allow you to read, write and append the files on the card, plus traditional DOS-like commands for changing directory, deleting files etc.

As part of the Arduino Slave Network project, I thought it would be neat to port OpenLog to a suitable serial slave, and have it act as a central fileserver for the rest of the network. As the slaves are somewhat limited in their RAM resource, and cannot hold much data locally, having a central storage facility will be an asset to the system.

In this way, slaves with sensors connected could record perhaps a hour's worth of data and then write it out to the communications network, where it could be loaded into the RAM buffer on the Master or Hub unit - but more importantly stored as a CSV file on the OpenLog slave.

With this in mind the first task was to make an OpenLog and get it working. I have a NuElectronics "Sensor" shield, which has an SDcard socket and a Dallas RTC on board. Most of the I/O lines on the NuElectronics board were directly compatible with those in the OpenLog design, apart for the chip select line, which is on Digital 5, rather than Digital 10 on OpenLog - which used D 5 for running a status LED.

The quick solution was to modify the OpenLog code to move the status LED down to Digital 2, and then set the pin mode of Digital 5 to be an input. I then hooked a wire jumper from Digital 10 to Digital 5 - so that D10 still selected the SD card, and D5 remained a high impedance input.
To test OpenLog, you need a terminal emulator programme. Microsoft no longer include a terminal with their bundled software, so you have to find one that works. I eventually chose Tera Term 4.67 which seems to work well. I had no success with Termite or the Arduino Terminal window - neither of which appear to send CRTL z correctly.

OpenLog Storage Control.

To access data from the Slave RAM, there will have to be a command from the Master which requests the slave to write a block of data. The Master can also be used to create the filenames for the slave data to be stored and put the OpenLog into append mode at the right time to receive the slave data.

Suppose we want to create a file called temperature1.csv to accept temperature data from Slave 1.

The Master will have created this filename using the new command:

new temperature1.txt

Then before the request to the slave for data, it should issue an append command:

append temperature1.txt

Followed by the command to slave 1 to send its RAM contents eg. write out the first 1024 16bit integers from RAM:

w,1,0,1023

The slave will then write these as comma separated integer variables finishing with a ? to inform the Master that it has finished the command process. The Master then needs to send the escape sequence to the OpenLog so that the logging to temperature1.csv is closed.

The escape sequence could always be incorporated into the end of the data issued by the slave - this would then need no intervention from the Master to close the appended file.