Stoker monitoring with open software and/or Linux

[Stew Ellis]

After a lot of searching and reading on this website I decided that the Stoker would be the most open platform within my limits of software and hardware knowledge. So I ordered a Stoker with 1 pit and 3 food probes, 5cfm fan, handle, and WSM adapter on Thursday. Called Friday and they said they have all parts in stock and it should ship Monday. Trying to get my ducks lined up before it gets here.

The goal is to be able to control, monitor and graph cook sessions in as open as possible a way, meaning it has to be able to be done with a standard browser (not IE or Safari) and/or software tools that are open-source and free that can run on Linux, OSX, BSD and even Windows. Amir helpfully pointed me to webapp discussions earlier in the Winter, and Curt Timmerman pointed to his BBQ Monitor website, http://curttimmerman.net/logger/bbqmonitor.html , which I had visited before, but I had missed the buttons across the bottom which included the source to all the modules for that service.

I have now read all those docs and code, and have a sense of how it works, although I have not ctag'ed the perl and stepped through it in emacs yet. As Curt states, it does not have any fan code since he is using it on an electric smoker. It also does not do any control functions, but that is something that can be done with just the browser and the builtin http server on the Stoker. I hope it will not be beyond my abilities to add a stunt to the cgi and to the javascript? code to store and graph fan condition (off, on, % ?). Although I only live in MI, not AK, we do get kind of cold here and I would like to be able to store and chart ambient temp as well.

The only problem I see is that you it looks like you have to be running a web server to be able to record and chart the Stoker. The logging part of bbqmonitor is a perl cgi script. Of course, the iphone is going to have to access the charts on a server somewhere since I cannot write my own iphone apps. How much data does the Stoker keep? Does it keep whole cooks? Can you step back and forth in the Stoker's own web interface?

I will definitely be using bbqmonitor for home cooks, but I am a little concerned about exposing my home network to the internet. Right now all I am letting through from o/side is ssh. Haven't had to worry about web security since my university hired a full time person to run the website and server about 12 years ago. I am less concerned about security for the Stoker since they now have the ability to lock the controls. I guess I am going to have to think about ssh tunnelling of http, which I think I used to do a long time ago.

Of course all this is affected somewhat by how I tie the Stoker to my home network. Right now I have found an ancient D-link bridge/AP in my basement, but I have not fired it up yet.

Is there anyway to load barebones session graphing to the Stoker's web server?

There is also the possibility of accessing the Stoker data with telnet or ftp and parsing it into a form that can be plotted by gnuplot.

Then RJ Riememsnider drew my attention to RRD for graphing and MRTG for gathering SNMP statistics. I have loaded all the rrd and mrtg packages available to Ubuntu Karmic and will be looking through them in the next few days.

If anyone else has any ideas or code to share, please post it here.

 

[Amir]

Answering your one question, Stoker does NOT keep any past data. You can only query its current status.

 

[Joe Keenan]

As Amir said, you only have current data available from the Stoker, no matter if you're using the HTML or JSON output, or you're getting the temp feed in a telnet session.

If you want to show historical data, like graphing for a cook, you'll need to the application will need to get it from the Stoker, save it somehow, and then use that data for the graphing display. Which means you do need a fairly complex application running somewhere. Either on a client computer (like StokerLog), or on a web/application server like bbqmonitor.

Most client computers will also run a full suite of web applications (web server, perl, etc), but doing an install on a PC or Mac is fairly complicated. Much more complicated than installing a normal application. And I don't think it'll work to try and do a centralized server for users to share. Getting the remote Stokers to talk to an Internet-based server is probably unworkable.

In order to add any code to the Stoker itself, I think you would have to replace the "bbq" program on the Stoker, which is likely to be impossible without the help of Rock's and Kaytat software.

joe

 

[Curt Timmerman]

Originally posted by Joe Keenan:

Most client computers will also run a full suite of web applications (web server, perl, etc), but doing an install on a PC or Mac is fairly complicated. Much more complicated than installing a normal application.

joe

Joe check this out:
WAMP or Mac
It is a very clean installation and it will do everything you have discussed.

Curt

 

[Joe Keenan]

Originally posted by Curt Timmerman:
Joe check this out:
WAMP or Mac
It is a very clean installation and it will do everything you have discussed.

Curt

I've done all that before. It's not a problem for ME. But I do think it would be a problem for your average "smoker", looking to install a logging program. And that's based on the questions I've seen regarding setting up a network to properly communicate with the Stoker.

Maybe I'm wrong. But I don't think you or I are the right people to judge how hard this is for the average Stoker user.

joe

 

[Curt Timmerman]

Thinking out loud here.
I am building a controller around this device:
Ethernet relay
It has a relay (you can get more) for control and 8 I/O sensors for measurement. I am controlling it from a CGI application but it has a simple web server that can server up java applications (class files).

This opens up the possibility of a self-contained Stoker like controller that works with any java-enabled browser. Saving the chart data may be a security issue, I'm not a java expert.

There are a number of possibilities.

Curt

Joe - didn't mean to underestimate your skill set.

 

[Amir]

$133 for just the remote switch? I suppose you could get it to turn the blower on and off. But then your control loop must run in the PC. Should you turn off the PC or have it become slow or crash, your system stops working.

And you need to buy and build the blower and the probes.

I am not seeing the appeal of this device.

 

[Stew Ellis]

While waiting for my Stoker I have been going crazy with 'man -k' and synaptic trying to find anything that talks about plotting or graphing, then reading the docs. rrd and about everything else seem to require times in "seconds since epoch" format, so I have been looking at time formatting options also. Somehow in all of the ruckus I had completely forgotten about my old pal gnuplot, so I checked him out anew.

Although gnuplot is extremely canted toward plotting functions, it does have enough data handling to do curve fitting. It is extremely flexible in accepting time input and in labelling the graph with time tics. It also is very forgiving regarding missing data. I took a hand log from a few weeks ago, where I recorded Pit, Meat, Lid and Bottom Vent % and was able to plot in a very nice output. I am still trying to figure out additional comment labelling, for example in this plot, I only had the Meat probe in the Meat about 5 hours.

I was going to post the plot, but do not have a photo posting host.

 

[Stew Ellis]

Since everyone else who has posted to this thread seem to be talented experienced programmers (Pros?), my little effort will be laughable.

I spent the day playing with gnuplot and sh and was able to get a kind of minimal plot that at least lets me visualize the results of the three smokes I have logged so far by hand recording the ET-73 and lid thermo readings. The data files had to be hand cleaned since I had made mistakes in my entries. I used jove, a small emacs-alike editor. It has strong regex search and replace capabilities, plus can record and playback macros.

gnuplot is a powerful program, but has limited ways of getting input, without many commandline switches. It handles loadfiles that can consist of any command that is legal at an interactive gnuplot prompt, with some limited variable substitution, including back-tick expansion and pipelines. I could not figure out any way to pass a datafile name to the loadfile, however. So in order to pass a datafile name to gnuplot, I wrapped it in shell, which takes the filename that you want to plot from the commandline and uses it to construct a filename for plotfile, construct a unique title for the whole plot, and create a uniquely named temporary loadfile.

Here is a plot of a butt I did week before last:

That's 12:00 Noon

 

[Amir]

Does gnuplot support a pipe input? If not, how are you going to use it as the smoker is going? Keep creating files and feed it to it? That gets kind of clunky

.

 

[Stew Ellis]

gnuplot does have the '-' file for stdin. What I posted last night did not take advantage of that: I was sending my gnuplot commands to a temp file and then loading gnuplot with the created file that set my output filename and graph label from the shell variable $1. It was 38 lines of shell and called 2 outside commands, /bin/echo and basename. Your question prompted me to redo the shell script using a here-document. Now it is 30 lines with only 1 external call to basename. If the user does not pass a filename it exits with a USAGE prompt describing the file format expected. I have it set to produce .png files, but gnuplot can produce 60 different kinds of graphical output by changing 2 words in my sh script.

As far as how I will use it, you have to understand my work environment. My terminal has 5 overlapping xterms, firefox, thunderbird, handbrake open most of the time. To run my script it involves mousing to an xterm, typing the first two letters of the commandline followed by ESC-P<RET>. Currently I have to then run xv, which I have to type xvESC<RET>. I will probably add an autodisplay to the sh.

I still do not have a data collection stunt for the stoker. USPS.GOV shows that it is out for delivery, so hopefully by the end of the day I will have a chance to look at all the data collection possibilities. telnet seems the most obvious for an sh program. One of the nice things about gnuplot is that it autoscales the x axis for time data that is at irregular intervals.

Of course, I will probably run bbqmonitor, plus interacting with the stoker using twitter and web browsers. I will probably even try out Stokerlog on my WinXP SP2 machine that is mostly dedicated to video capture and editing.

 

[Stew Ellis]

Stoker came about Noon. Hard-wired it to my router and began playing with it. Using the web interface and holding first one probe and then the other I was able to figure out which of the 2 probes I connected was which. Once I knew that I could monitor the telnet output. I was disappointed to find that each device is a tuple, rather than the whole data stream as one tuple. I have an easier time parsing domains from one tuple rather than have each device reporting its status as a row. I will figure it out. The format is documented on kaytat.com

Also, I am disappointed that there is no fan data, although I have not been able to figure out how to turn on the fan yet. (I have barely looked at the manual.) I think this is a function of me not having assigned probes and set target temps yet. I am having a bit of trouble with the menus, I always do with printers and similar kinds of menus.

I thought I might get some more insight by comparing the info from Stokerlog and/or bbqmonitor, but I stopped short on Stokerlog when it needed .NET 4.0, and I could not figure out what all I needed for bbqmonitor. All the java include files all over the place kind of spooked me since I have not run a webserver in about 12-13 years, and have never been able to get into java.

 

[j ostheim]

http://www.bbqbuds.com has plotting for stokers (both live and from stoker log output.

I haven't done controlling the stoker yet, but it wouldn't be difficult.

All you need for the site is your stoker on an external ip or URL, the server polls the stoker for temps every 60 seconds (I might lower this, though that is plenty for plotting).

Anyway it works and is available so, feel free to try it.

James

 

[Bryan Mayland]

James's site is great, but if you're looking to just automate a gnuplot, I do this for my homebrew controller currently (until I have time to work out a slick solution like bbqbuds).

A shell script:
<pre class="ip-ubbcode-code-pre">
#!/bin/sh

while [ -z "" ] ; do
DATE=`date --rfc-3339=seconds`
echo -n $DATE,
curl -s 192.168.1.252/csv
echo
gnuplot < temps.conf
sleep 10
done
</pre>
Which I run like ./poll > data.txt &

So every 10 seconds it pulls the data from my device and appends it to the data.txt and rebuilds a graph. The stoker data might have to be massaged a bit to fit into gnuplot format, but that's what sed / awk / etc are for!

Here's my gnuplot config
<pre class="ip-ubbcode-code-pre">
set terminal png truecolor noenhanced size 1024,480
set output "temps.png"
#set size ratio 0.75
set key horiz
set key bottom left
set autoscale y
set autoscale xfix
set xdata time
set timefmt "%Y-%m-%d %H:%M:%S-0400"
set style data lines
set datafile separator ","
set yrange [40:*]
set ylabel "Temperature (F degrees)" 2, 0
set ytics nomirror
set y2range [0:100]
set y2label "Blower %" -2, 0
set y2tics 0, 25
set format x "%H:%M"
set tmargin 0.5
set lmargin 6
set rmargin 8
set bmargin 2
#set style fill transparent solid 0.50 noborder

plot "data.txt" \
using 1:3:2 linecolor rgb "#eeeeee" notitle with filledcurves, \
'' using 1:9 linecolor rgb "#ffffcc" notitle axis x1y2 with boxes fs solid, \
'' using 1:8 linecolor rgb "#cceeff" notitle axis x1y2 with boxes fs solid, \
'' using 1:7 linecolor rgb "#3169c6" notitle axis x1y2, \
'' using 1:2 linecolor rgb "red" notitle, \
'' using 1:3 linecolor rgb "red" linewidth 3 title "Pit", \
'' using 1:4 linecolor rgb "green" title "Brisket", \
'' using 1:5 linecolor rgb "blue" title "Pit Top", \
'' using 1:6 linecolor rgb "purple" title "Ambient"
</pre>
Which makes a graph something like this

 

[Stew Ellis]

Intensely cool! I did have to stare at it for a few minutes until I figured out your fan plot was the light blue filled curve. Is there a narrow yellow vertical bar at 16:30 or is that my eyes/imagination.

I have not played with the Stoker today yet because I am trying to get my 10-year-old D-link wireless bridge working.

Too bad the stoker does not send date in tuple,domain,domain, etc., csv. I am going to have to stuff the data into arrays of some sort which I have done in awk probably 15 years ago to generate row oriented data from whatever you call the other. Perl may be easier. If I prototype in sh and awk, then recode in perl, it is easier for j random user to install perl and gnuplot on a Windows machine than sh and awk. For example neither cygwin tools not the M$ unix commandline tools are available for my wife's 64-bit Vista machine, and I am not sure they are available for any of the home versions of Win 7.

 

[Bryan Mayland]

Originally posted by Stew Ellis:
Is there a narrow yellow vertical bar at 16:30 or is that my eyes/imagination.

You are correct, sir! The yellow bar indicates the controller is in "lid open mode". If the controller thinks it is properly maintaining the temperature and the temperature suddenly drops 20F, it disables the fan for a few minutes.

The blue is the fan speed. I don't know how to get gnuplot to put a little blue box by where it says "Blower %" to signify that. At about 11:20am, I swapped the position of the greed and red probes, which is why it got sort of wonky there for a bit.

 

[Stew Ellis]

Originally posted by Bryan Mayland:

You are correct, sir! The yellow bar indicates the controller is in "lid open mode". If the controller thinks it is properly maintaining the temperature and the temperature suddenly drops 20F, it disables the fan for a few minutes.

The blue is the fan speed. I don't know how to get gnuplot to put a little blue box by where it says "Blower %" to signify that. At about 11:20am, I swapped the position of the greed and red probes, which is why it got sort of wonky there for a bit.

What about adding them to the key? I would think you would just have to give them a title in the 'using' command. I also suspect that the plot using key values are displayed in the key orthogonal to the order in your gnuplot command. Leave the label for the right y the same

Is this an arduino board or a PID?

 

[Stew Ellis]

I spent a significant amount of time this weekend trying to format the data returned by the stoker into csv that could be used as input to either my or Bryan Mayland's gnuplot "program". At first I used telnet to monitor the stoker 'bbq -temps' at the tini /> prompt. The problem with this is that tini only seems to support one instance of telnet, and it only reports temps, and it does not give all the info that is returned by accessing http://stoker/stoker.json .

'bbq -temps':
260000125918C830: 1 25.4 77.7 -9.4 0.2 1 0.8 20.5 69
DD000012591D2830: 1 25.4 77.7 -8.8 0.2 1 0.8 20.9 69.7
3500001259174130: 1 25.3 77.4 -9.4 0.2 1 0.8 20.4 68.8
38000012592A9930: 1 25.5 77.9 -8.8 0.2 1 0.8 21.1 69.9

Also, the constant stream of data is too much. It would be better to poll every 5 minutes or so, and the json interface would fit that scheme better.

json:{"stoker":{"sensors":[{"id":"260000125918C830","name":"Sensor 4","al":0,"ta":32,"th":32,"tl":32,"tc":69.3,"blower":null},
{"id":"DD000012591D2830","name":"Sensor 2","al":0,"ta":170,"th":32,"tl":32,"tc":69.9,"blower":null},
{"id":"3500001259174130","name":"Sensor 3","al":0,"ta":32,"th":32,"tl":32,"tc":68.8,"blower":null},
{"id":"38000012592A9930","name":"Sensor 1","al":0,"ta":250,"th":32,"tl":32,"tc":69.9,"blower":null}],
"blowers":null}}

I am not any good at hex, but I am pretty sure that neither of the above is in alpha order.

Over time the order also changes in unpredictable ways.

So, I decided not to bother with telnet anymore. I googled and found some sample code that extracted the sepisode info for a season of southpark by retrieving the json at southpark.com. That code is at South Park example

All he does is create hashes from the name value pairs and step through each episode and print the information in a readable formatted manner.

I figured that was at least close to what I wanted to do, so I got it to create an array that contained a comma-separated record on one line of all the information I might want for each probe and then print that. I am not sure of the etiquete for sharing code on this board, but do not really have a public website where I can post code, so I am going to post the short perl program here.
#!/usr/bin/perl -w
# $Rev: 11 $
# $Author: artem $
# $Date: 2009-05-23 23:09:47 -0700 (Sat, 23 May 2009) $
# Adapted to Stoker beginning Sun Sep 5 13:38:52 EDT 2010

#use strict;
use WWW::Mechanize;
use JSON -support_by_pp;
#use Sort::Fields;

fetch_json_page("http://stoker/stoker.json");

sub fetch_json_page {
my ($json_url) = @_;
my $browser = WWW::Mechanize->new();
eval{
# download the json page:
print "Getting json $json_url\n";
$browser->get( $json_url );
my $content = $browser->content();
my $json = new JSON;

# these are some nice json options to relax restrictions a bit:
my $json_text = $json->allow_nonref->utf8->relaxed->escape_slash->loose->allow_singlequote->allow_barekey->decode($content);

# iterate over each sensor in the JSON structure:
my $sensors = 1;
foreach my $sensors(@{$json_text->{stoker}->{sensors}}){
# create an array of sensors information:
@sensor = ("$sensors->{id},$sensors->{name},$sensors->{tc}");
push(@array_of_sensors , @sensor, "\n");
# push(@array_of_sensors , @sensor);
$sensors++;
}
print "\n";
};
# catch crashes:
if($@){
print "[[JSON ERROR]] JSON parser crashed! $@\n";
}
}
#my @sorted = fieldsort ',', [2, 1], @array_of_sensors;
#print @sorted;
print @array_of_sensors;

NOTE:
I know I should be using strict, but I could not seem to get @array_of_sensors into global scope and I just wanted to see the array. The # commented parts are my attempt get a sorted array. When I uncomment use Sort::Fields and the next to last and third to last lines and comment the last line and run
the program, I do get sorted output, but also a screenful of errors:
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.
Use of uninitialized value in string comparison (cmp) at (eval 66) line 6.


38000012592A9930,Sensor 1,70.4DD000012591D2830,Sensor 2,71.13500001259174130,Sensor 3,69.3260000125918C830,Sensor 4,70.4

Also the elements of the array have the terminating \n stripped.

 

[Curt Timmerman]

Stew

Just a quick suggestion. Include:
use Data:: Dumper ;
$Data:: Dumper::Indent = 1 ;

with the other 'use' declarations.

Then:
print Dumper (\@array_of_sensors) ;
to show exactly what is in this array, in Perl format.

Also - whenever you are getting data from an external source, you have to be prepared for unsorted data, no data (tripped over Stoker power cord), or partial data (sensor came unplugged). I learned my lesson the hard way and had to do some major rewriting.

Curt

 

[Stew Ellis]

Thanks. The array had a line for each sensor followed by an empty line (\n only?). I corrected the assignment of the json values to @sensors so that \n would be at the end of each row rather than being after each row which I was doing before when I was pushing the @sensors onto the @array_of_sensors.

That also cleared up the error from whoever about the cmp operator. Now all I get is what I think I want to get:
/food$ perl get-stoker-json.pl
Getting json http://stoker/stoker.json

38000012592A9930,Fire,70.6,
DD000012591D2830,Food1,70.9,
3500001259174130,Food2,69.2,
260000125918C830,Food3,69.7,

So I can sort on field 2. However, just now I tried to do a numeric sort on the first field:

DD000012591D2830,Food1,70.9,
38000012592A9930,Fire,70.9,
260000125918C830,Food3,69.9,
3500001259174130,Food2,69.9,

It did not complain about the format of the numbers in field 1 (they are hex, right?). How does perl want those numbers formatted? However the order does not make sense to me. If I do a num sort on field ['3n', 2] it is ascending with the identical values sorted by ascending subkey 2.

38000012592A9930,Fire,70.6,
DD000012591D2830,Food1,70.6,
260000125918C830,Food3,70.6,
3500001259174130,Food2,71.1,

Except for the probe ID, I am now on my way. I may even be able to fix things so I can restore 'strict'.

The next step is to figure out the easiest/best way to add the current time to the beginning of each record and then create a flat file of the information that I want to plot from my csv file.

Once that is done then all I will have left is to wrote a timed polling loop to write the data to a logger file.

Then I will take the ideas from mine and Bryan's gnuplot config approaches and redo them in perl.

In the end I will probably want to have some way of recording comments to go along with the data for adding foil times, meat types and sizes, weather (actually I will probably use my food3 probe for logging ambient temp most of the time.

The only things anyone else would need would be perl with appropriate CPAN modules and gnuplot, all of which are free and multiplatform.