New stoker-web

Gary Bak

TVWBB Fan
Hello All,

I’ve been working on a web-based program to monitor my stoker and change its settings. I like the stoker-log program, but did not want to be tied to viewing the status on a single machine. The program I’m working on is a web-app that will run on a web server and be accessible from any browser, including mobile devices. The home screen is read-only, so it can be viewed by anyone. After logging into the application, you will be able to change settings. Here is a short list of features that I plan to have in the first release.

Features:
• Web Interface / multi-browser / mobile device support.
• Large dials for viewing probe temperatures and alarm settings
• Easy recognition of alarm conditions
• Supports multiple cookers and temperature probes
• Logging of cook data, supports multiple logs simultaneously
• Default page is view only, login to change settings.
• Cook profiles to store and load favorite settings.
• Displays and logs local outdoor weather information
• Email updates
• Real time graphs
The temp data is provided real time to the browser as the temperatures change, and uses server-push technology. The accompanying graph is updated once a minute. It uses the telnet interface to the stoker since the JSON response times are pretty slow.

I must add, that this program is not for everyone. It will not be as easy to setup and configure as the stoker-log. It is a web application that requires a servlet engine like Jetty, Tomcat or any other application server. My plan is to run the program on my webserver 24/7. If / when it detects the stoker online, it will start a log and begin displaying stats when I open the page.

Edit, Project site is here: http://code.google.com/p/stoker-web/

I'll try to get a screen shot up.
 

Curt Timmerman

TVWBB Member
Looks great. Can you provide a simple block diagram of the software/hardware pieces in your application.

I am working on a similar application, but it's for an electric smoker.

Curt
 

Gary Bak

TVWBB Fan
The project is all Java and uses Google’s GWT as the framework. I choose GWT as primarily a training exercise since I wanted to try it out.

I am using gwt-comet for the server push.
Jericho for parsing the http admin page
Apache Commons Net for the Telnet libraries
Gwt-visualization for the graphs ( I may include a JFreeChart option also)

The current line graph is a flash based graph, so it will not work on Apple iPhones or iPads. I have plans to be able to add the option to swap that out with the simple line graph from gwt. The drawback to the gwt-visualization library is that it requires a live internet link to draw the images, so jFreeChart will be the backup for those who don’t have a connection. Here are the samples for those: http://www.jfree.org/jfreechart/samples.html. I have used jFreeChart before to create histograms, so I don’t see that as being too difficult to implement.

I’m using flat files for local storage of settings and logs. I resisted my every urge to use a database just to keep thing simple. A DB would simplify the program interface a bit, but I’m more worried about maintenance. Statistics, re-indexing, purging are things I don’t really want have to deal with.

The GUI has been the toughest part of the project thus far. GWT provides a Java Swing like interface for building the gui, but there are quirks when building every component and it takes time to work them out. I’ve got the main page kind of where I want it so I can focus on the guts again.

I have built in the use of multiple cookers by prefixing the device name with a #_ (1_, 2_, etc). This will assign the specified device to a cooker and build a section on the screen for each one. I have limited hardware so I’ll likely have to build a simulator to provide dummy data for additional hardware. As currently written (but not at all tested) stoker-web will support as much hardware as the stoker itself. I may change the layout of the gauges to be vertical if it detects more than 3 per cooker, this would allow it to grow vertically instead of making it wider.

I still have not designed a replacement for the stoker setup where a blower is assigned to a pit probe. I thought of a good possibility last night using drag and drop items, allowing devices to be dragged into a cooker. This will have to come after the base feature set.

I used stoker-web for a cook yesterday and it worked surprisingly well since it is only a few weeks old. I have a limited amount of time to work on the project ( usually late nights ) but I’ve been pushing hard to get it to the ready state. I need to add the login logic and read-only page before the first release, since I see that as the main feature with stoker-web. I have the login and password encryption written in a separate project and I just need to integrate it. Hopefully, I have something in < 2 weeks.

Gary
 

Curt Timmerman

TVWBB Member
Thanks Gary - lots of good information

After a lot of work and frustration with the Arduino board, have decided to take a different route. There is a lot of effort out there to turn embedded boards into a PC, so I thought why not just use a PC?

I purchased an asus mini-ati board that I plan to use with the Stoker (temperature sensors) and an ethernet controlled SSR to control the heating element.

Here is a diagram. It doesn't show the SSR interface.


Like your implementation, it is not for the casual user.

Curt
 

Gary Bak

TVWBB Fan
Hey Curt, Long ago, on another project to control 220v relays, I had been working on reading thermistor data through a Vellman board. I had to create a circuit for the thermistor, wrote code to talk to the Vellman, created a class to run the Steinhart-Hart equation and though it worked and the temps were accurate, I had no nice box for all this junk and no easy way to run cables in and out. Had it all working on the bench, but the whole process was too complex and I scrapped it. In the end, as much as I like mucking with hardware, I’m a software guy, so I’ll stick with what I know best.


From your diagram, I guess everything will act off of the data persisted in the database? How will you fire events from the Query API to the Update API, does it poll for changes?
 

Curt Timmerman

TVWBB Member
Originally posted by Gary Bak:
Hey Curt...

From your diagram, I guess everything will act off of the data persisted in the database? How will you fire events from the Query API to the Update API, does it poll for changes?
The temperature monitoring/recording is a standalone application (Perl) that fires at a regular interval.

The presentation and control are handled by an ajax javascript browser application. It polls, via the API, temperature readings that have occurred since the last poll. The browser page is updated and, if necessary, the PWM SSR is toggled. The ethernet controlled SSR comes from NCD

Coding is for the most part is complete, I just have to find time to put the pieces together.

Curt
 

Gary Bak

TVWBB Fan
Slowly making progress...

* Coded local weather retrieval in Json format from Yahoo.
* Stabilized the telnet controller. Can now handle failure conditions and reconnects.
* Added file logging, format will look something like this:
c|00|01|Name
c|01|db0000116f0bec30| FOOD|ALARM_FIRE|250|075|food sensor|
c|03|230000002a55c305| BLOWER| |000|000|blower 1|
c|02|e70000116f279030| PIT|ALARM_FOOD|320|070|pit sensor|230000002a55c305
#start
d|1312948017087
d|1312948074816:01|78.3:02|79.7|0
d|1312948134942:01|78.1
d|1312948194783:01|79.0:02|80.4|0
d|1312948254766:01|78.3:02|79.0|0
d|1312948314894:02|79.7|0
d|1312948377050:01|77.6:02|80.0|0

Every point will not be logged if the data as not changed. I also changed the time format from seconds since epoch (seen above) to a more readable format.

The project was finally large enough and changes frequent enough to necessitate the need for source control. Added the project to my SVN source control. I had been 7-zipping the files and throwing it on network storage for backup.
 

Gary Bak

TVWBB Fan
Project is still moving along

Coded all save and parsing of the log file. On browser refresh, the log is read and the graph is rebuilt from that data.

Spent a bunch of time refactoring to the code to use events instead of timers and polling. This cleans it up quite a bit and allows instant feedback on the screen and instant writes to the log. Blower indicator on the browser is less than a second behind the hardware event.
Also hardware status is now event driven, connects, disconnects, etc.

Added the login button and disabled editable fields on the screen when not logged in. Still need to add the guts to the login and add a browser cookie.

Added ability to name cookers and save the value.

I have run a few cooks with stoker-web and was quite impressed this past weekend with how well it worked. Still quite a few late nights before I have something I can release. Still have a few key items left.
 

Gary Bak

TVWBB Fan
Thanks.

I decided last night to scale back what will be in the first release and just get something out there so what I've coded thus far can be tested. Without any type of deadline, I could just continue forever...

Login code works now requiring a valid username and password.
Custom logging also just about ready.
 

Gary Bak

TVWBB Fan
A little note on progress. I replaced the flash based graph I had before with a javascript one. I had troubles with the old one not painting sometimes and updating on the fly. I'm using Highcharts now and they are pretty slick. Much easier to interface with as well.

Login is working now and so is the multiple log feature. The idea is to start a new log when the food goes on and add the necessary probes to the log. When the food is done, stop the logs and there will be an option to create a PDF report which will contain the graph, target temps and any notes added while cooking.

It also has the ability to attach to an existing log, say the stoker-web crashes (as if). When it is restarted, you can choose to attach to any existing logs so they can be continued so long as the configuration is the same.

Still a few critical things left before the alpha release, but it should not be long. I think under estimated the size of this project when I began, it is now at about 60 classes and over 5000 lines of code.

Here is the latest screen shot. It still needs some css work and formatting.

 

Adam W

TVWBB Member
Gary, I have been following this thread and think your app looks great, I am really looking forward to trying it out. Thanks for all the hard work that all of us software ignorant people can benefit from!!!
 

Gary Bak

TVWBB Fan
Hey Adam, I sure hope others find my efforts useful.

I intend for stoker-web to be free, but I'm a bit torn as to release it as no charge software or as an open source project. I'm kind of leaning toward throwing out on google code and use all the features it offers, svn, bug tracking, release downloads, but I'm still not sure.

I'm running a brisket cook tonight, so I don't want to work on stoker-web while it's running a "production cook" so I decided to write an alpha punch list instead.

Here it is with a time estimate as well. The times are in days which is 2-4 hour late night time blocks: ( so 1/2 day is 2 hours
)

Fix Manage logs button (Attach to existing log) 1d
Add Stoker unavailable page w/ countdown until next retry w/ try now button 1d
This only applies when a Default log is not currently writing.
Fix master menu button layout ½d
Add note functionality ½d
Fix bug in telnet reconnect causing second cooker to be added ( new bug ) ½d
Add weather pane ½d
Fix client session tracking.
Deploy and debug on web server 2d

BTW here is a screen shot of me waiting for an overshoot to come down before adding my brisket.

 

Gary Bak

TVWBB Fan
Here is a shot when I opened the lid. The fan fired and those data points appeared on the graph. I had zoomed in on the graph where the period was just a few minutes.

 

Gary Bak

TVWBB Fan
The forever brisket is pushing 16 hours now and still not done. Wiped any development time I may have had today.

 

Gary Bak

TVWBB Fan
Some good and not so good news to report.

Good news first, as part of testing out the bad news below, I tried deploying my stokerweb war file on my web server. After installing Jetty, the deployment was trivial. I wasn’t working at first so I moved the web server out of the DMZ and it worked perfectly. Idiot me forgot to open port 22 in addition to port 80 from the DMZ into the wireless network, once I did that, I moved the machine back and it worked! All I had to do is drop the war file, configure a context file, add a stokerweb.properties file and I was off and running. I was going to report on my success last night and saw that my images in this post had disappeared. I had dropped Apache for Jetty on port 80 and I was no longer serving my screen shots. I’ll likely just put Jetty behind Apache with mod_proxy.

Now for the not so good news:
While running the long brisket cook, I noticed that browser refreshes took forever to repaint the graph. On a refresh, all the local data, on the browser, is gone and must be fetched again. The Server side reads and parses the log file and send the information to the browser, this happens sub 2 seconds, but the browser takes forever to repaint the graph. The time was in minutes. Thinking it may be the overhead of the GWT debugging infrastructure, I created and deployed a war file on the web server, using the brisket log file on the web server the graph painted much quicker, but still way too slow.

In the graph there is a point for each probe every minute, that is not a lot, but there are 4 points for every fan on/off cycle. This adds up when fan is cycling on and off every few seconds. I still think Highcharts should be able to process this many data points, so I need to revisit how I am using it. I did port a non-recursive version of the RamerDouglasPeucker algorithm to Java to reduce the number of points in the graph, if necessary, but that still does not help with the fan events as that data can’t be pruned.

I’ve toyed with another way to get the data into the chart last night, but nothing is working yet. It is imperative that the graph loads quickly.
 

Curt Timmerman

TVWBB Member
Originally posted by Gary Bak:

Now for the not so good news:
While running the long brisket cook, I noticed that browser refreshes took forever to repaint the graph. On a refresh, all the local data, on the browser, is gone and must be fetched again. The Server side reads and parses the log file and send the information to the browser, this happens sub 2 seconds, but the browser takes forever to repaint the graph. The time was in minutes. Thinking it may be the overhead of the GWT debugging infrastructure, I created and deployed a war file on the web server, using the brisket log file on the web server the graph painted much quicker, but still way too slow.
It may be too late in your design to implement but I have had amazing results using jquery/jquery.flot to plot data graphs. This all takes place with javascript on the browser. The raw data is retrieved (ajax) from a server, the data points are reformatted (trivial) for flot, and the graph is plotted on the browser. It is very fast, even with a large number of data points.

Curt
 

Top