The Development Log


 
Only the last low and high peak are displayed on the graph, and only when your mouse is close enough to the peak. The extra values on there are to help you tune your PID, which would use the period of the oscillation (the time from one high to the next high, or the low to the next low). The "half" number is the last half of the period (the last low to high or high to low) which can be compared to the overall period to see if your low to high is short but high to low is long which would indicate you're giving it too much power and overshooting or you can't damper the fire enough to bring the temperature back down. The "deg" is the gain from the last period's high-low to this one. If you've got perfect oscillations where each time the high and low are the same, this is 0. If it is a positive number things are getting more out of control over time, if negative, they are dampening around the setpoint.

Edge and IE don't support server-sent event streaming so they poll HeaterMeter every 10 seconds, Chrome, FF, Opera, Safari all stream the data which updates every 1-5s as it changes. Therefore you and end up seeing more "noise" in the graph because it contains up to 10x more data. Edge has had this web standard "under consideration" for development for like 3 years now.
 
I've pushed out a new snapshot release of just the AVR firmware. This build adds support for longpress handlers in the menu system so it fundamentally changes some of the underlying button detection code that we love to break so much. That's why I need some testers before this gets rolled into the wrt/lede firmware as the default!

The way it works is that each menu state can declare that it supports a long press of a button (>1 second). When a button is pressed down, HeaterMeter sees if the current state has a longpress for that button. If it does not, the handler fires immediately so press is responsive as possible. If it is held down then it will start a repeat count which accelerates things like quickly changing the setpoint in 5, 10, or 20 degree increments. If there is a longpress handler, then HeaterMeter waits to see of you hold it for the full 1 second. If you release before 1 seconds, the short press button handler fires. For longer than 1 seconds, the long press handler fires. If a longpress event occurs, or a state transition (via shortpress) occurs into a state with a longpress handler for that button, the button must be released before that button will function again.

This is hard to describe so I've also added the first longpress handler to give you something to test with: PID Output Off mode. A left button longpress from the device's "home" state (the one that displays the temperatures or -No Pit Probe-), will toggle the HeaterMeter output on/off. This provides an easy way to turn off the HeaterMeter output when a cook is done without setting the setpoint to a very low value to effectively disable it. An often-requested feature finally makes it in!



  • A left longpress while in auto output mode will disable output, LCD home will display [Off] as the output
  • Left longpress again will return to auto mode with the original setpoint
  • Left short press in auto mode still toggles Lid Mode on and off, but only functions if not [Off]
  • A left longpress in manual mode will set the output to 0% (this is not a toggle, always 0%)
  • Left short press in manual mode lowers the manual output by 1% (as it had before)
  • Pressing left from any of the device menus should immediately return you to the home screen, even if you hold the button down and NOT toggle on/off even if you continue to hold it for 1 second, you must release it after entering the home state and then press and hold again to trigger on/off
  • Setting a setpoint or manual output automatically turns PID control back on
  • The [Off] state is not persisted across device power cycles, but most likely will be

This also fixes a couple small bugs in the AVR firmware:
  • Pressing "back" (left button) from the menus would not update the display with the home screen for up to 1 second if there were no food probes detected. This has been fixed so exiting the menus updates the display immediately.
  • If a setpoint of just "C" or "F" was sent via the webui, HeaterMeter would enter manual mode. It now stays in whatever mode it was in and just updates the units.

To test, just go to LinkMeter -> AVR Firmware from the webui and select "Online Repository" and flash
Code: 
Location: snapshots/trunk/heatermeter.hex
MD5: 136d1bcb469398f32cc3d183e4748c95
Version: 20170722B

You may return to the regular firmware if you experience any difficulties by flashing the "bundled firmware " hm.hex from the same page. If you do test and you do have problems, please report back with what menu state you were in, what button you pressed, what was supposed to happen and what did happen. Also include which probes were plugged in, and if you were in auto or manual mode, and any other details you can provide to help me reproduce it.
 
Last edited:
Long press left worked to shut my HM "OFF" tonight... NICE!
Great work Bryan, I'll let you know if I spot anything strange.
 
Since it sounds like I didn't screw anything up with the longpress code, here's another chance for me to have screwed something up! New full snapshot and AVR firmware 20170730B posted
  • Include "HeaterMeter" in mDNS advertisement (this was dropped when switching to the new umdns).
  • Include longpress firmware bundled into the Pi firmware.
  • Units (F, C, Resist, Adc, Off) sent through the webui is no longer case sensitive.
  • Add support for HeaterMeters in "Off" mode. Setpoint displays as "Off" in the home page and blank in the config page. The database will not store any setpoint while HeaterMeter is set to Off mode.
  • Now shows output percent in manual mode on both webui home and config pages when in manual mode (used to always show setpoint).
  • Current HeaterMeter configuration is also stored as part of a stash operation, and these are included in stash backups. Not doing anything with this yet though.
  • Fix broken AVR Firmware "Local File" broken when adding API.
  • Fix AVR Firmware "Online Repository" not loading if page was loaded over HTTPS. Now pulls data over https regardless of protocol the page was viewed with. Thanks Steve!
  • (AVR) Refactors HeaterMeter Auto / Manual / Off mode. Now is stored across reboots. This affects lid detect mode and startup mode as well (although those aren't persisted, they are new substates of the Auto mode).
  • (AVR) Prevent alarm arming and ringing when in Off mode.
  • (AVR) Fix going into Lid Mode when setting to Off mode.
  • (AVR) Longpress Left on the Home screen when in Manual Mode now switches to Off (was sets output to 0%). It just felt more natural after testing to have it do the same thing as when in Auto mode.

So now in any setpoint edit box (webui home and config) you can use these to switch modes
  • Just a raw positive number: Set to auto mode using current units
  • Positive number with units C or F (e.g. 225F): Set to auto mode and change units
  • Percent (e.g. 50%): Set to manual mode and set output percentage. 0% is still manual mode.
  • Negative number (e.g. -50): Synonym for percent
  • the word "off": Set HeaterMeter mode to OFF, disable output and alarms. Actually only the letter 'o' is needed (case-insensitive).

I'm going to spruce up the setpoint edit box on the home page so you don't have to remember all the keywords, but html is hard you guys. Now that OFF mode is in, I also want to extend the archive ability to have an option to automatically stash the database and HM config when switching to OFF mode and / or reset the the database when switching out of OFF mode.
 
  • Change from rtl8xxxu driver to rtl8192cu, which actually works on Pi2/3 and supports AP mode again
  • Allow changing of the graph probe colors from CSS with .pcolorX class (e.g. .pcolor1 for Food1)
 
Being able to reset your RPi's configuration to default is now back in the latest snapshot. In fact, the whole filesystem will now be reset back to default instead of the old way, which tried to capture the /etc/config directory somewhere during firstboot. The new images use a read-only squashfs system and mount a journaled filesystem on top of it. This allows us to completely wipe any filesystem changes and go back to stock. There should be nothing you can do now that "Reset Config?" from the device menus can not undo. This is also good because if your filesystem becomes corrupted somehow, you can just reset config and you'll be back to a bootable system.

This is now v14 Release Candidate 1. There's a lot that had to be changed to make this work as far as how the filesystems are laid out and the way we access this functionality. I've tested on a Zero (nonW) and Pi 3 as well as migrating a config from v13, which no longer requires a second reboot on the first boot.

Preconfigured images for your wifi network are at https://heatermeter.com/dl/
Clean install or upgrades can use the images from the snapshot images

- Technical Story Below -
I've been trying to get this working for a while now as a TODO item that would frustrate me and I'd put it aside. However, this was the last thing on my TODO for this release and it was the only thing left. Saturday I was able to get an ext4 overlay on an ext4 root running in userspace. Sunday I was able to hard code some stuff and get ext4 overlay on ext4 root but it was a chicken-egg problem. How can you write the initialization geometry to get the rootfs mount to just pivot to overlay if the rootfs is read-only? Both of these are extroot scenarios and not really native overlays by LEDE standards. Reading through a couple thousand lines of code, I found that one of the root volume drivers can subpartition the rootfs partition if the filesystem is squashfs and there is space left on the partition according to the msdos partition table. A bunch of tweaks to the build system and we've got it: a squashfs root with a real ext4 overlay.
 
Nicely done on the overlay. Some of the other development boards are doing this as well. Allows you to boot off the small on-board storage and then overlay the SD card as the new root partition.
 
Pushed out a new AVR firmware only to the Online Repository. This only adds one feature which is that the servo stays active all the time while the HeaterMeter is on. If the HeaterMeter is switched off (left long button press) then the damper should close and deactivate. I've been using this for a couple weeks and it feels like it works ok so unless anyone has a problem with it, I'll bundle it into the main Release Candidate and we'll call it done.
 
I've spent some time of the past few days getting everything squared away with allowing heatermeter.com as a mail relay for being able to send notifications without having to put your personal email account/password in HeaterMeter, which is not a secure device. See the wiki using-the-heatermeter-mail-relay.

There are quite a few restrictions to make sure that this system can't be abused (spoiler: it still can be abused).
  • Your HeaterMeter must be in the HeaterMeter Device Registry. If nothing appears there, no go.
  • Your device must have checked in within the past day. (may further restrict this)
  • Authentication must be ON. Username is heatermeter, and your password is your RaspberryPi's serial number, with leading zeros, no whitespace.
  • From address must be notify@heatermeter.com. I will not relay mail "from" another domain.
  • Your connection must use TLS.
  • Your ISP must allow port 587 out, un-proxied or filtered.
  • You can not send to multiple recipients in the same message.
  • Maximum message size is about 1KB, there is some flexibility due to header sizes here.
  • Connections will be throttled to prevent abuse-- do not send a lot of test messages within an hour of when you expect to send real notifications.

If anyone else has any ideas to further lock it down I'd be interested to hear them. I've tried to do my best to secure it, but this is an open source project so there's no real way to really restrict the client when anyone can be a client if they follow the proper registration steps. Looking down the road it might be required that users create an account at heatermeter.com and they can only send to registered email addresses or something. But let's see how this works because I think it is pretty trivial to get setup.

I'm holding the release on this because I think I'd like to have this information be the default SMTP configuration. This can all be setup automatically on firstboot if there's no other msmtp config file and everything seems to be working out for people.
 
I think it's a good idea in that it reduces the complexity. I would leave it disabled by default, allowing it to be easily enabled, with some wording explaining what you are agreeing to. Some people might not be ok with the idea of heatermeter.com knowing the email addresses of it's users. I would also suggest you purge or sanitize the mail server logs to reduce exposure.
 
To play Devil's Advocate, Steve..... don't use the service if you don't want the service to have any knowledge of you. Expect *EVERYTHING* to be logged. Personally, just to protect himself, I'd expect Bryan to keep the mail logs around for 30-90 days, at least, just on the chance a hole is found and it turns into an open relay.

Me, I'd override and use my own mailserver.... then again, I've also discovered that my ISP is apparently oh-so-helpfully dropping outbound messages from my HeaterMeter to my phone, using yet another address as a return. Setting the return address to my ISP's domain apparently allows it through. Yes, the SMTP connection is authenticated. And no other traffic is affected.
 
Another suggestion for the heatermeter.com email service would be to validate the recipient address before sending emails. This ensures that a) you know you've put in the correct email address and b) you reduce the risk of info@heatermeter.com becoming flagged as a spam account.

It might also be better to change the from address to something more specific, such as hm-alert@heatermeter.com
 
I kind of agree about the opt-in policy, but there isn't a way to really disable it because if it is configured it will work if you use it. I wouldn't want to leave it configured except leave out the password or something, because then it isn't clear how to enable it without reading the docs.

As far as logs, I'm going to have to keep some logs, at least 30 days to be able to run analytics on to check for abuse. I'll probably just put up a Privacy Policy page as a start that indicates that it won't be used for commercial purposes.

The validated email thing is also a good idea but then I need to start keeping a database of validated email destinations too, which isn't great because the email address is tied to a specific device. Also someone would have to create a whole UI for submitting your email address for validation and being able to remove addresses from the list and also some secure way of removing an email address tied to an old device but not allow someone else to remove your email address. I think it is too much to expect from this sort of anonymous interaction the user has. If the user had a login to HeaterMeter Live, the cloud-based real time platform, then we could do all these things because we're actually linking devices to users and therefore validated email addresses. But that isn't built yet!

I do prefer alert@heatermeter.com, you're right. I think I'll change it to that in the next pass.
 
Snapshots and you! Today's snapshot includes fixes for major security vulnerabilities in dnsmasq.

This also includes pre-configuring the SMTP client to use the new heatermeter.com gateway if no other server has been configured. If the msmtprc config file has the default host in it (mailhub.oursite.example) or is heatermeter.com, we'll autoconfigure the settings with the username and current Pi serial number. This happens each time the firmware is installed or upgraded so moving to a new Pi should just work if the user reflashes the firmware or resets the config. I was trying to decide if this should be done on every boot, but that seems unnecessary. I guess we'll see how that works in the long run.
 
I posted this yesterday here but maybe I closed the window after I viewed the preview? In any case there's a new feature in the latest snapshot: Windows Network device discovery!


I can't remember what I wrote here yesterday because it was 4 paragraphs long and this morning I've got nothing else to say about it. I've made an SSDP search responder, which is the Windows way of doing Zeroconf service discovery. HeaterMeter devices on your network will now show up in the Network section of windows explorer, and double clicking them will launch a browser to take you to the device home page.

I've also noticed that the umdns service is broken again in the last few snapshots. Guess what? They've added experimental seccomp support to it and this breaks umdns on 100% of OpenWrt/LEDE devices (which do not have seccomp compiled into the kernel)! I'm working to get it resolved upstream but the next snapshot will have a temporary band-aid to fix it for HeaterMeters.

I am still working on the 8192cu support, as the v14 release wifi driver is absolute garbage. It drops off and reconnects every few hours which makes it worthless unless you can do all of your smokin' in the first hour or two after bootup. Users with Edimax and other 8192cu adapters are advised to hold off on upgrading until this can be resolved. If you didn't see it in the other thread, the snapshots have updates for the wifi services to remove the KRAK attack vulnerabilities as well.
 
Looks like there is a bug to fix in the SSDP responder too, when the networks change (like when it switches from AP mode to Client on first boot) the socket rebind on the new address is failing so the program exits. I'll be fixing that but in the meantime it means the HeaterMeter won't show on the Network display on the first boot unless this is an upgrade install. After that it should be fine, as long as the IP address doesn't change.
 
  • Fixed SSDP crashing if network interface list changes and there's no multicast route available yet
  • Fixed umdns service failing to start on snapshots. Much thanks to Michal Sojka for fixing this promptly upstream
  • Temporarily switch back to rtl8xxxu driver for 8192cu (Edimax, etc) adapters for reliable client mode until I get the other driver fixed. This means no AP mode for these devices but I'd rather have a working client mode than one that does both AP and Client mode unreliably.
  • Renamed SSDP service description to suggested format: "hostname (HeaterMeter)"
 
You must log in or register to reply here.

 

Back
Top