Feature Spotlight: LidTrack Smarter Lid Mode Detection

Bryan Mayland

TVWBB Hall of Fame
I've been working on a new upgraded feature for the past week and I thought I'd put out a little preview as what's in store because I'm tired of looking at the code for today. This is not available in the snapshot yet (pending UI changes) but look for it to be available after Thanksgiving. Don't want to make such a big change right before one of the biggest smoking days of the year and also not be around to emergency hotfix it if needed!

What is (the current) Lid Mode?

HeaterMeter has Lid Mode detection for the purpose of automatically shutting off the output when the smoker lid is opened and the temperature drops. This can be activated manually by pressing the LEFT button on the device, or clicking the "Lid Closed" text on the top left of the webui home page. The current way the automatic lid detection works currently is by setting a simple threshold temperature that if the Pit probe goes under it, just disable the output for a specific number of seconds or until the temperature returns to the setpoint. My normal procedure when opening the lid though is to activate it manually with the button before opening the lid (it won't auto-disable for the first 30 seconds of lid mode so it is safe it the temperature is already above setpoint) then open the lid and do my business.

It is almost always advisable to let the lid mode tick down to completion on its own after closing the lid. Turning off lid mode manually just after closing the lid returns to automatic control immediately while the temperature is still very low, which leads to the controller going 100% full speed ahead and creating a lot of overshoot on the recovery. Letting the timer expire on its own allows the majority of the recovery to happen on its own at 0% output before picking control up from a position much closer to the setpoint.

"So sounds pretty OK already, what's the porblem?"
  • Lid Mode can activate even with the lid closed if the trigger percentage is too small to accommodate for normal dips in temperature, requiring a high activation percentage.
  • Lid Mode can take too long to activate when it is needed, causing the controller to run at full speed before the lid event is detected which stokes the fire too much and leads to overshoot on recovery. This is the opposite of the first problem and requires a low activation precentage.
  • The duration of the Lid Mode is fixed which means you have too much time sometimes and not enough time other times. Sometimes I reset the countdown mid-lid because the meat just won't behave and now I've got too much time and argh this is a nightmare. This requires a high lid mode duration.
  • Letting the timer tick down on its own with the output at 0% can start to snuff out the fire if left too long, so again we've got a conundrum where now we want a low lid mode duration.
Enter LidTrack

LidTrack is here to solve all those problems. Either that, or create something so complicated it fails to work properly ever! This new feature runs on the Raspberry Pi, in linkmeterd's Peak Detection plugin. It uses temperature peaks, trends, and recent history to make smarter choices about when the lid is opened and when control should be returned in a way that the AVR just can't fit (believe me, I wrote most of it there first before restarting from scratch).

To detect a lid open event, LidTrack watches the trend over the last 5-20 seconds and detects sharp drops in temperature. If the average temperature drop per update drops off by a mean more than 0.5F per second over at least 5 seconds following a period of lower variance, and is following a downward trend after a detected high peak. BOOM. Welcome to Lid Town.

Now in Lid Mode, LidTrack waits for the temperature to bottom out. The can be because the temperature has stabilized at with the lid open or the lid has closed and is starting to climb again. This is detected by a period of downward trend which reaches a minimum and recovers by a small amount. Establishing this as a baseline, it continues to look for peaks of large amplitude just in case the temperature bounces around while the lid is open. It patiently waits for the temperature to climb by a large amount from the lowest detected peak and then start to plateau. This is the natural recovery that occurs when the lid is closed. Before the temperature can start dropping again, LidTrack ends Lid Mode and resumes normal automatic smoker operation.

"I dunno it sounds like pretty much the same thing, what's the advantage?"
  • Lid Mode kicks in faster. With the old standard 6% setting and a pit temperature of 225F, the temperature would have to drop below 211.5F before activating and the output would be pegged at 100% for a few seconds before this happens, and would take close to 10 seconds. With LidTrack, Lid Mode kicks in in 5-7 seconds, usually before the output even reaches 100%.
  • Fewer false positives. Sometimes a smoker just gets a little cool, yo, and I've seen situations where the old Lid Mode would kick in as the controller is struggling to bring the temperature up. The temperature was dropping slowly, the controller was ramping up, and uh oh now we've hit 211.5F so we hamstring the controller by shutting off the output when it was needed most. No hard limit allows the controller all the time it needs to find the right output and avoid a poser lid event.
  • Lid Mode can activate quickly even if the starting temperature was above setpoint. The old hard 211.5F limit means that if you were hovering at 230F, you have to wait 5 extra degrees now. LidTrack doesn't care what the temperature was, just that it dropped fast and is below setpoint. It kicks in just as fast if the temperature was 220F or 230F.
  • Long Lid Mode durations are good. Take your time restacking that meat, saucing those ribs, or foiling your butts. LidMode knows when you're done working and won't timeout early.
  • Lid Mode ends exactly when the pit temperature with the lid closed is reaching equilibrium.
  • LidTrack adjusts itself based on if you're using Celsius or Fahrenheit and no longer screws users who use a sane measurement system like the current 6% hard limit. Remember that hard 211.5F limit? For Celsius users, that limit was 214.7F which was more easily falsely triggered.
Wow how are you still reading this wall of text? Let's see some example images.
LidTrack.png

Here's a graph with some triggers. Note that the width of the orange Lid Mode areas are different durations! Control resumes before the temperature even starts going down after the lid is closed. Now that's some well-coded logic. It even still works if you manually trigger Lid Mode like you're doing all the work, jumping in mid-lid to return control at the optimal time.

LidTrack-HighTemp.png

Works great at other temperatures too. 350F, 500F, 108C, or 260C (not shown). Notice the lack of a lid event during the 500F run where I closed off all the vents. The slow temperature drop meant that even passing the old 6% threshold didn't trigger anything and the controller was able to bring the temperature back up to setpoint. Well, until I screwed it by opening the lid, where it activated in just 5 seconds. Is that a Thermoworks Pro-Series probe at 560F? Eeee. Maybe? 🤐

I implemented this four times in different ways after some ideas I had proved to be too slow to activate, too slow to return control, generated too many false positives, or just didn't work right at all. It works with my regular thermocouple, it works with a slower-reacting thermistor. I think I've got it now though and all it cost was a bag of lump and everything smelling delicious for a week.

Look for this soon along with a couple other fixes and features in the next snapshot. Now is the time where you ask questions or make observations and show me how I wasted my time or overlooked something vital that's going to make this a nightmare for everyone else but me? I feel like this is the most advanced Automatic Lid Mode Detection scheme available today and you'll get it free because for some reason this stuff is fun for me.

EDIT: My friend who usually comes up with the best names for things has really let me down as she doesn't have a cooler better name for this than I came up with, so your suggestions for a better awesome name will also be considered. Wait. Lid Awesome? Lidlord? Marbel's The Incredible Lidulk? Hrm...
 
Last edited:

Gary V

TVWBB Fan
Holy s**t Brian. Nice piece of writing. You have come up with a great solution for the current lid detection algorithm. Any chance Turkey day usage will be available. My turkey is a calling.
 

Steve_M

TVWBB Guru
Very cool, Bryan! My day job is working with data scientists, most who have PhDs in all sorts of fields.

Don't forget to properly brand this new feature with AI and ML buzzwords!

"LidTrack(tm) powered by MeterAI(tm) is the worlds first fully autonomous bbq lid tracking system utilizing a complex set of algorithms that applies Machine Learning to predict and react to both human and non-human events"
 

Tom O.

TVWBB Member
Just reading that made my head hurt. Had I tried to really understand it, my head would have exploded. Please keep that in mind.

Tom
 

Bryan Mayland

TVWBB Hall of Fame
Holy s**t Brian. Nice piece of writing. You have come up with a great solution for the current lid detection algorithm. Any chance Turkey day usage will be available. My turkey is a calling.
I just pushed out a new snapshot with LidTrack built in. It is disabled by default and is still waiting on UI in the config page to enable it. I'm not going to finish the other changes I am making to the config page today, so I figure I will release it now for anyone wanting to try it. Since there's no way to turn it on in the webui, you'll need to go to System -> Startup -> Local Startup and add a line to turn it on. If you haven't added your own stuff to the startup script then just make it look like this:
Code:
# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.

uci set linkmeter.daemon.lidtrack_enabled=1

exit 0
If you've added your own stuff to the startup script, I bet you'll be able to figure out what the important line here is ;) IMPORTANT: You'll probably want to disable the existing lid tracking by setting "Activate at X% below setpoint." on the config page to 0, and extend the duration beyond the default, because LidTrack should now end Lid Mode early as it sees fit. I set mine to 600 seconds. Be sure to test it once before relying on it!

You'll see LidTrack messages in Status -> System Log, although it won't actually enable / disable Lid Mode unless you turn it on with the line above.
 

Bryan Mayland

TVWBB Hall of Fame
Just reading that made my head hurt. Had I tried to really understand it, my head would have exploded. Please keep that in mind.
You're right about that! The short version is the new automatic lid tracking code should:
  • Shorten the duration of the Lid Mode countdown when it detects the lid is closed and the temperature has recovered substantially, for the fastest setpoint recovery with less overshoot
  • Activate Lid Mode more quickly when the lid is opened
  • Prevent false triggering of Lid Mode due to slow pit temperature variations
The only important caveat is that the user has to let the Lid Mode end automatically, not override it by manually turning it off (by pressing the button on the device) when closing the lid.
 

IvanTrail

New member
So, what that says to me is:
Second derivative of the temperature curve over a certain negative threshold followed by a negative first derivative shuts it off.

Positive first derivative followed by a negative second derivative threshold turns it on.

Second derivative is concavity and first is slope for our non-calculus speaking friends.

Looks pretty solid to me man. Thanks for the hard work!
 

Bryan Mayland

TVWBB Hall of Fame
Positive first derivative followed by a negative second derivative threshold turns it on.
Actually it is closer to "Large positive deviation followed by a low positive first derivative threshold turns it on." The slope of the return is not considered, only that the value is much higher than the low peak. I don't track concavity due to the complexity of trying to curve fit a small number of points to what might not be a smooth curve with variable width (for lack of a better term) of the humps.

Ideally I'd be curve fitting it all and be able to detect inflection points and use them to make predictions about upcoming peaks but I didn't have much success with that.
 

Top