Help - HM broken?

[Tony L-Iowa]

For a while I thought I've had probe noise issues but now the HM don't log valid temps for the probes anywhere. Curious if I can replace a part to get things working again. I resoldered and it didn't help. My thermocouple seems ok, just the probe readings are off the charts. I'm in the middle of a dual Boston Button smoke and this is the past 9 hours graph. Near the end I switched over to a battery pack to rule out electrical noise from outlet.

 

[Tony L-Iowa]

One more note. I had Sept 2015 firmware. I just now (at the time of this reply) upgraded to the Feb 2016 repository update.

 

[Tony L-Iowa]

Just loaded the latest hm-noise.hex AVR and here's the pic from that:

 

[Steve_M]

How does it perform outside of using it with the electric smoker? Have you tried putting the probes in water and bringing them to a boil? What about your kitchen oven?

I wonder if there's some interference between the circuitry in the electric smoker + SSR that's causing the HM to behave oddly.

 

[RalphTrimble]

are you sure your probe(s) aren't just fried from taking on water or being over heated? The erratic temp graph you posted looks like what I got with just about every probe I've killed one way or another.

As for troubleshooting the HM board to solve this problem, there's not a whole lot to it. The probes come in through the input jacks, there are only 3 components between the probe jack and the ATMega, those are the sets of three components that stand on end behind the probe jacks (the resistor/capacitor from the RC filter and the pullup resistor). After the RC filter and pullup resistor the probe leads go directly to the ATMega pins, starting in the corner down behind the pit probe jack, working back from the corner Probe-0 (PIT) through Probe-3.

So you can reflow solder on the components that are standing on end, and the four pins of the ATMega in that corner, and the probe jacks. Then clean that area of the board with some isopropyl alcohol and a soft bristle brush (like an old toothbrush). It could also be erratic 3.3v supply to the pullup resistors causing the bad readings, so measure that to make sure it is stable.

If you still have the noise when running off a battery then you can rule out power supply interference... My first guess would be bad probes...

 

[Tony L-Iowa]

How does it perform outside of using it with the electric smoker? Have you tried putting the probes in water and bringing them to a boil? What about your kitchen oven?

I wonder if there's some interference between the circuitry in the electric smoker + SSR that's causing the HM to behave oddly.

I haven't tried that yet. When I first built the unit I don't remember having these kinds of issues and was using it with the same smoker. I believe things do work better (or at least used to) when I wasn't plugged into the same GFCI outlet as my smoker outside. But recently I've tried ruling out A/C power by switching over at times to my battery pack. I did that at 10am today and it did not stabilize.

I will try the boiling water test and report back. Still cooking the boston butts right now. May not be able to do the test today with company coming over tonight.

Strange thing happening now. For the past 30 mins things have started to stabilize with the probes and they are near the temp I expect (I have another battery sensor probe to gauge real temps since this is misbehaving during this smoke session). My non-HM sensor says 183 while my HM probes now say 189. You can see my HM here:
http://bbq.tonylyne.com/

 

[RalphTrimble]

It's not a bad idea to keep a new probe around to use as a sanity check if/when you have these kind of problems. When I used Maverick probes I had problems quite often and I would plug in the new probe to see if the issue was with the HM or the probes. Since I've been using Thermoworks Pro probes and TC pit probe I haven't had a problem in a very long time. To me it looks like you have bad probes...

 

[Tony L-Iowa]

Ralph, I have thermoworks probes and have a 3rd new one still around in my basement. When these issues first started up I thought they were the probes and then I got new probes and same thing happened. I will do the water boil test indoors soon and report back. That should rule out the probes and possibly confirm if there is some issue between my electric smoker and the HM interfering.

 

[Tony L-Iowa]

Also, I wouldn't expect 2 probes to be bad and behave EXACTLY like their counterpart. Below is a graph of this session showing that the probes are behaving the exact same. The only common piece there would then be the AtMega and the 3.3V power feeding pullup resistors. Maybe I need to look at the 3.3V power closer. It comes from the MCP-1700-33 regulator?

 

[RalphTrimble]

So I am confused, are you currently using the HM to control an electric smoker via SSR, or using the HM with a blower to control a real fire?
If you are using an electric smoker then you certainly did not power that with a battery, must have AC power there on the other side of the ssr. So perhaps there is interference coming back from the SSR? You could experiment a bit with isolating or grounding the SSR shield independently or grounding it together with the HM to see if that effects the interference.

 

[RalphTrimble]

I see now in those recent graph how the two probes track with each other, you're right, bad probes wont do that...

If you are using an SSR you might unplug the SSR and let the HM operate without it for a sec and see if the probes level out, if so then it is leading you in the direction of the SSR as the source of the noise

Inside the HM the probe input circuits are completely isolated from one another. The things they have in common are the 3.3v on their pullup resistors and ground, and the ATMega (though they each go to their own pin on the ATMega and should be isolated in theory)

Excess flux from soldering can cause interference on the probes pretty easily, you could have flux shorting out a couple probe signals at the same time causing a similar fluctuation. The alcohol bath should cure this if it is your problem, and again, measure the 3.3v on the pullup resistors to make sure it is stable.

 

[RalphTrimble]

One more thing that comes to mind after seeing the probe readings level out after some time. This could indicate a bad solder joint somewhere on the HM board that makes better contact after the HM board warms up a bit, most likely somewhere in the 3.3v line that goes to the pullup resistors. Unless off coarse you've made changes in the rig to cause the probes to level out, like isolating or grounding the probe braids, the ssr etc.

Here is an image that shows the path of the 3.3v on the board, it would be a good idea to reflow solder on those points to eliminate possible cold solder joints, particularly on the ones where the trace goes from one side of the board to the other.

PS this image was created to help troubleshoot a flakey TC, so I didn't highlight the 3.3v leads going the the standard probe pullup resistors. They all come off of pin 20 there on the ATMega by the oscillator.

 

[Tony L-Iowa]

So I am confused, are you currently using the HM to control an electric smoker via SSR, or using the HM with a blower to control a real fire?
If you are using an electric smoker then you certainly did not power that with a battery, must have AC power there on the other side of the ssr. So perhaps there is interference coming back from the SSR? You could experiment a bit with isolating or grounding the SSR shield independently or grounding it together with the HM to see if that effects the interference.

I am using the HM to control an electric smoker via SSR. The AC power obviously drives the heating element load. I have had the HM powered by AC/DC adapter in same outlet box and I have tried HM powered by batteries. Do you think interference coming back from the SSR is possible? I could definitely try different grounding methods as you suggested. Right now the SSR is on a heat sink under my smoker. Here's some links to my setup (if you have a smokingitforums account):

http://smokinitforums.com/index.php?topic=2765.msg23809#msg23809

 

[Tony L-Iowa]

I see now in those recent graph how the two probes track with each other, you're right, bad probes wont do that...

If you are using an SSR you might unplug the SSR and let the HM operate without it for a sec and see if the probes level out, if so then it is leading you in the direction of the SSR as the source of the noise

Inside the HM the probe input circuits are completely isolated from one another. The things they have in common are the 3.3v on their pullup resistors and ground, and the ATMega (though they each go to their own pin on the ATMega and should be isolated in theory)

Excess flux from soldering can cause interference on the probes pretty easily, you could have flux shorting out a couple probe signals at the same time causing a similar fluctuation. The alcohol bath should cure this if it is your problem, and again, measure the 3.3v on the pullup resistors to make sure it is stable.

I reflowed solder on the entire board about 2 months ago and cleaned it with alcohol. Made no change in this problem. I'm now done with my smoke and the meat is resting. Preparing to bring the HM/probes inside and do an ice bath to boiling water check. If this passes it would show that things work with the SSR out of the loop and that it's most likely the culprit, yes? If that is the case do you think my best best is to ground the SSR somehow directly to the HM? Maybe I need to use one of those wires in the ENET cable bundle for that? Is there a ground on the RJ45 pins anywhere? I only currently have the blower + and - hooked up to the SSR currently. Which isn't pin 4 the ground already and that should be grounding the low voltage side of the SSR with the HM?

Also, I have the 60Hz noise cancellation enabled that Bryan added to the configuration page. Not sure if that helps/hinders for this problem but I remember it making a difference when I originally enabled that feature.

 

[Tony L-Iowa]

This is interesting...

So, I moved the HM inside. Plugged in two probes from last cook room air temp. They both bounce around a much higher than room temp readings (first blue circle). Then I put the probes into an ice bath (red line after first blue circle). Probe 1 goes high (~450) and probe two goes near the ice bath water temp (~37). Then I unplug probe 1 (second blue circle). Grab my new and unused thermoworks probe and plug it in (3rd blue circle). Temp falls inline with probe 2 (~37) which should be roughly accurate for ice bath. Then I unplug new probe and plug back in the bad probe and it goes to bad readings again. Maybe my probe(s) are shot. I thought these were supposed to be quality probes. I haven't submerged them and they've only had maybe 20 smoking sessions put on them (all I use them for). I'm going to start the boil test soon and keep the results coming in.

 

[Tony L-Iowa]

So, the ice to boiling water test was interesting. I found that it does appear I had a bad probe. The thing that gets me is how one probe going bad can throw off the other probe? Shouldn't these probe inputs be complete independent? In the picture below I swapped probes from bad probe 1(red circle area) to my spare good probe. Then the probes start following each other consistency as I was heating the water to boiling. One note of interest is that probe 2 took an immediate dip when probe 1 was plugged in. This makes me wonder more about how they are coupling behavior together rather than acting mostly independent. Anyway I'm going to mark and/or throw away my bad probe since it's obviously probably of no more use based on the data shown. I'll hang onto it for a while just to make sure this isn't a fluke. Thanks for the support and suggestions to get this narrowed down!

Also, I noticed when I reached boiling point my probes only read 208 degrees rather than 212. Does this mean I should think about using a probe offset or maybe calibrate the 10k resistance values on the config page? Or just ignore that it's off on the top end? Ignore everything to the right of the blue line and arrows. I was playing with the bad probe again to make sure it was bad everywhere.

 

[Tony L-Iowa]

Anyone with thoughts about how what appears to be one defective probe could have caused the good probe to misbehave along with it? That's the part of my whole fiasco that I'm baffled by. In the early part of my smoke the good and bad probes were in lock step synchronization of presenting bad data.

 

[RalphTrimble]

perhaps the bad probe has the "signal" lead shorted to the metal braid and that metal braid is touching the other probes braid? Drawing at straws here, bottom line, toss the bad probe and use a good one....

 

[Tony L-Iowa]

Thanks Ralph. I did toss it. I just like to learn from some of these things to watch out for again in the future.