Coefficient wont update correct value

[RBennett]

For whatever reason I cannot get Probe 2 "Coefficient C" to update what I input.

It's almost as if its too big of number. I want to put in "9.9680614e-8" but instead it will update and reappear as "9.9680604e-8" . All the other probe input fields do not have this issue. I've tried re-flashing both the card and the micro controller with no luck.

 

[Bryan Mayland]

It's actually because it is too small a number. The code running on the atmega is only single precision float, meaning that only the first 6 significant digits are likely to be preserved when converting from a string. The coefficients are calculated in the Pi in double precision, so there will be some rounding in the microcontroller.

The difference is 0.000000099680604 vs 0.000000099680614 so it makes no practical difference in our calculations.