Integrating Time, Memory, and Heading Capability, Part V

Posted 10 August 2018

Well, it appears I spoke too soon about having solved the I2C hangup problem on my Wall-E2 wall-following robot.   In my last post on this subject, I described all the troubleshooting efforts I employed to nail down the cause of intermittent hangups when trying to use the MPU6050 6DOF IMU on the robot, along with several other I2C devices (a Teensy 3.5 used for IR homing, and Adafruit RTC, and FRAM modules).

After (I thought) figuring out that the I2C SCL/SDA line lengths were the root problem of the hangups I had been experiencing, my grandson Danny and I spent some quality time reworking Wall-E2’s layout to accommodate shorter line lengths.   Instead of mounting the IMU and it’s companion sensors on the second deck as before, we 3D printed a small plastic plate to attach to one of the hexagonal 2nd deck standoff posts and provide a 1st deck mounting area for the sensors.   The previous and new mounting locations are shown below:

2nd deck mounting location. The MPU6050 is the module with the illuminated blue LED toward the rear of the robot

1st deck mounting location for I2C sensors (lower right-hand corner of the photo). The Teensy 3.5 IR homing module is shown mounted on the IR detector housing (above the red plastic plate)

Unfortunately, as I was doing some final tests on this setup, I started experiencing hangups again.   After a day or so moping and some very choice words, I started all over again trying to figure out what happened.

On previous searches through the i-verse, I had run across several posts indicating that the Arduino Wire library had some basic problems with I2C bus edge conditions; there were several places where it uses several blocking ‘while()’ loops to transmit and receive data on the I2C bus, and there was no way to recover from a ‘while()’ loop where the exit condition was never satisfied.    After literally exhausting all the other possibilities, it was becoming apparent that this must be what was happening – the MPU6050 must occasionally fail to respond correctly to a I2C transaction, causing the associated ‘while()’ loop to never exit.

So, I started looking for solutions to this problem.   Again, I found some posts where folks had modified the low-level I2C bus handling code found in twi.c/.h, the code underlying the Android Wire class.   I found a post by ‘unaie’ (http://forum.arduino.cc/index.php/topic,19624.0.html) with the same complaint, but he also posted modified versions of twi.c and twi.h that solved these problems by forcing the ‘while()’ loops to exit after a set number of iterations, and resetting the I2C bus when this happens.   His modified versions can be downloaded at:

http://liken.otsoa.net/pub/ntwi/twi.h

http://liken.otsoa.net/pub/ntwi/twi.c

I downloaded these files and tried to replace the ‘stock’ twi.c/h with the modified versions. Unfortunately, unaie’s modifications were made on a quite old version of the files, and conflicted with the later ‘repeated start’ versions of these files that are in the current ‘wire’ library.

So, I did a ‘diff’ between the ‘repeated start’ version and unaie’s version, and created a modified version of the latest ‘repeated start’ twi.c/h.   In addition, I added a couple of functions to allow monitoring of the number of times a bus reset was required due to a ‘while()’ loop timeout.   When I was finished, I ran the sensor for over 24 hours with no failures, but in that time there were three instances where a ‘while()’ loop timed out and a I2C bus reset was required.   A small snippet of this run is shown below.   The blue line is the yaw value, and the plot snippet shows where I manually rotated the sensor just after 24 hours, and the horizontal orange line shows the number of bus resets.

Small snippet of 24-hour sensor run. blue line is reported yaw value; orange shows the I2C bus reset counter

So it is clear that, absent the lockup recovery modifications, the I2C bus would have locked up long before, and that with the modifications ‘while()’ loop deadlocks have been successfully handled.

11 August 2018 Update:

The sensor is still going strong after 44 hours with no hangups, and the reset counter is still holding at 3.

The complete twi.c & twi.h codes are included below:

 

Stay tuned!

Frank

 

9 thoughts on “Integrating Time, Memory, and Heading Capability, Part V

    1. paynterf Post author

      Ku,

      Thanks for the note, and I’m glad my code worked for you. You might also want to read this post for some later information related to the I2C lockup saga.

      Regards,

      Frank

      Reply
  1. Gsary

    Hi, Frank
    I tried your solution..
    I got this error below..

    Any idea or help will be great!

    Arduino: 1.8.5 (Windows 7), Board: “Arduino Pro or Pro Mini, ATmega328P (5V, 16 MHz)”

    In file included from C:\Program Files (x86)\arduino-1.8.5\arduino-1.8.5\hardware\arduino\avr\libraries\Wire\src\Wire.cpp:26:0:

    C:\Program Files (x86)\arduino-1.8.5\arduino-1.8.5\hardware\arduino\avr\libraries\Wire\src\utility/twi.h:2:1: error: expected unqualified-id before numeric constant

    1

    ^

    C:\Program Files (x86)\arduino-1.8.5\arduino-1.8.5\hardware\arduino\avr\libraries\Wire\src\Wire.cpp: In member function ‘void TwoWire::begin()’:

    C:\Program Files (x86)\arduino-1.8.5\arduino-1.8.5\hardware\arduino\avr\libraries\Wire\src\Wire.cpp:62:12: error: ‘twi_init’ was not declared in this scope

    twi_init();

    ^

    exit status 1
    Error compiling for board Arduino Pro or Pro Mini.

    This report would have more information with
    “Show verbose output during compilation”
    option enabled in File -> Preferences.

    Reply
    1. paynterf Post author

      Gsary,

      Sorry, but your post really doesn’t contain enough information to make a good guess at the source of the problem. Are you sure you are actually compiling the version of twi.h/c from my post? The error message above suggests you are compiling the library files from the original Arduino IDE installation.

      Also, just for completeness, I no longer use this code. I have long since changed to using Jeff Rowberg’s I2Cdevlib code, configured to use the SBWIRE library (the SBWIRE library includes the required fixes for the twi.c/h hangup bugs).

      Hope this helps,

      Frank

      Reply
  2. Pingback: Known defect in Arduino I2C code causes hangup problems | Paynter's Palace

  3. Craig Larson

    Thank you for your contribution to this maddening issue. I’m late to the game and enjoyed your I2C Freeze page. Too late for me, I was following your comments on SBWire and have stable code for once: 8 hours and counting! I have been working this issue for the last week with great frustration until I found your materail.

    Reply
    1. paynterf Post author

      Craig,

      Glad to hear my posts have helped. Fortunately for everyone in the Arduino world, the ‘stock’ Wire library was FINALLY revised last summer to include timeouts, although you still have to manually add the code to your sketch to enable the timeout feature. See this post for the details

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *