Posted 23 September 2021

For some time now I have been following developments on Paul Stoffregen’s Teensy forum with respect to OTA updates for Teensy products. As it turns out, doing OTA updates is less than straightforward due to the Teensy architecture (I’m waving my hands here).

However, in recent months, thanks to Teensy forum member joepasquariello, there is a new set of programs that seem to address the Teensy OTA issue in a much more comprehensive manner. See this thread (starting with post #85) on the Teensy forum for the details. After reading through the posts, and asking some more dumb questions (which joe was fortunately willing to answer), I decided to give this a try to see if I could achieve OTA updates for a Teensy controller, and maybe make it as convenient as OTA updates are now with Arduino products using the Pololu Wixel RF serial pipe.

The FlasherX package is actually three different files; FlasherX.ino is a demo program that #includes FlashTxx.h and FlashTxx.cpp. The FlasherX.ino program is loaded into the target Teensy device in the normal manner, just like any other Teensy program. However, once it is there, it adds the capability to update the Teensy’s firmware using the .HEX file (produced by the normal compile process) transmitted over any serial port. Of course, in order to make this magic happen, the updated Teensy program must also have this capability, so that means that any Teensy program destined for OTA updates must include the basic ‘FlasherX’ components.

In order to get started, I dl’d all three files and programmed a Teensy 3.5 with the FlasherX.ino program using my normal VS2019/Visual Micro setup. Then I created a new Arduino Project in VS2019 called ‘TeensyFlasher_V1’ and copied the entire contents of FlasherX.ino. I compiled it for a T3.5 target and used my normal Teensy bootload process to transfer it to the Teensy 3.5. Here is the serial port output after the upload:

Next I used Tera Term’s ‘file send’ capability to send the FlasherX.hex file to the Teensy using the wired USB connection as just another serial port. When I first launched Tera Term I got an error dialog saying it “Cannot open COM11. Access denied” – reminding me that I had to close the normal connection in order for Tera Term to operate properly. Once I had done this, I used the ‘File->New Connection menu item in Tera Term to create a new connection to COM11 (the Teensy 3.5 USB port), as shown:

Next I used ‘File->Send File… to select the .HEX file created by the compiler, and transfer its contents over to the Teensy using the COM11 serial port

After selecting the .HEX file and clicking ‘Open’, the HEX contents were sent over the serial connection as shown below:

The reason for using the HEX file created from compiling FlasherX.ino (as opposed to a random Teensy app) is because FlasherX.ino already contains all the pieces needed to do OTA updates. Although a bit recursive, using the FlasherX app demonstrates that a Teensy device can be updated via a serial connection repeatedly, as long as the newly updated firmware contains the pieces needed to support the next OTA update.

The next step was to figure out how to get VS2019/Visual Micro to do the update automatically, so that the process emulates what happens with an Arduino controller when F5 is pressed. VS2019 supports post-build scripts, but they aren’t used in the Visual Micro environment. Instead, as I learned from VM guru Tim Leek, VM supports post-build operations via a special file called ‘board.txt’ in the project directory. If this file is present, properly formatted lines are treated as ‘build property overrides’. For instance, this is the line in ‘board.txt’ that calls a C# console app to actually communicate with the target Teensy.

This command launches the console app, and provides it with the path to the project, the path to the sketch, and the currently selected port number. See this thread on the VM forum for all the gory details.

I ran into a problem when I tried to use the above command line with my C# console app, because the arg[] string array in the console app was interpreting spaces in the strings in the calling command differently than expected, so when I sent three strings to the command, the console app thought I was only sending two – somehow concatenating the com port number into the second string (the sketch path). After playing around some more, I found that the following treatment:

when passed as an argument to this small code snippet

produced the following output:

With the correct number of arguments, and the comport number on its own line – yay. I have no idea why this works and the other formulations don’t, and why the second argument has to be have it’s quotes escaped with ‘\’, but hey – it works.

As it turns out, this exercise was a bit academic anyway, because both the {ProjectDir} and {sketch_path} expand to identical results, so I only need one of them and the comport number to proceed. So, the ‘final’ (for now) call argument list looks like:

which produces:

27 September 2021 Update

In order to do OTA updates, I need to be able to connect to one of the Teensy’s hardware serial ports. Up til now I have been just using the USB serial port for convenience, so I thought I would try using a FTDI Pro USB/Serial converter module to connect to Serial1 on the teensy, as shown in the following photo:

In the above photo, the normal USB connection provides 5V power for the Teensy, and also allows ‘direct’ programming via the normal VS2019/VM route. The FTDI module connection shows up as COM3, a ‘regular’ USB serial port. With this setup I was able to upload a .HEX file to the Teensy using Tera Term, confirming that I can indeed use a hardware serial port for this purpose.

04 October Update:

Between the last update and now, I spent a lot of time trying to get the command-line post-build console app working, mostly to no avail. I kept getting wrapped around the axle trying to deal with three different serial connections; two to the teensy itself, and one from the command-line app to VS2019’s Console service. This got to the point where at one time the ‘Console’ output was going to my computer display, which made the mouse jump around wildly, occasionally launching or closing apps! Even trying to restart the PC was a trial-and-error process, because about half the time the screen focus would jump away from the target button before I could get it clicked!

So, I decided to go back to basics and use just the FTDI serial adaptor to run the Teensy. This required using the Vcc connection from the FTDI module to power the Teensy, which worked OK once I figured out how to switch the FTDI module from 3.3V out to 5V out.

With this setup I am able to upload programs to the Teensy using Tera Term, as before, but now I don’t have the other USB connection to confuse things. It does mean however, that now I MUST use Tera Term to upload sketches, as I no longer have the capability to use the Teensy bootloader from VS2019.

With this simpler setup, ‘Console’ statements from the command-line app should go only to the VS2019 debug session, and serial port operations from the command-line app should go only to Teensy’s ‘serial1’ hardware serial port.

05 October 2021 Update:

I expanded the above operation to restore the USB connection to the PC, and created a small Arduino program to (at first) just echo serial data from one serial port to the other. Here’s the program:

Once I had this going to my satisfaction, I started adding pieces from my ‘TeensyFlashDemoV2’ project until I had a complete working program, as shown below:

Here’s the Arduino code:

and here is the companion C# command-line app:

And here is the command-line output from my last run:

In the above printout, most of the HEX lines were removed just to keep the line count down, but all 2977 lines were successfully sent to the Teensy. The lines

were emitted from the Teensy at the conclusion of the HEX file upload. The line

was from the command-line app comparing the number of lines sent with the number of lines received. The numbers matched, so ‘2977’ was sent to the Teensy to complete the update.

At this point I have a working automated update system except the COM port number and HEX file location was passed to the C# program as a set of fixed arguments (as shown below), to disconnect the command-line app from the Arduino sketch, instead of dynamically providing the same parameters from VM build properties

Now that I have some confidence that the command-line app does what it is supposed to do, I can go back and make that connection again, hopefully with better results this time.

A problem with the Arduino-to-C# command-line app is that the ‘build properties’ exposed by Visual Micro aren’t always the same, so what works for one sketch may or may not work for another. Here’s the ‘build properties’ output from my expanded ‘TeensySerialBridgeV1.ino’ file:

I need two things from the build properties – the COM port number, and the path to the .HEX file. From the above list, it appears that I could use either of the following properties,

but neither is very convenient, as the COM port number has to be extracted from a ‘known’ string. On other projects, I have seen a build property like ‘serial.port.number = 11’ which would be much more convenient, but it seems to have disappeared. For the path to the .HEX file, I should be able to use

or

but it appears the ‘final_output_path’ is a bit more directly related to the .HEX file location. So, to make this work using the VM post-build facility, the required ‘board.txt’ file should look like:

When the above is placed in ‘board.txt’ and the Arduino sketch is run, I got the following output:

Which appears to be correct, and the Teensy was obviously updated (got the USB disconnect/connect tones, and the LED tell-tale stopped and then resumed blinking). It looks like I might actually have a working program now – woo hoo!

06 October 2021 Update:

One minor snag occurred with the build property used to pass the COM port number to the command-line program; I had been using {serial.port.caption} and it turns out that this parameter (along with many other serial.port parameters) isn’t always available – things change depending on the COM port chosen in the Arduino program and the actual device connected (or not connected) to the PC. After another round of posts in the Visual Micro forum, Tim Leek pointed out that the ‘serial.port’ parameter should always be available. I had missed this on this because it isn’t grouped in with the rest of the ‘serial.xxx.xxx’ parameters. With the benefit of hindsight, I can see why this is so; the ‘serial.port’ parameter is probably pulled from the VS2019 IDE, while the other ‘serial.port.xxx’ parameters are hardware-specific. Once I switched my ‘board.txt’ to use the ‘serial.port’ parameter vs ‘serial.port.caption’, I started getting much more consistent results. Here’s the ‘final’ (I hope) board.txt configuration:

One other small snag; my ‘TeensySerialBridgeV1’ program was originally intended to just pass serial data back and forth between two terminal programs (Tera Term on one side, the Teensy sketch on the other). As a result, there was a built-in assumption that both serial connections would always be available, as shown in the following code snippet:

The first ‘while()’ statement causes the program to hang if the physical Teensy USB connection is missing. This gave me some grief when I removed the Teensy USB cable to prove I could do the entire update cycle with only the Serial1 hardware serial device connected – oops!

The final Teensy program is shown below:

09 October 2021 Update: The final product

After encountering and solving a few more ‘gotchas’, I think I now have a reliable Teensy OTA mechanism using the VS2019/Visual Micro IDE and Joe Pasquariello’s wonderful ‘FlasherX’ code . The ‘trick’ is using Visual Micro’s ‘board.txt’ feature to add a post-build command that runs each time the Teensy sketch is compiled, and a small C# command-line program to trigger the FlasherX update code and feed it the correct Hex file.

I created a very simple Teensy project called ‘TeensyOTADemo’ that does only two things; it flashes the built-in LED, and launches the FlasherX update code when triggered by a ‘U’ (for Update) ascii code on Serial1 (pins 0 & 1 on a T3.5). Then I created a small C# command-line program to actually send the trigger character and then transmit the .HEX file created as part of the normal compile process.

Then I created a ‘board.txt’ file containing a post-build command that launches the C# command-line program and feeds it two arguments – one containing the build path, and the other containing the COM port number to be used to connect to the Teensy’s Serial1 port.

Here’s the Teensy sketch:

And the C# command-line program:

And last, the ‘board.txt’ file contents that launches the C# command-line program:

Here’s a short video showing the update process. The clip starts with the ascii ‘U’ trigger code arriving at the Teensy Serial1 port, and ends with the update finished and the Teensy rebooted back into the ‘blink’ code.

09 October 2021 Update: HC-05 Bluetooth link:

The next step is to replace the FTDI module with an HC-05 Bluetooth link. I had some HC-05 modules from previous work, and I had done a similar project with an Arduino some time back – so how hard could it be?

I hooked up a HC-05 to the Teensy as shown in the following photo, using a bench power supply to power both the HC-05 and the Teensy 3.5

Then I paired the HC-05 with my laptop; this took a bit of fumbling around, as the laptop thought it was still paired with an HC-05 even though it’s been over two years since I last used it, and I had since forgotten that the default password was ‘1234’. In any case, I eventually got it paired, as shown, with an automatically generated COM port on COM4.

Next I fired up Tera Term and created a connection to COM4, as shown:

And then tried sending some ascii text through to the Teensy. This appeared to work fine, but when I tried a program update via the HC-05 link I got an error that indicated the FlasherX code didn’t like something I was sending it.

The ‘abort – bad hex line’ output happens before I try to send it the .HEX file contents, so it clearly isn’t choking on that; maybe some extra characters getting sent instead of just the “U” character?

Looking at the ‘FlasherX’ code, it appears that the offending characters/line is supposed to be printed out with the lines

which implies whatever caused the problem is non-printable – maybe an extra CR/LF? So, I added some debug code to update_firmware() to print out each line as it is read into the ‘line’ character array, so maybe I can see the problem.

Hmm, more mysteries. I reconnected the regular USB connection, and uploaded the new version of the sketch. Then I disconnected the USB connector, and used Tera Term to send the update command (“U”). However, this time the OTA Update code didn’t choke, and AFAICT performed a completely successful update via BT and the HC-05 – yay! Of course, I now have no idea what caused the problem to start with, and whether or not my addition of a few lines of debugging code made the difference.

So, I undid the changes to the update code and tried again. And, as shown below, the OTA update worked perfectly – double Yay!!

So now to try an OTA update using my command-line program instead of Tera Term. I disconnected TT from COM4, and then, after selecting COM4 in the VS2019 IDE, pressed F7 to start the process. This worked as well, but it appeared the transfer rate was considerably slower – no idea why.

Here’s a short video showing the OTA update using the command-line program and BT OTA.

Just comparing the overall times from the two videos above the one using the command-line program runs approximately twice as long. It’s hard to believe it is only the use of the command-line program vs Tera Term that is causing the delay – but it is possible that TT was simply running at a higher baud rate – more study required.

In summary, however, slow rate or no, it is clear that we now can implement completely automated, seamless OTA update processes using VS2019 and Visual Micro. Now that the groundwork has been laid, I would assume that this same strategy could be implemented for other IDE’s as well.

I have started GitHub repositories for the Teensy demo sketch TeensyOTADemo and my TeensyFlash C# command-line program on my GitHub site, and I would welcome suggestions/improvements.

10 October 2021 Update:

I’m trying to determine what caused the slow update when using my command-line program and the HC-05 when doing an OTA update relative to the situation when using Tera Term to manually transfer the .HEX file to the Teensy. In both cases, the OTA link is through the HC-05, so that module shouldn’t be the cause. However, just to check I went back and checked the HC-05 settings using the ‘AT Command Mode’ trick discussed in this post. AFAICT, the HC-05 was set to 115200 baud rate. I also checked to see if I had somehow misconfigured the serial port created in my command-line program, but it is set to create a port at 115,200 BPS as well. So, I’m officially baffled.

To investigate the problem, did an OTA update via the HC-05 BT link, but using Tera Term to trigger the update process and send the .HEX file. Using this method, it took approximately 14 sec to transfer the entire 3142-line .HEX file. Then I did an OTA update using my command-line program, using the same HC-05 BT link. With this setup, the transfer took approximately 35 seconds – over twice the time. Watching with my scope, it is clear that the transfer takes place at a much slower rate when using the command-line program, even though (at least AFAICT) the programmed baud rates are identical in both cases. The two photos below show the transfer rate for both cases, using the same time scale for both.

After a bit of web research on C# .NET serial port programming, I came across this post, which indicates that using .NET for this task was not a very wise move – bummer!

Stay tuned,

Frank