Michael's Project Blog

Project blog of things that Michael thinks about or researches

Pool Fill: Debugging ESP8266 with no serial port

I realized that once I hook in the command library it will take over the serial port. If I write debugging information it will send that to the connected AVR which should ignore it as garbage. (note to self: make sure command library can detect garbage) There is a solution using a second serial port on the ESP8266, but it is only one way because the RX pin can not be used. I would also need to make an adapter for my FTDI cable. Then I had an idea; why not just publish an MQTT debug topic? It seems like this might be handy even after development is done for debugging live issues. I planned to add MQTT support regardless so it should be relatively easy to add another topic.
  1. Research and install MQTT broker on local LAN
  2. Configure the MQTT Arduino client
  3. Add MQTT configuration parameters to the web configuration screen

Pool Fill: SNTP Client

I spent quite a bit of time thinking through various implementations and the resulting user experience for DST time shifts. I don't like any of the options I've studied. The idea of shifting the scheduler time, skipping (spring forward) or duplicating (fall back) set points in the process, is a non-starter for me. I could write a bunch of complex code that would try to schedule all the otherwise skipped items and know not to duplicate set points, but that seems like a lot of testing, debugging, and places for code defects that I'll find a decade from now. I've settled on a solution that at its core comes down to this one design philosophy:
  • Always operate the software in either UTC or local standard time; apply DST shifts when viewing or setting the schedule and clock
For the controller I plan to always operate it in local standard time because again testing and debugging the scheduler where local time and UTC can be +/- 12 hours seemed like way too much work. This also means that the controller does not need to be told the timezone or timezone offset. It only needs to be told about the DST offset (0 or some positive integer minutes). The user will always interact with the controller in local time (DST or ST). The controller will store and schedule set points in standard time. For the user this is all as one would expect except for one effect. The previously scheduled entries will shift an hour at the DST shift. This is no worse than a controller that doesn't accommodate DST at all and a big improvement when the user is trying to set the schedule.
For the web server I plan to always operate it in UTC. The user will need to set a timezone with offset from UTC and DST start /end data (all the timezone data for the Timezone.h library). That should be relatively easy with the web interface. Timezone information is needed regardless since the SNTP client only receives UTC and must be offset for local time. Handling DST shifts on the Web Server are trivial since there is no scheduling logic. Again the internals will all use UTC and will be offset to local time when interacting with the user.

Some fun facts I learned along the way:
  • The DST begin and end points vary all over the world and have changed many times in history; even in recent history. The shift has not and is not always 60 minutes; even today at least one country (Australia) has a region using 30 minutes
  • The ESP8266 internal SNTP client can be set for timezone and automatically adjust for DST; however the DST shift points are fixed to align with some undocumented region, making the capability practically useless

Pool Fill: SNTP Client

Thought through this some more today. I considered several approaches and settled on the approach of making local time a display artifact. All internal time will be in UTC (same as NTP servers) and adjusted for the user when viewing the time. This has a couple of implications for the pool fill controller:

  1. controller needs to be told the current local time shift (including DST)
  2. controller needs to shift all interactions with the user
  3. does the controller store set points in localtime or UTC?

Pool Fill: SNTP Client

TimeLib and the Timezone library don't really work together like the Timezone examples imply. All of the TimeLib transformation functions use the currently stored TimeLib value. If TimeLib is tracking UTC then all of the transformation functions will only work for UTC. The approach here will be to place the Timezone library functions into the TimeLib SyncProvider function. Every time the SyncProvider is called it will grab the UTC time from the ESP library and convert it to local time before returning that time to TimeLib. TimeLib will always track local time. This has one potential side-effect. At the DST shift, the time returned by now() (i.e. sysTime) will jump forward or backward causing any timing functions that use now() to possibly break. This is user beware.... Timing functions should use the Arduino millis() function for checking elapsed time! See this for an overview of timing best practices: How can I handle the millis() rollover?

Need to debug the scheduler for days past sunday. Found a real doozie of an error and corrected it but the fix is untested.

Always set the clock to a date past 1/1/1971 or the schedule timing functions will not work correctly. The best thing here is to force the clock to something like 1/1/2017 at startup; then free run from there until something sets the time.

Need code to handle inhibit > unix epoc rollover; also scheduling for the week following the unix epoc rollover needs to be studied.

On a DST spring forward; anything scheduled between 2am and 3am will be skipped.
On a DST fall back; anything scheduled between 1am and 2am will be scheduled twice.

remainingRunTime will also break across the shift

Pool Fill: SNTP Client

First a little rant. There are a couple NTP libraries for Arduino. None of them are actually NTP. All of them are SNTP but probably not even fully SNTP compliant.

Need to combine:
  • TimeLib
  • Arduino Timezone Library
  • ESP Library time.h (includes sntp.h)

  1. Need to work out the linkage between Arduino time library and the ESP time library
  2. Need to work out the linkage between the Timezone Library and the ESP time library
    • It seems that the ESP time library uses some fixed DST start/end values....
  3. Need a prototype for testing

Cell Phone Quiet Zone


While watching a movie yesterday I noticed that the movie theater chain had convinced Cingular to co-develop a "please turn off your cell phones" segment. I started thinking that, if the major carriers wanted to, they could develop a system that would make this automatic. Let's be honest, while we want to think the people that let their phones ring in quiet areas are inconsiderate, in truth, most of those people are just forgetful and/or lazy. I fall into both categories occasionally. What is really needed is a way for concenting phone users to have their phone automatically transition to a quieter mode after entering an area where it would be inconsiderate to have the phone ring.
A system could be devised where a small transmitter placed in quite locations would signal the phone that it should transition to a quite mode. The user would have the option of ignoring the signal, where for example doctors may want to ignore the quite mode.

Why is this important?

More and more businesses and other orginizations meeting places are being overrun by the distraction of a ringing cell phone. This does not happen because the users are inconsiderate, but because they are either forgetful, lazy, or both. This technology can supplement our basic human nature to create a more productive and satisfying world.


Who is going to pay for this feature?
After all nothing is free and someone must pay for the software R&D and the transmitters. I believe that the software for the phone will be implemented by the cell phone providers or eventually by the manufacturers themselves. If there were an opensource reference application (say in java) then the R&D for each phone type would be minimal. As for the tranmitters, I believe that the orginizations and businesses would purchase these directly. Movie theaters, churches, quiet restraunts, and other orginizations would be willing, I believe, to spend ~$100 / room to enable the quite feature. Some businesses would be willing to spend something similar for conference rooms.


Implementation Method

I belive that this entire service could be implemented with Bluetooth.

.....describe the beacon tranmitter and behavior of the phone......

While some may question the range of Bluetooth, I believe the range of the system would be sufficient if using the higher power Bluetooth modes. If the fixed tranmitter were just a beacon tranmitting a one way message, the transmitter could use the higher power Bluetooth modes not implemented in the phones. The phone would not need to communicate back to the tranmitter, but would simply register that the beacon was heard.

Implementation Issues

The current Bluetooth security modes have most Bluetooth radios either powered down or in a non-discovery mode. The phones that are powered down would of course need to be enabled but the non-discovery mode would be just fine since the phone is only looking for the beacon but not required to respond.

Having the phone's Bluetooth mode enabled leads to a question of battery power. In my phone I have the Bluetooth mode enabled because I am constantly using the Bluetooth mode for synchronization with my PC and for my Bluetooth headset. However I can imagine that many users to not need to have Bluetooth enabled on the phone. Once Bluetooth is enabled, the battery life is reduced in many phones by 10%-20%. Requiring that Bluetooth be enabled by default may turn into a "my phone lasts longer than your phone" marketing nightmare.

Today many phones in use do not have Bluetooth capabilities. More than half of the models being sold do have Bluetooth, but that may not be enough to make a serious dent in the noise problem. This technology will only be successful when most phones in use are Bluetooth capable.

Project Tracker for stovenour.net

Projects are scattered over stovenour.net based on the current status of the project (e.g. gleam in my eye - on the blog, work in progress - Hardware or Software Development, might make me $$$ - Private Projects, etc.) What I really need is a project blog that lists every project I've documented and the location for the relevant information. The fields are:
Date / date
Description / text
Status / combo - gleam, active, hold, abandoned, etc.
Blog Link / wiki link
Page Link / wiki link

I can not figure out how to create page links in a tracker. I can store the link in a text box, but the links are not clickable. Maybe there is a way around that?

For now I will create a wiki page with a simple table. ProjectList

NOAA Weather Alert Automation


While watching the NOAA radar page for my area, I realized that they offer NOAA,:::::.gov/aboutrss.html" rel="external">RSS feeds of the weather, watches, and alerts. Now this seems like a cool idea. Can I automate the announcement of watches for my home? Clearly I could just go out and purchase a NOAA,:::::.gov/nwr/nwrrcvr.htm" rel="external">weather alert radio. Honestly, I already own one, but radio? that's so last century. How about making it part of my home automation? That's the wave of the future.

Why is this important?

Many homes and nearly all offices do not have a way to receive timely weather updates. This service could be integrated into home based automation devices and corporate communications systems (e.g. IP PBX) to provide both visual and audible indication of dangerous weather conditions.



As I would expect from the government, the feed content and formats are somewhat useful but, IMHO, not well thought out. The whole purpose of something like XML is to make the data parsable by a machine without ambiguity. In NOAA's defense, it is fare to say that the data that must be unambiguously parsed is dependent upon how the data is used. For instance the NOAA,:::::.gov/products/spcwwrss.xml" rel="external">base RSS feed is just a headline service with title and a link. This is suitable for inclusion on a headline service, but not suitable for automating alerts. Automation of alerts will require at least the type of information included in the header of a NOAA,:::::.gov/directives/010/pd01017012b.pdf" rel="external">SAME broadcast on the emergency radio broadcast service.

I found a page that lists information by county. This feed is location specific, but an interface that allows a user to select the correct zone might be difficult depending upon how well the zones are defined. Ideally there would be an XML document somewhere on the NOAA or other governmental site that defines the zones. This format also does not specifically call out the event type so for instance giving different treatment to a warning vs. alert would require parsing ASCII coded fields in an undefined name space. This is an example of the title element from a tornado watch item within the channel:
Tornado Watch - Baxter (Arkansas)
Ideally NOAA feeds would use the NOAA,:::::.gov/directives/010/pd01017012b.pdf" rel="external">SAME coding defined for the emergency radio broadcasts.


Turns out that there is a somewhat coordinated effort to consolidate all of the efforts. The Common Alerting Protocol (CAP) is an XML based document definition that appears to have more well defined information elements. This is the ticket. The National Weather Service supports some version of CAP although the <geocode> element is not quite up to the current revision of the specification. The CAP feeds are separated into states. The index is http://www.weather.gov/alerts/. The county FIPS codes are included in the <geocode> element. The FIPS location codes can be found here NOAA,:::::.gov/geodata/catalog/wsom/html/cntyzone.htm" rel="external">http://www.nws.NOAA.gov/geodata/catalog/wsom/html/cntyzone.htm along with a link to the delimited dump file. I also found the <cap:event> coding http://www.weather.gov/alerts/product_list.txt. This information is all at the bottom of this page http://www.weather.gov/alerts/.