[From David Benavente's LAOLAO Bay ICON Station Maintenance log -- June 26, 2012:]
Accessed Station through cliff. David B. used SCUBA, while Rod C., John I., and Steven J. snorkeled. Found some evidence that fishermen were in close vicinity to the station. Also a knife was found below station, this could be because the base of the station appears to have become a congregation area for Trochus mussels. Because these are a harvested species fishermen may be visiting the station more often.
This log is a description of site surveys, station log maintenance, and biological observations at Laolao Bay, Saipan, at approximately 15° 09' 24.4" N, 145° 46' 10.6" W.
Wednesday, June 27, 2012
Saturday, June 23, 2012
Docomo cellular connection spotty, error-prone
Following the recent modem outage (May 12th - 29th), I attempted my usual practice of a gradual re-enabling of the data table downloads.
A note about data tables: the datalogger has six data tables, each with its own collection schedule. One very simple table stores one record per day, with only a few data values that are used to monitor the memory card's diagnostics and confirm that it is formatted/storing correctly. At the other end of the spectrum, one data table stores a new record every five seconds, to record each individual reading from the analog sensors (air temperature and barometric pressure).
Historically most of the CREWS data record has come from the one-hour data table, since most CREWS stations have relied on once-hourly, twenty-second windows of communication on the GOES East satellite. Data were summarized as necessary and transmitted in a format designed to fit comfortably in our communications window.
With the advent of larger-capacity memory cards, we began to store more granular records in the datalogger's memory for later retrieval. With our "always on" connection to the Saipan CREWS station (apart from the cellular network problems, that is), we began to access all of those data in real time.
As it might be expected, the 5-second data table contains a very large number of records, although each record stores relatively few data values. There is also a 30-second, a 1-minute, a 6-minute, a 1-hour and (as mentioned before) a 1-day data table.
Since our data feeds require only the 1-hour and 6-minute data tables for processing, I generally disable the download of all other data tables during long modem outages, in order that when service resumes our feeds can come up as quickly as possible. Later I will manually re-enable download of the larger data tables and monitor their progress to ensure that our main data feeds aren't negatively impacted. This was more or less how things worked following the April 12th - 19th outage earlier this year, with all data tables recovered by April 24th.
However, after the May 29th re-establishment of communications with the modem, two differences were observed. For one, the LoggerNet monitoring software was reporting a large number of error conditions and failed connections, although it did not seem to be impacting the download of the 1-hour or 6-minute data tables. What was worse, however, is that attempts to re-enable the download of the other data tables appeared to be knocking the modem entirely offline, requiring a request to Docomo for modem/network reset.
The first happened on the morning of Wednesday, May 30th, 2012. I re-enabled the first of the data tables for download, and all communications ceased. I alerted Ross and he replied later that same day:
Following this clear pattern of outages, I did not attempt to access the other data tables right away. Instead, I allowed the main 1-hour and 6-minute data tables to run normally and populate our data feeds. However, several weeks later I glanced at the LoggerNet status readouts and noticed that the pattern of error conditions and failed connection for the Saipan modem appeared to have resolved itself. All communications appeared to be normal (for example, in comparison with our other cellular modem which is deployed at Port Everglades near Fort Lauderdale, FL).
When I noticed this change, I re-enabled download of the other data tables and this time, they all downloaded beautifully. This caught us up back to the beginning of the May 12th outage for the first time, and we once again were keeping current with all data tables in real time. I sent out this by email on Friday, June 22nd:
A note about data tables: the datalogger has six data tables, each with its own collection schedule. One very simple table stores one record per day, with only a few data values that are used to monitor the memory card's diagnostics and confirm that it is formatted/storing correctly. At the other end of the spectrum, one data table stores a new record every five seconds, to record each individual reading from the analog sensors (air temperature and barometric pressure).
Historically most of the CREWS data record has come from the one-hour data table, since most CREWS stations have relied on once-hourly, twenty-second windows of communication on the GOES East satellite. Data were summarized as necessary and transmitted in a format designed to fit comfortably in our communications window.
With the advent of larger-capacity memory cards, we began to store more granular records in the datalogger's memory for later retrieval. With our "always on" connection to the Saipan CREWS station (apart from the cellular network problems, that is), we began to access all of those data in real time.
As it might be expected, the 5-second data table contains a very large number of records, although each record stores relatively few data values. There is also a 30-second, a 1-minute, a 6-minute, a 1-hour and (as mentioned before) a 1-day data table.
Since our data feeds require only the 1-hour and 6-minute data tables for processing, I generally disable the download of all other data tables during long modem outages, in order that when service resumes our feeds can come up as quickly as possible. Later I will manually re-enable download of the larger data tables and monitor their progress to ensure that our main data feeds aren't negatively impacted. This was more or less how things worked following the April 12th - 19th outage earlier this year, with all data tables recovered by April 24th.
However, after the May 29th re-establishment of communications with the modem, two differences were observed. For one, the LoggerNet monitoring software was reporting a large number of error conditions and failed connections, although it did not seem to be impacting the download of the 1-hour or 6-minute data tables. What was worse, however, is that attempts to re-enable the download of the other data tables appeared to be knocking the modem entirely offline, requiring a request to Docomo for modem/network reset.
The first happened on the morning of Wednesday, May 30th, 2012. I re-enabled the first of the data tables for download, and all communications ceased. I alerted Ross and he replied later that same day:
I sent another email to Luigi [at Docomo] this morning to reset the modem. I will take down the modem settings and have Sierra Wireless take a look. It isn't clear whether the Docomo network or the device is the cause of these outages. No reports of network issues have been received, but as David mentioned, they can be frequent. I don't quite understand how we had good service for the initial months and not now.This apparently got a response from Docomo immediately and the modem was back online again the next morning. However, it once again went offline when I attempted to re-enable download of the other data tables. From my message to Ross on May 31st:
You might want to check the modem again today! It seems like it might be locking up whenever I try to pull data files from the logger. I don't know why it would suddenly be reacting this way because this was S.O.P. last year (August/Sept) and during the uptimes earlier this year (March 19th through May 12th). And most notably on April 24th, when I did some very intensive data transfers from the modem and everything seemed fine.This message may have caught Ross away from his email, for his reply arrived on June 4th (wherein he said he would contact Docomo) with an update on June 5th (to report that the modem was online again).
Following this clear pattern of outages, I did not attempt to access the other data tables right away. Instead, I allowed the main 1-hour and 6-minute data tables to run normally and populate our data feeds. However, several weeks later I glanced at the LoggerNet status readouts and noticed that the pattern of error conditions and failed connection for the Saipan modem appeared to have resolved itself. All communications appeared to be normal (for example, in comparison with our other cellular modem which is deployed at Port Everglades near Fort Lauderdale, FL).
When I noticed this change, I re-enabled download of the other data tables and this time, they all downloaded beautifully. This caught us up back to the beginning of the May 12th outage for the first time, and we once again were keeping current with all data tables in real time. I sent out this by email on Friday, June 22nd:
LoggerNet's error rate on the docomo modem suddenly relaxed this past week and I took a chance and re-enabled the download of those more granular files I'd mentioned. That worked beautifully, so we're all caught up with the missing data back to May 12th, and all files are once again downloading in realtime. I don't know what was causing the modem to cycle offline before but it seems to be okay for now.Mike J+
Friday, June 1, 2012
Maintenance log: May maintenance
[From David Benavente's LAOLAO Bay ICON Station Maintenance log -- May, 2012:]
I was away on travel for most of May. To my understanding the MMT performed regular maintenance on the ICON station during this time. Steven J. reported that the station and its instruments were cleaned.
I was away on travel for most of May. To my understanding the MMT performed regular maintenance on the ICON station during this time. Steven J. reported that the station and its instruments were cleaned.
Subscribe to:
Posts (Atom)