Tech Staff Meeting Notes May 9, 2011
The top priority is always repair of down stations.
NM problem stations
ET problem stations
Eight stations are currently down due to flooding and eighteen more are at risk. Jim prepared this table. Mitch is working on disaster relief funds to help replace the stations, perhaps with a version 3 of ISIS. We won't be able to access these stations for several weeks to make assessments much less repairs so we will instead focus on making preventive maintenance visit to dry stations now, so we can focuse on flooded stations later.
Sign up for WinLink training. Two slots per day are available. All slots are 3:15-3:45.
Date for the Fish Fry is July 8. The cost was $5 last year. Jim and Chris are checking notes to set the cost for this year.
Some items, mostly comm issues, are languishing on this list for an exremely long time. We're starting to make progress on these and need to keep going.
Steve has ASO (network code AO) data flowing into one waveserver. He's working on getting it into the rest of the system. ASO's earmark is eventually to have two central AR broadbands, a northeast AR strongmotion, and 6 adopted TA stations, all sending to CERI in real-time.
Top priority stations - RDGT, then CTCT and GLST
MPH LL channels are the old WWSSN style sensors in the basement. These are digitized on mudworm. Given other modern sensors in operation, Mitch decided to decommission these channels. Al will remove them (just the MPH.LL?.NM.00 channels) from mudworm, waveservers, and heli. Jim et al will remove the unneeded hardware. We will keep the sensors for possible display in the middle house.
Flooded
HCAR
EPRM
CWPT
GOBM
SJBM
MORT
MIST
TNMT
HDBT
Down
RDGT-sporadic /Dave
CATM-sporadic /Dave
CTCT-sporadic /Dave - see image
Misc Problems
BRNM - see image
CTCT - see image
Gain Channel Problems
CHRM/Dave
MORT/Dave
FLPT/Dave (doesn't gain range)
CATM/Dave -gain ranges every time (canister in the shop being rebuilt)
Isis Oxidation
GLST.EHZ
Muted Amplitude
BLAR.EHE
DWDM.EHE
HCAR.EHN
JHAR.EHZ
KEWM.EHZ
LRAR.EHE
MSAR.EHZ
MARM.EHN
TMAR.EHZ
WALK.EHN
Strongmo Down
HDAR
MKTA
need a padlock on the door.
AC unit in the penthouse isn't working. Probably just needs cleaned.
DSL is installed but the networking is slightly different from lnxt. Steve is working out the problems making both the dsl and the dbb simultaneously accessible.
Construction is finished at the mill. We can get the phone line permanently buried now.
LNXT:
Chris got a quote for the service contract for the generator. It raised some questions and he'll get more detail on what it actually includes so we can decide whether to purchase it.
NMAD:
Need alternate comm path. Telecom says dsl isn't available (though we once had ISDN).
Does G and D provide internet service? Would it be worth purchasing service from them?
G and D and the local EMA installed a 25kW generator to power that side of the building. If there's room power-wise would they let us power the rack from it too? We could share the maintenance of the generator. We'd also have to install a transfer switch.
AZTN: Replaced XLAK clock. Still doesn't work. The new clock tested successfully in the shop. Bad antenna?
CERI:
P-plant has a spare Carrier AC unit that we could use a backup for the Liebert. It consists of a large internal unit like the Liebert and an external condensor. Chris will check it out.
The DBB firewall installed and operating. Only specific ip's are allowed through. You need to use the route command to manually add a route. Evertying behind the firewall is the same D ip number but with 10.156.1 instead of 141.225.156. Remaining items include:
Ownership, Operation, and Maintenance of the four Charleston, SC stations (C1SC, C2SC, ADSC, and TRSC) will be turned over the Univ. of SC at Columbia. Initial December 8-9 went well. John contacted site owner and IT people for firewall and permissions. He requested CERI access be continued for a tbd transition period. Mitch will work with Phillip to get data flow redirected to Columbia. Then those feeds will get added to the CO network export to CERI. The stations will keep the same name but the network code will change (from NM to CO).
Paperwork is missing for some visits so the state of some upgrades is
not completely known. Remember, the job's not finished til the paperwork's
done!
I added the firmware column to the upgrade list as that seems to be causing problems at some sites. Are there problem sites that have all the hardware but need new firmware that should be added to this list? The worst offenders of this list are CBHT, MCAR and RDST.
Steve will pursue getting CBHT on an isolated subnet as suggested by the CBHS IT person. Mitch will put RDST on gcf2ew to see if that works.
Year3 upgrades
| station | yr3 hardware | router | mss100 | firmware | plug | comment |
|---|---|---|---|---|---|---|
| RDST | Y | Y | N | ? | ? | Al has determined that other RDST IP's are trying to access our router, and Steve will look at routers external connection and security settings |
| MCAR | Y | Y | Y | ? | ? | upgrade not documented |
| CUET | Y | Y | Y | ? | N | upgrade not documented |
| MKAR | Y | Y | Y | ? | N | comment |
| CBHT | Y | N | Y | ? | ? | Steve torequest our 192 IP be put on a private VLAN |
| COLT | N | N | N | ? | ? | none |
| GILT | N | N | N | ? | ? | none |
| NAIT | N | N | N | ? | ? | still not on internet |
| NMEM | N | N | N | ? | ? | none |
| TUMT | N | N | N | ? | ? | none |
Replace foam rubber plugs at all year 1 stations (need to notch it to accomodate the GPS pods): ASAR, CBHT, CUET, CVTN, HDBT, LNXT, MCAR, NAIT, SFTN & TUMT.
NAIT: Couldn't get the NWS cable modem to cooperate at NAIT (firewall?). Dean was going to ask Comcast about it and maybe get a new modem. Steve will follow up.
The latest in the continuing CVVA saga:
CVVA was visited and verified that data is still coming out of the lantronix card and all of the settings are as specified. The security table was opened to 255.255.255.255 so it won't come into play. Data is still not getting to Rhydra1 though. Steve has sent another email to Debbie Cottrell at NG Security to see if she can help resolve the problem. I also left he a voice mail but did not hear back from her. CVVA is upgraded but there remains an unresolved issue with the firewall Steve is working with Merritt Reagan NGC to get them resolved. Merritt says their it security has been slow to respond but they're still working on this. Its due for a routine maintenance visit.Steve followed up with this email and is still waiting to hear back.
We need a single document (access database excel spreadsheet) to document information on all freefield strongmotion stations to include contact(s), current IP's(eternal/external), what ports are allowed in from what IP's here at CERI, telnet yes or no, trigger levels etc. etc.
HDBR
Awaiting funding approval from TDOT to replace the DAS's with Kinemetrics Granites.
HDAR
Bad CMG5 E/W.
Not getting broadband feed.
On hold for Boswell comm.
HDBT
Replace two bad mss100's.
Bad 100' accelerometer.
On hold for Boswell comm.
How many of these are bad batteries or awaiting power upgrades?. Passing a load test in the summer is not always predictive of performance in the winter.
Down
TRYN-reinstall after tower is rebuilt.
RBNC-sporadic
WSNC-sporadic
Strong Mo Down
BCRT
CVVA -firewall issues
CSTN -very gappy
Doesn't degain in Xpick/Mitch
ETT
CCNC
WSNC
RBNC
Corrected mishandling of 3-char station codes and error in gain_station.dat file.
Others had gaps in the sac EHG file.
Remaing upgrade stations are DYTN, CCRT, and CPRT (remaining ARRA upgrade is
GTTN). See map.
CCRT is next on the list.
CPRT is trenched and ready for upgrade. Repeats through GTTN so we want to do
these in the same trip.
DYTN permission is Nashville awaiting a decision
GTTN is awaiting internet permission (Dollywood is top prospect).
STAR Node
star node needs tower work and power supply upgrade.
BHT move. Latest update from John:
- We had planned on moving BHT from it's present location because access has gotten so bad.
- I have been working on getting a permission to move it about 3 miles to the E NE to Black Mtn. There is some issue with the permission, but my contact with the Cumberland Trail State Park says he believes we can work it out.
- If this station is ever upgraded to a RefTek, it is doubtful we would be able to make the 45 mile radio shot from Black Mtn. to Starr Mtn. so we would have to find a new place to transmit to. It will not repeat through any other station, because it is either blocked by mountains or too long a shot.
- Previously, we had discussed trying to find another site to relocate BHT to in order to center it between WMTN and DYTN.
- I found a site in the Catoosa Wildlife Area called Pilot Knob Mtn, (36 deg. 322'N, 84 deg. 45'W). It is about 18 miles NE of BHT. It is at an old fire tower site and apparently there is still a good road up to it.
- I have also found a couple of potential sites to transmit to where we could get internet.
So, before I proceed any further, we need to decide if a replacement station for BHT should get a RefTek and if so, do we want to put the station at Black Mtn. or at another site.
BRBC Node
Power upgrades: RBNC, and TRYN.
TRYN: Hardware is removed from the tower and will be reinstalled when the new one is complete.
There seems to be a strange, repeating signal on many of the stations, most notably on WSNC/RBNC. It repeats about every 14 minutes and lasts about 1 sec. Example data are in /gaia/home/mwithers/public_html/STAFF/Analog/2010/20100910_051500_MAN. Click this picture of it on CCNC. John listened for them on the spectrum analyzer at the BRBC node and reports:
Every 13 min. 53 sec. there is a noise burst that raises the noise floor and knocks out our stuff. I'm wondering if it could be a sideband from something. I listened and I couldn't hear anything but white noise when it happens. I'm wondering if cavity filters and pre-amps will help.John identified the source of the interference as an amateur radio operator who has a repeater at 224.540 MHz which has a side band right about 217.5 MHz. He'll be meeting with him at the tower in two weeks to determine how best to resolve it. He's transmitting 2W and we're pointed right at his omni, so either the ham radio needs tuned or a filter, or we need to move our antenna.
MTTN Node
Greentop is getting a new tower too. There will not be internet up at the mountain, so we need to find a place to transmit to. We should be able to hit just about anything in Knoxville, Maryville or Pigeon Forge; is there a preference?
USGS provide RT147 accelerometers for all 6 stations. Two are installed (which ones). Need to install the remaining four as we visit the station (i.e. don't need to make a special trip).
Only GTTN remains. Tied to CPRT upgrade (see above) and awaiting comms permission. If we can't get the permission for local internet by the end of June, we need to move ahead with the repeat through ASTN. There is new infrastructure at ASTN that could be a prospect for putting it on AC power.
Due to remodeling, we pulled K2 1738 from Firestation #2 in Dyersburg, TN in February 2009. It is in the shop with accessories.
The seisnet group on umdrive is at http://umdrive.memphis.edu/mwithers/seisnet You will need your uuid and password.
The pink palace display now includes a heli connection for MPHB. Need to retrieve any remaining hardare (e.g. antenna and reciever on roof).
