Tech Staff Meeting Notes March 9, 2011
The top priority is always repair of down stations.
NM problem stations
ET problem stations
There will be an external review panel at CERI sometime around the end of April. The review was requested by Chuck to learn, from an outsider's perspective, what we're doing right and what we can do to improve. Input from staff on specific questions that should be answered and areas that should be evaluated are welcome--give them to Mitch or Chuck sooner rather than later.
Steve Horton lead answered questions about the Guy earthquake Swarm. We currently have one permanent AG station, three tempoary real-time stations, and two non-telemetered portable stations in the vicinity. There is hardware from USGS for another real-time station but where it will be installed depends on how the seismicity evolves. We're going to wait and see for a week or two before siting it.
The NLE will be May 16, 2011. We're be exercising our WinLink system.
Some items, mostly comm issues, are languishing on this list for an exremely long time. Those responsible for those items need to find a solution to the problems or we need to decommission those stations.
Is this entry still true?
ISIS routine maintenance & service 2010:
NFAR, DWDM, and EPRM remain.
Top priority stations - LVAR, and RDGT
Down
BOAR (sporadic- probably bad battery)
LVAR (sporadic -probably bad battery)
MPH.LL / Mitch
LRAR.EHE /Dave
RDGT-sporadic /Dave
CATM-sporadic /Dave
CTCT-sporadic /Dave
Misc Problems
APRT has 20Hz noise. Its not as bad as it was but its still there. -
see previous image from 1/26
BRNM - see image
CTCT - see image
Gain Channel Problems
CHRM/Dave
MORT/Dave
FLPT/Dave (doesn't gain range)
CATM/Dave -gain ranges every time
Isis Oxidation
GLST.EHZ
Muted Amplitude
BLAR.EHE
GOBM.EHZ
HCAR.EHN
MSAR.EHZ
TNMT.EHZ
MARM.EHN
WADM.EHN
WALK.EHN
Strongmo Down
HDAR
Node reset instructions are at
/gaia/smeagol/home/seisadm/ew/node.notes/phone.reset.proc
Note that using the pc reset option with the GX620 system can cause it
to hang. Only power cycles are recommended. The reset option has been
disabled at most nodes.
MKTA
need a padlock on the door.
AC unit in the penthouse isn't working. Probably just needs cleaned.
The marked phone reset is working again. There was a problem with the transient suppression card, it probably took a hit. The fuse that provides power to the reset module was also blown. Jim emailed a copy of my reset procedure which is now at /gaia/smeagol/home/seisadm/ew/node.notes/MKTA__phone_reset.docx. The PC power cycle is on relay 3. The GX620 smart power switch on the front of the PC is still hooked up to relay1 but I wouldn't advise using that.
DSL is now installed but the networking is slightly different from lnxt. Steve is working out the problems making both the dsl and the dbb simultaneously accessible.
Construction is ongoing at the mill.
LNXT:
Chris got a quote for the service contract for the generator. It raised some questions and he'll get more detail on what it actually includes so we can decide whether to purchase it.
NMAD:
Need alternate comm path. Telecom says dsl isn't available (though we once had ISDN).
Does G and D provide internet service? Would it be worth purchasing service from them?
G and D and the local EMA installed a 25kW generator to power that side of the building. If there's room power-wise would they let us power the rack from it too? We could share the maintenance of the generator. We'd also have to install a transfer switch.
AZTN: Replaced XLAK clock. Still doesn't work. The new clock tested successfully in the shop. Bad antenna?
CERI:
The DBB firewall installed and operating. Only specific ip's are allowed through. You need to use the route command to manually add a route. Evertying behind the firewall is the same D ip number but with 10.156.1 instead of 141.225.156. Remaining items include:
Ownership, Operation, and Maintenance of the four Charleston, SC stations (C1SC, C2SC, ADSC, and TRSC) will be turned over the Univ. of SC at Columbia. Initial December 8-9 went well. John contacted site owner and IT people for firewall and permissions. He requested CERI access be continued for a tbd transition period. Mitch will work with Phillip to get data flow redirected to Columbia. Then those feeds will get added to the CO network export to CERI. The stations will keep the same name but the network code will change (from NM to CO).
I added the firmware column as that seems to be causing problems at some sites. Are there problem sites that have all the hardware but need new firmware that should be added to this list? The worst offenders of this list are CBHT, MCAR and RDST.
Paperwork is missing for some visits so the state of some upgrades is
not completely known. Remember, the job's not finished til the paperwork's
done!
Year3 upgrades
| station | yr3 hardware | router | mss100 | firmware | plug | comment |
|---|---|---|---|---|---|---|
| RDST | Y | Y | N | ? | ? | Al has determined that other RDST IP's are trying to access our router, and Steve will look at routers external connection and security settings |
| MCAR | Y | N | N | ? | ? | upgrade not documented |
| CUET | N | ? | Y | ? | N | upgrade not documented |
| MKAR | Y | Y | Y | ? | N | comment |
| CBHT | N | N | N | ? | ? | Steve torequest our 192 IP be put on a private VLAN |
| COLT | N | N | N | ? | ? | none |
| GILT | N | N | N | ? | ? | none |
| NAIT | N | N | N | ? | ? | still not on internet |
| NMEM | N | N | N | ? | ? | none |
| TUMT | N | N | N | ? | ? | none |
Replace foam rubber plugs at all year 1 stations (need to notch it to accomodate the GPS pods): ASAR, CBHT, CUET, CVTN, HDBT, LNXT, MCAR, NAIT, SFTN & TUMT.
NAIT: Couldn't get the NWS cable modem to cooperate at NAIT (firewall?). Dean was going to ask Comcast about it and maybe get a new modem. Steve will follow up.
CVVA is upgraded but there remains an unresolved issue with the firewall Steve is working with Merritt Reagan NGC to get them resolved. Merritt says their it security has been slow to respond but they're still working on this. Its due for a routine maintenance visit.
We need a single document (access database excel spreadsheet) to document information on all freefield strongmotion stations to include contact(s), current IP's(eternal/external), what ports are allowed in from what IP's here at CERI, telnet yes or no, trigger levels etc. etc.
Al and Jim have all the issues resolved and we're now able to download 200 sps triggers. These are the guidelines.
HDBR
Funding has been approved to replace the DAS's with Kinemetrics Granites.
HDAR
Bad CMG5 E/W.
Not getting broadband feed.
HDBT
Replace two bad mss100's.
Bad 100' accelerometer.
How many of these are bad batteries or awaiting power upgrades?. Passing a load test in the summer is not always predictive of performance in the winter.
Down
BCRT
BHT-sporadic
TRYN-reinstall after tower is rebuilt.
RBNC-sporadic
WSNC-sporadic
Strong Mo Down
BCRT
CVVA -firewall issues
CSTN/John got a new lantronix card and is working now, but is very gappy-
still true
Doesn't degain in Xpick/Mitch
ETT
CCNC
WSNC
RBNC
FPAL is about 10 degrees warmer than the Arkansas stations so some shade or forced ventilation may be in order.
Upgrade stations are DYTN, CCRT, and CPRT (ARRA upgrades are BCRT, GTTN, and GMG). See map. John is working on the permissions and paths for them.
DYTN is now on property owned by Cumberland trail state park. Is our permission to be there grandfathered in? John submitted a proposal for the upgrade.
STAR Node
Clock needs a new battery. Need to swap the entire clock then replace the battery back in the shop.
BHT move. Latest update from John:
- We had planned on moving BHT from it's present location because access has gotten so bad.
- I have been working on getting a permission to move it about 3 miles to the E NE to Black Mtn. There is some issue with the permission, but my contact with the Cumberland Trail State Park says he believes we can work it out.
- If this station is ever upgraded to a RefTek, it is doubtful we would be able to make the 45 mile radio shot from Black Mtn. to Starr Mtn. so we would have to find a new place to transmit to. It will not repeat through any other station, because it is either blocked by mountains or too long a shot.
- Previously, we had discussed trying to find another site to relocate BHT to in order to center it between WMTN and DYTN.
- I found a site in the Catoosa Wildlife Area called Pilot Knob Mtn, (36 deg. 322'N, 84 deg. 45'W). It is about 18 miles NE of BHT. It is at an old fire tower site and apparently there is still a good road up to it.
- I have also found a couple of potential sites to transmit to where we could get internet.
So, before I proceed any further, we need to decide if a replacement station for BHT should get a RefTek and if so, do we want to put the station at Black Mtn. or at another site.
BRBC Node
Power upgrades: TVNC, RBNC, and TRYN.
TRYN: Hardware is removed from the tower and will be reinstalled when the new one is complete.
There seems to be a strange, repeating signal on many of the stations, most notably on WSNC/RBNC. It repeats about every 14 minutes and lasts about 1 sec. Example data are in /gaia/home/mwithers/public_html/STAFF/Analog/2010/20100910_051500_MAN. Click this picture of it on CCNC. John listened for them on the spectrum analyzer at the BRBC node and reports:
Every 13 min. 53 sec. there is a noise burst that raises the noise floor and knocks out our stuff. I'm wondering if it could be a sideband from something. I listened and I couldn't hear anything but white noise when it happens. I'm wondering if cavity filters and pre-amps will help.
MTTN Node
Greentop is getting a new tower too. There will not be internet up at the mountain, so we need to find a place to transmit to. We should be able to hit just about anything in Knoxville, Maryville or Pigeon Forge; is there a preference?
Update from James:
RLAP had a short in the solar panel cable and was without power for about 17 days. The Freewave and Moxa at RLAP also suffered damage, maybe from static. The cable, Freewave, and Moxa have been replaced and operation restored. The problem now is the receiver batteries, which aren't being recharged by the solar panel. The easiest solution is to replace the two batteries there with fresh ones, and bring the old ones back to CERI for a good recharge.
I've updated the QC pages for each of our GPS stations by adding figures for receiver voltages and temperatures. The main page is still at http://www.ceri.memphis.edu/people/jpdavis/gama/results.html
Both of our NetR8's have been cold-tested and have passed. The NetR8's have already been phased out of production. We're receiver two new Trimble NetR9 receivers (hopefully) within the next month. The NetR9 does more and costs less than the NetR8. Go figure.
All monuments still need general PM, mainly painting.
Bob's ordered some VLF Designs BC-12 power supplies to replace the BC-1 and BC-2 chargers at NMKM, STLE, and NMKM, and perhaps a few of our older GPS-100's.
These stations still need to be steinerized: CVMS, MACC, HCES, MAIR, NWCC.
STLE, NMKM, and RLAP all still need a charger upgrade. Bob's purchased the new equipment, so it's on the way.
We need to schedule a second GPS braindump to bring Steve Brewer and Al up-to-date on all GPS network operations before I'm gone. I'm now scheduled to be completely done by end of May.
USGS provide RT147 accelerometers for all 6 stations. Need to install.
BCRT is complete but only transmits intermittently. When its down, we have access to the radio on star mountion but not the radio at the station.
GMG upgrade and the power run for CPRT are scheduled for the last week of March.
All replaced DM24's and CMG40T's will be donated to Bob for distribution to his Argentinian colleagues. These donations are "as-is".
Jonah submitted two trouble tickets to Chris after the 4.7 near Guy. Both are phone/download problems.
Due to remodeling, we pulled K2 1738 from Firestation #2 in Dyersburg, TN in February 2009. It is in the shop with accessories.
We pulled the k2 at Obion (station 2460) because the building it was located in is being torn down. This one is ready for reinstall.
The seisnet group on umdrive is at http://umdrive.memphis.edu/mwithers/seisnet You will need your uuid and password.
The new truck is here. Need to put 148 on gov deals.
The pink palace display now includes a heli connection for MPHB. Need to retrieve any remaining hardare (e.g. antenna and reciever on roof).
