Tech Staff Meeting Notes November 28, 2010
The top priority is always repair of down stations.
NM problem stations
ET problem stations
The CERI Holiday party is scheduled for December 3 at 11:30.
December Show n Tell is monday, December 6 at 1:30 in the war room. Jim will show us all about mss100's.
The January 1, 2011 holiday will be observed on December 31, 2010. So UofM paid Holidays are December 24, 27-31.
The January staff meeting will be at 1:30 on Monday January 3, 2011.
ISIS routine maintenance & service 2010:
NFAR, DWDM, and EPRM remain.
Down
EBZ/Dave
NHIN/Steve (bad time in adsend, need new xlak)
MPH.LL*/Jim
Strongmo Down
HDAR/Chris comm issues between mud island and the weigh station?
RDST/Steve ip/mss issues. CBHT/Chris mss is connected, no contact with cmg5
MCAR/Chris down most of the time. Occasional brief periods of connectivity.
The staff meeting notes show it needs a router and mss100 upgrade.
GSAR requires frequent resets. Somehow port forwarding config got bolluxed. Needs a site visit to reconfig.
Muted Amplitude
NWCT.EHN
LRAR.EHE
BLAR.EHN
WADM.EHN/Dave
BETM.EHN/Dave
WALK.EHN/Dave
DWDM.EHE/Dave
NHIN.EHN/Dave
GOBM.EHZ/Dave
Gain Channel Problems
CHRM/Dave
MORT/Dave
FLPT/Dave
Isis Oxidation
TMAR.EHE/Dave
GLST.EHE/Dave
BACM.EHE/DAVE
Unknown problem
CTCT-drops out occasionally
MIST has sporadic telemetry dropouts.
CATM.EHE/Jim image
TOPM/Jim
MLDM/Jim
MARM.EHN see image
Node reset instructions are at
/gaia/smeagol/home/seisadm/ew/node.notes/phone.reset.proc
Note that using the pc reset option with the GX620 system can cause it
to hang. Only power cycles are recommended. The reset option has been
disabled at most nodes.
MKTA
DSL is now installed but the networking is slightly different from lnxt. Steve is working out the problems making both the dsl and the dbb simultaneously accessible.
Construction is ongoing at the mill.
LNXT:
Chris is checking on the cost of a service contract for the generator.
NMAD:
Need alternate comm path. Telecom says dsl isn't available (though we once had ISDN).
AZTN: Replace XLAK clock.
CERI:
The DBB firewall installed and operating. Only specific ip's are allowed through. You need to use the route command to manually add a route. Evertying behind the firewall is the same D ip number but with 10.156.1 instead of 141.225.156. Remaining items include:
Ownership, Operation, and Maintenance of the four Charleston, SC stations (C1SC, C2SC, ADSC, and TRSC) will be turned over the Univ. of SC at Columbia. Initial tour will be December 8-9. John contacted site owner and IT people for firewall and permissions. He requested CERI access be continued for a tbd transition period. Data flow will be redirected to Columbia. Then those feeds will get added to the CO network export to CERI. The stations will keep the same name but the network code will change (from NM to CO).
I added the firmware column as that seems to be causing problems at some sites. Are there problem sites that have all the hardware but need new firmware that should be added to this list? Year3 upgrades
| station | yr3 hardware | router | mss100 | firmware |
|---|---|---|---|---|
| RDST | Y | Y | N | ? |
| MCAR | Y | N | N | ? |
| CUET | N | ? | Y | ? |
| MKAR | Y | Y | Y | ? |
| CBHT | N | N | N | ? |
| COLT | N | N | N | ? |
| GILT | N | N | N | ? |
| NAIT | N | N | N | ? |
| NMEM | N | N | N | ? |
| TUMT | N | N | N | ? |
Replace foam rubber plugs at all year 1 stations (need to notch it to accomodate the GPS pods): ASAR, CBHT, CUET, CVTN, HDBT, LNXT, MCAR, NAIT, SFTN & TUMT.
NAIT: Couldn't get the NWS cable modem to cooperate at NAIT (firewall?). Dean was going to ask Comcast about it and maybe get a new modem. Steve will follow up.
CVVA is upgraded but there remains an unresolved issue with the firewall Steve is working with Merritt Reagan NGC to get them resolved. Merritt says their it security has been slow to respond but they're still working on this.
Al and Jim have all the issues resolved and we're now able to download 200 sps triggers. These are the guidelines.
HDBR
Funding has been tentatively approved to replace the DAS's with Kinemetrics Granites.
HDAR
Bad CMG5 E/W.
Not getting broadband feed.
HDBT
Replace two bad mss100's.
Bad 100' accelerometer.
Down
TRYN-reinstall after tower is rebuilt.
Strong Mo Down
ADSC- changing ip issues
CVVA -firewall issues
Gain Channel Problems
CCNC- see new images
Unknown
RBNC.EHE - new image
WSNC -same
CMGA- same
CSTN/John got a new lantronix card and is working now, but is very gappy- still true
FPAL is about 10 degrees warmer than the Arkansas stations so some shade or forced ventilation may be in order.
Mitch tentatively selected WMTN, ASTN, and RCGA. Possible alternatives include CPRT, CCRT. These selections were made solely on geography. We need to get together to discuss other issues such as noise, telemetry, permission, etc.
STAR Node
BHT. Chris reports that it is time to seriously pursue another site as access is even difficult by 4-wheeler. David reports moving it will take all the fun out it, what's the worst that could happen? John said that folks at Cumberland Trail State Park said we can put the station at another spot on Black Mtn. which is just a couple of miles NE of the present location. It is accessible by a paved road. We can pursue that or I can look for something that will put it further NE to get it more equidistant between DYTN and WMTN. Right now it is awfully close to DYTN and BCRT. Mitch likes the idea of splitting the difference between DYTN and WMTN. John will dig out his old notes and starting looking around.
BRBC Node
Power upgrades: TVNC, RBNC, and TRYN.
TRYN: Hardware is removed from the tower and will be reinstalled when the new one is complete.
There seems to be a strange, repeating signal on many of the stations, most notably on WSNC/RBNC. It repeats about every 14 minutes and lasts about 1 sec. Example data are in /gaia/home/mwithers/public_html/STAFF/Analog/2010/20100910_051500_MAN. Click this picture of it on CCNC. John listened for them on the spectrum analyzer at the BRBC node and reports:
Every 13 min. 53 sec. there is a noise burst that raises the noise floor and knocks out our stuff. I'm wondering if it could be a sideband from something. I listened and I couldn't hear anything but white noise when it happens. I'm wondering if cavity filters and pre-amps will help.
MTTN Node
Greentop is getting a new tower too. There will not be internet up at the mountain, so we need to find a place to transmit to. We should be able to hit just about anything in Knoxville, Maryville or Pigeon Forge; is there a preference?
A new prof at UNC-CH, Lara Wagner, might be interested in co-locating some of here equipment (Nanometrics broadbands & digitizers), with as at VHTN or LRVA. She is setting up a transect array of 10 stations in western NC.
Need to do spend the GPS pm money by february (i.e. do the preventive maintenance).
James has a very nice SOH website at http://www.ceri.memphis.edu/people/jpdavis/gama/results.html.
PTGV was losing satellites indiscriminately. There another sensor package transmitting right at the gps stations that could be causing interference. The hosts assured James that they aren't the source of the problem though is seems to be clearing up as their experiement completes and more of their sensors go offline.
Most monuments need a new paint job and weed control. Mineral Area Colege is the top priority.
MAIR is now cabled. Jim and David ran the cable and got assistance from their location contractor to connect it. James will mediate with Chris to contact the contractor to get them in touch with Jenny to enter into tigerbuy to pay the bill.
There may be an issue with the NetR8 stopping satellite tracking at temperatures below -5C. Ain't that special. Presumably those in the west are applying heat and there will be a fix forthcoming from trimble.
Need to replace older GPS-100 supply systems.
These stations still need to be steinerized: CVMS, MACC, HCES, MAIR, NWCC.
STLE, NMKM, and RLAP all need a charger upgrade. They are currently working (and have been for a long time) so this is not necessarily a top priority.
Now that the AG network has a maintenance budget, Jim will order one of the web enabled power switches that has a configurable watchdog timer and a spare set of Avalan radios.
vent all cabinets.
Install network WatchDog Timers. Meremonte uses iboot. Is he talking about the ones from dataprobe? http://dataprobe.com/iboot-remote-reboot.php
The HALT link is working well with the new cables, connectors, and dish antenna.
We have permission to use the South Pemiscot School system Admin building internet. We'll repeat throught the GPS station STLE. Requires radio upgrade for STLE. May also need to buy some hardware to clean up the cabling in the network closet at the building. James will contact Mike Lee at the airport for perimission to add the antenna at STLE pointing at PEBM.
CPCT is delayed until January. Greg and Jim have a new plan that considerably simplifies power and hardware aquisition and assembly is in progress. BCRT, GMG, and GTTN along with the 3 TBD non-ARRA ET upgrades will be done in the spring.
All replaced DM24's and CMG40T's will be donated to Bob for distribution to his Argentinian colleagues. These donations are "as-is".
Due to remodeling, we pulled K2 1738 from Firestation #2 in Dyersburg, TN in February 2009. It is in the shop with accessories.
We pulled the k2 at Obion (station 2460) because the building it was located in is being torn down.
The neic has a new metadata website at https://geohazards.usgs.gov/smds/index.php?message=1. John is finished populating it with info from our stations.
The seisnet group on umdrive is at http://umdrive.memphis.edu/mwithers/seisnet You will need your uuid and password.
Rotational sensor is feeding an earthworm but still has IRIG crosstalk problem to fix. Its fine on an Oscope hooked to the output of the mux and looks fine in LabView. Its bad when viewed with wave_viewer. Need to see what the sac files look like to make sure it isn't just a wave_viewer feature.
Parker's old Suburban was sold on gov deals and our cut was about $1800. Chris is checking on doing the same with old blue and, much to David's chagrin, 148. Funds will go toward a new truck. Chris will begin specing the new vehicle (should it be a pickup with a shell or another suburban, or a panel truck, or something else?).
The pink palace display now includes a heli connection for MPHB. Need to retrieve any remaining hardare (e.g. antenna and reciever on roof).
3890 Power Chris is going to make a diagram for 3890. Good luck son.
3876 Power Chris is going to make a diagram for 3876. Good luck son.
Should we have a trouble ticket system? Yes, all we need is someone with enough time to decide which to use, acquire it, install it, learn it, and train everyone else how to use it. O.T. Guy won the nomination.
