Tech Staff Meeting Notes February 27, 2012
The top priority is always repair of down stations.
NM problem stations
ET problem stations
The President's 2013 (or is it 2014, I get confused) budget has funds proposed for adoption of 100 to 250 earthscope TA stations east of longitude 100W. Plans are being developed to make a prioritized list of stations. Plans for O&M will be developed when there's a better idea of what is going where (there are a host of technical and logistical issues to resolve).
We may be getting two Basalt/shortperiod upgrades. We need to make sure we get the right one. There is a version of the basalt w/ no internal episensor and either 4 or 8 external channels. The strong motion Basalts with the internal episensor have only 1 external channel.
There are several stations that have a problem with arbitrarily hanging. Mitch sent an email to the netops list but the resulting suggestions didn't really help to solve the problem. Many of the reftek stations mysteriously cleared up after reloading the firmware.
GTTN/CPRT. These are two refteks behind a single router. They use udp to transmit data. The router opens a port when a broadcast is initiated by the reftek. The rtpd on achilles acknowledges back to the reftek on that port. When the refteks creates a new connection for whatever reason, sometimes the router port used by the router isn't the same as what the reftek or rtpd thinks it is. Steve has a "fanless pc" in house with ubuntu installed and he is working on rtpd/rtp2rtpd and on Scream. Jim updated the firmware on the GNAR reftek and its been behaving ever since (even though we were previously at a newer release than what reftek support said was needed to fix this problem). So while we were going to test the fanless pc at GNAR first, we can now go straight to GTTN/CPRT (though new building construction is underway there now).
RDST and MCAR are also prone to hang. They're now on gcf2ew along with a script that forces a kill and restart when necessary. Need a fanless pc.
EDIT, NNAR & CHNM: NNAR and CHNM repeat through EDIT and the received signal is very bad due to trees. NNAR and CHRM are very noisy and fade out quite a bit. EDIT is now located in the Chicksaw National Wildlife Preserve so cutting trees is not an option. There isn't a clear path to a repeater on the Edith forest tower either. Need a new plan (maybe go directly to the node?).
WALK: Chris and Manning were unable to reach WALK up in the KY bend. Mr Lynn indicated that the station was completely underwater but it is still up? (yes still up).
Reinstall the rest of HDBT when internet connectivity is established. We're bailing on Boswell and sending everything to AZTN.
Search for quieter site for CTCT.
PLAR was decommissioned. Marion Haynes gave Chris contacts for a replacement site.
Down
HCAR - flooded, need to verify that well stays dry.
PGVM.EHN
MPH.LHZ- weren't we getting rid of this station?
Unk Problem
HDAR
WADM.EHZ - see image
Isis Oxidation
GLST.EHE
Muted Amplitude
WALK.EHE
BRGM.EHN
LRAR.EHE
TWAR.EHE
BLAR.EHE
BETM.EHN
HOVM.EHE
MCAM.EHZ
Mag. 4.0 gain range problems
These stations had problems with the gain channel during the Magnituded 4 Missouri
event on Feb 21, 2012. It isn't clear to me whether they data channels saturated
or whether there was excessive chatter in the gain channel.
See image.
CHRM
DWDM
MATM
PGVM
FLPT
LFRT
HTAR coax problem.
Weak Battery/Solar
There were 10 stations down for this event. I'm guessing that for a
majority of them it was just the battery. Chris fixed 4 of them. HCAR was
flooded and HTAR is a likely coax problem. The remaining 4 are listed
/gaia/smeagol/dat2/Working/NM/201112/20111224_124800_MAN
LFRT, LVAR, TNMT, and WADM.
DBB
MKTA
LNXT:
Generator had a full service and load test. Overall its in very good condition. It needs a blockheater, new muffler and exhaust manifold, and vaporizor solenoid. The MOIS and INTR alarms are not being used so they'll be retasked to alarm on loss of line power, and on generator power. It will require a minor code enhancement to env_alarm.
NMAD:
Steve will cancel phone line.
G and D and the local EMA installed a 25kW generator to power that side of the building. If there's room power-wise would they let us power the rack from it too? We could share the maintenance of the generator. We'd also have to install a transfer switch.
AZTN:
CERI:
Twisted pair from Richardson is extremely noisy causing problems with
the analog backbone. Richardson is also on the UM 5-year capital
improvement plan to be replaced. We got a tour of Wilder and its doable.
We need to see if we can get a path to CERI so we know whether we just
need to move what's on Richardson now, or whether we need to turn Wilder
into an entirely new node (i.e. new mudworm). Then we can get Planning
to help us with a design and then we can get permission.
We could just move the firewall from 3904/218 to Wilder and get the DBB
on the campus internet that way (if there's no aironet line of site).
There are good telephone pairs from from the Wilder roof to the basement
and we could move the ceri node/mudworm to the basement. Otherwise we'd
have to find and trust connections between the basement and CERI.
We have line of site from the southeast corner of 3904 except there are powerlines in the way. Can we put up a tower on the east side of 3904 to get over the trees and power lines on central?
NMEM two way comm restored.
LNXT: Basalt is ready to go.
Ownership, Operation, and Maintenance of the four Charleston, SC stations (C1SC, C2SC, ADSC, and TRSC) was turned over the Univ. of SC at Columbia. At Phillip's request, we letting the data flow through CERI for the time being. Eventually it will get switched to Columbia and the network codes will then change from NM to CO.
RDST, CBHT, and MCAR are coming in via gcf2ew on anssworm. CBHT occassionally loses its connection but is overall, ok. MCAR loses its connection about twice as much so while its mostly functional, it should be better. At RDST, a datapacket sneaks through a couple times per day.
Replace foam rubber plugs at all year 1 stations (need to notch it to accomodate the GPS pods): ASAR, CBHT, CUET, CVTN, HDBT, LNXT, MCAR, NAIT, SFTN & TUMT.
NAIT: Couldn't get the NWS cable modem to cooperate at NAIT (firewall?). Dean was going to ask Comcast about it and maybe get a new modem. Steve will follow up.
CVVA. John contacted DMME again. He's also looking into the possibility of a freewave link to UVA.
We need a single document (access database excel spreadsheet) to document information on all freefield strongmotion stations to include contact(s), current IP's(eternal/external), what ports are allowed in from what IP's here at CERI, telnet yes or no, trigger levels etc. etc.
HDBR
Granites are in house and powered up. Steve assigned ip's (.158.50-57) and David installed rock2slink on them. Data are flowing to earthworm on deagol.
HDAR
Bad CMG5 E/W.
Not getting broadband feed.
HDBT
Replace two bad mss100's.
Bad 100' accelerometer.
Steve will discontinue the Boswell dsl and retreive the pc. We'll make a wireless link from the weigh station to AZTN and look into getting additional bandwidth at AZTN. The building and bridge channels don't need to be exported continuous to CERI but the mud island and weigh station channels do. We might be able to get funding to expand the bandwidth and ship all the channels to NSMP in real-time.
GTTN move is schedule for March 26. Chris will be present to guide the move and make sure its working afterward.
Traces taken form this event: /gaia/smeagol/dat2/Working/ET/201202/20120225_225600_MAN/
Down
TRYN
WSNC
Gain Channel Problems - see images
DYTN
SMNC
TVNC
Unk Problem
RBNC.EHE - see image
Strong Motion station down
CVVA - no internet
GTTN - unk problem - see image
Only the DYTN upgrade remains.
See map.
Forest service returned agreement. Mitch is working on it.
STAR Node
BHT move. John has a verbal permission from the Cumberland Trail State Park to move it about 3 miles to the E NE to Black Mtn. This likely will need to be a written permission.
BRBC Node
Power upgrades: RBNC, and TRYN.
TRYN: Hardware is removed from the tower and will be reinstalled when the new one is complete.
MTTN Node
STLE landowner has asked that we move eq. cabinet. Chris is looking for options.
The VLF power supplies were installed CJTR and PIGT. Still needed
at NMKM and RLAP. Chris is trying to find out if we have them or if we need to purchase.
recover the wireless hardware.
LCHS and HCES will get upgrades to NetR9.
The new receiver at HCES is not happy and comms come and go. Most likely its an issue with the wireless DLink. Do we want to run a conduit before they pour the sidewalk?
Granites are streaming to test earthworm on deagol.
The rotation sensors leaked. PMD no longer makes them and isn't willing to repair or replace. New sensors are not in the budget.
10 of the 11 CMG6TD's are serviced, on the shelf, and ready to go. Chris is looking for suitable bags to protect them during burial.
What equipment in the FDL is no longer needed. Mitch will call a meeting of FDL users to discuss culling the equipment. Also, the check out/in process is largely ignored--what system could be implemented that people will actually use.
The prototype in LEPT had excessive gain range chatter during the M3.9 MO event on 2/21. VLF is going to modify all the ordered hardware to increase the delay from 2s to 4s.
Union City is back and triggered on the 2/21 Mw3.9
Jer-Ming's passcal hardware is packed up and on its way back to Socorro.
Dark room refurbish is done. Its now the ring laser lab.
The seisnet group on umdrive is at http://umdrive.memphis.edu/mwithers/seisnet You will need your uuid and password.
The pink palace display now includes a heli connection for MPHB. Need to retrieve any remaining hardare (e.g. antenna and reciever on roof).
