Tech Staff Meeting Notes July 9, 2013
The top priority is always repair of down stations.
NM problem stations
ET problem stations
Multiple simultaneous failures compounded the power outage on June 13. Notes from the Post-Mortem are here
The students are making good progress with the tech staff providing support. See the installation progress chart in the break area outside the maclab.
Routine Maintenance: Top Priorities for the new year are these 2012 leftovers.
NMAD: 1 ISIS and 0 bb
LNXT: Done
MKTA: 6 ISIS and 1 bb
CLTN needs an earthen berm.
Down
NNAR
CHNM
HOVM.EHZ
SEAR offline
Misc Problems
TYAR- dropping out
CPAR - dropping out
BACM.EHE - isis oxidation (scheduled for upgrade)
KEWM.EHE- isis oxidation (scheduled for upgrade)
LNXT.HNE- drifting - see image
FLPT - spikes - see image
GNAR.HH* still drifting
AG station problems
CCAR.HN*- see image
Mag. 4.0 gain range problems
These stations had problems with the gain channel during the Magnitude 4 Missouri
event on Feb 21, 2012. It isn't clear to me whether they data channels saturated
or whether there was excessive chatter in the gain channel.
See image.
CHRM
DWDM
MATM
FLPT
GNAR needs better insulating berm.
EBZ was upgraded to ISIS3.
DBB:
MKTA:
LNXT:
We're using propane too fast. We need to let the level, currently 24%, go down before repairing the leak. Will call Cummins when we're under 20%, probably this month at current rates.
NMAD:
G and D and the local EMA installed a 25kW generator to power that side of the building. If there's room power-wise would they let us power the rack from it too? We could share the maintenance of the generator. We'd also have to install a transfer switch.
AZTN:
CERI:
Need a Generator fail alarm in 218/3904. We currently get a pflr alarm when the old rack loses power. There is no way to know if the generator took over other than systems start turning off when the ups dies. Note, that an otmp alarm will follow a pflr alarm unless someone switches the Liebert to generator.
Generator fuel lift pump, battery & air filter should be replaced. Jim sent a quote to PP for that follow up work but no word yet. Cummins is now on a UM agreement and Jim is working with purchasing to get LNXT generator included. This is all done.
Richardson is being replaced. Construction to begin this summer. Richardson will be demolished after the move to the new dorm--possibly sometime in 2014.
The vlink device servers from B&B work very well and replace the lantronix and routers.
Replace foam rubber plugs at all year 1 stations (need to notch it to accomodate the GPS pods): CBHT.
CVVA. Now has an IP thanks to the VA DMME. John will visit next week. Dave will program a vlink and send it to John. Done.
HDBR
The comm problems for HDBR are a result of cost cutting and a desire to not expand
the footprint on the roof of aztn. We have 7 freewaves on the bridge, transmitting
to 5 freewaves at aztn. This worked well in the lab but doesn't provide sufficient
bandwidth when deployed. Mitch will ask Shahram for $4k to add two more freewaves.
All rotation sensors are corroded, flooded, and not operational.
Note from Steve on May 15:
I spent some time looking at the bridge recorders, and wanted update everyone with regards to how things are working. Only two recorders of the seven are completely unreachable, and in both those cases the freewave at the recorder site is reachable. All seven recorders can be power cycled (ie Unit answers to the phone key tone commands) I downloaded a 3min segment of data available at: /gaia/home/sbrewer1/sac/bridge.dat/20130515_151000_MAN Here is a summary of the state of things, arranged by recorder. hdbr-hd1 (pier A) ok waveserver keeping up to date, ssh works fine. hdbr-hd2 (pier B) ok waveserver keeping up to date, ssh works fine. hdbr-hd3 (pier C) unreachable, freewave at site is reachable, power cycle did not help. (bad radio link?) hdbr-hd4 (pier D) ok?, waveserver behind ~2min, ssh doesn't really work. Did password prompt but could not connect. (Bad radio link?) hdbr-hd5 (pier E) no waveserver data, power cycle did not restore, reachable by ssh. Bad configuration? hdbr-hd6 (pier F) ok?, waveserver behind ~2min, ssh doesn't work at all. (bad radio link?) hdbr-hd7 (pier G) ok, came back after power cycle. (Not so good of a radio link?) Looks like most problems could be resolved, if the freewave radio links worked better. The only recorder completely unreachable is hdbr-hd3 (pier C) Is there be a way to add two more freewaves to the AutoZone end, and reconfigure the links without visiting the bridge? thanks, steve.
HDAR
HDBT
LRVA and VHTN are on deck for upgrades with the Basalts from the Depot.
Down
BHT- sporadic
CCRT
CCNC
TVNC-sporadic
SMNC- still experiencing some type of interference - see image.
BCRT.HN* - see image
Unk Problem
CPCT.HN* - see image
GFM - see image
FPAL - sporadically drifting
RBNC.EHE and EHG - see image
GTTN.HNE- see image
There is a TA station, V52A is installed nearby and a radio link should work to the top of the Green Top tower. Mitch will contacted EarthScope and got this reply:
Hi Mitch,
I am cc'ing this message to Katrin Hafner, as she handles all adoptions (vault or complete station). Our basic process is that we contact the landowner to see if they are amenable to an extended permit and if they are, we would work to transition the permit to CERI.
I am not sure if V52A is a new style tank (one piece, roto-molded, white plastic) or the old style (corrugated black plastic tube). Either way the diameter is ~ 1 m and essentially the entire floor of the vault is usable (apart from space for a bilge pump). My impression is that the vault floor could accommodate the 4 sensors you have, but I think you could easily measure this. Katrin can give you the exact diameter, since a few inches could make the difference, one way or the other.
You can also check out the QUACK system on the DMC web pages to see how the PDFs look for V52A (or maybe you are already using this site in your processing).
STAR Node
Need to replace vandalized windows in the fire tower.
Generator is not functional despite EMA's effort to repair. Consider additional battery for starworm.
If they get the generator working, a generator on alarm would be useful. Do we have this at other nodes (e.g. Memphis and LNXT)?
BHT move. There was discussion over the permission for PMTN. Jim is ok with fixing the fire tower. Mitch will verify with John when he returns from vacation and then submit the agreement for approvals.
John suggests that we consider abandoning CMGA and moving it to the TA station TA-X51A. It has already been installed and a radio link between that location and GMG will work. This is in reference to TA V52A above, right?
BRBC Node
RBNC and CCNC need new radio pairs.
MTTN Node ASTN: Change to AC power. We have permission and can install anytime.
Have $1500 to by a 1u unit to replace the VM's.
HHAR, FCAR, and LCAR got an annual service and new batteries. CCAR, WHAR, and WLAR still need an annual service.
Jim and Steve H. are working with AGS and AOGC to add 3 additional portables to the AG network. These will be CMG6TD's with cell modems.
One 6TD returned, has flat traces. Two others forwarded to England for repair.
The seisnet group on umdrive is at http://umdrive.memphis.edu/mwithers/seisnet You will need your uuid and password.