Tech Staff Meeting Notes July 9, 2013

The top priority is always repair of down stations.
NM problem stations
ET problem stations

Duty Table updated July 2013


Priorities


News & Highlights

Multiple simultaneous failures compounded the power outage on June 13. Notes from the Post-Mortem are here


NELE

The students are making good progress with the tech staff providing support. See the installation progress chart in the break area outside the maclab.


NMSZ

Short Period & Broadbands

Routine Maintenance: Top Priorities for the new year are these 2012 leftovers.

NMAD: 1 ISIS and 0 bb

LNXT: Done

MKTA: 6 ISIS and 1 bb

CLTN needs an earthen berm.



Down
NNAR
CHNM
HOVM.EHZ
SEAR offline

Misc Problems
TYAR- dropping out
CPAR - dropping out
BACM.EHE - isis oxidation (scheduled for upgrade)
KEWM.EHE- isis oxidation (scheduled for upgrade)
LNXT.HNE- drifting - see image
FLPT - spikes - see image
GNAR.HH* still drifting

AG station problems
CCAR.HN*- see image

Mag. 4.0 gain range problems
These stations had problems with the gain channel during the Magnitude 4 Missouri event on Feb 21, 2012. It isn't clear to me whether they data channels saturated or whether there was excessive chatter in the gain channel. See image.
CHRM
DWDM
MATM
FLPT

GNAR needs better insulating berm.

EBZ was upgraded to ISIS3.

NODE Tasks

DBB:

MKTA:

LNXT:

We're using propane too fast. We need to let the level, currently 24%, go down before repairing the leak. Will call Cummins when we're under 20%, probably this month at current rates.

NMAD:

G and D and the local EMA installed a 25kW generator to power that side of the building. If there's room power-wise would they let us power the rack from it too? We could share the maintenance of the generator. We'd also have to install a transfer switch.

AZTN:

CERI:

Need a Generator fail alarm in 218/3904. We currently get a pflr alarm when the old rack loses power. There is no way to know if the generator took over other than systems start turning off when the ups dies. Note, that an otmp alarm will follow a pflr alarm unless someone switches the Liebert to generator.

Generator fuel lift pump, battery & air filter should be replaced. Jim sent a quote to PP for that follow up work but no word yet. Cummins is now on a UM agreement and Jim is working with purchasing to get LNXT generator included. This is all done.

Richardson is being replaced. Construction to begin this summer. Richardson will be demolished after the move to the new dorm--possibly sometime in 2014.


ANSS SM

The vlink device servers from B&B work very well and replace the lantronix and routers.

Replace foam rubber plugs at all year 1 stations (need to notch it to accomodate the GPS pods): CBHT.

CVVA. Now has an IP thanks to the VA DMME. John will visit next week. Dave will program a vlink and send it to John. Done.

HDBR
The comm problems for HDBR are a result of cost cutting and a desire to not expand the footprint on the roof of aztn. We have 7 freewaves on the bridge, transmitting to 5 freewaves at aztn. This worked well in the lab but doesn't provide sufficient bandwidth when deployed. Mitch will ask Shahram for $4k to add two more freewaves.

All rotation sensors are corroded, flooded, and not operational.

Note from Steve on May 15:

I spent some time looking at the bridge recorders, and wanted update
everyone with regards to how things are working. Only two recorders of the
seven are completely unreachable, and in both those cases the freewave at
the recorder site is reachable. All seven recorders can be power cycled (ie
Unit answers to the phone key tone commands)

I downloaded a 3min segment of data available at:
/gaia/home/sbrewer1/sac/bridge.dat/20130515_151000_MAN

Here is a summary of the state of things, arranged by recorder.

hdbr-hd1 (pier A) ok waveserver keeping up to date, ssh works fine.

hdbr-hd2 (pier B) ok waveserver keeping up to date, ssh works fine.

hdbr-hd3 (pier C) unreachable, freewave at site is reachable, power cycle
did not help. (bad radio link?)

hdbr-hd4 (pier D) ok?, waveserver behind ~2min, ssh doesn't really work. Did
password prompt but could not connect. (Bad radio link?)

hdbr-hd5 (pier E) no waveserver data, power cycle did not restore, reachable
by ssh. Bad configuration?

hdbr-hd6 (pier F) ok?, waveserver behind ~2min, ssh doesn't work at all.
(bad radio link?)

hdbr-hd7 (pier G) ok, came back after power cycle. (Not so good of a radio
link?)

Looks like most problems could be resolved, if the freewave radio links
worked better. The only recorder completely unreachable is hdbr-hd3 (pier C)

Is there be a way to add two more freewaves to the AutoZone end, and
reconfigure the links without visiting the bridge?

thanks, steve.

HDAR

HDBT


ETSZ

LRVA and VHTN are on deck for upgrades with the Basalts from the Depot.



Down
BHT- sporadic
CCRT
CCNC
TVNC-sporadic
SMNC- still experiencing some type of interference - see image.
BCRT.HN* -
see image

Unk Problem
CPCT.HN* - see image
GFM - see image
FPAL - sporadically drifting
RBNC.EHE and EHG - see image
GTTN.HNE- see image

There is a TA station, V52A is installed nearby and a radio link should work to the top of the Green Top tower. Mitch will contacted EarthScope and got this reply:

Hi Mitch,

I am cc'ing this message to Katrin Hafner, as she handles all adoptions (vault or complete station). Our basic process is that we contact the landowner to see if they are amenable to an extended permit and if they are, we would work to transition the permit to CERI.

I am not sure if V52A is a new style tank (one piece, roto-molded, white plastic) or the old style (corrugated black plastic tube). Either way the diameter is ~ 1 m and essentially the entire floor of the vault is usable (apart from space for a bilge pump). My impression is that the vault floor could accommodate the 4 sensors you have, but I think you could easily measure this. Katrin can give you the exact diameter, since a few inches could make the difference, one way or the other.

You can also check out the QUACK system on the DMC web pages to see how the PDFs look for V52A (or maybe you are already using this site in your processing).

STAR Node

Need to replace vandalized windows in the fire tower.

Generator is not functional despite EMA's effort to repair. Consider additional battery for starworm.

If they get the generator working, a generator on alarm would be useful. Do we have this at other nodes (e.g. Memphis and LNXT)?

BHT move. There was discussion over the permission for PMTN. Jim is ok with fixing the fire tower. Mitch will verify with John when he returns from vacation and then submit the agreement for approvals.

John suggests that we consider abandoning CMGA and moving it to the TA station TA-X51A. It has already been installed and a radio link between that location and GMG will work. This is in reference to TA V52A above, right?

BRBC Node

RBNC and CCNC need new radio pairs.

MTTN Node ASTN: Change to AC power. We have permission and can install anytime.


GPS

Have $1500 to by a 1u unit to replace the VM's.


AG Network

HHAR, FCAR, and LCAR got an annual service and new batteries. CCAR, WHAR, and WLAR still need an annual service.

Jim and Steve H. are working with AGS and AOGC to add 3 additional portables to the AG network. These will be CMG6TD's with cell modems.


FDL

One 6TD returned, has flat traces. Two others forwarded to England for repair.


NSMP

NSMP Station List.


MISCELLANEOUS

The seisnet group on umdrive is at http://umdrive.memphis.edu/mwithers/seisnet You will need your uuid and password.