Tech Staff Meeting Notes September 13, 2010
The top priority is always repair of down stations.
NM problem stations
ET problem stations
NetOpsV will be focused on AQMS on November 3 and 4 in Menlo Park. It will likely be delayed until February, 2011.
R&O Meeting this afternoon at 3 in the seminar room.
See Healthcare email from Zettergren for important changes to insurance.
Footware is required at CERI. In lab and field environments, footwear must include protection for the toes and top of the foot.
Starting in October, we're going to have a show and tell each month. It will be nominaly scheduled for 1:30 the afternoon of the first monday of each month subject to change based on the needs of fieldwork. The first one will be a CMG5TD/MSS100 show an tell by Jim on October 11. The one for November will be getting data and viewing with pql and will be held on November 8 (Mitch is out of town the first week of November). Other potential topics include: Shaketable, discriminator alignment, CMG6TD, HowTo K2, Computer room power and generators, node resets dbb and other checks, Instrument response and metadata, ANSS performance goals, ANSS self assessment, GPS receivers, FDL tour.
We're going to add the name of the person responible for each entry in the NMDown and ETDown lists. Note that everyone is still needed to keep things going but we need someone who is keeping track of each problem.
David gave an update on what went on at GFM. The tree is on its last legs but is still pretty stable. If we choose GFM for an S-13/RT130 upgrade, and assuming we could get forest service permission and a decent place to land, we could probably get it all up there in two trips with a helicopter for a couple hundred bucks an hour.
ISIS routine maintenance & service 2010:
14 of the 25 New Madrid node stations have been visited and 6 canisters
were rebuilt and reinstalled.
10 of the 24 Lenox stations have been visited and 6 canisters were
rebuilt and reinstalled.
4 of the 21 Marked Tree stations have been visited and 6 canisters were
rebuilt and reinstalled.
Down
PARM/Jim
PEBM/Jim (bad timing but status stream says its locked and good to 4us)
NHIN/Dave (bad time in adsend, need new xlak)
JHAR/Dave
MPH.LL*/Jim
Strongmo Down
HDAR/Chris comm issues between mud island and the weigh station?
RDST/Chris no contact with mss. power reset didn't fix it.
CBHT/Chris mss is connected, no contact with cmg5
MCAR/Chris down most of the time. Occasional brief periods of connectivity.
The staff meeting notes show it needs a router and mss100 upgrade.
Muted Amplitude
BLAR.EHE/Dave
BLAR.EHN/Dave
LVAR.EHN/Dave
STAM.EHE/Dave
WADM.EHN/Dave
BETM.EHN/Dave
WALK.EHN/Dave
DWDM.EHN/Dave
NHIN.EHN/Dave
Drifting
HICK.HHE/Dave - see image
PARM.HHN/Dave
Gain Channel Problems
CHRM/Dave
MORT/Dave
FLPT/Dave
Isis Oxidation
JHAR.EHE/Dave
TMAR.EHE/Dave -see image
Unknown problem
CATM.EHE/Jim - see image
TOPM/Jim -see image
MLDM/Jim - see image
HALT/Jim - see image
HENM/Jim has a strange "vibration" on the vertical only (do these sensors somehow
know they're headed for the graveyard).
Node reset instructions are at
/gaia/smeagol/home/seisadm/ew/node.notes/phone.reset.proc
Note that using the pc reset option with the GX620 system can cause it
to hang. Only power cycles are recommended. The reset option has been
disabled at most nodes.
MKTA
The phone line is recabled temporarily and the reset is reenabled.
DSL is now installed but the networking is slightly different from lnxt. Steve is working out the problems making both the dsl and the dbb simultaneously accessible.
Construction is ongoing at the mill.
LNXT:
Chris is checking on the cost of a service contract for the generator.
NMAD:
Need alternate comm path. Telecom says dsl isn't available (though we once had ISDN).
AZTN: Replace XLAK clock.
CERI:
The DBB firewall installed and operating. Only specific ip's are allowed through. You need to use the route command to manually add a route. Evertying behind the firewall is the same D ip number but with 10.156.1 instead of 141.225.156. Remaining items include:
Ownership, Operation, and Maintenance of the four Charleston, SC stations (C1SC, C2SC, ADSC, and TRSC) will be turned over the Univ. of SC at Columbia possibly by September 1. John will need to give Phillip Crotwell a show and tell at each station. Data flow will need to be redirected to Columbia (firewalls and etc taken care of). Then those feeds will get added to the CO network export to CERI. The stations will keep the same name but the network code will change (from NM to CO).
Year3 upgrades
| station | yr3 hardware | router | mss100 |
|---|---|---|---|
| RDST | Y | Y | N |
| MCAR | Y | N | N |
| CUET | N | ? | Y |
| MKAR | Y | Y | Y |
| CBHT | N | N | N |
| COLT | N | N | N |
| GILT | N | N | N |
| NAIT | N | N | N |
| NMEM | N | N | N |
| TUMT | N | N | N |
Replace foam rubber plugs at all year 1 stations (need to notch it to accomodate the GPS pods): ASAR, CBHT, CUET, CVTN, HDBT, LNXT, MCAR, NAIT, SFTN & TUMT.
NAIT: Couldn't get the NWS cable modem to cooperate at NAIT (firewall?). Dean was going to ask Comcast about it and maybe get a new modem. Steve will follow up.
CVVA is upgraded but there remains an unresolved issue with the firewall Steve is working with Merritt Reagan NGC to get them resolved. Merritt says their it security has been slow to respond but they're still working on this.
Al and Jim have all the issues resolved and we're now able to download 200 sps triggers. These are the guidelines.
HDBR
The aztnworm upgrade broke the disk sharing with the win98 box. Mitch
sent a budget and justification to Sharam. This project needs to be
either upgraded to replace the substandard PMD dataloggers, or it
needs to be decommissioned (see HDBR notes from
).
Recorder #5 has a bad disk bay.
HDAR
Bad CMG5 E/W.
Not getting broadband feed.
HDBT
Replace two bad mss100's.
Bad 100' accelerometer.
Down
StrongMo Down
Drifting
Gain Channel Problems
Unknown
FPAL is about 10 degrees warmer than the Arkansas stations so some
shade or forced ventilation may be in order.
Mitch needs to get off his butt and select the 3 non-ARRA ET stations
for upgrade.
STAR Node
BHT. Chris reports that it is time to seriously pursue another site as
access is even difficult by 4-wheeler. David reports moving it will take
all the fun out it, what's the worst that could happen? John said that
folks at Cumberland Trail State Park said we can put the station at
another spot on Black Mtn. which is just a couple of miles NE of the
present location. It is accessible by a paved road. We can pursue that
or I can look for something that will put it further NE to get it more
equidistant between DYTN and WMTN. Right now it is awfully close to DYTN
and BCRT. Mitch likes the idea of splitting the difference between DYTN
and WMTN. John will dig out his old notes and starting looking around.
BRBC Node
Power upgrades: TVNC, GFM (done), RBNC, and TRYN.
GFM The NC state park service is purchasing GFM. We already have a couple
permissions from them and John initiated the paperwork for the GFM permission.
TRYN: Hardware is removed from the tower and will be reinstalled when
the new one is complete.
There seems to be a strange, repeating signal on many of the stations,
most notably on WSNC/RBNC. It repeats about every 14 minutes and lasts
about 1 sec. Example data are in
/gaia/home/mwithers/public_html/STAFF/Analog/2010/20100910_051500_MAN.
Click this picture of it on CCNC.
This from Jim:
MTTN Node
Greentop is getting a new tower too. There will not be internet up at
the mountain, so we need to find a place to transmit to. We should be
able to hit just about anything in Knoxville, Maryville or Pigeon Forge;
is there a preference?
A new prof at UNC-CH, Lara Wagner, might be interested in co-locating
some of here equipment (Nanometrics broadbands & digitizers), with as at
VHTN or LRVA. She is setting up a transect array of 10 stations in
western NC.
James has a very nice SOH website at
http://www.ceri.memphis.edu/people/jpdavis/gama/results.html.
PTGV was losing satellites indiscriminately. There another sensor
package transmitting right at the gps stations that could be causing
interference. The hosts assured James that they aren't the source of
the problem though is seems to be clearing up as their experiement
completes and more of their sensors go offline.
Most monuments need a new paint job and weed control. Mineral Area
Colege is the top priority.
MAIR is currently wireless. We have a cat5 surge arrestor and
permission to run the cable into the office (less than 50' though
it does need a hole in the wall).
There may be an issue with the NetR8 stopping satellite tracking
at temperatures below -5C. Ain't that special. Presumably those
in the west are applying heat and there will be a fix forthcoming
from trimble.
Need to replace older GPS-100 supply systems.
These stations still need to be steinerized: CVMS, MACC, HCES, MAIR, NWCC.
STLE, NMKM, and RLAP all need a charger upgrade. They are currently
working (and have been for a long time) so this is not necessarily a
top priority.
vent all cabinets.
Install network WatchDog Timers. Meremonte uses iboot. Is he
talking about the ones from dataprobe?
http://dataprobe.com/iboot-remote-reboot.php
Rough-ins are complete for all NM broadbands except LPAR and GNAR.
GNAR needs a custom cabinet on stilts. Can also remove the computer
and put both GSAR and GNAR behind a router pulled directly into scream
and rtpd at ceri.
Near-term tentative schedule: SWET is done.
Need to test rtpd on wintel. Scream/scream2ew will be replaced with
rtpd/reft2ew.
We are approved for the following in approximately the same order as we
hope to complete them:
Note that using other funds, we're also upgrading 3 additional ET stations to
digital with episensor and FPAL to the same hardware as the other broadbands.
Mitch hung a chart in the war room to track our progress (we're
required to make quarterly progress reports and to meet the proposed schedule).
We need to be very careful to keep AG, ARRA, and CERI hardware separate and
be certain that hardware purchased by each project goes only to that project.
All replaced DM24's and CMG40T's are to go to south campus and taken off
inventory. This hardware is not to be redeployed anywhere in the CERI
seismic network nor FDL. Mitch is checking whether they can be donated
to Bob's Argentinian colleagues.
Guralp is in Tigerbuy so we'll be able to pay them for the repair
of the two CMG6TD's.
Need to review power loads and line balance after adding the new
AQMS servers. Review needs to consider addition of new landmark
servers, mac server, and seisnet deep waveform buffers. The Earth
Science Library (ESL) in 3892 is moving to the main branch and has a
liebert. We should consider carving out rack space in that room (though
backup power is an issue). Mitch will set up a meeting with Jim B., Steve B.,
and Bob D. We will have to move into an era of not having all servers
(e.g. landmark servers) on the generator.
We're now officially providing maintenance support for NSMP stations
in the NMSZ area. There are 22 stations. SLU has 11 to the north
and CERI has 11 in AR and TN. They are all dial-up K2 or Etna. Data
continue to go to NSMP in Menlo who will continue to monitor SOH.
They will email or call when there is a problem. We're the boots
on the ground.
Due to remodeling, we pulled K2 1738 from Firestation #2 in Dyersburg, TN in
February 2009. It is in the shop with accessories.
We pulled the k2 at Obion (station 2460) because the building it was
located in is being torn down.
The neic has a new metadata website at
https://geohazards.usgs.gov/smds/index.php?message=1.
John is finished populating it with info from our stations.
ISTI was contracted by USGS to support INV. Mitch is working with
Sid to get initial dataless SEED going. John will be our INV czar.
(though now I hear rumblings that there may be a different ANSS
supported tool for this).
The seisnet group on umdrive is at
http://umdrive.memphis.edu/mwithers/seisnet
You will need your uuid and password.
Rotational sensor is feeding an earthworm but still has IRIG crosstalk problem to fix.
Its fine on an Oscope hooked to the output of the mux and looks fine in
LabView. Its bad when viewed with wave_viewer. Need to see what the sac
files look like to make sure it isn't just a wave_viewer feature.
Chris continues to hound the supply to try to get Parker's old Suburban
onto govdeals.
The pink palace display now includes a heli connection for MPHB. Need to
retrieve any remaining hardare (e.g. antenna and reciever on roof).
3890 Power Chris is going to make a diagram for 3890. Good luck son.
3876 Power Chris is going to make a diagram for 3876. Good luck son.
Should we have a trouble ticket system? Yes, all we need is someone
with enough time to decide which to use, acquire it, install it,
learn it, and train everyone else how to use it. O.T. Guy won the
nomination.
No contact with #7
ETSZ
TVNC/John
GRBT/John- sporadic
CCNC/John- see image
TRYN/John-reinstall after tower is rebuilt.
ADSC/Steve changing ip issues?
CVVA/Steve firewall issues?
ATTN/John
ETT.EHG/John - same as before see image
SMNC/John - same as before see image
RBNC/John - interference
VHTN-EHN/John - Same as before w/ new image
WSNC/John - see image
CMGA
CSTN/John got a new lantronix card and is working now, but is very gappy
The 5 sec. noise bursts appear simultaneously approx. every 14 minutes
on the stations ( RBNC, WSNC, MGNC & CCNC) from receive site #2. We
should check the phone cable shield ground in the receiver cabinet and
rack. Also look at the Balun/TSP card in the rack. It might be RF
interference coming in via the receivers?
GPS
AG Network
ARRA Upgrades
FDL
Computer Room Power
NSMP
Metadata
MISCELLANEOUS
CERI
