Tech Staff UnMeeting Notes November 16, 2009
The top priority is always repair of down stations.
NM problem stations
ET problem stations
The task list updated February 6, 2009 needs to be updated again. Note that everyone is still needed to work on essentially all tasks. There is however, a single person who is primarily responsible for each one. Mitch of course, is ultimately responsible for everything.
Repair down stations
SM sample rate fix/yr 3 hardware upgrade
AG Network
In order to avoid hindering progress on field work, this meeting was an unmeeting. That is we updated the notes without actually meeting face to face. Mitch supplied the undonuts for the meeting.
Mark Meremonte will be visiting December 1st to provide advice and instruction on RT130 and RefTek software O&M. Unfortunately he got called to jury duty today (11/16)--hopefully that won't interfere with his visit.
The CERI Christmas party will be at 11:30 on Friday December 4. Sign up sheet is outside Jenny's office.
Chris is arranging annual safety training for himself, David, and Jim. Unfortunately none of the staff were able to attend so Chris will arrange another session.
Its likely that some of these station problems are at the nodes and can be fixed by checking the discriminators in detail.
Holly's NM list:
Muted Amplitude
LRAR.EHE
DWDM.EHE
TNMT.EHN
GOBM.EHN
MARM.EHE
MIST.EHE
Unusable
CHRM
PLAR
Gain Channel Problem
CHRM
MORT
Drifting
HICK.HHE
Isis Oxidation
SJBM.EHN
MARM.EHN
FLPT.EHE
MCAM.EHN
WYBT.EHZ
Several of the stations can get improved telemetry by transmitting directly to the node. The original configuration changed enough so that there is room in the band. Jim will investigate.
MKTA:
Need alternate comm path. Telecom says dsl isn't available (though we once had ISDN).
LNXT:
The last 12 Lenox node ISIS/BB stations received their routine yearly service visit and a couple of outages were also taken care of.
DSL is running at lnxt but for an unknown reason we can't see it from Memphis. Steve will call tech support to help get the problem resolved.
Still need to tie generator status alarms into lnxtworm
NMAD:
Need alternate comm path. Telecom says dsl isn't available (though we once had ISDN).
AZTN: Upgrade PC and replace XLAK clock
CERI:
Need to gather up a contact list (e.g. everyone's home phone, power companies, etc) and put it on webworm (password protected). Jim has a draft of this that so far has all the node contacts. Mitch suggested that the umdrive seisnet space might be a better place than webworm for it.
Some traffic on the CERI network was causing problems on the DBB. Steve met with Liliana and it looks like a better way to go is a firewall. Steve has one and will install it.
NAIT: Couldn't get the NWS cable modem to cooperate at NAIT (firewall?). Dean was going to ask Comcast about it and maybe get a new modem. Steve will follow up.
Need to complete firmware upgrade on CVVA. The facility changed hands. They are willing to let us piggy back on their internet and John is working out the technical details with them (i.e. which side of the firewall we'll be on, hopefully the outside).
Al is making a practice run on the strongmo trigger download for data from the 3.4 near Dyersburg on Nov 6. Mitch sent these instructions are archving triggered strongmotion data:
HDBR
Recorders 1, 6, and 7 are repaired and awaiting Steve to reconfigure.
After that we can ask Bob for the next lane closure for reinstallation.
Al reports that of the remaining four, one is not associating.
HDAR
DSL is running at Bosewell and we can see the PC there (bowlworm). But we still can't see the hardware at the weigh station (nor mud island) on the VLAN. Steve reports that Lylah thinks its a wiring problem and will call her for an update.
Need to determine what import machine and wsv all these data should be on. How do we trigger?
Bad CMG5 E/W. (on hold for vlan comms)
HDBT
Mud Island vertical array is operational. Not really, functional is a better
word until we can get it online to qc/vet data (see HDAR).
Replace two bad mss100's. (on hold for vlan comms)
Bad 100' accelerometer. (on hold for vlan comms)
Holly's ET list:
Stations down
MGNC
SMNC (John has a replacement sensor)
TVNC (John has a replacement sensor)
Chris teamed up with John again during the week of October 19. John sent this report:
Chris and I had a rather productive week last week.
GRBT - Maintenance
BCRT - Maintenance and Replaced Battery
DYTN - Maintenance, Replaced Battery, adjusted VCO frequency
CMGA - Maintenance & adjusted SR2, (it was undercharging)
ETT - Maintenance, Found bad breakout card, continued moisture in PII box, Put in old style BABs.
Starr Mtn. CR - Replaced AC/Heat Thermostat, balanced radios, Line amps,
CPRT - Maintenance, new Battery, Adjusted FSK (had drifted off frequency).
LRVA - Maintenance, Cleaned badly corroded Battery box, replaced BAB & Solar Regulator.
GTTN - Maintenance, Replaced dead CPRT receiver.
Apparently, we cannot get on any internet up at Greentop because of security issues. I don't think it will be a big deal to send it down to Sevierville or Pigeon Forge. The state is replacing the tower and building there (sounds similar to what is supposed to happen at TRYN) so we need to be prepared to move our stuff once it is ready.
The folks at CVVA are ready for us to get back online. I will try to get up there this week.
Chris added this:
All batteries were new AC Delco style
I thought CMGA received a new battery also?
STARR mtn. tower got another new window.
I'll try to get to BHT soon. That is the last Star Mtn station out.
Morristown Node all back up now.
STAR Node
RCGA charging circuitry at is old and primitive and in need of an upgrade.
BHT. Chris reports that it is time to seriously pursue another site as access is even difficult by 4-wheeler. David reports moving it will take all the fun out it, what's the worst that could happen?
BRBC Node
GFM The NC state park service is purchasing GFM. We already have a couple permissions from them and John initiated the paperwork for the GFM permission.
TRYN: the forest service wants to remove the tower. There will be a new 180' tower and we have permission to be on it. We also have permission to be in the building (rack mount only). Can we convert this station to AC?
NCCU: Gordana wants to get this working. John will schedule with her to come in and try to get it back up.
James has a very nice SOH website at http://www.ceri.memphis.edu/people/jpdavis/gama/results.html.
James provided these unmeeting notes:
ALL sites - All sites need general maintenance. PIGT and CVMS need their batteries checked, and may be up for battery replacement.CJTR - Steve Brewer's worked with Randy Porter and Michael Bonds on this, and we have the IP addresses we needed. There is a configuration issue somewhere on either Camp Robinson's or the State of Arkansas' side that's preventing us from communicating. I'm working with Randy Porter at Camp Robinson on the issue.
STLE, NMKM, RLAP - These sites are still using early versions of Greg's regulator/charger, and should be considered for upgrades when possible.
HCES/HCEX - The site was hit by lightning this last summer and the coax for HCEX was damaged. The coax run for Glen Mattioli's station HCEX has been measured to 150'. Bob wants Glen to provide parts for the coax replacement. Once we have the new coax and connectors, we'll be looking to replace the current damaged coax with the new.
RLAP - We've looked at the data with the new antenna, and RLAP's position has jumped back to where it should be. The weirdness we were seeing in the timeseries between 2005 and 2008 was due to a bad antenna and NOT the monument.
Vaults, enclosures, and radios are here.
Five of six sites are permitted and ready for rough in. Wooly Hollow is first up this week and another stations is planned for next week (shortened T-day week).
RTPD and reftek2ew are installed and working on Mitch's office sun workstation. So is RTCC which allows command and control via a browser (i.e. no pda). Mitch will migrate these to their permanent home on achilles.
The other five sensors and refteks are on the bench. Mitch will hook them to a hub and get them onto rtcc.
The 2nd revision of VLF's new solar regulator/battery charger/dual output load management w/ LVDC is done and 6 should be here in a week or so. There is a 3rd revision design in the works for the NMSZ BB upgrades (it will run much cooler).
ASN3 is down and the others have significant timing issues. Mitch checked ASN1 and it appears to be seeing satellites but it doesn't look like its actually syncing and adjusting the clock (though Mitch is not a comptetent time code reader).
We are approved for the following in approximately the same order as we
hope to complete them:
Note that using other funds, we're also upgrading 3 additional ET stations to digital with episensor and FPAL to the same hardware as the other broadbands.
Mitch hung a chart in the war room to track our progress (we're required to make quarterly progress reports and to meet the proposed schedule).
Mitch acquired an anemometer for inclusion in the FDL. Heather is installing the software on laptop #3 and testing/calibrating the unit. It will be installed at the NVT array initially and then checked into the FDL when they're done.
There are problems with the Generators and wiring in 3904/218. Symptoms include surges on circuits not backed up by the generators during the transfer test every other monday. This resulted in some damaged hardware on September 30. Also there are the ups problems and rebooted machines from the June storm power outage. Greg guesses there could be loose neutrals. Chris will reconfigure the biweekly test to not transfer power and Mitch will see about getting Greg down here to spend some quality time reviewing and rehabbing the entire mess.
Greg went over everything with a fine tooth comb and found nothing wrong. We're back to running transfer tests and haven't had any additional problems. Freaky.
David, Chris, and Jim got hijacked for two weeks by the Langston NVT array. Fortunately its in and done and things went very well. Steve Horton had this to say:
With great skill, proficiency and hard work, several members of the network technical staff (Jim, Chris, and Dave) have completed installation of the 19 element NVT array in Mooring, TN. We are now collecting data and should plan a service visit within the next couple of weeks.
Tom Brackman at NKU has a shiny new CMG6TD that he wants to install and get integrated with ANSS. He'll run scream and Steve will help him get a minimal earthworm going to export data (only need startstop, statmgr, scream2ew, and export_scnl).
What to do with the Steiner brain dump DVD's?
Mitch made a seisnet group on umdrive. Go to http://umdrive.memphis.edu/mwithers/seisnet You will need your uuid and password. There are several things on there including John's database. We can put contact info there too when its available. Only people in the seisnet group have access to this area. If you download something for update, don't forget to upload it again so that we all have the same version of the files.
There is now a ceri_seis email group that includes all the seismic networks staff.
The OGS connection is complete (they're running an earthworm on 4 different machines because of the way the DAS processes are spread around). Mitch will make a separate OGS heli page and verify NEIC is getting them.
Woops, not so fast. Connecting their scream with ours somehow clobbered their kiosk display so they turned off the feed. Steve will help them, simplify things and use an earthworm export instead. New hardware may be needed.
Rotational sensor is feeding an earthworm but still has IRIG crosstalk problem to fix. Its fine on an Oscope hooked to the output of the mux and looks fine in LabView. Its bad when viewed with wave_viewer. Need to see what the sac files look like to make sure it isn't just a wave_viewer feature.
Supply is checking the condition of Parker's old Suburban, Chris doubts it will run after a year sitting over there. He told her we would try to crank it up if needed. McGoldrick needs to stop worrying about whether or not it will be adopted into a good home and get Old Blue off our lot ASAP. David pointed out that Chris already sold one vehicle this year (his van) and parting two might be too much to bear. Mitch says tough noogies dude, get rid of the damn thing.
The pink palace display now includes a heli connection for MPHB. Need to retrieve any remaining hardare (e.g. antenna and reciever on roof).
We're not allowed to store dead batteries in the open on the pallet outside Chris's garage. We need to get rid of them within 3 or 4 days and/or keep them (short term only; no long term storage) in a labeled enclosure (does it need to be vented?). Chris will look into what's required and make it so. Done.
Chris called P Plant electrical shop. They filled the generators soon after the outage. He said when there is a power failure as long as the generator is running that powers fuel tank pumps on campus they will bring fuel to keep us online. Chris will get all contact information posted near transfer switch and added to Jim's list. Done.
3890 Power Chris is going to make a diagram for 3890. Good luck son.
3876 Power Chris is going to make a diagram for 3876. Good luck son.
Should we have a trouble ticket system?