Tech Staff Meeting Notes January 3, 2007

Current Top Priorities are:Station Maintenance. (Also wish to get INV populated.)

There are four levels of priority.
PrioritiesWish to try to get these done (or make progress on) before the next meeting (yeah right)
Second TierFit this in too if an opportunity arises. May be good to do instead of priorities but check with Mitch. Likely to be a priority in a previous and/or subsequent month
As Time PermitsFit these in when you can but not instead of first or second priorities
OtherStuff we hope to get to and don't want to forget about

This task list as of September 12, 2006. Note that everyone is still needed to work on essentially all tasks. There is however, a single person who is primarily responsible for each one. Mitch of course, is ultimately responsible for everything.

Greg's todo list is incorporated into these notes.

Fourth Annual Feast of Flesh will be Friday February 23, 2007 at 6:30 at the Butcher Shop on Germantown Road. Priorities

Holly's list of stations from 20070108:







Lenox Agreement. Jim noticed the 12 year agreement for the LNXT node is about to expire. Done.

James will coordinate with Bill or Chris when he will be going to the field and needs assistance. Notes from James:

Firstly, the radio tests in and around Tiptonville are complete. We'll likely do a relay shot from RLAP to New Markham, and then bring both sites back to the highschool in Tiptonville. We have permission from the school to use their 50' tower on-site. We'll need a second shot from our site at the Corrections Complex (NWCC) to the highschool. For this, we'll need either a roof or building mount antenna at at least 30' height. The MIS building itself (where out stuff is) can give us almost twenty feet. We'll need to buy high-gain antennas and mounting hardware, and some _serious_ lightning protection.

The second set of radio tests will be at Steele, Missouri, hopefully on 10/20. I visited this last Friday and spoke again with superintendent Mitchell Fisher. WREG has a short mast on one side of the administration building of the Steele schools for their weather station. I'm going to email our contact there to see if we can mount our antenna on that mast. The building's mechanical room has roof access and plenty of space for our radio equipment enclosure. Bob seems to be waiting for the results of this test before he buys radios and antennas.

We've not been able to secure permission to get back inside the terminal building at RLAP. To winterize the station, we need to replace the five aircells with another solar panel, battery, and a charger that will take two panels. Sure, we could trench AC power out to the station, but that would be wasted effort if we _do_ get permission to move back inside. Meanwhile, having the charger, battery, and panel hardware on-hand would be nice for one or both of our new sites around Tiptonville. Mitch and I are in agreement that we can't keep sponging off the seismic networks. We'll have to buy our own stuff.


The map shows stations with yellow triangles being new ones.
SMNC may be getting interference and the gain channel is showing up on the vertical (but doesn't look right). Turns out its interference with a nearby digital transmitter. Need to switch frequencies.
SMNC4done (see note above)
BHT8move (eventually)

Flat gap and GMT are done.
Van hill is on deck. Weren't able to get the to grade the road to the top but it at least goes much farther up now that before. CCRT has been acting strangely. Its now flat. Critter damage has been repaired twice, might need a major rebuild.
Chris got an insurance letter, need to make copies and put them in the vehicles. Done.
TZTN needs a gate and have episensors swapped. The one there now belongs to SLU. Sensor has been swapped, John will ship the the replaced one back to SLU.
GFM has a cable problem. Will visit in the spring.
BRBC was down for a week starting 11/9 while they do work on their internet. Its still down. Clearly the necessary time was underestimated. Al is looking into it. Done
Chris and tbn will join John the week of Jan 22. They're deciding which stations to hit.

Bill is getting familiarized with these so he can take them over.
GPS pods changed at ADSC and TRSC. SHTN is the only one left. Mitch ran the code correct the leap second error. Stations corrected are: ADSC TRSC SHTN GILT CSTN JCMO STIL HAIL SWET. Only data between 200509040000 and 200601010000 exclusive were modified. The sac header value kuser0 was set to "lp fix" and the code will not modify any data that has that value (to avoid doing it twice) if it is run again. See ~mwithers/src/fixleapsec.c for details on the mod.
CVTN and ASAR optical isolation. Need to upgrade to year 3 hardware. Hardware ready. James brought back some nice pictures from CVTN to help figure out how to do it (e.g. the conduit is way too small).
John will give Bob a ring to get the new SLU shipping address (to send back the episensor) and ask about MVKY. Done, SLU now has MVKY.
CUSEC now has business class DSL and an ip address for CUET. May need to get a switch to hang between the dsl router and the mss100. Steve will visit to determine hardware needs.
That leaves NAIT as the only non-real-time station. Wish to get it online in 2007. Routine Maintenance.

Unscheduled Maintenance
Several of these stations are problematic. Jim replaced that canister on a couple and they work fine in the shop. Greg's current theory is that there is a microscopic film of corrosion on some that forms between the copper springs at the bottom and the aluminum canister. He surmises that this causes some kind of goofy current or rf feedback internal to the canister. A small movement reestablished conductivity between the springs and canister and the problem goes away. Whatever, its all black magic to me.
These stations have lots of glitches on them (mitch took them out of the sta/lta trigger subnets),
I need a new list, the above is from the previous meeting. Jim reports that the "glitchy" stations all have lower receive levels than before. Possibly age? Or are the trees higher now? Greg presented the possibility of nearby power line insulators that are bad. This would make arcs that could appear as impulses. Investigation continues.

GNAR had a several second timing error but the DAS indicated all was fine. Jim rebooted and it corrected itself. Weird but watch out for this. Seisnetwatch should catch this and indicate a large latency if the timing is out far enough.

Mitch was having problems with last 3 dm24/cmg40t tests in the test vault. Ew2mseed kept seeing large gaps and overlaps and reallocing memory to enormous values til eventually there was a bus error. Flushed tanks and restarted several times in December. Flushed tanks, and verified that dm24's were set resuse flash in FIFO mode on. They were. Cleared the flash on the dm24 and flushed the waveserver tanks and restarted on 20070102 and ran ew2mseed on the tanks as the data come in. So far its working flawlessly. Will let it run over the weekend to get data for the pdf's, then try to break it.
Note that a contract was let with ISTI to develope an enhanced waveserver that allows overlaps and is more robust with gaps. This should eliminate this sort of problem and allow use to run in adaptive mode. This is the mode that sends current data first and back fill with oldest unacked packets in flash as bandwidth permits.

Second Tier Priorities

NCCU Done.

Steiner Brain Dump Seminar Greg is ready. Mitch acquired a voice recorder and Voice-to-text software. Then we'll take digital photos of anything significant on the white board. Combined, with Greg's drawings, documentation, etc, this should make a nice searchable document. Video would be nice but costs are prohibitive as well as effort required to process and archive (and subsequently retrieve relevant info).
Greg requested hot topics from everyone.

FDL. Mitch needs to talk with Horton about what to do about FDL laptops. Bill is in charge of everything in the fdl. He'll be developing SOP's to keep track of who has what where.

3890 Power Chris is going to make a diagram for 3890. Good luck son.

As Time Permits

Mitch needs to write an ew module to monitor the environmental channels newly installed at the nodes. Done and deployed at all nodes with the env channels. Env_alarm runs at the nodes and sends alarm messages to mainworm. Send_env_alarm listens for them and sends an email when it sees one. MKTA and MTTN still need the gizmo to generate these channels. It appears that the module (env_alarm) sends an OTMP off message on reboot (but not on ew restart). Mitch's theory is that it takes longer for adsend to sync to IRIG and start sending when there is a system reboot than it does when there is just an ew restart. This causes an apparent transition from low to high and env_alarm thinks it is a change of state to off. Will increase time to ignore on startup. Of greater importance is why it only sends an OTMP off instead of one for every channel when this happens? Mitch will investigate. Done. Al is installing.

Generators Old generator needs an exhaust pipe.

Inventory Jim has all the broadband sensors defined and is working on additional equipment (e.g. isis) and the connections. Mitch will install Bill's account in mysql and inv so he can start building stations. Mitch needs to sit down with Jim and figure out what we need to have ISTI help us with.


All hardware is in house to replace the nodes in the coming year. We'll upgrade to ew v7.1 at the same time. This means we'll be using scnl instead of just scn. Steve, Al, and Mitch are working on the details (headaches) of that. Glasworm is the converter and is operational. New switches for the scnl trace-wire are in house. Steve will install.

I-40 Bridge.
Greg will arrange and supervise regrading of HDBT. The regrading was started and there were, shall we say, problems (more like a CF). Hopefully the contractor will have better luck next time. Chris is arranging to have a fence installed (after any pad, gravel or whatever is installed). Cost is $1100 installed (must be real silver). We don't want to do it ourselves since it has to tie into the existing dilapidated fence.

MPHB. Withers needs to determine current orientation so we can pull it and orient correctly. Helicorders look strange. Need to verify the data. Recentered, data are fine now.

VBMS. ASL crew will install sensor, das, and vsat in November. Problems with the VSAT carrier delayed installation...the saga continues.

3904/207. Need two new power circuits in that room. There is one free breaker in the panel. There is a single 4 outlet circuit to the darkroom that isn't used and will be cannibalized. Chris will make it so. One should be on the west wall. Done.

SeisNetWatch ( applet wasn't loading. Mitch installed a boot script and its working now.

There is a new chair at ASU in Jonesboro and he's interested in getting a modern display for Laredo. Mitch is deciding what to do.