CERI8104/CIVL8126-  Data Analysis in Geophysics

 Fall 2019

Tu-Th 11:20-12:45

CERI New/Long Building MAC - Student Computer Lab



Instructor: Bob Smalley
Office hours: when you see me in my office.



frequency domain view of traveling delta functions


Animation of Frequency Domain integrand (top) and Fourier sum to the Time Domain (bottom) of two traveling delta functions.

Course Description:

The course provides an overview of common tools used by Geoscientists.

Homework assignments are designed to develop a working knowledge of a wide range of computer tools.

Topics will include programming in

MATLAB
UNIX/LINUX (Mac OS-X Darwin)
Seismic Analysis Code (SAC)
scripting (sh and csh)
AWK/NAWK/GAWK
Generic Mapping Tools (GMT)
network tools such as FTP, VNC
Python
FORTRAN
C/C++
Adobe Illustrator
EXCEL
Web page development

and Requests.

The course will be "lab based", a short presentation followed by hands on programming exercizes on the CERI computer system for the remainder of the class period.

Texts:

No "official" text. Some useful references and documentation follow.

Please see the web pages from previous years for additional links to online textbooks, articles, tutorials, etc.

Eric Daub's 2018 class notes
Bob Smalley's 2013 class


The Univerity is licensed to access many UNIX books online at safaribooksonline. Access to this website is free from the Univerity of Memphis network.

UNIX Visual Quickstart Guide, Fifth Edition, D.S. Ray and E.J. Ray.
This book will serve as the "textbook" for the introduction to UNIX part of the course.
It is available at safaribooksonline.com (through the link given).
Reading assignments from this book will be assigned.

We will also reference:

Classic Shell Scripting, A. Robbins and N.H.F. Beebe.

Course requirements:

Students must attend all labs. Class absences must be discussed PRIOR to the missed class.

Homework assignments must be handed in on time; 10% will be taken off for each day late.

The class will include completing a computer based research project that uses the tools discussed in class.

Homework:

Homework assignments will be posted on the class web page.

Project:

Each student will design, implement and present a small-scale data analysis project.
The project should use materials and/or techniques discussed in the class.
Use of SAC, and/or MATLAB, and/or Python, and/or GMT as part of the project is mandatory.
Geophysics data is preferred and the project can (is encouraged to), but is not required to, be related to your primary research interest or topic.
The topic should be discussed with me, decided on and approved no later than Nov 1st.
The project should not also be used for another class you are currently taking.
Each student will give a 20 minute PowerPoint presentation during the last day of class.
The project should be thoroughly documented and all scripts/programs/macros turned in as part of the project.


Grading:

50% homework
10% class participation
40% final project.
There is no final exam.

Classes:

        (Clicking on links opens a ppt of the class. Notes are uploaded after class.)
Aug    27   Tu    LAB 1a    Class Information;  LAB 1b - Intro Matlab; LAB 1c - Lab Exercise.
Aug    29   Th    LAB 2a    Continue Matlab; there are lab exercises integrated into the presentation. ; LAB 2b - Lab Exercise.
Sep       3   Tu    LAB 3a    Programming, Computer "Math" counter_v1 m file, and Matlab, LAB 3b vectorizing Stein and Wysession synthetic seismogram (fortran, matlab translation - not vectorized).

Sep       5   Th    LAB 4a    Matlab, LAB 4b vectorizing Stein and Wysession synthetic seismogram.
Sep    10    Tu    LAB 5a    Matlab,  Lab exercise - write a function to calculate the square root using, Hero's or Babylonian method
Sep    12    Th    LAB 6a    Matlab. Lab exercises out of power point.
Sep    17    Th    LAB 7      Matlab. Lab exercises out of power point.
Sep    19    Th    LAB 8      Matlab. fscanf examples, textscan examples, matlab graphics - handles, how2handlegraphics.m, scan1.dat, az backaz notes.
Sep    24    Tu    LAB 9      Matlab. finish how2handlegraphics, helper file for how2handlegraphics CreateFigure.m, input files for lab- mixedin1.dat, mixedin2.dat.

Sep    26    Th    LAB 10    Matlab - finish readfile_exercise, Introduction to UNIX - exercise mixed into presentation.
Oct      1    Tu    LAB 11    Introduction to UNIX - exercise mixed into presentation. Edited copy of list of "all" standard UNIX commands List of Unix commands - Wikipedia, with the ones you will use on a daily basis shown with a yellow background.

Oct      3    Th   LAB 12    Introduction to UNIX - exercise mixed into presentation. Mac UNIX commands List of Mac Unix commands, some files for exercises ToyClassFolder
 
Oct      8    Tu   LAB 13    Continue Introduction to UNIX - exercise mixed into presentation. Review MATLAB homeworks. Continue MATLAB - See the demo m-file for testing and selecting data in matrices row_col_val_log.m. Now follow what is in the demo m-file to replace the string XXXX that shows up in 5 lines (lines ~64 to ~79) of m-file select_ex_hw.m. These two m-files were written to demo a number of features of how to manipulate matrices without loops. Read the code carefully - is is documented for instruction - and ask if you don't understand what is happening.

Oct    10    Th   LAB 14    Continue Introduction to UNIX - exercise mixed into presentation. Continue reviewing MATLAB homeworks. Continue review m-file for test and select data in matrices row_col_val_log.m. Review m-file select_ex_hw.m. Look at vectorized lat/long testing lltest.m and testing your distance and azimuth routines test_dist_az.m distaztsts.dat distazmoretsts.dat.

Oct    15    Tu -- NO CLASS - FALL BREAK

Oct    17    Th   LAB 15    Finish Introduction to UNIX - exercise mixed into presentation. Continue reviewing MATLAB homeworks (solutions select_ex_soln.m. Various ways to test and report bad data checking). Continue review m-file
(I sometimes make changes to the notes presented in class - based on questions/comments during the class or because I found something interesting. The files at the links above are therefore not completely static during the semester.)

Oct    22    Th   LAB 16    "Done" Introducing UNIX, introduction to shell scripts, GMT, awk - exercises mixed into presentation. Download for class gmt_ex_1.sh, gmt_ex_2.sh, sam_map_class.sh and the zip file with plate boundaries in gmt format. pb2002gmt. There are 8 files, with extension ".gmt" in the zip file.

Oct    24    Th   LAB 17    "Done" Introducing UNIX, introduction to shell scripts, GMT, awk - exercises mixed into presentation. Download for class gmt_ex_1.sh, gmt_ex_2.sh, sam_map_class.sh and the zip file with plate boundaries in gmt format. pb2002gmt. There are 8 files, with extension ".gmt" in the zip file.

Oct    29    Th   LAB 18    Advanced shell scripts, GMT, awk - exercises mixed into presentation. Download lab18.zip, india_ex_files.zip, MySecondMap.sh. Using dem topographic datasets on the MACLAB computers - Using GMT topo data from MACLAB.

Oct    31    Th   Lab 19    Review scripts making maps with topography: india_ex, MySecond Map. Connect to folder with topographic databases.
Nov     1    Tu   LAB 20  Introduction to Seismic Analysis Code - SAC. Download sacdata.zip.
Nov     7    Th   LAB 21  Advanced Seismic Analysis Code - SAC. Download updated sacdata.zip. and exercise file class exercises lab 21.
Nov   12    Th   LAB 22  Advanced Seismic Analysis Code - SAC. Download updated sacdata.zip. and exercise file class exercises lab 22.
Nov   14    Th   LAB 23  Advanced Seismic Analysis Code - SAC. Download updated sacdata.zip.
Nov   19    Th   LAB 24  Introduction to Python. Download python1.zip.
Nov   21    Th   LAB 25  Introduction to Python. Download python2.zip.
Nov   26    Th   LAB 26  Introduction to Python. Download python3.zip.
 Homeworks:

        Aug   29                    Due: Tu, Sep 3. Read Eric's datanotes.pdf (Eric Daub's notes) Ch 1 - Introduction to the Course, and Ch 7 - MATLAB 1.
        Sep      3    HW-1:     Due: Th, Sep 5. Run the fortran and matlab versions of the synthetic seismogram program of Stein & Wysession in the class links above.
        Sep      5    HW-2:     Due: Tu, Sep 10. Vectorize the Stein & Wysession synthetic seismogram, Due: Tu, Sep 10.
        Sep    10    HW-3:     Due: Tu, Sep 17.
        Sep    12                    Due: Th, Sep 17  Read Eric's datanotes.pdf (Eric Daub's notes) MATLAB 2-5.
        Sep    17    HW-4:     Due: Th, Sep 26, extended to Oct 1, further extended to Oct 3. math notes for delta and az/back az
        Sep    24                    Due: Tu, Oct. 1.   read the 9 page unix-intro by Satten
        Oct      1                    Due: Th, Oct. 13. What Unix Shell Should I Use?   Read Eric's UNIX notes 1 and 2.
        Oct      3    HW-4a:   Due: Tu, Oct. 8.   Continue HW-4.

        Oct      8     HW-4b   Due: Tu, Oct. 10.   Modify the m-file select_ex_hw.m so it selects the subset of data with the bad points removed. There are comments as to what has to be done in each line. Compare this code to that in the example row_col_val_log.m to see what needs to go into each of the places with XXXX.

        Oct    10     HW-5a   Due: Fr, Oct. 11 afternoon.   Follow the steps in Setting up Mac GIU to get your home and "root" (Macintosh HD) folders in the dock and on the desktop respectively. Next run the command in the file to turn on displaying the invisible files in the GUI display of files. Please email me when you have accomplished these tasks and they are working, or come to get help before tomorrow afternoon.

        Oct    10     HW-4c   Due: Tu, Oct. 17. We are getting closer on the Dist-Az homework - HW-4x. Here is a checklist of what the final version of the homework should do - specifically: I would like an m-file that I can run that will produce a printout of the answers (deltas and azimuths for the 8 test and 8 simulated station-earthquake pairs). hw4_del_az_continue_c.docx. The zip file you submit should therefore have the input data files, the program that reads the input data files and prepares the data for the dist and az functions (or one that does both - your choice), your functions, any documentation or other information in a word file.

        Oct    10     HW-6   Due: Th, Oct. 17. This is for all those who will be using a Windows PC to do their UNIX homeworks. It will be best for you to use a virtual machine (VM) to run unix/linux. I have a Ubuntu VM that has GMT installed (and I'm working on installing SAC).
First install the program "Virtual Box" by Oracle. Do a Google search for it (or go to virtualbox.org), download and install it. Next upload this file Ubuntu.18.04_GAMIT.ova. This is a BIG file - 13GB. Once that file is downloaded Google "install ova on virtualbox" or follow this link techjunkie.com/ova-virtualbox/ to install Ubuntu.
Running Ubuntu inside VirtualBox will have no effect on your Window OS.
The file Set up Ubuntu.docx has how to test that it is installed and gmt is running, or how to fix a common error if it does not run. If you have any problems, come see me.

        Oct    19     HW-7   Due: Tu, Oct. 22   Read Eric's AWK 1 and 2 notes. We will be having homeworks that need UNIX starting this week. See if you can get both  VirtualBox and Ubuntu working on your laptop or home computer or you will be limited to working in the lab or using ssh (a terminal only, UNIX command line interface that will allow you to connect from outside the lab - but you will only be able to work in the UNIX terminal - no GUI. I will show you how to connect during the next class.)

        Oct    22     HW-8   Due: Tu, Oct. 24.  HW8, and read Eric's notes GMT 1 and 2. In HW8.docx, type your answer to the question (what does each flag and its arguments do?) about the shell script we looked at today and the file gmt_ex_2.sh that we did not get to today (run gmt_ex_2.sh to see that it works), directly into the word file and submit it by email. The second file (gmt_ex_2.sh) makes the same plot we made in class with gmt_ex_1.sh, but does it by first making a base map (using psbasemap) and then does the plotting of the line and points with a gmt call for each. Read the comments in the files to see the differences. The second plot also has a title (and including the title seems to break the -BWeSn axis labeling so I had to remove that command and there are axes labels on all 4 sides).

        Oct    24     HW-8a   Due: Th, Oct. 29.  Read Eric's notes GMT 3 and 4. 

        Oct    24     HW-9   Due: Tu, Oct. 29.  HW9.  HW9 zipped folder, and read Eric's notes Shell Script 1 and 2.

        Oct    29     HW-10   Due: Th, Oct. 31.  HW10.  HW10 zipped folder, and read Eric's notes Shell Script 3. UPDATES: Using GMT topo data from MACLAB,  HW10 update,

        Nov     1    HW-11   Due: Th, Nov. 7.  HW10. Read Eric's SAC notes 1, 2 and 3.

        Nov     7    HW-12a Due: Tu, Nov. 12.  Read Eric's SAC notes 4, review notes 1, 2 and 3.

        Nov     12    HW-13 Due: Th, Nov. 14.  Flesh out the psuedo code to plot the Hilbert transformed radial component overlaid on the vertical component - this is same thing that is in the PowerPoint in the class_exercises_lab_22 link above, Lab 22 HW exercise.docx. The seismograms you need for the homework and the sac macros we looked at today are in the sacdata directory.

        Nov     14    HW-14 Notice this has 2 parts: Part 1 Due: Tu, Nov. 19.  Read Eric's Python notes 1. Part 2 Due: Th, Nov. 21, do the exercises at end of the notes.
        Nov     19    HW-14 Notice this has 2 parts: Part 2 Due: Th, Nov. 21, do the exercises at end of the notes.
        Nov     20    HW-15 TBA. Due: Tu, Nov. 26. Read Eric's Python notes 2 and 3. More short python exercises.


Since the HW is on the computer the easiest way to deal with it is electronically. Unless there is something that can't be done electronically, all future homeworks should be submitted without hard copy. This also makes it easy for me to verify your codes.

What to turn in:

1) Program codes
2) Copies of files with output (textual, graphic)
3) Any input data files necessary that I did not provide (e.g. you got them from the internet or your research or made them up).
4) Information on how to run your code so I can reproduce your outputs.

Put the above in a folder with your name and the assignment number, compress it into a zip file and email it to me by the due date.


Links and other useful info:

        Mathworks
        Getting started with Matlab: html, PDF documentation for Matalb
        An Interactive Introduction to MATLAB, U of Edinburgh
        Numerical Computing with MATLAB, Clive Moler (author/founder of MATLAB/Mathworks)


Additional UNIX files/link:

        UNIX_tutorial (Havill, 2006)
        UNIX_book (Fiamingo et al, 1998)
        UNIX_in a nutshell (3rd ed., not latest, Robbins, 1999)
        unix-intro (Satten, 1989. Unlike a traditional introduction to Unix, the emphasis of this one is on philosophy and brevity. When
                           you understand how the creators of Unix intended you to use it, you’ll approach Unix on it’s "best side". This
                           introduction intends to help a new Unix user get started on the right foot quickly.)
        The Linux Development Project Excellent collection of documentation, tutorials, etc. See especially the The Advanced Bash Scripting Guide
        The Grymoire - home for UNIX wizards
        UNIX SHELL Quote Tutorial (Grymoire)
      
       The UNIX Haters Handbook
       The truth about UNIX: The user interface is horrid
       UNIX is a four letter word
       The rise and fall of UNIX
We will also reference:

Classic Shell Scripting, A. Robbins and N.H.F. Beebe.

The "AWK book"
Awk Article 1
Awk Article 2
Awk Article 3
         IBM LINUX tutorials

Geophysical programs that run under UNIX:

        Generic Mapping Tools (GMT)

        Seismic Analysis Code (SAC) - actually a link to all IRIS Software Manuals

 
     Fortran
       Fortran tutorial - Stanford
       history
       Recursion in Fortran - Andrew Miller, Penn State

    History

    Number representation
      Integer representations
      Common Floating Point Representations  
      Floating point Hidden Bit   
      What Every Computer Scientist Should Know About Floating-Point Arithmetic  

    Finding old web pages. An archive of some old links that have gone away. Beta version.

       Wayback page

    Missing links.

Just about everything on the Web is "under construction" as links are ephemeral. I usually link to the source pages, rather than offer a local copy. This means they can go away. Please report broken links. Some are recoverable, but many just disappear into the ether.

If you find this web site useful and have any questions, have found any errors, etc., please feel free to contact me - rsmalley at memphis dot edu