### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# Advanced Functions of Temporal GIS ENVR 468

UNC

GPA 3.81

### View Full Document

## 25

## 0

## Popular in Course

## Popular in Environment

This 152 page Class Notes was uploaded by Thelma Dickens on Sunday October 25, 2015. The Class Notes belongs to ENVR 468 at University of North Carolina - Chapel Hill taught by Marc Serre in Fall. Since its upload, it has received 25 views. For similar materials see /class/228866/envr-468-university-of-north-carolina-chapel-hill in Environment at University of North Carolina - Chapel Hill.

## Similar to ENVR 468 at UNC

## Popular in Environment

## Reviews for Advanced Functions of Temporal GIS

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/25/15

Calculus useful for Statistics 1 The set of real numbers xe R is a real number Examples X2 X130333333 2 Function x is a function which for any xeR associates a unique value x Example k Vx 6 01 f x 0 aw otherw1se 3 Derivatives dfx dx xx0 whichfxis changing with respect to x at the point x0 and is de ned as f xg is the derivative of x at x0 The derivative represents the rate at fyx0 fx0 53fx0 x Note that by rearranging another de nition is fx0 dx fx0f x0dx Following are some properties of the derivatives 2 dfx dgoc d amxngm dx dx Examples d 2 i 2x dx Exercise Calculate 2x3 a 4 Integrals V0 The primitive F x0 I f x dx of the function fx is de ned as the area under the x curve from 00 to x0 Note that by simple geometric consideration we have F x0 dx F x0 f x0 dx By Fx0 dx Fx0 dx I rearranging we have f x0 This means by the de nition of w Hence the dx xx0 derivative that xg is the derivative of F x at xxg ie f x0 primitive of x may be de ned within a constant as the function F x I f x dx such 61F x dx that it s derivative is equal to x Following are some properties that follow from the de nition of primitives f x m Frx I fxdx dx within a constant a xquot b x xn1 xm1 X2 We now de ne the integral I f x dx as the area under the fxcurve from x1 to x2 X1 which can naturally be calculated from the primitive F x as x2 jfxdx Fx2 Fxl X1 which we nd convenient to write as x2 I fxdx Fm Fx2 Fx1 X1 Some properties of integrals that are easily obtained are x2 x2 Iafxdx a Ifxdx x1 x1 x2 Iltfxgx dx fxdx ilgxdx Jfxdx Ifxdx Jfxdx Using the above properties we can now calculate integrals Examples 2 0dx012 0700 1 20 adx ax12820a710a10a 10 3 x33 33 23 19 xzdx 7 7777 3 3 3 3 5 2 3 5 2 5 2 4x43xdx 4i3i 437w 4i3i 2 5 2 2 5 2 5 2 k Vx 6 01 4 0 1 4 th d0dkd0dxk 0 owotherwise en LizW x 2 x i x i Let f x Exercises 1 1 Calculate x2 ldx 0 1 no 2 Calculate f x dx and I f x dxwhere x is given by 0 no 05x Vx 6 02 0 aw otherwise f x 5 Bivariate Integrals fx y is a bivariate function which for any xeR and yeR associates a unique value fxy Example k 2xy v x 6 01 and y 6 01 0 aw fxy y2 x2 The bivariate integral Idy Idx f x y is calculated by rst treating y as a constant and yl V X2 evaluating the integral I f x y dx Fyx1 x 2 with respect to x and then calculating y2 the integral Idy F yx1 x2 with respect to y yl Examples y2 x2 y2 x2 2 y2 x x12 Idy Idxocy Idy yY jdy y 7 y1 x1 y1 xx1 n 4 1 2 1 x2 quot2 1 F1 jdy jdxxy ldy 74 ldy 22y l2yy2lyo 3 0 0 0 0 0 Exercises 1 1 1 Calculate Idy Idx 2x 3y2 0 0 2 2 2 Calculate Idy Idx xy 2y 1 0 1 1 no no 3 Calculate Idy Idx f x y and Idy Idx f x y wherefxy is given by 0 5 0 foo foo k 2xy v x 6 01 and y 6 01 0 aw fxy Creating amp Exploring a Data Library for SpaceTime Geostatistics This lecture will introduce you to some basic techniques for querying environmental data from the internet and importing it into ArcGIS for exploratory analysis You will learn how to download and create a data library suitable for use within ArcGIS BMEGUI amp MATLAB You will also learn some basic and advanced techniques for manipulating the data once in ArcGIS We will use Surface Water Phosphorus in New Jersey as a case study to develop these skills Homework 1 will test these skills on a different parameter of interest for groundwater in New Jersey TOPICS I Obtaining Data a Downloading from the internet b Importing to Excel II Creating Data Library for ArcGISBMEGUI a Creating Database File b Creating BMEGUI Data File c Exporting to ArcGIS III Exploring Data in ArcGIS a Downloading Shapeflles b Joining Data Tables c Adding XY Data d Setting up Data Queries Obtaining a Dataset Obtaining Surface Water Quality Data for the state of New Jersey Selecting surface water quality parameters of interest The New Jersey Department of Environmental Protection NJDEP monitors the water quality across the entire state using a network of 115 surface water monitoring stations This work is done by the Bureau of Fresh Water and Biological Monitoring which is part of the Office of Water Monitoring amp Standards httpwwwstatenjusdepwmm of the NJ DEP The work is part of the larger water quality monitoring program of the USGS and EPA and part of a national effort to calculate TMDL s total maximum daily load for compliance with the federal Clean Water Act To select a water quality of importance to society one should investigate a water quality parameter that has 1 a large spatial coverage and 2 for which high values are often observed A value is considered high if it poses a risk to human health or to the environment This may happen if concentration values are found to often exceed a health standard or if it they are often above detection limit etc Therefore it is important to examine the pertinent literature about water quality concerns in the location you are investigating in this case New Jersey The literature should provide insight into both government and public perceptions about which water quality parameters are most important to study As an example we will focus on Total Phosphorus in the state of New Jersey for the period of 19992003 This water quality parameter was chosen because the corresponding monitoring data has a large spatial coverage over the state of NJ as compared to other water quality parameters waterusgsgov and because high monitored values were collected as indicated in chapter 2 of the 2004 Integrated Water Quality Monitoring amp A Report 1 39 quot 39 J by the NJDEP Office of Water Monitoring amp Standards httpwwwstatenjusdepwmm This report is available at the following link httpwww state hi I n waL 39 Jquot 2004renort html Chapter 2 on the Chemical Water Quality Assessment has figures on Phosphorus particularly Fig 211 showing that Fecal Coliform and Total Phosphorus most often exceeds their standard and Fig 21a l showing maps showing the location of monitoring stations in nonattainment of the standard for Total Phosphorus The current standard from httpwww state hi I den wmmsgwqt 39 NJDEP is stated for Total Phosphorus in streams html as determined by the phosphorus as total P shall not exceed 01mgL in any stream unless it can be demonstrated that total P is not a limiting nutrient and will not otherwise render the waters unsuitable for the designated uses In the following we will therefore focus obtaining data for Total Phosphorus in the state of New Jersey We will further focus on the Raritan River Basin area as our mapping area of interest Obtaining Phosphorus Concentration in the Raritan River Basin of New Jersey 1 Create a working folder Create a Folder NJRaritanGIS in the following directory dtemp You will store all of your data and GIS shape les in this folder 2 Download NJ Surface Water Monitoring Network and Phosphorus Data from USGS rap96x r onH wasp arr 54m Go to httpwaterdatausgsgovnwisgw In Data Category choose Water Quality for Geographic Area choose New Jersey Click on FieldLab Samples Select Site Type under Site Attribute Select Period of Record amp Parameter Groupings under Data Attribute 7 leave all else blank and click submit Under Site Type choose StreamRiver Under Period of Record choose 01012000 to current date Under Choose Output FormatSummary of Selected Sites select Site Description Information Displayed in Tab Separated saved to file Select Site ID Decimal Longitude and Decimal Latitude using Ctrlclick in the scroll down box Click on Submit Save the file in the NJRaritanGIS folder as NJiUSGSiStationLoctxt Go to httn39 nwi waterdata nsos v data Select Site Type under Site Attribute Period of Record amp Parameter Grouping under Data Attribute Click on Submit Select StreamRiver under Site Type Enter a Period of Record from 01012000 to current date Under Parameter Groupings select Nutrients Skip the next section and under Retrieve Water Quality Samples for Selected Sites and then retrieve data from put in the Period of Record 20000101 to current date Then choose the last option tab separated data one sample per row with remark codes combined choose MMDDYYYY from the drop down list save to le from the next drop down list Click on Submit Save the file in the NJRaritanGIS folder as NJiUSGSiNutrientstxt Optional steps for EPA data you may skip these steps step u to step ee u Go to httpwwwepagovstoretdbtophtml Click on Browse or Download M odernized STORET Data Under STORETRegular Results click on Regular Results by Geographic Location Under Geographic Location Select by State and choose New Jersey and ALL counties Under Date choose 01011999 to present Under ActivityMedium choose Water Under Characteristic type in Phosphorus and Click on Search Choose Phosphorus as P and Click on Select Click on Continue dd On the next page choose site ID Location Information Activity Start Characteristic Name Result Value as Text and Units click on Continue 3 V W x y QWQN QWQ ee Click on 39download le39 and save the le in the NJRaritanGIS folder as 39NJiEPAiStati onL ociNum39ents txt39 Creating a Data Library 3 Set up the Phosphorus and Monitoring Network Database a Open i ii iii iv V vi 0 3 1 quot1 ram PW 39NJiUSGSiStationLoctxt in Excel Data Tab 9 Get External Data Box 9 From Text At the prompt choose delimited click next Choose Tab click next Click on the column that has the site ID numbers in the preview box Change format to Text Click on nextfmish to complete the import Finish the import Copy the entire worksheet into a 2quotd worksheet by clicking on the upper left comer of the table label the rst sheet RawData Working in the 2quotd worksheet Make sure to leave the row with the column headings starting with siteino and delete the extra comment rows in the le rows 124 or up to the column headings and the row just below the column headings starting with 5s Select the 39Latitude39 and 39Longitude39 columns and right click 9 format cells 9 number with 4 decimal places Rearrange the columns such that column A siteID column B Longitude column C Latitude Save the file in the NJRaritanGIS folder as NJiUSGSiStationLocxls Open MS ACCESS On the getting started screen click on blank database i On the right of the screen navigate to our working folder and create a name for the new database ie NJiUSGS A blank table will appear go to the External Data Tab 9 Import box 9 Excel Choose the NJiUSGSiStationLocxls file you just created Select the rst option Import to a new table in the existing database i ii i i iv v v39 Click 0k on the next screen choose the worksheet that contains the final data ie sheet2 On next screen make sure First row contains column headings is checked On the next screen make sure each column is formatted correctly site no text etc On the next screen choose No Primary Key click Next Name the new table and click fmish you can choose to save the import steps or not Double click the new table to open it In Go to External Data Tab 9 Export box 9 the more button 9 dBase 9 save as dBase IV and name it the same name as the Excel table I These steps are for Excel 2003 anal earlier versions starting from Step F Save again except this time save as type 39DBF4 albaseIV 39 you will have to click on 39Save39 anal 39Yes39 every time it appears The DBF is not fully saveal until you close the file completely Repeat steps ac for the NJiUSGSiNutrientstxt39 file We only want the phosphorus parameter which is code 39p00665 so delete all other parameter columns Delete the 39sampletm39 and 39samplecd39 columns as well 5 ltlt lt FF V HP N Insert a new column between sampledt and p0066539 Title this column 39month Insert another column and title it year Insert one nal column and title it days In the month column first cell down enter the formula MONTHMMDDYYYY39 where MMDDYYYY is the cell containing the date of measurement Apply the formula to the entire column Repeat step pq for the year column in place of MONTH in the formula put YEAR In an empty cell enter the starting MMDDYYYY ofthe data ie 01012000 In the days column first cell down type the formula DAYS360startingdateMMDDYYY where MMDDYYYY is the cell containing the initial dates of measurement Select the 3 new columns month year days and change cell format to number with 0 decimal places Copy this worksheet and paste it into a new one i Right click upper left comer of sheet copy ii In new sheet right click in the upper left cell and choose paste special iii Choose Values and number formats and paste into the new worksheet iv Delete the starting date cell used to calculate the days column in the previous worksheet Sort the data by the 39p0066539 column Data tab 9 Sort amp Filter box 9 Sort 9 expand selection 9 sort by Phos column Delete all rows where no data exists should be all at the end of the sorted list or an 39M exists Do a find and replace on the p0066539 column to replace all lt with and replace all 39E with 39 empty space go to Home tab 9 Editing box9 Find and Select button 9 replace Create a new column called 39p00665hardened and apply the following equation P0066ShardenedIFP00665lt0ABSP006652 P00665 Reformat the data columns as numbers with 3 decimal places Copy the worksheet and place in a new worksheet using paste special 9 values and number formats remove the original date column and the original phosphorus data column such that Column A siteID column B month column Cyear column Ddays column E phoshardened Make sure the site ID column is named exactly the same as the site ID column in the station location file Save the file in the NJRaritanGIS folder as 39NJiUSGSiPhosxls39 39 Repeat steps G I to create the DBF file Optional steps for EPA data you may skip these steps step i to step xx Open 39NJiEPAiStationLociNutrientstxt39 by importing into Excel Re format the Station ID column as text Re format longitude and latitude as numbers with 4 decimal places Delete columns 39state39 39county39 39HUC39 39zone39 and 39characteristic name39 Rename the 39value as text39 column as 170066539 to match the USGS code Reformat the p00665 column as a number with 2 decimal places Sort the data in the p00665 column and delete all rows containing 39NonDetect39 and Present Sort the data in the unit column and delete the rows that do not contain units i39 iii iv v v39 vi39 H viii 1X1 xvii XViii Resave as DBF4 39 Add a new column after units name it Phos ugl use Format cells Number to set it s format to number with 2 decimals and apply the following equation Phos ugl IFLEFTunits 2 quotmg quot p00665 1000 p00665 Highlight the 39activity start39 column and go to Format cells custom and enter 39yyyymmdd 39 39 Insert an empty column after 39activity start39 name it year and use Format Cells Number to set it s format to number with 0 decimal and apply the following equation yearYEARactivity start Save this file as 39NJiEPAiStationLociPhosxls39 Also save thisfile as DBF4 Copy the 39site id39 39latitude39 and 39longitude39 columns into a new excel file and save this file as 39NJiEPAiStationLocxls39 Resave this file as DBF4 like before 39 In the 39NJ EPAiStationLociPhochls39 le insert a column at the beginning called 39Org ID39 andfill with 39EPA39 Copy the 39Org ID39 39Site ID39 39activity39 39year39 and p0066539 columns into a new excel file make sure to reformat 39site ID39 as text Save this file as 39NJiEPAiPhosxls39 4 Setting up ArcGIS and T 39 quot a b r39WWQP p Explorinq Datg in ArcGlS NJ m quot39 G0 to httpwww state hi I den gis 39 html Download the Hydrography State Boundary and Watershed Management Area shape les by selecting them from the drop down menu and clicking on download download them into the NJRaritanGIS folder Unzip the downloaded les into the NJRaritanGIS folder Open ArcMap Go to le 9 map properties 9 data source options 9 store relative path names Go to view 9 toolbars 9 spatial analyst to open the spatial analyst toolbar Go to spatial analyst 9 options 9 and set working directory to dtempNJRaritanGIS Click on the Add Data button and navigate to the NJRaritanGIS directory Add the depwmasshp staterivshp les in that order 5 Adding the USGS Database into ArcMan a b Click on Add Data and add the NJiUSGSi StationLocdbf39 and NJiUSGSi Phosdbf39 tables into ArcMap Right click and open each table Which table has the Phosphorus measurement data and how many measurements are there in that table Which table has the longitude and latitude of each monitoring station and how many monitoring stations are there 6 Adding the XY Point Data onto the Map W999 FQ39r39P QO m H Right click the NJiUSGSiPhosdbf39 table 9 Joins amp Relates 9 Join For no 1 choose the Site No eld from NJiUSGSiPhos For no 2 choose the 39NJiUSGSiStationLocdbf39 table For no 3 choose the Site No eld from 39NJiUSGSiStationLocdbf39 Click Advanced 9 keep only matching records 9 ok Right click on the joined table and see that new columns have been added Note which column has the longitude NJiUSGSiStationLocLongitude and which column has the latitude eg NJiUSGSiStationLocLatitude Sort the table by the site column click on Options 9 Export 9 rename StationPhosData and add the table to GIS Go to tools 9 add XY Data 9 select the new exported table StationPhosData Set the X Field to longitude and the Y Field to latitude Under coordinate system click edit 9 select 9 Geographic Coordinate System 9 North America 9 North America Datum l983prj and apply A second new layer has been created with point features representing the location of each of the Phosphorus monitoring events Note that if you set X to latitude and Y to longitude the points will not plot correctly Note that the points and other shape les may not coincide we will x this problem later Save the new layer as a shape le by right clicking on it 9 data 9 export data 9 replace exportioutputshp39 with NJiUSGSiPhosLoc and click OK you can now remove the old point layer from the table of contents 7 Create BMEGUI Data Files a 99 FrPPW W9 The typical format for BME analysis is the SpaceTime Vector GeoEAS format i Line 1 descriptive title of le ii Line 2 of columns in the le iii Line 3 to number of columns column names iv Data columns 1 typical order is side ID longitude latitude measured value time In ArcMap select the attribute table of the joined layer with station location and data values in the same table Open the attribute table Click on Options amp Exp01t change the name in the output box before saving to the working directory ie StationPhosData2 Go to the newly created dbff11e open in Excel and resave as an Excel xls flle Close the le and reopen in Excel Rearrange the columns so they are in the following order site ID long lat data time Save as a text le 7 tab delimited clicking yes when prompted Open the text le delete the rst row Add the additional rows as described in pa1t a Save with the extension dat 8 CheckingChangingMatching Projections for all shapefiles a PPWWQQQV 3 PW F V5939 Check to see which layers have a de ned coordinate system by right clicking each layer 9 propeIties 9 source Depwmasshp should be the only layer with a predef1ned coordinate system Record the coordinate system that has been de ned Remove all layers from ArcMap Open ArcToolbox Go to Data Management Tools 9 Projections and Transformations 9 De ne Projection Set the Input Dataset to the stateshp shapeflle Click on the icon to select a coordinate system Click 39Imp01t and select depwmasshp as the shapeflle that has the coordinate system that you would like to copy Click 39Add39 and Click OK39 and OK39 again to nish You have de ned a coordinate system for stateshp that is the same as that of depwmasshp Repeat the same steps f to j for the staterivshp shapeflle For the XY data you will need to de ne a different coordinate system geographic coordinate system in order for the points to match up with the downloaded shapeflles Go to ArcToolbox 9 Data Management Tools 9 Projections and Transformations 9 De ne Projection Choose the NJiUSGSiPhosLocshp shapeflle Click on the icon to select a coordinate system Click 39Select Navigate to Geographic Coordinate SystemNorth AmericaNorth America Datum l983prj Click 39Add 39OK39 and OK39 again to nish You have de ned a coordinate system for NJiUSGSiPhosLocshp Go back to ArcMap and add back shapeflles as need in order to display all 4 shapeflles of interest depwmas state stateriv NJiUSGSiPhosLoc 9 9 Exploratog Analysis for Phosphorus a Fquot 0 First we need to select only the Raritan Basin in New Jersey and only those stations and river network which fall within the Raritan i Select the Watershed Management Areas corresponding to the Raritan WMA 8 9 and 10 by right clicking the depwmas shapeflle 9 open attribute table 9 highlight WMA 6 8 and 10 ii Close the table you should see the WMA39s highlighted on the map iii Right click the depwmas layer 9 selection 9 create layer from selected features and rename the resulting layer as 39RaritanWMA iv Go to Selection 9 Select By Location and select features from stateriVshp that intersect features from RaritanWMA V Save the selection as a new layer and rename as RaritanRiver vi Repeat the above selection by location for the USGS monitoring events except instead of 39interesect choose 39are completely within the depwmas features vii Save the new layer files as 39RaritanUSGSPhos viii You can now remove all the layers not corresponding to the Raritan and zoom in on the Raritan Basin only Now we want to explore the data for a select year or range of years and visualize this on the map i Go to Selection 9 Select By Attribute 9 choose the RaritanUSGSPhos layer and create a query selecting phosphorus data NJUSGSi4 for 2001 quotNJiUSGSi4quot 2001 ii Those stations monitoring in 2001 are highlighted so create a layer based on this selection named 39RaritanUSGSPhos7200139 iii Right click this new layer 9 properties 9 symbology 9 quantities 9 and select the phosphorus field to visualize either using graduated color or graduated symbols Geoprocessing and Buffers i To combine features in a layer highlight the RaritanRiver layer and go to ArcToolsBox 9 DataManagement Tools 9 Generalization 9 Dissolve Select the layer RaritanRiverlyr set the Output Feature Class to DtempNJRaritanGISRaritanRiverDshp Set DissolveiField to NUMBER Select which fields you would like to keep in the new attribute table and the function you would like to do when combining e g set Field to LENGTH and Statistics Type to SUM and click on OK iv Now to add a 05 km buffer around the combined river network go to ArcToolsBox 9 Analysis Tools 9 Buffer Select the layer to add a buffer around RaritanRiverlyr set the Output Feature Class to DtempNJRaritanGISRaritanBuffer705kmshp Set the buffer Distance to 05 Kilometers set Side Type to FULL End Type to ROUND and Dissolve Type to ALL and click on OK ii39 lt 10 Saving the Project a There are now several new layers which should rst be saved as shape les RaritanWMA RaritanRiver RaritanUSGSPhos RaritanUSGSPhos72001 RaritanEPAPhos72001 RaritanBuffer705km right click each layer 9 data 9 export data and save them with the same namesshp b Go to le 9 save as and you can now save the project for later use Measure Theory useful for Statistics We shall consider sets of elements or points The nature of the points needs not be de ned they may represent elementary events real values etc A set is an aggregate of such points The following operations on two sets A and B are useful AU B union of A and B is the set of all points in either A or B Am B intersection of A and B is the set of all points that belong to both A and B A B difference of A minus B is the set of all points in A that are not in B DEFINITION A1 If 2 is a given set then a o algebra F on 2 is afamily F of subsets of 2 with the following properties i Q e F where Q is the empty set ii FeF 3 FE eF where FE 2 F is the complement ofF in 2 m eF 2 AE1H eF The pair 2 F is called a measurable space A probability measure P on a measurable space 2 F is afunction P F gt 01 such that a He 1 b 0 S PAS1forallsets A 6F c if 4 n A Q zit then n n14 1P4 The triple 2 F P is called aprobability space In a probability context the sets are called events and we use the interpretation PF quotthe probability that the event F occursquot An important aalgebra used to de ne random variables is the Borel 039algebra on SR1 where SR1 is the real line The Borel aalgebra on SR1 noted B1 is the smallest 039algebra containing the collection of all the open intervals of SR1 of the form a b2alt Ilt b oolt as blt oonamely B1rwH2 H aalgebraof SR1 CH It can be shown that the Borel aalgebra B1 does indeed eXist The elements B e B1 are called Borel sets and they include all real intervals open closed semiclosed nite or in nite If 2 F P is a given probability space then a function yQ gt SR1 is called F measurable if f1Ua ea yaeU e F On the strength of the notions introduced we de ne the random variable as follow DEFINITION A2 Let QF P be a given probability space and SR1B1 be a measurable space where B1 is a a algebra of Borel sets on the real line SR1 A real valued random variable Xa where a 69 are elementary events is a F measurable mappingfrom 2 F to SR1B1 ie XQ gt SR1 and VBEB1 261bjwe2 web e F Every random variable induces aprobability measure Lix on SR1 B1 de ned by XB PX lb PX M e81 From the probability measure ux we de ne the cumulative distribution function of the random variable X FXUt 411 if r SID PXS it Modeling spatial variability 1 The covariance of a Spatial Random Fields SRF 11 Definition of SRF and associated bivariate pdf The Spatial Random Field SRF Xs is a random variable that is a function of location ss1s2 Xs x is a random variable X s x is another random variable X s and X s are correlated they are characterized by their bivariate pdf fXsXs Z1Z cx Z1263 Note For SRF we like to also write purely a notation issue fXsXs Z1Z ZfXWJZ is 15quot 12 The mean trend of a SRF The mean trend ofXs is ms mm I dz I dz39 zfxxrz 239 mm EXs I dz I dz39 239 fxx z 139 Note For SRF we like to write mas EXs I dz I dz39 z A z z39 s s39 Also we can obtain the marginal pdf from the bivariate and use that to eXpress the mean trend fXzs ffwdz39 fXzz39ss39 ms mm I dz 2 fX z s 13 The covariance of a SRF For random variables x and x recall that de nition of the covariance is covxx ExmxXx mc I dz I dz39Cz mx z39 va fxxv z 2539 Properties covxx Ex x mx mx proof c0Vxx Ex x Ex m f mex mx m covxx varx 6x2 proof c0Vxx E x mx2 varx Hence for a SRF X the covariance of X between s and s is CXSS 00VXSXS EXS mxsXs mxs I dz If dz 39 z mXltsltz39 mXlts39 gt ong z s s39gt Notation convention c for covariance subscript X for the SRFX and ss to express that the covariance is taken between points s and s The covariance denotes the spatial correlation Properties cxss E X SX S mxsmxs cXss varXsc5Xs2 1e the covariance of X s between points s and s is the variance of X s at s Example 1 Let X s be a SRF lets and s be any two points and assume that the bivariate pdf of X at s and s is llsS39H fXZ Zs s kzzeXp T 1f ze01andze01 O Ow Write the equations for mXs mXs and cXss Example 1 Let X s be a SRF lets and s be any two points and assume that the bivariate pdf of X at s and s is llsS39H fXZ Zs s kzzeXp T 1f ze01andze01 O Ow Write the equations for mXs mXs and cXss lls s39ll a ms fjdz fgdz39zkzz39 exp m s I dz f dz 39 2539 kzz39 exp m cXss fdfd139z mX sz39mX skZZeXp HS s 14 Homogeneous SRF A homogeneous SRF has the following properties It s mean trend is constant mXs m It s covariance is only a function of the distance cXss cX rs s Example 2 Are the following SRFs homogeneous l A SRF with mean trend mXsa s s0 2 A SRF with covariance cXss eXp lls s 3 A SRF with mean trend mXs4 and covariance cXss eXp lls s ll 4 The SRF of Example 1 O 14 Homogeneous SRF A homogeneous SRF has the following properties It s mean trend is constant mXs m It s covariance is only a function of the distance cXss cX rs s Example 2 Are the following SRFs homogeneous 5 A SRF with mean trend mXsa s s0 NO 6 A SRF with covariance cXss eXp lls soll NO 7 A SRF with mean trend mXs4 and covariance cXss eXp lls s ll YES 8 The SRF of Example 1 NO Properties of homogeneous SRF It s variance of is constant 526on proof OX2S EXs mXs2 cXss cX rs s cX 0 is not a function ofs CX0 ZGXZ When r gt 00 then cX r gt 0 0X0 EXSXS l HH H r mX2 A useful equation to estimate the covariance from site specific data 15 Models of covariance for homogeneous SRFs The behavior of the covariance at the origin is important Gaussian model ct7 sill variance cXr co eXp 3r2ar2 a spatial range the distance at which the cov drops by 95 of the sill value Behavior at the origin the correlation remains at a smooth processes Exponential model ct7 sill variance a spatial range cXr co eXp 3rar Behavior at the origin the correlation is linear 4 1 1 10 Nugget effect model re variabihty c if r0 woman 0 ow ct7 sill variance Behavior at the origin the correlation drops 4 purely random 0 0560 0 050 Nested covariance models 0X r 010 02r where 01r 02r etc are covariance models Example 0X r 001 exp 3rar1 002 57 co variance contribution from the exponential component a spatial range 002 variance contribution from the nugget effect 16 Estimation of covariance for homogeneous SRFs Recall that a useful equation to calculate the covariance of a homogeneous SRF is 0X0 EXSXS l HH H r mX2 When having sitespeci c data and assuming the SRF is homogeneous use the following estimator A 1 Nr 2 C 7quotz 7 XeaiXaii m X Nr 24 h d t z X where Nr is the number of pairs Xhead Xta 1 separated by a distance of r dr that is such that r dr 5 shead sta 5 rdr 17 Modelin the covariance Modeling the covariance of the SRF is the exercise of tting a covariance model or nested models through the set of experimental covariance values 2 c r X 15 1 I 05 39 D 7 77 7 7 2 4 8 1D 12 2 39 15 1 05 f t D 1 2 4 r e 12 2 5x 15 1 05 r o i quot 39 2 4 r s 10 12 17 Modelin the covariance Modeling the covariance of the SRF is the exercise of tting a covariance model or nested models through the set of experimental covariance values 1 CX 39 Exponential model cXr co exp 3rar 391 c0 21 39 39 a 10 05 39 D 2 4 r 6 777 71707 2 1 3X CX 7 Gaussian model cXr co exp 3r2arz 39 39 c0 7 21 1 a V a 10 05 77quot D 2 4 r e g i 1 712 2 cxr 39 Nested model cXr C01 exp 3rarj 002 6r 15 7 C01 105 1 7 Cir 05 rxxr r r 002 105 o 2 4 r 6 1 710 3912 18 Modeling the covariance of SRF in BMEIib In BMEIib the function used to estimate the covariance of a homogeneous SRF is covario m SYNTAX clCo covario c Zclmethodoptions INPUT clrl r2 r3 r4 defines classes oflags r1 r2 is the first class oflags ie those such that r1 5 llshead sta ll 5 r2 r2 r3 is the second class oflags etc ch zh are coordinates and values of the site speci c data OUTPUT 01 average length of pairs in each class C values of the experimental covariance for each class 0 number of pairs in each class Hints when estimating the covariance from sitespecific data 1 Choose your classes well Wide enough so you have enough pairs small enough to have a good resolution 2 The maximum value distance of a spatial lag should be less than half the domain size 3 You want to capture the behavior at the origin In BMElib the functions used to for the covariance models of homogeneous SRFs are Gaussian model cXr co eXp 3r2ar2 dCogaussianCDcO ar T Covariance range I Covariance sill Distance values eg D r1 r2 r3 Covariance values eg K K1 K2 K3 where Kl c0 eXp 3r12ar2 To plot a covariance model use the function mode lplot m as follow covmodel gaussianC covparaml 10 silll range10 r00l20 values of r modelplot r covmodel covparam Exponential model cXr co eXp 3rar covmodel exponentialC 39 I covparamcO ar 0000 arar Nugget effect model cXr co 6r covmodel nuggetC covparam CO Nested exponential nugget cX r c0 eXp 3rar1 002 6r covmodel exponentialC nuggetC covparam cOl arl 002 See mode 1 syntax m for more info on building covariance models in BMElz39b Appendix The covEstimationS m Tutorial Exploratory analysis and covariance estimation for Spatial Random Fields SRF Xs A Spatial Random Field SRF X s takes a value that changes as a function of the spatial location sx y Hard data are exact measurements of the SRF collected at a set of spatial location with coordinates chx1y1 xnyn The markerplot m command may be used to provide a map showing the hard data with circles that have a size proportional to the value of the hard data The covariom command may be used to estimate experimental values of the covariance as a function of spatial lag One then need to select a covariance model that ts the experimental covariance values The exponential Gaussian and Nugget covariance are example of some popular covariance models Generally a sill variance value Cg and a spatial range a specify these covariance models The covEstimationS m command is a tutorial showing how to use these commands for a few selected case of SRFs The case corresponding to four different SRFs are presented in gures on the following pages The program covEstimationS m itself is printed after those figures Spatial Random Field SRF X s with exponential covariance Markerplot of data 16 0 3 l 8 O o o O y Km 3 O o l l l l l l l 10 12 14 16 18 20 XKm Covariance cm I 2 VariograrnCova39iance Distance The model selected is CrCg eXp 3 rar where the sill variance C 0 10 and the spatial range a 45 Km Spatial Random Field SRF X s with Gaussian covariance Covariance Cir Markerplot of data 0 O 0 l 10 12 XKm Covarlance Cr spallal lag r Km The model selected is CrCg eXp EV2 ar2 where the sill variance C017 and the spatial range a 65 Km iii Spatial Random Field SRF X s with a nested covariance structure with nugget and Gaussian components Markerplot of data n l l l l l l l l 10 12 14 16 18 20 XKm Covariance cm VariogamlC anance O O 39u a Distance The model selected is Cr Cmgge 509 Cg eXp r2 ar2 where the nugget sill Cmgge043 a Gaussian sill C0022 the spatial range a 10 Km Spatial Random Field SRF X s with exponential anisotropic covariance Markerplot of data 207 o o O 39 o 0 o 0 18 0 0 o o O 9 O 154 o O O o o o 14 o 039 o 9 o o 0 g 127 0 9 00 A 0 E 0 39 g1or 0 gt a 0 o 39 87 o 0 o O 39 0 39 o o 6 o o 0 0 0 390 47 f 0 0 0 0 24 O 0 0 o 0 2 O o o 39 o n k C 1 1 1 1 1 1 1 1 0 2 4 6 10 12 14 16 18 20 XKm 1 r Covariance cm 3 l w all alrecuons asdegree gamma 5 3 5 1 04 02 o 1 0 15 Distance The model selected is anisotropic exponential The major direction is at 45 degree counterclockwise for the horizontal direction The covariance in that direction is CVFCU eXP39ramty39or where the sill variance C 0 155 and the spatial range amajor 7 Km In the minor direction the sill is the same and the range is ammo 3 Km which corresponds to an anisotropy ratio of amajor ammo 23 The covEs timationS m Program ESTIMATION OF AN EXPONENTIAL COVARIANCE Read the hard data from le and display it using circles having a size which is proportional to value of each hard data valvalname letitlereadGeoEAS coveXpdat chval 12 zhval3 gurehold on markerplotchzh 4 30 Xlabel39X Km ylabele Km title Markerplot of data Estimate the covariance by using pairs in all directions and plot the result on a new graphic window gure hold on cl015 4 7 20 dCocovariochzhcl39kron variancevarzh hl lot0 d39variance C r plot0 150 0 k title Covariance Cr A reasonable model is an exponential covariance function with a sill variance ccl00 and a range aa45 Km r0 2 15 covmodel eXponentialC ccl 00 aa4 5 covparamcc aa modelplotrcovmodelcovparam Type any key to continuing pause clc vi ESTIMATION OF A GAUSSIAN COVARIANCE WITH NUGGET EFFECT Read the hard data from file and display it using circles having a size which is proportional to value of each hard data valvalnamef11etitlereadGeoEAS covnuggaudat chval 12 zhval3 figurehold on markerplotchzh 4 30 Xlabel39X Km ylabele Km title Markerplot of data Estimate the covariance by using pairs in all directions and plot the result on a new graphic window gure hold on cl010 22 4 6 ll dCocovariochzhcl39kron variancevarzh hl lot0 d39variance C r plot0 150 0 k title Covariance Cr A reasonable model is an gaussian covariance function with a sill variance cc022 and a range aa10 Km and a nugget effect with sill 043 r0 2 15 covmodel39nuggetC 39gaussianC39 covparam 043 022 10 modelplotrcovmodelcovparam ESTIMATION OF AN ANISOTROPIC COVARIANCE Read the hard data from le and display it using circles having a size which is proportional to value of each hard data valvalname letitlereadGeoEAS covanisdat chval l 2 zhval3 gurehold on markerplotchzh 4 30 Xlabel39X Km39 ylabele Km title Markerplot of data Estimate the covariance by using pairs in all directions and plot the result on a new graphic window gure hold on cl0 2 4 615 dCocovariochzhcl39kron variancevarzh hl lot0 d39variance C3939r plot0 150 0 k title Covariance Cr Estimate the covariance by using pairs in the direction with an angle45 degree clockwise for the horizontal direction with a tolerance of 45 degrees and plot the result in a same graphic window cl0 2 4 815 dCocovariochzhcl kron 0 0 90 h2 lot0 d variance C39 r legendh all directions39 45 degree Estimate the covariance by using pairs in the direction with an angle45 degree clockwise for the horizontal direction with a tolerance of 45 degrees and plot the result in a same graphic window viii cl012 415 dCocovariochzhcl kron 0 90 0 h3 lot0 d39Variance C r legendh all directions39 45 degree 45 degrees A reasonable model is an exponential covariance function with a sill variance ccl55 and a range aa7 Km is the major direction at 45 degree and a range aa3 Km in the minor direction at 45 degree which corresponds to an anisotropy ratio of 7323 r0215 covmodel exponentialC ccl55 aa7 covparamcc aa modelplotrcovmodelcovparam aa3 covparamcc aa modelplotrcovmodelcovparam Introduction to TGIS Reading Temporal GIS Chapter 1 Questions What is a Temporal Geographic Information System What is a FieldBased TGIS What are fundamental functions and advanced function of TGIS What are examples of graphical representations of fieldbased TGIS What are BMEbased advanced functions Definition of Temporal Geographic Information Systems GIS Database of information linked to geographical objects TGIS GIS changing over time TGIS has purpose content and context We adopt the following rather broad definition of TGIS The combination of scienti c modelling and information technology in order to process knowledge about phenomena that occur in a geographotemporal domain and to satis particular user needs in the most e icient way FieldBased TGIS There is a variety of TGIS classifications in the literature An important classification TGIS which is particularly relevant to the theme of this course is as follow a TGIS dealing with objects which possess geometrical and topological features and nonspacetime attribute values representing points lines arcs areas etc b TGIS dealing with elds natural social or epidemiological which are functions taking their values in a geographotemporal domain eg attribute data representing the distribution of contaminant concentration soil erosion properties hydrologic parameters eXposure fields ecological patterns and disease rates Although an important topic in its own right TGIS of the category a above will not be considered in this course Instead This course Will focus on fieldlbased TGISy since the scientific disciplines of interest are concerned mainly with fields distributed in space and time Visualization of FieldBased TGIS GIS fields can be visualized as a b V 0 V d e V An array of attribute values assigned at the centers of representative areas An isocontour map A choropleth map A set of elevated points at the centers of the areas A set of prisms erected over the areas A smooth surface of the actual and interpolated densities Fig Page 5 and 6 TGIS Functions The T GIS functions are usually classified into two major groups of socalled fundamental functions and advanced functions 1 The fundamental functions involve loworder geometric operations and may be viewed as tools establishing relationships between spatial and temporal entities Measurement functions eg measuring the distance between two points etc Classi cation functions eg classification as a function of shape etc Scalar functions e g adding a specified constant to each attribute value Overlay functions e g combining attribute values into an output layer Neighborhood functions eg search of elements within specified distance Connectivity functions e g connectivity of two points in a network TGIS Functions con t Q The advanced functions also called compound operations provide mathematical techniques for advanced data processing accounting for spatial and temporal variability and the uncertainties of the attributes of interest Classical statistics 0 Time series analysis Classical geostatistics and spatial statistics 0 composite spacetime 0 account natural laws and theoretical models BMEbased advanced functions Within the above general framework the following three BME based advanced functions will be discussed in this course 0 Knowledge base KB development This function involves the acquisition and storage of the various forms of data and other knowledge sources physical laws ecological principles epidemiological relationships etc relevant to the problem at hand 9 Spatiotemporal representation In this function one chooses the appropriate spacetime geometry establishes the epistemic rules of KB integration and processing selects the appropriate mathematical tools and clarifies the goals of the study 9 Scienti c mapping This function demonstrates the ability of BME to offer valuable information about the phenomena of interest in terms of composite spacetime maps and probability distributions and to aid scientific interpretation and decisionmaking via theoryladen visual representations Software of advanced functions of TGIS Tables page 9 and 12 ARCINFOGRID IDRISI SpaceStat SSpatialStats The Gaussian Distribution 1 Univariate Gaussian distribution A Gaussian random variable x is completely defined by its mean In and variance 02 and its pdf is 1 1 z my 6X 7 fxoc TM p 2 62 The moments of the Gaussian pdf are given by x m2nl 0 I 2n62n x m 2n Dug 1X3XX2n 102quot n quotQ See proof below Additionally the expected value of CXPOC is given by eXpx expo 02 2 73 72 n 1 2 a A 73 72 4 n 1 2 a Gaussian pdf Gaussian cdf Proof of the formulae for the even moments Write ga I d J eXp a m2 2 a where a 1 o2 Taking the derivative we have 4 n M no L380 d11 m exp 120 m2 2 w a which after substituting a 0 2 gives 50 2 J2 7r 2 O39X m 211 1o 239 1 which completes the proof 2 Multivariate Gaussian distribution A vector of 1 Gaussian random variables x x1 xn T is completely defined by its mean vector mm1 mn T and it s 11 by n covariance matriX C The multivariate pdf of x is given by 71 12 i 1 TC71 f x W6Xp quotx m x m quot 2 2 x 0 1 0 Example Plot of the bivariate pdf of x I with mean In 0 and covariance C 0 x2 21 Moments of a Gaussian vector of random variables E1 1 See proof in Appendix Exi mi Eximixjmj cij See proof in Appendix Exi39mixj39mjxk39mk 0 Where 017 is the ithj th element of the covariance matrix C 22 Marginal distribution xa T T Where xa x1 xna and x1 xna1 xnmnb W1th x Let x be partitioned so that x b ma Caa Cab n nanb If x is multivariate normal w1th mean In and covariance matrix C C mb ba bb then the marginal distribution of xa is also multivariate Gauss1an W1th mean ma m 1 mm and covariance matrix Cm Mathematically this property is expressed as follow 12 fxaota IdefxXaXbW I dxbeXp x mTClx m Rub 27 Rub 1 12 71 C a 1 T 71 z eXp 7Xama Caa la ma 277115 2 2 23 Conditional distribution Let x be partitioned so that x a If x is multivariate Gauss1an W1th mean m a and xb mb covariance matrix C a ab then the conditional distribution of xa given that xbzb is also ba bb 39 39 m m C C 1 m 39 multivariate Gauss1an W1th mean ab a ab bb lb b and covariance 71 Caalb Caa Cabeb Cba Mathematically this property is expressed as follow See proof in Appendix C 1 1 T 71 eX i m C m My2 p 2 x x lbfxXaaxb 12 fxb lb Cbb l 1 1 Wexp2lb mbTCbbi lb quot710 fxalxb Ia I xb 12 71 aalb l T 71 Z eXp 7Xama Caa la ma 27am 2 2 lb lb lb Notes 39 0 The variance of a scalar random variable x1 given xbzb is always smaller than the variance of x1 Varx1 xb S Varx1 0 In the case of two scalar random variables x1 and x2 the graph of Ex1 12 with respect to X2 is called the REGRESSION LINE of x1 on X2 Ex1 Z2 m1C1262271Z2 m2 3 Examples Example I 39 Let x be a vector of Gaussian random variables representing the concentration of a pollutant at 4 T locations xx1 x2 x3 x4 The mean vector mEx and covanance matrix CCovx of x equal to 33 1296 1458 846 1581 27 2025 1175 2224 m C 25 841 1592 31 sym 3721 Calculate the expected value and variance of x1 given that x2 x3 and x4 have the following measured concentration values ppm T T lb 262 9633 Z4 2 202221 Solution 39 Ex11lb m1C1ngz1Xb mb 1 2025 1175 2224 20 27 331458 846 1581 841 1592 22 25 sym 3712 21 31 2941 Varlxdlb 1 C11 Clegblclb 2025 1175 2224 71 1458 1296 1458 846 1581 841 1592 846 sym 3712 1581 2246 Note that Varx1 xb ltlt Varxl The knowledge of the measured values for x2 x3 and x4 has reduced the uncertainty of the value taken by x Example 2 39 Let x representing relative humidity and x2 representing temperature C be a Gaussian T vector of random variables x x1 36 With the followmg mean and covariance Etcqu MW 623 l98 507 Obtain the regression line of relative humidity versus temperature Solution 71 ElxilZzlz m1012 722 962 m2 198 205 7 623 507 962 4483 039 Z2 Note that When 12 2 m2 we have Ex1 2 Ex1 Example 3 39 Let x be a vector of Gaussian random variables representing the concentration of a pollutant at 4 T locations xx1 x2 x3 x4 The covariance matrix CCovx of x is equal to l 07 08 05 l 05 08 l 07 sym l C0vx 2 Out of the random variables x2 x3 and x4 which one contributes the most information to the estimation of x Solution Varx1 x2alz612a71612 1 072 051 Varlxilx3lz 7121 71403071013 1 082 2036 Varxilx4 712 71494071014 1 052 075 The smallest conditional variance is obtained when using x3 which is the most correlated with x1 and therefore it brings the most information Example 4 39 Using the same variables as in example 3 nd among x2 x3 and x4 the couple of variables that contribute the most information to the estimation of x Solution 39 2 71 Varixilx2x3lz 712 012 0132 623 612 024 23 73 713 2 71 2 0 2 0 24 all Varx1 x2x4al 012 014 2 05 724 74 K614 2 71 Varixilx3ax4lz 712 713 01 6324 61320 35 The smallest conditional variance is obtained by using the variables x and x3 Appendix Let x X1 X17 be a multivariate normal random vector taking valuesX X1g7 The probability distribution of X is given by I 12 1 T 2 2 2 exp EY AY fXX where m Ex is the vector of expected values where Y X m and where A is a symmetric positive definite matriX that can be written as AHTH The distribution can also be written using the covariance matriX c A391 HTH 1 as 12 IC II fxn PW exp YTC 1Y Note that since A is symmetric definite positive it can be diagonalized by an orthogonal matriX A STAS STS1 S and H are related by AzSAsTSA12A12TSTHTH HzAlZTST H71SA712 Proof of the Normalization property To verify the normalization property we integrate the distribution Ide XM I We iYTAY 39A39m IdYe lYTHTHY Rn xn 2 172 Rn Xp 2 70172 Rn Xp 2 Making the transformation U HY with jacobian YT UT IH391 we find IAll2 1 T IAIl2 20172 RindexnX 2 n2gnwlAl126XPL 2U J 2 172 IAI12 Proof of the formulae for the centered second order moment r1T1 RIndXO 1211ij mpfxnm mm RIndYYIICexpL 2Y AYJ aY5 1 Makmg the transformatlon U HY w1thJacob1an aw T W we nd Rjndxm 1211ij mjfxnX Zion2 RI dUIAIU2 H UH UexpL 2U U J 1 71 71 I 1 T 0U Hv U H v U U U 2 172 RIquot 1k k2 11 zeXPL 2 J 1 71 71 l 1 T Hv H v dUU U U U 2 172 1k 111quot k IeXPL 2 J 1 71 71 2 2 n2 sz W T T H12H i m 11391ltH 1 IgvH 1H1 vA 5101 so that as expected X 111139 39 my Q T he Marginal distribution The marginal distribution of the subset X1X1X 1 of the multivariate normally distributed random vector x x1x2 where x2 X X1 MI and n 111 122 is also multivariate normally distributed with covariance matrix 011 HI The covariance matriX C11 for X1 is expressed as a function of the covariance matriX C for x as follow C C C 1T1 12 C12 C22 This property leads to the following equation 7112 n2 KL 0K2 em YTC 1Y fX1X1 RLdXzngF 2 IC illlZ 1 11 T 71 Wexp3Y1 C11 Y1 Proof Let A C 1 and define the following partition l A11 A12l A172 A22 We may write Y YTCilYYTAYYlTYZTiAlT1 A12 1 A Y 12 22 2 Y1TA11Y12Y1TA12Y2 Y2TA22Y1 Noting that A22 is symmetric pd we can use a Cholesky factorization and write YZTAZZY2 YZTHTHY2 UTU where U HY2 Y2 H IU and H is an upper triangular matriX Using this relationship we have YTAYY1TA11Y1 2 WUTU I39I39I39l vT YfAqu VTV aam2 manm m I39I39I39l W W Y1TA11Y1 VTVWTW T T T T Y1A11Y1 Y1A12 W21Y1W W m A21 T 7 T Y1A11 A12 A221 A21Y1w w Making the transfonnation W U V HY2 V Y2 H 1W V with Jacobian 502 i 1 1n the 4W H IAzzl 2 integral we get IA 2 7 IdXZfXXW de2 exp Y AY an an 27139 mm W 1 e 1YTA A A IA Y WTW 2 172 an IA2212 XP 2 1 11 12 22 21 1 IAIN2 1 L T 71 172 e Y A A A A Y 27139 2 2 172 IA2212 XP 2 1 11 12 22 21 1 1 W 2 T 71 W WeXP Yi A11 A12A22A21Y1 22 Now we note the following property of the inverse of a partitioned matrix Aim Ari oi i0 I ii S71A21Aii 571i where S A22 A21A1 11A12 is the Schur complement Using the properties of matrices and determinants we can see from the above equation that 71 C22 S 1 A22 A21A111A12 or equivalently 71 71 C22 A22 A21A11A12 and 20 C22 IA IA11ISI Similarly considering A11 Aleg A21 as a Schur complement we can write 71 71 C111 811 A12A22A212 and A I in C11 Substituting those relationships in the expression obtained for IaXzfX X we get the required result that Rquot2 12 f x jde x M 1YTC lY X1 0 an 2 X 2 712 6XP 2 1 11 1 21 Introduction to Mapping Exploratory analysis Analysis of space time variability Spatiotemporal estimation Exploratory analysis Import usually from les and Visualize the raw data Get a feel for the mapping situation In BMEG U1 or BMElib or you may use marker plots pole plots color plots etc Example BMElib color plots of annual airborne lead concentration over the US in 1985 Data of annual airborne lead concentration in 1985 0 Monitoring Stations S ate boundaries National lead standard Quarterly Average lt 15 pigm3 120 110 100 90 80 70 51 Analysis of space time variability Model the mean trend Obtain the residual data Calculate estimates of the covariance function using the residual data Model the covariance function using permissible covariance models Use BMEGUI or use the stdtlib and modelslib directories in the BMElib package Modeling the covariance function provides general knowledge about the spacetime variability of the spatiotemporal eld of interest Spatiotem poral estimation Obtain estimates of the field at the node of a mapping grid and construct the corresponding map Provide an assessment of mapping uncertainty Use BMEGUI or use the bmeproballb bmez ntllb and bmehrllb directories in the BMEll39b package Example color map of spatioternporal estimated of annual airborne lead concentration in 1985 Spatiotemporal estimate of annual airborne lead concentration in 1985 0 Monitoring Stations S ate boundaries 50 037 45 022 A 40 014 U n E g 0082 g 35 S 005 30 a 003 25 0018 National lead standard 0011 20 120 110 80 70 100 90 Longitude deg SpaceTime mapping with Soft data Soft Data Xp Xs z is a STRF Soft data forXav is available at the soft data points p50 p1 pm The vector of random variables xso x1xns represents the 81quot RF at the soft data point ie xsoftXpla a Soft data of probabilistic type are eXpressed using the soft pdffsogso as follow S Isoft 1 Pxsoftltlsoft IZZOR du f5 1 FSZsoft Usually the soft pdf is independent between soft data points so that fsqso f51 f5ns EXAMPLE At point p1 the soft pdf is Gaussian with mean m12 and variance 0123 and at point p2 the soft pdf is uniform from a24 to 326 What is the soft pdf for ZSOft 11 92 The answer is as follow lZ m12 1 X 7 X 2 m2 fSOKl 61 6 p 2 all me p 6 0512 if4SZZS6 0 otherwise fSZ2 fSUtso fSZ1fSZ2 Coding soft data in BMEIib In BMEIib the soft pdf015011 1 ns is coded in a discretized form using 4 variables softpclftype n1 limi and probdens The reader can type help probasyntax for a detailed explanation of how these variables work The rst variable softpclftype is an integer taking values 1 2 3 and 4 It speci es the type of soft pdf as follows 1 for histogram 2 for linear 3 for histogram on a regular grid and 4 for linear on a regular grid Along each of the MS dimension the univariate pdffsogi is de ned using intervals of values for If The interval limits are speci ed using the matrix limi and the value of f5 If in these intervals is speci ed by the matrix probdens 0 n nle vector of the number of interval limits nlz39 is the number of interval limits used to de ne the so pdffs at point Pi limi nle matrix of interval limits where l is equal to either maxnl or 3 depending on the softpclftype If softpclftype l or 2 then limi is a ns by maxnl matrix and limiil nlz39 are the interval limits for soft data 139 If softpclftype 3 or 4 then limi is a ns by 3 matrix The interval limits are on a regular grid and limiil3 are the lower limit increment and upper limit of the interval limits for soft data 139 probdens nsXp matrix of pdf values where p is equal to either maxnll or maxnl depending on the softpclftype If softpclftype l or 3 then probdens is a ns by maxnll matrix The pdf value is constant in each interval and probdens i nlil are the value of the pdf in each interval If softpclftype 2 or 4 then probdens is a ns by maxnl matrix The pdf value varies linearly between interval limits and probdens i nli are the value of the pdf at each interval limit EXAMPLE with softpdftype 1 soft pdf of histogram type gtgt so pd ype1 gtgt nl4 gtgt limi0 2 3 4 gtgt probdens1 3 27 gtgt h r0baplotso pd ypenllimipr0bdens o m 91 probdsns EXAMPLE with softpdftype 2 soft pdf of linear type gtgt so pd ype2 gtgt nl4 gtgt limi0 2 3 4 gtgt probdens0 4 1 07 gtgt h r0baplotso pd ypenllimipr0bdens probdsns EXAMPLE with softpdftype 3 soft pdf of histogram type on regular grid gtgt so pd ype3 gtgt nl5 gtgt limi0 1 4 gtgt probdens1 2 3 28 gtgt h r0baplotso pd ypenllimipr0bdens prubuens EXAMPLE with softpdftype 4 soft pdf of linear type on regular grid gtgt so pd ype4 gtgt nl5 gtgt limi0 1 4 gtgt probdens0 3 1 2 06 gtgt h r0baplotso pd ypenllimipr0bdens probdens 0 D 05 1 15 2 Ilml Writing and reading soft data fromto files The writeProbam and readProbam functions allow the user to read and write soft probabilistic data fromto a file Syntax gt gtwritePr obac sisSTsoftp dftypenllimiprobdensfiletitle datafile gt gt csisSTsoftp dftyp enllimiprob densfiletitle readPr obadatafile EXAMPLE the following file named somesoftdatatxt contains the soft data at two points BME Probabilistic data 7 s1 s2 code for the variable equal to 1 Type of soft pdf equal to 1 corresponding to histogram number of limit values nl limits of intervals nl values probability density nl 1 values 1 09 1 1 4 01 03 07 11 10 15 05 01 02 1 1 2 01 03 5 0 Plotting soft data The probaplotm function allowing to plot soft data has the following syntax gtgt hprobapl0tsoftpdftypenllimiprobdensSidx EXAMPLE gtgt csisSTsoftpdftypenllimiprobdensfiletitlereadProba39somesoftdatatxt39 gtgt subplot211 hprobaplotsoftpdftypenllimiprobdens gtgt subplot212 hprobaplotsoftpdftypenllimiprobdens 39392 1 05 u 4 12 1 4 4 a 2 1 o 1 u 15 02 0 25 o 3 035 Generating soft data The probaGaussianm and probaUniformm generate soft data of with Gaussian and uniform distributions respectively For example the following code generate a two soft data points the first is Gaussian with mean 2 and variance 3 the second data point is Gaussian with mean 1 and variance 4 gtgt softpdftypenllimiprobdensjarobaGaussiam21034 gtgt subplot211 hprobaplotsoftpdftypenllimiprobdens 1 gtgt subplot212 hprobaplotsoftpdftypenllimiprobdens 39392 02 015 01 005 7 n 75 o 5 10 0157 01 0057 o 4 95 1h 1 2 1 4 15 15 The SRF Xs is a function of space only in a 2D spatial domain This SRF has a mean trend equal to zero and a covariance Crc0eXp3rar with c01 ar5 Additionally we have hard data at two hard data points At s14 Xs12 and at s52 Xs17 And we have soft data 1 9 and 23 We want to estimate the posterior pdf and it s moments at 11 specify the general knowledge orderNaN39 The mean trend is equal to zero covmodel eXponentialCquot covariance is exponential Crc0eXp3rar covparam1 539 parameters for the covariance model c01 ar5 specify the specificatory knowledge ch1 45 239 Hard data has two data points at 14 and 52 zh12391739 Value of hard data at 04 is 12 and at 52 it is 17 cs1 9392 3 Soft data has two data points at 1 9 and 23 softpdftype239 Soft pdf type2 correspoinding to linear nl439339 Number of limits for each soft data points limi0 2 3 61 2 4 NaN Limits for each soft data points probdens0 2 10 023390 2 0 NaN339 soft pdf value for each limit value specify calculation parameters nhmax10 maX number of hard data in estimation neighborhood nsmax10 maX number of soft data in estimation neighborhood dmax100 dmaxmaX spatial search radius for estimation neighborhood optionsBllEoptions Use default options specify the coordinate of estimation point ckl l The estimation point is 11 calculate BME posterior pdf using BNlEprobanf zpdfinfoBMEprobanfckchcszhsoftpdftypenllimiprobdenscovmodelcovparamnhmaXnsmaXdmaXor deroptions calculate moments of BlVlE posterior pdf using BMEprobaMoments momentsinfoBMEprobaMomentsckchcszhsoftpdftypenllimiprobdenscovmodelcovparamnhmaXnsmaX dmaXorderoptions eXpecvalkmoments l varkmoments2 Lucauun oflhe mapping poin39s v Hard dala paints 0 on dale pmms r Estimation puims 3 y cuurdinate X7 1 05 03 02 0 1 So pm at 1 9 1 2 3 4 5 6 500 pm a 2 3 BME pageer pm a Gunmaan xy11 SpaceTime mapping Spacetime mapping with BMEIib using the bmeprobalib functions The bmeprobalib directory contains functions to perform space time estimation using both hard and soft data Two of there functions are BMEprobanf m and BMEprobaMoments m The input of these functions are General Knowledge 9 Mean trend of Xp mxpEXp Covariance 0fXp Cxpp E XpMxp Xp MxP l Specificatory knowledge S Hard data Zhard er Zrnhla Pxhard Zhard 1 Soft probabilistic data Zso I Pxso ltzso FSZso Some calculation parameters The coordinate of the estimation point The output calculated are BMEprobanf m the posterior pdffKng BMEprobaMoments m the moments expected value variance of the posterior pdf General knowledge ms cxrT Specificatory knowledge Hard data Soft proba data BMEprobanf m Posterior pdf at the estimation point f1ltxk I BMEprobaMoments m Moments of the posterior pdf Expected value variance Spatial example The SRF Xs is a function of space only in a 2D spatial domain This SRF has a mean trend equal to zero and a covariance Crc0eXp3rar with c0l ar5 Additionally we have hard data at two hard data points At s04 Xsl 2 and at s52 Xsl 7 We want to estimate the posterior pdf and it s moments at one estimation point of coordinate 11 specify the general knowledge orderNaN39 The mean trend is equal to zero covmodel eXponentialCquot covariance is exponential Crc0eXp3rar covparaml 5 parameters for the covariance model c0l ar5 specify the specificatory knowledge ch0 45 239 Hard data has two data points at 04 and 52 zhl 239l739 Value of hard data at 04 is 12 and at 52 it is 17 cs39 There is no soft data softpdftypel39 no soft data nl39 no soft data limi39 no soft data probdens39 no soft data specify calculation parameters nhmax10 maX number of hard data in estimation neighborhood nsmax0 maX number of soft data in estimation neighborhood dmax100 dmaxmaX spatial search radius for estimation neighborhood optionsBllEoptions Use default options specify the coordinate of estimation point ckl l The estimation point is 11 calculate BME posterior pdf using BNlEprobanf zpdfinfoBMEprobanfckchcszhsoftpdftypenllimiprobdenscovmodelcovparamnhmaXnsmaXdmaXor deroptions figurehold on p10tzpdf title BllE posterior pdf of variable zk at coordinate Xyl l Xlabel zk ylabel fKzk calculate moments of BlVlE posterior pdf using BMEprobaMoments momentsinfoBMEprobaMomentsckchcszhsoftpdftypenllimiprobdenscovmodelcovparamnhmaXnsmaX dmaXorderoptions eXpecvalkmoments l varkmoments2 Spacetime example The STRF Xst has mean trend equal to zero and the following covariance Crtc0eXp3rareXp3Uat with c0l ar5 atlO Additionally we have hard data at two hard data points At s04 and t30 Xsl 2 and at s52 and tlO Xsl 7 We want to estimate the moments at two points of coordinate 1115 and 1116 specify the general knowledge orderNaN The mean trend is equal to zero covmodel eXponentialCeXponentialC covariance model covparaml 5 10 covariance parameters specify the specificatory knowledge ch0 4 305 2 10 Hard data coordinate zhl 2l 7 Value of hard data cs There is no soft data softpdftypel no soft data nl no soft data limi no soft data probdens no soft data specify calculation parameters nhmax10 maX number of hard data in estimation neighborhood nsmax0 maX number of soft data in estimation neighborhood dmax100 10 510 dmaXlmaX spatial search radius for estimation neighborhood dmaX2maX temporal search radius for estimation neighborhood dmaX3spacetime metric usually equal to arat optionsBllEoptions Use default options specify the coordinate of estimation point ckl 1 151 1 l6 The estimation points calculate moments of BlVlE posterior pdf using BMEprobaMoments momentsinfoBMEprobaMomentsckchcszhsoftpdftypenllimiprobdenscovmodelcovparamnhmaXnsmaX dmaXorderoptions eXpecValkmoments l yarkmoments2 Spacetime example with nested covariance model The mapping situation is the same as for the STRF Xst above Except that we now use the following covariance Crtc02exp3rarleXp3Uatl c02eXp3r 2ar2 2eXp3Uat2 with c01l arl5 atllO and c02l ar250 at2lOO all the other Input parameters are the same as above except for the covariance model given as follow covmodel eXponentialCeXponentialC gaussianCeXponentialC covparaml 510l 50100 calculate moments of BlVlE posterior pdf using BMEprobaMoments momentsinfoBMEprobaMomentsckchcszhsoftpdftypenllimiprobdenscovmodelcovparamnhmaXnsmaX dmaXorderoptions eXpecvalkmoments l varkmoments2 Spatiotemporal Modelling Reading Temporal GIS Chapter 2 Questions What is a Spatiotemporal Continuum What is a coordinate system What is a spacetime metric O O O o What are examples of composite spacetime metrics Spatiotemporal Continuum A spatiotemporal continuum or domain E is a set of points associated with a continuous spatial arrangement of events combined with their temporal order A useful spatiotemporal continuum is equipped with a coordinate system and a measure of spacetime distance E coordinates p metric dpb The coordinate system A coordinate system identifies a point in the spatiotemporal domain E can be identified by means of the spatial coordinates S Sp35quot E S C Rquot and the temporal coordinate t along the temporal axis T c R pszeESgtltT An interesting classification of the spatial coordinate systems S can be made in terms of the Euclidean and the nonEuclidean group of coordinate systems Example Traditionally geographic coordinates are expressed in terms of latitude and longitude Mendjans Green W39s1 mendjan Mendjan 391 a P N 1 La tirade Equator 4 Longitude gt The metric The second feature of a spatiotemporal continuum E is its metric that is a mathematical expression that defines distances in spacetime There is the separate metric and the composite metric The separate metric includes a spatial distance idsl and a time interval at a p a s a teRLO X T The composite metrical structure assumes that the space and time parameters are connected by means of an analytical eXpression g ie a p a gds1a sna t Examples of spatial metrics e 211 dsi2 lnorm metric ids a 2 211 d5 See and Reproduce Example 211 with BMElib Spatial Euclidean metric a s Max permeability path ldsl pm 2 maX ds PM D Qq g Eo gg P2 Examples 0f composite spacetime metrics Some physical applications dp m Spacetime metric in BMElib dp ds dmaX3 dt A rapid introduction to ENVR 468 The course focuses on the development of advanced functions for fieldbased Temporal Geographical Information Systems TGIS These elds describe natural epidemiological economic and social phenomena distributed across space and time The course Will introduce computer programs for GIS it Will provide the spacetime concepts and mathematical framework and it Will be dealing with realworld TGIS applications The computer programs The computer programs for GIS and TGIS considered in this class are 1 ESRI arcGIS version 92 for basic GIS work 2 The BMEGUI numerical library of spacetime Geostatistical functions written in Python and VIA T LAB ArcGIS The Comprehensive Geographic Information System ArcGIS has a three part interface 0 ArcCaIalog for navigating spatial data 0 ArcMap for creating presentation graphics 0 ArcToolbox powerful geoprocessing tools Concepts and mathematical tools Review of statistics useful for Geostatistics o The random variable x takes different possible values 1 realizations o Pxlt is the probability of the event x takes a values smaller than 1 o F xPxlt z is the cumulative distribution function CDF of x o og de d 1 is the probability density function PDF of x o fxyog liS the bivariate pdf of the random variables x and y o cwa liS the conditional pdf of x given that y 1 SpaceTime Random fields STRF ps I is the coordinate of a point in the spacetime domain The STRFXp is a random variable indexed by the location p Distance in the spacetime domain the spacetime metric Xp 1 and Xp 2 are two random variable how are they correlated Variability in space and time and covariance function 0 mxpEXp is the mean trend ofXp 0x2p E Xp mxp2 is the variance ofXp o cxpp E Xp mxp Xp mxp is the covariance of X between points p and p 0 Experimental values of examp are estimated using available data of Xp o A theoretical model for cxpp is obtained by tting permissible covariance models to the experimental values of cxpp Spatiotemporal estimation and uncertainty assessment 0 The mapping problem Obtain estimates of Xp at the node of a mapping grid 0 Prior stage Gather the general knowledge about Xp moments phys law 0 Integration stage Process the information available at the data points hard data o Interpretive stage Select an appropriate estimated value at each estimation point 0 Uncertainty analysis Use the posterior pdf to completely describe mapping uncertainty The applications The applications will consist of realworld mapping TGIS projects Using skills acquired in basic GIS ie arcGIS each student will 0 Research a spacetime dataset of concern for society 0 Formulate the spacetime mapping problem and 0 Use concepts and mathematical tools together with the BMEGUI software of spacetime Geostatistics in order to provide a realistic representation of the eld over space and time 0 Write a high quality final report that could be considered as a conference proceedings paper The spacetime datasets corresponds to any fields describing natural epidemiological economic and social phenomena distributed across space and time A dataset of concern for society could for example consist of monitoring data for an environmental contaminant found at a level that is higher than the allowed standard The student may use herhis datasets or use one considered in class such as water quality data sampled by the North Carolina Department of Environmental and Natural Resources in its surface water and groundwater The aim of this course is to prepare its participants to use its concepts mathematical formulation and computer software to analyze their own TGIS dataset Textbook Christakos G P Bogaert and ML Serre Temporal GIS Advanced Functions for FieldBased Applications SpringerVerlag New York NY 217 p CD ROM included 2002 Philosophy and grading The students should learn the concepts and not use the tools as a black box They will be graded on solving conceptual problems rather than just applying the programs The students will do homework s and a project which will count for the final grade as follow Homework 50 Studentde ned project 50 Links to material needed for the class of fall 2008 httpwwwuncedumserreteachingfallZOO8envr468 Creating amp Exploring a Data Library for SpaceTime Geostatistics This lecture will introduce you to some basic techniques for querying environmental data from the internet and importing it into ArcGIS for exploratory analysis You will learn how to download and create a data library suitable for use within ArcGIS BMEGUI amp MATLAB You will also learn some basic and advanced techniques for manipulating the data once in ArcGIS We will use Surface Water Phosphorus in New Jersey as a case study to develop these skills Homework 1 will test these skills on a different parameter of interest for groundwater in New Jersey TOPICS I Obtaining Data a Downloading from the internet b Importing to Excel II Creating Data Library for ArcGISBMEGUI a Creating Database File b Creating BMEGUI Data File c Exporting to ArcGIS III Exploring Data in ArcGIS a Downloading Shapeflles b Joining Data Tables c Adding XY Data d Setting up Data Queries Obtaining a Dataset Obtaining Surface Water Quality Data for the state of New Jersey Selecting surface water quality parameters of interest The New Jersey Department of Environmental Protection NJDEP monitors the water quality across the entire state using a network of 115 surface water monitoring stations This work is done by the Bureau of Fresh Water and Biological Monitoring which is part of the Office of Water Monitoring amp Standards httpwwwstatenjusdepwmm of the NJ DEP The work is part of the larger water quality monitoring program of the USGS and EPA and part of a national effort to calculate TMDL s total maximum daily load for compliance with the federal Clean Water Act To select a water quality of importance to society one should investigate a water quality parameter that has 1 a large spatial coverage and 2 for which high values are often observed A value is considered high if it poses a risk to human health or to the environment This may happen if concentration values are found to often exceed a health standard or if it they are often above detection limit etc Therefore it is important to examine the pertinent literature about water quality concerns in the location you are investigating in this case New Jersey The literature should provide insight into both government and public perceptions about which water quality parameters are most important to study As an example we will focus on Total Phosphorus in the state of New Jersey for the period of 19992003 This water quality parameter was chosen because the corresponding monitoring data has a large spatial coverage over the state of NJ as compared to other water quality parameters waterusgsgov and because high monitored values were collected as indicated in chapter 2 of the 2004 Integrated Water Quality Monitoring amp A Report 1 39 quot 39 J by the NJDEP Office of Water Monitoring amp Standards httpwwwstatenjusdepwmm This report is available at the following link httpwww state hi I n waL 39 Jquot 2004renort html Chapter 2 on the Chemical Water Quality Assessment has figures on Phosphorus particularly Fig 211 showing that Fecal Coliform and Total Phosphorus most often exceeds their standard and Fig 21a l showing maps showing the location of monitoring stations in nonattainment of the standard for Total Phosphorus The current standard from httpwww state hi I den wmmsgwqt 39 NJDEP is stated for Total Phosphorus in streams html as determined by the phosphorus as total P shall not exceed 01mgL in any stream unless it can be demonstrated that total P is not a limiting nutrient and will not otherwise render the waters unsuitable for the designated uses In the following we will therefore focus obtaining data for Total Phosphorus in the state of New Jersey We will further focus on the Raritan River Basin area as our mapping area of interest Obtaining Phosphorus Concentration in the Raritan River Basin of New Jersey 1 Create a working folder Create a Folder NJRaritanGIS in the following directory dtemp You will store all of your data and GIS shape les in this folder 2 Download NJ Surface Water Monitoring Network and Phosphorus Data from USGS rqowrvgop s wasp arr 54m Go to httpwaterdatausgsgovnwisgw In Data Category choose Water Quality for Geographic Area choose New Jersey Click on FieldLab Samples Select Site Type under Site Attribute Select Period of Record under Data Attribute 7 leave all else blank and click submit Under Site Type choose StreamRiver Under Period of Record choose 01012000 to current date Under Choose Output FormatSummary of Selected Sites select Site Description Information Displayed in Tab Separated saved to file Select Site ID Decimal Longitude and Decimal Latitude using Ctrlclick in the scroll down box Click on Submit Save the file in the NJRaritanGIS folder as NJiUSGSiStationLoctxt Go to httpnwiswaterdata usos vusanwisqwdata Select Site Type under Site Attribute Period of Record amp Parameter Grouping under Data Attribute Click on Submit Select StreamRiver under Site Type Enter a Period of Record from 01012000 to current date Under Parameter Groupings select Nutrients Skip the next section and under Retrieve Water Quality Samples for Selected Sites and then retrieve data from put in the Period of Record 20000101 to current date Then choose the last option tab separated data one sample per row with remark codes combined choose MMDDYYYY from the drop down list save to le from the next drop down list Click on Submit Save the file in the NJRaritanGIS folder as NJiUSGSiNutrientstxt Optional steps for EPA data you may skip these steps step u to step ee u Go to httpwwwepagovstoretdbtophtml Click on Browse or Download M odernized STORET Data Under STORETRegular Results click on Regular Results by Geographic Location Under Geographic Location Select by State and choose New Jersey and ALL counties Under Date choose 01011999 to present Under ActivityMedium choose Water Under Characteristic type in Phosphorus and Click on Search Choose Phosphorus as P and Click on Select Click on Continue dd On the next page choose site ID Location Information Activity Start Characteristic Name Result Value as Text and Units click on Continue V W x y QWQN QWQ ee Click on 39download le39 and save the le in the NJRaritanGIS folder as 39NJiEPAiS tatz39 onL ociN u m39 ems txt39 Creating a Data Library 3 Set up the Phosphorus and Monitoring Network Database a Open 39NJiUSGSiStationLoctxt in Excel using Data 9 Import External Data within Excel i Be sure to change site no to text before completing the import Finish the import Copy the entire worksheet into a 2quotd worksheet label the first sheet RawData Working in the 2quotd worksheet Make sure to leave the row with the column headings starting with siteino and delete the extra comment rows in the le rows 124 or up to the column headings and the row just below the column headings starting with 5s Select the 39Latitude39 and 39Longitude39 columns and go to Format 9 Cells 9 and format these cells as numbers with 4 decimal places Save the le in the NJRaritanGIS folder as NJiUSGSiStationLocxls Open MS ACCESS and click on Data from Excel Choose the Excel worksheet with just the station locations and site no Go to export 9 more types 9 DBF4 and save with same name as the xls file These steps are for Excel 2003 anal earlier versions starting from Step F Save again except this time save as type DBF4 albaseIV 39 you will have to click on 39Save39 anal 39Yes39 every time it appears The DBF is not fully saveal until you close the file completely Repeat steps ac for the NJiUSGSiNutrientstxt39 file We only want the phosphorus parameter which is code 39p00665 so delete all other parameter columns Delete the 39sampletm39 and 39samplecd39 columns as well Insert a new column between sampledt and p0066539 Title this column 39month Insert another column and title it year Insert one final column and title it days In the month column first cell down enter the formula MONTHMMDDYYYY39 where MMDDYYYY is the cell containing the date of measurement Apply the formula to the entire column Repeat step pq for the year column in place of MONTH in the formula put YEAR In an empty cell enter the starting MMDDYYYY ofthe data ie 01012000 In the days column first cell down type the formula DAYS360startingdateMMDDYYY where MMDDYYYY is the cell containing the initial dates of measurement Sort the data by the 39p0066539 column DatagtSort Delete all rows where no data exists should be all at the end of the sorted list or an 39M exists Do a find and replace on the p0066539 column to replace all lt wit and replace all 39E with 39 empty space go to edit 9 find 9 replace 9 y Create a new column called 39p00665hardened and apply the following equation P0066ShardenedIFP00665lt0ABSP006652 P00665 z Reformat the data columns as numbers with 3 decimal places aa Save the file in the NJRaritanGIS folder as 39NJiUSGSiPhosxls39 bb Repeat steps G I to create the DBF file 5 Fquot 0 99 rp qo H 4273093 wr arts17 aglt Optional steps for EPA data you may skip these steps step i to step xx H N xvii XViii Resave as DBF4 1 Re format longitude and latitude as numbers with 4 decimal places 39 Delete columns 39state39 39county39 39HUC39 39zone39 and 39characteristic name39 Rename the 39value as text39 column as 170066539 to match the USGS code 39 Reformat thep00665 column as a number with 2 decimal places Open 39NJiEPAiStationLociNutrientstxt39 by importing into Excel Re format the Station ID column as text Sort the data in the p00665 column and delete all rows containing 39NonDetect39 and Present Sort the data in the unit column and delete the rows that do not contain units Add a new column after units name it Phos ugl use Format cells Number to set it s format to number with 2 decimals and apply the following equation Phos ugl IFLEFTunits 2 quotmg quot p00665 1000 p00665 Highlight the 39activity start39 column and go to Format cells custom and enter 39yyyymmdd 39 39 Insert an empty column after 39activity start39 name it year and use Format Cells Number to set it s format to number with 0 decimal and apply the following equation yearYEARactivity start Save this file as N EPAiStationLociPhosxls39 Also save thisfile asDBF4 Copy the 39site id39 39latitude39 and 39longitude39 columns into a new excel file and save this file as 39NJiEPAiStationLocxls39 Resave this file as DBF4 like before 39 In the 39NJ EPAiStationLociPhosxls 39 file insert a column at the beginning called 39Org ID39 andfill with 39EPA39 Copy the 39Org ID39 39Site ID39 39activity39 39year39 and 170066539 columns into a new excel file make sure to reformat 39site ID39 as text Save this file as 39NJiEPAiPhosxls39 4 Setting up ArcGIS and T 39 quot a b r39WWQP p Explorinq Datg in ArcGlS NJ m quot39 G0 to httpwww state hi I den gis 39 html Download the Hydrography State Boundary and Watershed Management Area shape les by selecting them from the drop down menu and clicking on download download them into the NJRaritanGIS folder Unzip the downloaded les into the NJRaritanGIS folder Open ArcMap Go to le 9 map properties 9 data source options 9 store relative path names Go to view 9 toolbars 9 spatial analyst to open the spatial analyst toolbar Go to spatial analyst 9 options 9 and set working directory to dtempNJRaritanGIS Click on the Add Data button and navigate to the NJRaritanGIS directory Add the depwmasshp staterivshp les in that order 5 Adding the USGS Database into ArcMan a b Click on Add Data and add the NJiUSGSi StationLocdbf39 and NJiUSGSi Phosdbf39 tables into ArcMap Right click and open each table Which table has the Phosphorus measurement data and how many measurements are there in that table Which table has the longitude and latitude of each monitoring station and how many monitoring stations are there 6 Adding the XY Point Data onto the Map W999 FQ39r39P QO m H Right click the NJiUSGSiPhosdbf39 table 9 Joins amp Relates 9 Join For no 1 choose the Site No eld from NJiUSGSiPhos For no 2 choose the 39NJiUSGSiStationLocdbf39 table For no 3 choose the Site No eld from 39NJiUSGSiStationLocdbf39 Click Advanced 9 keep only matching records 9 ok Right click on the joined table and see that new columns have been added Note which column has the longitude NJiUSGSiStationLocLongitude and which column has the latitude eg NJiUSGSiStationLocLatitude Sort the table by the site column click on Options 9 Export 9 rename StationPhosData and add the table to GIS Go to tools 9 add XY Data 9 select the new exported table StationPhosData Set the X Field to longitude and the Y Field to latitude Under coordinate system click edit 9 select 9 Geographic Coordinate System 9 North America 9 North America Datum l983prj and apply A second new layer has been created with point features representing the location of each of the Phosphorus monitoring events Note that if you set X to latitude and Y to longitude the points will not plot correctly Note that the points and other shape les may not coincide we will x this problem later Save the new layer as a shape le by right clicking on it 9 data 9 export data 9 replace exportioutputshp39 with NJiUSGSiPhosLoc and click OK you can now remove the old point layer from the table of contents 7 Create BMEGUI Data Files a 99 FrPPW W9 The typical format for BME analysis is the SpaceTime Vector GeoEAS format i Line 1 descriptive title of le ii Line 2 of columns in the le iii Line 3 to number of columns column names iv Data columns 1 typical order is side ID longitude latitude measured value time In ArcMap select the attribute table of the joined layer with station location and data values in the same table Open the attribute table Click on Options amp Exp01t change the name in the output box before saving to the working directory ie StationPhosData2 Go to the newly created dbff11e open in Excel and resave as an Excel xls flle Close the le and reopen in Excel Rearrange the columns so they are in the following order site ID long lat data time Save as a text le 7 tab delimited clicking yes when prompted Open the text le delete the rst row Add the additional rows as described in pa1t a Save with the extension dat 8 CheckingChangingMatching Projections for all shapefiles a PPWWQQQV 3 PW F V5939 Check to see which layers have a de ned coordinate system by right clicking each layer 9 propeIties 9 source Depwmasshp should be the only layer with a predef1ned coordinate system Record the coordinate system that has been de ned Remove all layers from ArcMap Open ArcToolbox Go to Data Management Tools 9 Projections and Transformations 9 De ne Projection Set the Input Dataset to the stateshp shapeflle Click on the icon to select a coordinate system Click 39Imp01t and select depwmasshp as the shapeflle that has the coordinate system that you would like to copy Click 39Add39 and Click OK39 and OK39 again to nish You have de ned a coordinate system for stateshp that is the same as that of depwmasshp Repeat the same steps f to j for the staterivshp shapeflle For the XY data you will need to de ne a different coordinate system geographic coordinate system in order for the points to match up with the downloaded shapeflles Go to ArcToolbox 9 Data Management Tools 9 Projections and Transformations 9 De ne Projection Choose the NJiUSGSiPhosLocshp shapeflle Click on the icon to select a coordinate system Click 39Select Navigate to Geographic Coordinate SystemNorth AmericaNorth America Datum l983prj Click 39Add 39OK39 and OK39 again to nish You have de ned a coordinate system for NJiUSGSiPhosLocshp Go back to ArcMap and add back shapeflles as need in order to display all 4 shapeflles of interest depwmas state stateriv NJiUSGSiPhosLoc 8 9 Exploratog Analysis for Phosphorus a Fquot 0 First we need to select only the Raritan Basin in New Jersey and only those stations and river network which fall within the Raritan i Select the Watershed Management Areas corresponding to the Raritan WMA 8 9 and 10 by right clicking the depwmas shapeflle 9 open attribute table 9 highlight WMA 6 8 and 10 ii Close the table you should see the WMA39s highlighted on the map iii Right click the depwmas layer 9 selection 9 create layer from selected features and rename the resulting layer as 39RaritanWMA iv Go to Selection 9 Select By Location and select features from stateriVshp that intersect features from RaritanWMA V Save the selection as a new layer and rename as RaritanRiver vi Repeat the above selection by location for the USGS monitoring events except instead of 39interesect choose 39are completely within the depwmas features vii Save the new layer files as 39RaritanUSGSPhos viii You can now remove all the layers not corresponding to the Raritan and zoom in on the Raritan Basin only Now we want to explore the data for a select year or range of years and visualize this on the map i Go to Selection 9 Select By Attribute 9 choose the RaritanUSGSPhos layer and create a query selecting phosphorus data NJUSGSi4 for 2001 quotNJiUSGSi4quot 2001 ii Those stations monitoring in 2001 are highlighted so create a layer based on this selection named 39RaritanUSGSPhos7200139 iii Right click this new layer 9 properties 9 symbology 9 quantities 9 and select the phosphorus field to visualize either using graduated color or graduated symbols Geoprocessing and Buffers i To combine features in a layer highlight the RaritanRiver layer and go to ArcToolsBox 9 DataManagement Tools 9 Generalization 9 Dissolve Select the layer RaritanRiverlyr set the Output Feature Class to DtempNJRaritanGISRaritanRiverDshp Set DissolveiField to NUMBER Select which fields you would like to keep in the new attribute table and the function you would like to do when combining e g set Field to LENGTH and Statistics Type to SUM and click on OK iv Now to add a 05 km buffer around the combined river network go to ArcToolsBox 9 Analysis Tools 9 Buffer Select the layer to add a buffer around RaritanRiverlyr set the Output Feature Class to DtempNJRaritanGISRaritanBuffer705kmshp Set the buffer Distance to 05 Kilometers set Side Type to FULL End Type to ROUND and Dissolve Type to ALL and click on OK ii39 lt 10 Saving the Project a There are now several new layers which should rst be saved as shape les RaritanWMA RaritanRiver RaritanUSGSPhos RaritanUSGSPhos72001 RaritanEPAPhos72001 RaritanBuffer705km right click each layer 9 data 9 export data and save them with the same namesshp b Go to le 9 save as and you can now save the project for later use Modeling spacetime variability 2 The covariance of SpaceTime Random Fields SITRF 21 Definitions The SpaceTime Random Field STRF Xp Xsl is a random variable that is a function of location ss1s2 and time I The spacetime covariance between spacetime points p and p is cxpp EXP mxPXP mxP l 22 Homogeneous I Stationary SITRF A homogeneousstationary STRF has the following properties It s mean trend is constant mXav mX It s covariance is only a function of the spatial lag r and temporal lag r 0XpP CXSJ SCI cx rllss ll Z I I l Are the following st covariance homogeneousstationary Homogeneousstationary Separable chMP co eXpSS eXplll l cXpp c0 eXp lls s ll eXp ll I l c0 eXp r eXp 139 chMP co eX10SS llI I co eXp VT 2 The covariance of SpaceTime Random Fields SITRF 21 Definitions The SpaceTime Random Field STRF Xp Xsl is a random variable that is a function of location ss1s2 and time I The spacetime covariance between spacetime points p and p is cxpp EXP mxPXP mxP l 22 Homogeneous I Stationary SITRF A homogeneousstationary STRF has the following properties It s mean trend is constant mXav mX It s covariance is only a function of the spatial lag r and temporal lag r cxpp cxSJ S J cx rllsS ll z I I l Are the following st covariance homogeneousstationary Homogeneousstationary Separable cXpp c0 eXp ss eXp lll l NO na cXpp c0 eXp lls s ll eXp ll I l c0 eXp r eXp 139 YES YES cXpp c0 eXp s s ll I l c0 eXp FT YES NO 23 Spacetime separable covariance for homogeneous I stationary SITRF A homogeneousstationary STRF is spacetime separable if it s covariance can be written as 0X01 chr 0X3 cXr is called the spatial covariance component eggDis called the temporal covariance component Exercise Consider the following covariance models cXr 239 c0 eXp 3r2ar2 eXp 3 701 cXr 239 c0 eXp 3ra1eXp 3 Ta1 002 eXp 3rarg eXp 3 7013 Are these models homogeneousstationary Are they spacetime separable If the covariance is separable What is the covariance model of the spatial and temporal components If covariance is not separable is it a nested structure of st separable covariance models 05 cXr0 T is a slice of that surface for r0 It shows the temporal component 15 23 cXr 10 is a slice of that surface for i0 It shows the spatial component 05 15 1 02 25 10 15 26 02 cxr 01 1b cxrt 0 o 393 3 oo 3 e cXr T is a 2D function that can be displayed as a surface 3 24 Displaying a separable st covariance for homogeneous l stationary STRF 25 Estimating a sit covariance A useful equation to calculate the covariance of a homogeneousstationary STRF is 0X0 7 EXSJXS J7 HH HW izz ir mX2 When having sitespeci c data and assuming the SRF is homogeneousstationary we may estimate the covariance value for a given spatial lag r and temporal lag 239 using the following estimator A 1 N r 17 CXr71 Nr 1 211 XheadiXtaili where Nr 239 is the number of pairs of points with values Xhead Xta separated by a 2 mX distance of r and a time of 239 In practice we use a tolerance dr and dz ie such that V dquot S llshead Staiill S Vd7 and 1quotde them Stan S Td7 21 26 SpaceTime covariance estimation in BMEIib Spacetime data formats in BMEIib Spacetime vector format When data are collected irregularly in space and time each data point has a separate spacetime location An example of this format is when data is collected at different times along a path e g samples collected on a moving plane or ship In this case the data is stored in BMElib in the spacelime vector format Each data point is specified with its specific spatial location and temporal coordinate Spacetime grid format When the data is collected at synchronized times for a set of fixed monitoring station we can take advantage of the regularity in the spacetime data An example of this format is for PMlO data collected every 6 days at set of EPA monitoring station with fixed spatial location In this case the data is stored in BMElib in the spacelime grid format In this format the fixed spatial location of the Monitoring Station MS are stored separately from the times at which samples are collected the Monitoring Events or ME 22 Conversion of spacetime data formats in BMElib Conversion between spacetime grid and spacetime vector formats stgridsyntaX SyntaXical help for st grid format valstg2stv Converts values from st grid coord to st vector coord valstv2stg Converts values from st vector coord to st grid coord probastv2stg Converts probabilistic data from st vector coord to st grid coord probastg2stv Converts probabilistic data from st grid coord to st vector coord A game Where should all these functions be located help iolib 23 help stgridsyntax stgridsyntax Syntaxical help for st grid format Jan 1 2001 CONTENT OF HELP FILE When the data are collected at a xed set of nMS Monitoring Sites with coordinates cMS for nME time events then one can use the st grid format to store the spacetime data In this case the locations of the Monitoring Sites are stored in the nMS by 2 vector cMSslMS s2MS the time of the nME Monitoring Events are stored in a l by nME vector and the spacetime data is stored in a nMS by nME matrix Z as shown in the following example tME1OO 200 300 400 cMS1O 10 Z 1 4 7 10 2O 20 2 5 8 11 3O 30 3 6 9 12 One can convert the data from st grid format to st vector format where each data value is associated with a st location Hence the values are in a nMSnME by 1 vector 2 to which is associated the locations chsls2t where s1 s2 and t are nMSnME by l vectors for the SI spatial coordinates s2 spatial coordinates and time respectively For example converting the above st grid data to st vector format is done by using the following command chzvalstg2stvZcMStME And the result is as follow in st vector format s1 52 time 2 ch1O 10 100 z1 2O 20 100 2 3O 30 100 3 1O 10 200 4 2O 20 200 5 3O 30 200 6 1O 10 300 7 2O 20 300 8 3O 30 300 9 1O 10 400 10 2O 20 400 11 3O 30 400 12 Additionally the data can be converted from st vector to st grid format as follow Z valstv2stg ch 2 cMS tME Note that the commands probastv2stg and probastg2stv do a similar conversion on probabilistic data 24 BMElib Covariance estimation using spacetime vector data In BMElib the function used to estimate the st covariance of a homogeneousstationary ST RF using data in st vector format is 0105 scovarioST m SYNTAX cls dt 0 o crosscovarioST cl 02 zl 22 cls clt options cl Spacetime coordinates of hi data points Column 1 and 2 are spatial Xy coordinates and colum 3 is time c2 for covariance estimation 02 and cl are the same z 1 vector of values of data at the cl spacetime coordinates z 2 for covariance estimation 2 2 and z 1 are the same cl 5 ncsl by 1 vector giving the limits of the spatial distance classes that are used for estimating the covariance or cross covariance The distance classes are open on the left and closed on the right clt nctl by 1 vector giving the limits of the temporal quotdistancequot classes that are used for estimating the covariance or cross covariance As for cls the classes are open on the left and closed on the right BMElib covariance estimation using spacetime grid data In BMElib the function used to estimate the st covariance of a homogeneousstationary ST RF using data in st grid format is stcov m SYNTAX C npstcovZicMSitMEiZjcMSjtMEjrLagrLagToltLagtLagTol INPUT cMsi spatial coordinates of nMS monitoring stations tMEi time of nME monitoring events Z i nMS by nME observed values may contains NaN for missing values Forcovariance estimation cMSi tMEi and Zi are the same as cMSj tMEj and Zj rLag r1 r2 define the spatial lag classes rLagToldrl dr2 u rl drl r2 dr2 tLag t1 t2 define the temporal lag classes tLagToldtl dt2 u tl dtl t2 dt2 3 The crosscovariance between two Random Fields 31 Definitions Let Xav and Y an are two random elds The crosscovariance between X at point p and Y at pointp is CXYQW EXP mxPYP MYOD D EXPYp maymm If Xp and Y p are homogeneousstationary STRF then the covariance is given by CXYpp CXYSJ S J cXY rllsS ll z I I l 32 Estimation a crosscovariance Using sitespecific data for two homogeneousstationary STRFs the crosscovariance is given by A 1 Nrr CH0 139z Nr TZ1 Xheadthaizi meY where Nr 239 is the number of pairs of points with values Xhead Y tail separated by a distance of r and a time of 239 In practice we use a tolerance dr and dz ie such that V dquot S llshead Staiill S Vd7 and 1quotde them Stan S Td7 20 33 BMEIib crosscovariance estimation The function to estimate the crosscovariance between two homogeneous SRFs is 0105 scovario m The function used to estimate the st cross covariance between two homogeneousstationary STRFs using data in st vector format is cros scovarioST m SYNTAX ds dt 0 o crosscovarioST cl 02 zl 22 cls clt options The inputs are the same as for covariance estimation except now cl z 1 are for fieldX and c2 z 2 are for field Y The function used to estimate the st cross covariance between two homogeneousstationary STRFs using data in st grid format is stcov m SYNTAX C rip stcov Zi cMSi tMEi Zj cMSj tMEj rLag rLagTol tLag tLagTol The inputs are the same as for covariance estimation except now cMS i tMEi and Zi are for fielannd cMSj tMEj and Zj are for field Y 21 34 Review of the BMEIib functions for covariance estimation pairsplot display pairs of points separated by a given distance interval pairsindeX finds pairs of points separated by a given distance interval covario covariance and cross covariance estimation a set of variables which are known at the same set of spatial coordinates crosscovario single cross covariance function estimation crosscovarioST spacetime cross covariance estimation for st vector data stcov spacetime cross covariance estimation for st grid data 22 Appendix The covEstimationSTv m and covE stimationSTv g Tutorials Exploratory analysis and covariance estimation for SpaceTime Random Fields SRF Xst SpaceTime Random Field 8 TRF X SJ take values that changes both as a function of the spatial location sx y as well as time t The hard data exact measurements of the STRF may typically be available in two type of format In the first format which we will call STg for Space Time grid there are a set of mm monitoring stations at location CMSx1y1 and the measurements occurred at a set of nm monitoring event at time tMEt1 that are the same for all monitoring stations hence resulting in nMsmME measurements In the second format which we call STV for Space Time vector each location is sampled at a different time hence in this case the data is listed as a set of 7111 measurements corresponding to spacetime 7111 points Phx1y1t1 The marke rplot m command may be used to provide a maps showing the hard data at different time intervals Hence a sequence of such maps will provide a quotmoviequot showing the spatiotemporal variation of the data When using spacetime data in the STV Space Time vector format the crosscovarioST m command may be used to estimate experimental values of the covariance as a function of spatial lag and temporal lag The covEstimationSTv m command is a tutorial showing how to use this command and an example is presented in the following pages The program covEstimationSTv m itself is printed at the end of this appendix When using spacetime data in the STg SpaceTime grid format the stcov m command may be used to estimate experimental values of the covariance as a function of spatial lag and temporal lag The covEstimationSTg m command is a tutorial showing how to use this command and an example is presented in the following pages after the example for the STg format The program covEstimationSTg m itself is printed at the end of this appendix SpaceTime Random Field ST RF X s t in STV format With a separable exponential covariance Hmuwavmmtmnmsem l39wlphln39rmmhmsmmhya m 39 u 5 5 u W 11y 39 gm gm quot n u 39 I 39 I 39 n I r u n o v I 39 a i a i a i a u u u m 5 n z i a a w 1 v is I 27 mm m N mmmmuumwnmzmzw15 quot kmwmwmlmm 7 m wr 5 2 2 wr Em I u 39 I 4 a i n z 1 o a39 1 x s a I u a n quota a m u u is m u my um Cumin 1 5 7 393 1 8 g 05 r E 7 r x 7 2 w m sum 59 r mm 1 5 7 3 M g as a i i i 7 y 5 1n 5 tempom W n my The model selected is Cr1j Cg eXp3 ra eXp3 tat Where the sill variance Cgl3 the spatial range a 8 Km and the temporal range a 9 days SpaceTime Random Field ST RF X s t in STg format With a separable exponential covariance mmmmam mmmmmm l7 u 23 m z in mm xmuuu39lzc mm m mamma a w l mme mm m cm w 39 m I m gm a I I o 2 u 7 I a 1 l l l n 2 i i m 2 t m n z t m u A i m M W covariance rL50 5 1O 15 5pm lug r mm 15 T 1 a 05 o i 7 7 7 7 o 5 1o 15 mme m u my The model selected is Cr1j Cg eXp3 ra eXp3 tat Where the sill variance Cgl3 the spatial range a 13 Km and the temporal range a 11 days The covEs timationS TV In Program covEstimationSTv covariance estimation tutorial using data in ST vector format ESTIMATION OF COVARIANCE USING SPACE TIME VECTOR DATA Read the hard data from le valvalname letitlereadGeoEASCcovstvdat39 chval 13 zhval4 thch 3 Open a new graphic window and display the hard data that were measured between day 0 and day 5 using circles having a size which is proportional to value of the hard data gure markerplotch0lt hampthlt5 12zh0lt hampthlt5 4 30 Xlabel39X Km ylabel y Km title Markerplot of data for time 0 to 5 days39 Open a new graphic window and display the hard data that were measured between day 5 and day 10 using circles having a size which is proportional to value of the hard data gure markerplotch5lt hampthlt10 1 2zh5lt hampthlt10 4 30 Xlabel39X Km ylabel y Km title Markerplot of data for time 5 to 10 days39 Open a new graphic window and display the hard data that were measured between day 10 and day 15 using circles having a size which is proportional to value of the hard data gure markerplotch10ltthampthlt1512zh10ltthampthlt154 30 Xlabel39X Km ylabel y Km title Markerplot of data for time 10 to 15 days39 Open a new graphic window and display the hard data that were measured between day 15 and day 20 using circles having a size which is proportional to value of the hard data gure markerplotch l 5ltthampthlt20 l 2zh l 5ltthampthlt20 4 30 Xlabel39X Km ylabel y Km title Markerplot of data for time 15 to 20 days39 Estimate the covariance by using pairs in all directions cls0 5 10 20 clt0 6 12 20 dsdtcocrosscovarioSTchchzhzhclsclt variancevarzh Plot the estimated covariance on a new graphic window gure subplot2llhold on plot0 dsl variance cl 39r plot0 150 0 k ylabel39Covariance Crt039 Xlabel spatial lag r Km39 title Covariance39 subplot2l2hold on plot0 dtlvariance cl r plot0 150 0 k ylabel39Covariance Cr0t39 Xlabel39temporal lag t Day A reasonable model is an separable spacetime covariance with a sill variance ccl3 an exponential spatial component with a range aa8 Km an exponential temporal component with a range at9 Day r0 2 15 covmodel eXponentialC covparaml3 8 subplot2ll modelplotrcovmodelcovparam ylabel39Covariance Crt039 Xlabel spatial lag r Km39 t0 2 15 covmodel exp0nentialC covparam13 9 subp10t212 mode1p10ttcovmodelcovparam ylabel39Covariance Cr0t39 XlabelCtemporal lag t Day Vi The covEstimationSTg m Program ESTIMATION OF COVARIANCE USING SPACE TIME GRID DATA Read the hard data from le valvalname letitlereadGeoEASCcovstgdat39 cMSval l 2 Zhval3 end nMEsizeZh2 tMEl nME Estimate the covariance by using pairs in all directions rLag 00 20 40 80 140 rLagTol00 15 20 40 60 Cr nprstcovZhcMStMEZhcMStMErLagrLagTol00 tLag 010 tLagTol0 00 l ones llengthtLag 1 Ct nptstcovZhcMStMEZhcMStME00tLagtLagTol Plot the estimated covariance values on a new graphic window gure subplot2llhold on plotrLagCr39r plot0 150 0 k ylabel39Covariance Crt039 Xlabel spatial lag r Km39 title Covariance39 subplot2l2hold on plottLagCt39r plot0 150 0 k ylabel39Covariance Cr0t39 Xlabel39temporal lag t Day A reasonable model is an separable spacetime covariance with a sill variance ccl3 an exponential spatial component with a range aal3 Km an exponential temporal component with a range at11 Day r0 2 15 covmodel exponentia1C covparam13 13 subplot211 modelplotrcovmode1covparam ylabel39Covariance Crt039 Xlabel spatial lag r Km39 t0 2 15 covmodel exponentia1C covparam13 11 subplot212 mode1plottcovmodelcovparam ylabel39Covariance Cr0t39 X1abe139temporal lag t Day Viii SpaceTime mapping 2 Mapping of HomogeneousStationary STRF 21 The mapping situation XpXst is a STRF Data measurements of Xp is available at the data points pdata p1 pn At this points Xp is viewed as a vector of random variables xdata x1 xn Xp1 Xavn We denote the estimation point as pk At the estimation point the random field is represented by the random variable xk X pk The mapping problem On the basis of some general knowledge 9 ie covariance function etc and sitespeci c knowledge S the data get some estimate 2k for xk Related question What is a good estimator 92k for the value of xk at pk A o Ihta points I I Estimation points III I 0 O o U o S 2 0 0 S1 0 o o Mapping situation showing available data points and a set of estimation points 22 The Mapping points The mapping points include the data points pdata p1 pn AND the estimation point pk hence pmp pdata pk spacetime location of the mapping points xmap xdam xk vector of random variables representing Xp at pmap zmap Zdata 1k a deterministic realization ie a set of possible values for xmap The data points pdata p1 pn are further divided among hard data points phard p1 pmh and soft data points pso pmh1pn Hence we finally have pmap phard psoft Pk xrnap xhard 3 xsoft xk Zmap Zhard 3 Zsoft 3 Zk 23 The estimation process Prior stage Using general knowledge 9 obtain the prior pdf of xmap xdlam xk ie f Zmap f Zdataa Xk Meta prior Organize the sitespecific knowledge S into hard data soft data etc ie Zdata Zhard 3 Zsoft Hence the prior pdf becomes f gotmap fg Zhard Zso Xk Integration or posterior stage Update the prior pdffg by integrating the sitespecific knowledge S to obtain the posterior pdf at the estimation point fgZhard Zsoft X Integrate S fKgUS Xk prior pdf posterior pdf providing a complete stochastic description of xk Xp at the estimation point The interpretive stage From the posterior pdff7lt xk extract some estimated value 92k for xk Xpk Practical notes for using the estimation process in spacetime mapping Typically spacetime mapping is the exercise of selecting an adequate grid of estimation point and an adequate estimator to construct maps of the estimated values Example of estimator The mode of the posterior pdf Example of mapping grid Regular 30x40 grid all estimation points delaunay triangulation of estimation points 24 The knowledge bases The total knowledge base Kabout the mapping situation is divided between general knowledge base 9 and sitespecific knowledge base S so that Kg U S The general knowledge base 9 includes all knowledge bases that are of general nature about the random field Xp of interest They can be generalizable over several mapping situations as they characterize global properties of Xp Examples of general knowledge bases for Xp are It s mean trend mxpEXp It s covariance cxpp E XpMxp Xp Mxp l A physical law or epidemiological relationship RXpYp0 with another field Y p The sitespecific knowledge base 3 includes all knowledge bases that are speci c to the mapping situation at hand They refer to the data measurements etc available in the specific mapping region of interest Usually the sitespecific knowledge bases is composed of the hard and soft data ie S includes The hard data hard 11 zmh The soft data 15011 1mm Zn which can be of interval type or probabilistic type Hard data are exact measurements ie at phard we have 3 39 Zhard Zlquot393 th Pxhard Zhard 1 Example We are mapping the rainfall Xp over Chapel Hill We have 5 rain gages and on 100702 at 330pm it is not raining Hence the hard data at these five spacetime hard data points is hard 0 0 0 0 0 0 and PlxhardZZhardl 1 Soft data 15m 1mm Znof interval type are intervals I with lower and upper bound of the measurements a and b ie hence at pm we have S Isoft 1 Paltxsoftltb1 where aamh1 an is the deterministic vector of lower bound value of the measurement bbmh1 3 is the deterministic vector of upper bound value of the measurement IImh1 1 are the intervals Iia 31 for imhl to 11 Example At two soft data points pso p2 p3 we known that the concentration Xp in the air of particulate matter is below the detection limit of 5 ppm hence a0ppm Oppm b5ppm 51313111 xso x2 X3 and Paltxso ltb1 Soft data 1 1mm znof probabilistic type is the socalled soft cdf of xso representing the random field Xp at the at the soft data points pso ie Squot Isoft 1 PxsoftltlsoftFSZsoft Example We have one soft data point pso p4 Where X p4x4 is a random variable such that 0 if Z4 lt 0 Px4ltZ4FSZ4 Where F5064 96420 if 0 S 964 31 1 if Z4 gt1 Draw F 304 and calculate f 504 25 Prior stage Processing the general knowledge base The general knowledge base 9 represents a knowledge of general nature about the random eld Xp that can be expressed in terms of stochastic moments equations of Xp ie they may written by means of the following constraints a 0313 3NC with gt xmap lidxmpga xmapvmmpmmp where fg is the prior pdf we are seeking for the vector of random variable xmap representing Xp at the mapping points ha are stochastic moments known from the general knowledge base 9 The functions gaxmap 051 Nc are chosen such that the corresponding ha are known from Q We are looking for a prior pdf with the following form f Zrnap eXp OZ1ll tagaXmap exp luolul glwmap luNcchZmap where ya are Lagrange coefficients which we need to solve for Procedure to process general knowledge 1 Find as many as gon possible for which we know the corresponding ha from g 2 The prior pdf is given by fgocmap exp yozifguagaltxmpgt 3 Solve for the unknown Lagrange coef cients ya by writing NC equations using the NC ha Values apmap I dxmapgaxmapfsxmappump 061 Nc Note that the equation corresponding to is the normalization constant ie 1 If deapr Xmap Example 1 LetXI be a temporal random field Let map1 12 at pmp I1 Ik with I1l and tk 2 9 XI m0l with m010 Xt2 V012 with V0100 Derive an expression for the prior pdf and a set of equations to solve for the Lagrange coefficients

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

#### "When you're taking detailed notes and trying to help everyone else out in the class, it really helps you learn and understand the material...plus I made $280 on my first study guide!"

#### "There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

#### "It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.