Today I took time to fix all the DEM data so that I would have one large file that covered the entire study area. Before i executed the merge operation, I had to compile all the DEM quads that I had for merging.
After i had all the data collected I did the Merge command for every 15 DEM files. I did this because last night I set all the files to Merge and it ended up not finishing when I left it overnight. Since i did the DEM merges in sections, I ended up with about 15 new DEM files, which I ended up merging into the final, large DEM file.
All this took quite a long time since the processing required to merge the DEM files is very high. The data size of the DEM is also another issue since it takes up more that 1GB on my computer.
Tomorrow I will be creating slope, aspect, and insolation files from the new merged DEM files that I created today.
Currently completed 34/90 hours as of 3/29/2009
Sunday, March 29, 2009
Tuesday, March 24, 2009
Progress for 3/24
Today I focused on correcting the DEM data so that it would fill out the entire study area. In order to accomplish this task, I took the 10m DEM data from each county and found a way to mosaic it across the entire study area. When I was finished I ended up with 252 different DEM maps that were representative of the Quadrangle indexes the area covers. I am currently looking for a way to merge these into one file, but at the moment I am having no luck in the matter.
If it is in fact possible to merge these files, I will be able to create the aspect, slope, and other maps from this DEM data to aide in the research. If not i will have to find another method to consolidate this data since 252 different files for each file type is not sufficient.
Today also gave me a chance to sort through all of my data and put them into identifiable folders with appropriate file names. This took a very long time, however, I know that it will save me alot of time in the future now that the file names and folders are representative of what's actually in them. Now I have folders sorted by county and the files named by what their function is. This makes my database much more usable and efficient.
My goals for the next time I work on the project are:
If it is in fact possible to merge these files, I will be able to create the aspect, slope, and other maps from this DEM data to aide in the research. If not i will have to find another method to consolidate this data since 252 different files for each file type is not sufficient.
Today also gave me a chance to sort through all of my data and put them into identifiable folders with appropriate file names. This took a very long time, however, I know that it will save me alot of time in the future now that the file names and folders are representative of what's actually in them. Now I have folders sorted by county and the files named by what their function is. This makes my database much more usable and efficient.
My goals for the next time I work on the project are:
- find a way to consolidate DEM data
- use a sample county to start the process of identifying rock glaciers and examine the attributes of the natural features in these areas
- using the information I find, start the process of making a model builder to automate the process of looking for rock glaciers
Tuesday, March 17, 2009
Today's progress
Today I finished the final downloads of all of the county level data sets that I need. I now have a huge collection of data for the study area which is about 9GB total. Now that I have finished collecting and organizing the data, I can begin to do analysis and start to map rock glaciers. I will hopefully be able to begin analysis this weekend.
Since the orthophotos that I have collected are organized by grid (high resolution) it seems that they are very large and make the computer very slow when loading. Not to mention that each county has about 30 raster files associated with it (see screenshot below). I will be using these for now, but I am in the progress of downloading the composite images for each county that are clipped to the county boundary. Unless there is some way that I can combine all the grid images from the current ortho-photos that I do have, I don't think they will be very useful and they take up alot of space. Once I have the smaller, newer NRCS composite orthos I can compare the 2 and see which is best.
It's obvious in this screen shot that the orthos are defined by the ortho quad grids. This is useful when indexing the files on a server, but it makes it so that adjacent counties often share the same grids. I could manually clip these but for the time being I am going to check on the other orthos available to see if they will suffice.
I have also finally been able to view the DEM elevation data that Brad sent me, along with the hill shades for that area. It seems like the current extent of his data doesn't entirely fill out the study area so additional elevation files will be needed (see screenshot below). I already have a few of these 10-m DEM's for the area that is missing and I will add them in later when I know exactly what is missing.
By looking at this screen shot you can see that the upper north and west side of the study area are not completed. I am currently in the process of downloading the 10-m DEM's for the counties that are not included.
While I had time today I went ahead and calculated slope, aspect, and zonal statistics for the DEM's that I do have. I will post a few screenshots of this stuff below as well.
This huge amount of data that I have now collected is much too large for my flash drive, which was my backup storage previously. I do have an Adrive account with 50GB and I have (slowly) been uploading the data there for a backup. Currently the main copies are residing on my local machine in the visualization center where I do most of my work.
Update:
I have determined the following counties need DEM data since the current extent isn't large enough:
Completed 26/90 hours as of March 17th, 2009
Since the orthophotos that I have collected are organized by grid (high resolution) it seems that they are very large and make the computer very slow when loading. Not to mention that each county has about 30 raster files associated with it (see screenshot below). I will be using these for now, but I am in the progress of downloading the composite images for each county that are clipped to the county boundary. Unless there is some way that I can combine all the grid images from the current ortho-photos that I do have, I don't think they will be very useful and they take up alot of space. Once I have the smaller, newer NRCS composite orthos I can compare the 2 and see which is best.
It's obvious in this screen shot that the orthos are defined by the ortho quad grids. This is useful when indexing the files on a server, but it makes it so that adjacent counties often share the same grids. I could manually clip these but for the time being I am going to check on the other orthos available to see if they will suffice.
I have also finally been able to view the DEM elevation data that Brad sent me, along with the hill shades for that area. It seems like the current extent of his data doesn't entirely fill out the study area so additional elevation files will be needed (see screenshot below). I already have a few of these 10-m DEM's for the area that is missing and I will add them in later when I know exactly what is missing.
By looking at this screen shot you can see that the upper north and west side of the study area are not completed. I am currently in the process of downloading the 10-m DEM's for the counties that are not included.
While I had time today I went ahead and calculated slope, aspect, and zonal statistics for the DEM's that I do have. I will post a few screenshots of this stuff below as well.
This huge amount of data that I have now collected is much too large for my flash drive, which was my backup storage previously. I do have an Adrive account with 50GB and I have (slowly) been uploading the data there for a backup. Currently the main copies are residing on my local machine in the visualization center where I do most of my work.Update:
I have determined the following counties need DEM data since the current extent isn't large enough:
- Montezuma County
- Dolores County
- San Miguel County
- Montrose County
- Gunnison County
- Saguache County
- Ouray County
- La Plata County
Completed 26/90 hours as of March 17th, 2009
Tuesday, March 10, 2009
DEM data, related studies, and rule based glacier model
I was able to use the NED DEM datasets Brad gave me last week to add to my ever growing database of spatial data for SW Colorado. Since I do not currently have a full version of ArcInfo that I can use this week since I am home for Spring Break, I will have to wait until next week to product the datasets I need from these DEM's. Once I do have access to ArcGIS tools I should be able to start the first part of a rule-based model that will allow me to identify potential rock glacier fomation areas. Once this is acquired, I will manually search these areas for indications of rock glaciers and map them as polygon features to later be added to the database.
In the meantime I have been putting in orders for more data at USDA NRCS for the remaining counties that I posted earlier here: http://kclyons09.blogspot.com/2009/02/study-area-defin.html . I am currently waiting on them to send me a link to download the rest of the counties containing the same data files I requested for Ouray county last week. Since they have download quotas on all of their spatial data I am only able to download about 1 county dataset every day. Hopefully by the time I am back from Spring break I will have this data collection completed.
Since I do not currently have access to ArcGIS I have been looking ahead at how to complete the process of rock glacier identification. I started today by looking up all of the relevent journal articles pertaining to rock glaciers, remote sensing, and GIS.
I have also discovered a worldwide glacier database which I hoped would identify a few of the rock glaciers in the study area. Unfortunately, it seems that due to the debris, lichen, and other issues they are not easily identifiable as other glaciers the system identified from remote sensing. The database I have been using is called GLIMS and is accessible at: http://nsidc.org/cgi-bin/get_metadata.pl?id=nsidc-0272 .
After discovering this information I started to research other Journal articles pertaining to rock glaciers, their formation and how to find them. I found several good resources that are good starting points in developing my rock glacier database. I have listed these articles below and the PDF files are accessible through the library or my Adrive data store.
After reading a Journal article by Brenning (2009) entitled "Benchmarking classifiers to optimally integrate terrain analysis and multispectral remote sensing in automatic rock glacier detection" in which he benchmarks several different algorithms to identify rock glaciers in the San Juan Mountains in Colorado, which comprise most of our study area. He concludes that a combination of remote sensing and terrain analysis are the best methods for rock glacier identification. It is also noted that current remote sensing techniques do not have the ability to map rock glaciers since they do not have a distinct spectral signal.
I will be posting more journal articles and links to related websites as I come across them.
Related studies relevant to this project:
Brenning (2009) -- "Benchmarking classifiers to optimally integrate terrain analysis and multispectral remote sensing in automatic rock glacier detection"
Download PDF here
Millar and Westfall (2007) -- "Rock glaciers and related periglacial landforms in the Sierra Nevada, CA, USA; inventory, distribution and climatic relationship"
Download PDF here
Konrad, et al. (1999) -- "Rock glacier dynamics and paleoclimatic implications"
Download PDF here
Esper-Angillieri (2008) -- "A preliminary inventory of rock glaciers at 301S la
Cordillera Frontal of San Juan, Argentina"
Download PDF here
GIS and Rock Glaciers in Norway:
http://nsidc.org/cgi-bin/get_metadata.pl?id=ggd284
Currently 19/90 hours completed as of March 10th, 2009
In the meantime I have been putting in orders for more data at USDA NRCS for the remaining counties that I posted earlier here: http://kclyons09.blogspot.com/2009/02/study-area-defin.html . I am currently waiting on them to send me a link to download the rest of the counties containing the same data files I requested for Ouray county last week. Since they have download quotas on all of their spatial data I am only able to download about 1 county dataset every day. Hopefully by the time I am back from Spring break I will have this data collection completed.
Since I do not currently have access to ArcGIS I have been looking ahead at how to complete the process of rock glacier identification. I started today by looking up all of the relevent journal articles pertaining to rock glaciers, remote sensing, and GIS.
I have also discovered a worldwide glacier database which I hoped would identify a few of the rock glaciers in the study area. Unfortunately, it seems that due to the debris, lichen, and other issues they are not easily identifiable as other glaciers the system identified from remote sensing. The database I have been using is called GLIMS and is accessible at: http://nsidc.org/cgi-bin/get_metadata.pl?id=nsidc-0272 .
After discovering this information I started to research other Journal articles pertaining to rock glaciers, their formation and how to find them. I found several good resources that are good starting points in developing my rock glacier database. I have listed these articles below and the PDF files are accessible through the library or my Adrive data store.
After reading a Journal article by Brenning (2009) entitled "Benchmarking classifiers to optimally integrate terrain analysis and multispectral remote sensing in automatic rock glacier detection" in which he benchmarks several different algorithms to identify rock glaciers in the San Juan Mountains in Colorado, which comprise most of our study area. He concludes that a combination of remote sensing and terrain analysis are the best methods for rock glacier identification. It is also noted that current remote sensing techniques do not have the ability to map rock glaciers since they do not have a distinct spectral signal.
I will be posting more journal articles and links to related websites as I come across them.
Related studies relevant to this project:
Brenning (2009) -- "Benchmarking classifiers to optimally integrate terrain analysis and multispectral remote sensing in automatic rock glacier detection"
Download PDF here
Millar and Westfall (2007) -- "Rock glaciers and related periglacial landforms in the Sierra Nevada, CA, USA; inventory, distribution and climatic relationship"
Download PDF here
Konrad, et al. (1999) -- "Rock glacier dynamics and paleoclimatic implications"
Download PDF here
Esper-Angillieri (2008) -- "A preliminary inventory of rock glaciers at 301S la
Cordillera Frontal of San Juan, Argentina"
Download PDF here
GIS and Rock Glaciers in Norway:
http://nsidc.org/cgi-bin/get_metadata.pl?id=ggd284
Currently 19/90 hours completed as of March 10th, 2009
Tuesday, March 3, 2009
County Data Collection Progress
After searching around for the best sites to get county level data, I have come to the conclusion that http://datagateway.nrcs.usda.gov/GatewayHome.html is the best one. Provided by the USDA and NRCS this site allows FTP downloads by county and offers a wide range of GIS shape files.
Today I have been trying to collect the data for one county (ouray county) to try to set up a standard dataset that I will use to find the rock glaciers and explain their existence. At the moment I have the following data for this county:
Also, I have re-referenced the original map of the study area to line up with the lat/long lines instead of the rivers which gave considerable error before. The current raster also has a high error but it fills the study area correctly and is easier to read. Here is the image:
Today I have been trying to collect the data for one county (ouray county) to try to set up a standard dataset that I will use to find the rock glaciers and explain their existence. At the moment I have the following data for this county:
- 12-Digit Watershed Boundary Dataset 1:24,000
- Digital Ortho Quad County Mosaic by APFO
- Digital Ortho Quad County Mosaic by NRCS
- Enhanced Digital Ortho Quad TerraServer
- 2006 National Ag. Imagery Program Mosaic
- Geographic Names - Populated Places
- National Land Cover Dataset by State
- Soil Survey Spatial and Tabular Data (SSURGO 2.2)
- U.S. General Soil Map (STATSGO)
- U.S. General Soil Map (STATSGO) - State Subset
- Annual Average Precipitation by State
- Monthly Average Precipitation by State
- Annual Minimum Temperature by State
- Annual Maximum Temperature by State
- Annual Average Temperature by State
Also, I have re-referenced the original map of the study area to line up with the lat/long lines instead of the rivers which gave considerable error before. The current raster also has a high error but it fills the study area correctly and is easier to read. Here is the image:
Currently 16/90 hours completed as of March 3rd, 2009
Subscribe to:
Posts (Atom)