The Fiducial Mark

Steven Romalewski of the CUNY Mapping Service at the Center for Urban Research reached out after my previous post, which outlined UMapper’s approach for mapping real-time Twitter posts.  The CUNY Mapping Service recently launched the Census Hard to Count 2010 site, designed to assist in increasing participation in the 2010 census.  The application displays areas determined as “hard to count” areas (methodology here), along with several layers of thematic data.

The site was recently updated with a Twitter feed that uses the #census and #census2010 hashtags.  This serves two purposes: acts as a geo-referenced feed for census information, and also helps create increase communications between census advocates and citizens interested in the 2010 census.  It’s also a great real-world application of georeferenced Twitter data.   While the percentage of users that have enabled geotagging is estimated to be under 1%, the total number of users in the USA is estimated to be 18 million.  So even a small fraction of users still represents a large quantity of georeferenced data.  Also, geotagging is still relatively new to Twitter – so the adoption rate may stand to grow as more people turn it on, and as more client application add the option to geotag posts.

To display the Twitter feed in the Hard to Count site, you just need to click on the Twitter tab on the upper left portion of the application (see screenshot to the left).  You can also choose the hashtag you would like to display, and the relevant post are displayed on the side panel as well as on the map.  Additionally, you can select a post on the left and choose the “Show on Map” option.  This will display the selected side panel post on the map.

It’s great to see a real-world use case for geotagged micromedia data – so feel free to check it out at www.censushardtocountmaps.org.  Hopefully this is a sign of more to come!

I had the opportunity to check out the new update from UMapper today, which adds the ability to layer real-time tweets over your maps.  While there are a lot of real-time tweet-mapping applications out there, the interesting thing about UMapper’s implementation is that you can create a map with a specific search term built into the map metadata.

See below for an example: this map focuses on the Washington DC area, showing tweets with the word “Toyota” in them.  In the map below, you can also pan and zoom – which will show the posts from any area of interest.

UMapper has come along quite away since the last time I looked at it, which was around a year ago.  The map above uses OpenStreetMap as a base layer, and there are quite a few improvements: different base data providers, several templates, as well as a flexible user permissions system that allows for collaborative mapping.  While I didn’t include anything fancy in the map above, it is also possible to mash in other data sources (e.g. KML, GeoRSS, GPX).  This allows you to easily design and share maps.

The ability to embed maps, such as the one above, is also a nice capability.  For example, it would be great for event monitoring – where people all over the world may be using a Twitter hashtag to talk about the event, but you are only interested in tweets from the specific event location.  While geotagged tweets have been possible to view in Google Maps for awhile now, I think the UMapper approach presents an attractive alternative for homing in on specific topics of interest.

After reading about Autodesk’s Project Butterfly, I took some time to give it a whirl.  Project Butterfly is an on-demand system designed for users that want to experience AutoCAD through a web browser.  Being a technology preview with limited functionality at present, it isn’t a product available for sale – but you can try it out for free and see how the experience differs from the usual desktop (on-premise) experience.

I’m not a heavy CAD user, but there are several reasons why I think Project Butterfly is compelling.

1) The fact that it is an on demand solution:

• No installation required.  A secondary benefit is that updates are pushed out rather than downloaded and installed.
• Platform independence: enables accessibility on desktops, laptops (netbooks!), and a number of operating systems.
• Project Butterfly utilizes Amazon Web Services.  This enables online data storage, but also allows users to download models to local computers.

2) SaaS solutions are nothing new, but I’m not aware of a precedence of a solidly entrenched geo-related desktop application being offered as an on-demand service in addition to the traditional desktop ownership model.  I wouldn’t bet on the decline of AutoCAD as a desktop solution anytime soon, but Project Butterfly provides and attractive glimpse of future possibilities in terms of CAD and geospatial data production and editing.  It will be interesting to see if and when Product Butterfly can graduate into a commercial product!

3) Online data hosting:

Data Download Capability Via Amazon Web Services

Files are stored using the Amazon Simple Storage Service.  It is possible to upload your own data (e.g. I was easily able to add a JPG image as a backdrop), or download data in a number of formats.  These include DWG, PNG, JPG or Zip (with Xrefs).

4) Data production and editing tools: I played around with the geospatial sample data in Project Butterfly.  While the current functionality is limited (it is a tech preview after all), the available features are presented in an nice user interface that is easy to navigate.  Basic drawing and editing tools are available, as well as modes such as snapping and ortho.  It is also possible to upload (import) local data.

Editing in Project Butterfly

5) Collaboration: I didn’t actually try it out, but there is an option to invite others to edit and also set permissions for invitees.  These include the ability to edit and download the data – great features for showing the data to someone without actually allowing them to edit or download the data.

It’s great to see a major vendor moving in this direction.  From a product perspective, an application like this should be able to dramatically reduce the typical software release cycle (e.g. 6 months or more for a heavy desktop application), and create greater efficiencies in terms of product rollouts, technical support, removing the need to ship traditional media discs, and more…

I initially learned about MapAction during an Infoterra event in the UK about a year a half ago.  Impressed by their work, I’ve been following their activities ever since.  After reading they had a team en route to Haiti to assist in the humanitarian effort there, I took a look at their Field Guide to Humanitarian Mapping.  It’s an excellent introduction to mapping methodologies and GIS on a budget, with a heavy focus on concepts, data collection, and specific workflows.

Looking through their list of Open Source GIS applications, I thought it might be interesting to run through a small project: gather data for the affected parts of Haiti and think about data sources, workflows, and considerations in the field.

A few observations at the start, and then I’ll walk through the process I went through:

• Satellite Imagery: Very difficult to attain…  GeoEye has graciously released post-earthquake imagery, but it is still difficult to get the full resolution processed imagery (not an ungeoreferenced jpeg).  I think that remote sensing satellite operators would want to (a) process post-catastrophe imagery as quickly as possible, and (b) get the imagery into the public domain as soon as possible.  It’s great that we can see post-earthquake imagery as a network-link in Google Earth, but people on the ground are not necessarily going to have internet access.  I know we cannot rely on private companies to provide free data as a service, but I do believe there is a need to acquire imagery quickly and get it into the public domain.
• SRTM terrain data is a tremendous resource, but getting at the data can be tricky.  More on that below…
• OpenStreetMap only needs a one-word description: fantastic.  I read earlier today that there have been over 400 edits made to Haiti since the Earthquake.  People around the world are donating their time to help out the cause.  Another great thing about it is that it is extremely easy to check out data and then pull it into another application – more on that below.

The Goal

Nothing fancy: I just wanted to see how long it would take and if it would be challenging to pull together base mapping data and then view it all together.  No real geoprocessing, but just data acquisition and setup using open source software and publicly available data.  This would be a similar to real-world workflows one could use for in-the-field mapping applications.  With this software and data configuration, it would be possible to begin updating data in the field, performing analysis (e.g. slope analysis for areas that could have a greater potential for mudslides), and providing spatial resources to other humanitarian groups in the field.

Ingredients

Software: no better time to try out the new QuantumGIS version, 1.4.0 ‘enceladus.’  It’s a desktop GIS application, and the download and installation process is quick and easy.

Terrain: I decided to use SRTM as a terrain layer and primary base data layer.  Why?  Height information is valuable when combined with vector data.  For example: a road network layered on top of an orthophoto won’t tell you that the road is on a steep slope – and terrain will.

Vector data: OpenStreetMap data was an obvious choice here.  The coverage is pretty good, and it is also being rapidly updated.

Imagery: I thought about downloading some Landsat imagery but took a pass: for an urban application, the medium/low resolution publicly available data isn’t very helpful.  What’s needed is 0.5 meter resolution imagery from the latest generation of sensors, which isn’t yet public at the time of writing.

Process

Here’s a step-by-step overview of the process:

1) Download an install Quantum GIS from www.qgis.org.  No extra instructions needed: this is easy.

2) Find the appropriate SRTM data.  As much as I love working with it, this is my pet peeve with SRTM: it’s available from multiple host sites, with multiple processing levels, and it can be quite a challenge to find the data you need.  Maybe it’s just me, but every time I grab some I’m left thinking about how much easier it could be to access.  In this case I first went to the Consortium for Spatial Information site and downloaded the Google KML link, displayed below. 

Looking at the KML link identifies srtm_21_09 as the file required for Haiti.  Instead of navigating the maze of SRTM sites, I just Googled the filename, which took me here: http://collections.sdsc.edu/dac2/telascience/telascience_data/elevation/cgiar_srtm_v4/tiff/.  I then downloaded the appropriate TIF file.

3) Acquire vector data.  This was very straightforward: I’ve never tried to download or use OpenStreetMap data offline, but fortunately it is a fairly simple process.  I went to www.openstreetmap.org, zoomed into Port-au-Prince, and then selected the Export button at the top.  I exported the data in the OpenStreetMap XML format.

4) The next step is to start assembling the data in QGIS.  I launched the application and loaded the SRTM data.  If you’ve used a desktop GIS application before, QGIS is fairly intuitive.

The image above shows the SRTM data with a MinMax contrast stretch applied.

5) I had to use “Manage Plugins” and load the OpenStreetMap plugin prior to adding the OSM data (Plugins > OpenStreetMap > Load OSM from file).  Now it is possible to view the vectors over the terrain data.

6) Once the OSM data is loaded, we’re ready for field mapping.  Note that it is possible to query the OSM data as well.  The image below shows a query on a hospital, which is identified in the table on the right.

This workflow isn’t very sophisticated, but it does demonstrate the ability to get up and running relatively quickly.  SRTM and OSM data are both invaluable resources – ideal for humanitarian work in disaster areas.  As for the timing: if you know where to get the data, I think the simple example above could be completed in under an hour.  That includes software installation, data downloads, and then assembling the data in a GIS.

Last October I reviewed Google’s new Building Maker application, and just recently I gave it another whirl.  While there are some new technical improvements (freeform polygons, new block options, and six new cities), it’s the concepts and implications of the system that continue to impress me…

3D Feature Extraction in Google's Building Maker

A few more thoughts on the implications:

• One of the best things about the system (and hopefully geospatial vendors are thinking about this as well) is that it’s 100% web-based.  All you require is a browser plug-in.  Compare this with other government or private mid to large-scale mapping efforts: these typically involve setting up local image servers in each office, and then shipping imagery around on hard drives.  Not ideal, but these are the realities when a single raw image can be 1GB in size.  While the geospatial market has a lot of image server solutions, not many organizations are delving into the imagery hosting/warehousing/serving business.  The current model is largely based on setting up your own infrastructure and hosting environment.
• I have a feeling this may represent the beginning a shift in how large/mid-scale mapping is performed.  While Building Maker is a  rudimentary toolset for 3D feature extraction, the idea of delivering browser based tools instead of desktop apps will open up a lot of opportunities.  For one thing, Building Maker is a great proof of concept for web-enabled mapping tools that don’t require thick desktop software installations.
• While a SOCET SET or PRO600 user may find Building Maker tools to be relatively basic, we shouldn’t underestimate the level of complexity in developing such a solution.  Mapping technology pre-dated software, and the commercial tools that are currently available have a high level of sophistication.  Hence, I don’t have any sort of expectation for a web-based replacement anytime soon.  The use of oblique imagery instead of creating some sort of stereo WMS viewer is a clever move by Google though.

Automatic Building Textures

Certainly interested in thoughts on this – and if you haven’t tried it yet, give Building Maker a whirl!