Archive for the ‘Geography’ Category

Coloring US Hacker Bigotry (Test Your Geographic Ignorance)

Thursday, April 27th, 2017

I failed to mention in How Do Hackers Live on $53.57? (‘Hack the Air Force’) that ‘Hack the Air Force’ is limited to hackers in Australia, Canada, New Zealand, and the United States (blue on the following map).

The dreaded North Korean hackers, the omnipresent Russian hackers (of Clinton fame), government associated Chinese hackers, not to mention the financially savvy East European hackers, and many others, are left out of this contest (red on the map).

The US Air Force is “fishing in the shallow end of the cybersecurity talent pool.”

I say this is “a partial cure for geographic ignorance,” because I started with the BlankMap-World4.svg map and proceeded in Gimp to fill in the outlines with appropriate colors.

There are faster map creation methods but going one by one, impressed upon me the need to improve my geographic knowledge!

Restricted U.S. Army Geospatial Intelligence Handbook

Friday, August 26th, 2016

Restricted U.S. Army Geospatial Intelligence Handbook

From the webpage:

This training circular provides GEOINT guidance for commanders, staffs, trainers, engineers, and military intelligence personnel at all echelons. It forms the foundation for GEOINT doctrine development. It also serves as a reference for personnel who are developing doctrine; tactics, techniques, and procedures; materiel and force structure; and institutional and unit training for intelligence operations.

1-1. Geospatial intelligence is the exploitation and analysis of imagery and geospatial information to describe, assess, and visually depict physical features and geographically referenced activities on the Earth. Geospatial intelligence consists of imagery, imagery intelligence, and geospatial information (10 USC 467).

Note. TC 2-22.7 further implements that GEOINT consists of any one or any combination of the following components: imagery, IMINT, or GI&S.

1-2. Imagery is the likeness or presentation of any natural or manmade feature or related object or activity, and the positional data acquired at the same time the likeness or representation was acquired, including: products produced by space-based national intelligence reconnaissance systems; and likenesses and presentations produced by satellites, aircraft platforms, unmanned aircraft vehicles, or other similar means (except that such term does not include handheld or clandestine photography taken by or on behalf of human intelligence collection organizations) (10 USC 467).

1-3. Imagery intelligence is the technical, geographic, and intelligence information derived through the interpretation or analysis of imagery and collateral materials (10 USC 467).

1-4. Geospatial information and services refers to information that identifies the geographic location and characteristics of natural or constructed features and boundaries on the Earth, including: statistical data and information derived from, among other things, remote sensing, mapping, and surveying technologies; and mapping, charting, geodetic data, and related products (10 USC 467).

geospatial-intel-1-460

You may not have the large fixed-wing assets described in this handbook, the “value-added layers” are within your reach with open data.

geospatial-intel-2-460

In localized environments, your value-added layers may be more current and useful than those produced on longer time scales.

Topic maps can support geospatial collations of information along side other views of the same data.

A great opportunity to understand how a modern military force understands and uses geospatial intelligence.

Not to mention testing your ability to recreate that geospatial intelligence without dedicated tools.

Spatial Module in OrientDB 2.2

Tuesday, August 23rd, 2016

Spatial Module in OrientDB 2.2

From the post:

In versions prior to 2.2, OrientDB had minimal support for storing and retrieving GeoSpatial data. The support was limited to a pair of coordinates (latitude, longitude) stored as double in an OrientDB class, with the possibility to create a spatial index against those 2 coordinates in order to speed up a geo spatial query. So the support was limited to Point.
In OrientDB v.2.2 we created a brand new Spatial Module with support for different types of Geometry objects stored as embedded objects in a user defined class

  • Point (OPoint)
  • Line (OLine)
  • Polygon (OPolygon)
  • MultiPoint (OMultiPoint)
  • MultiLine (OMultiline)
  • MultiPolygon (OMultiPlygon)
  • Geometry Collections

Along with those data types, the module extends OrientDB SQL with a subset of SQL-MM functions in order to support spatial data.The module only supports EPSG:4326 as Spatial Reference System. This blog post is an introduction to the OrientDB spatial Module, with some examples of its new capabilities. You can find the installation guide here.

Let’s start by loading some data into OrientDB. The dataset is about points of interest in Italy taken from here. Since the format is ShapeFile we used QGis to export the dataset in CSV format (geometry format in WKT) and import the CSV into OrientDB with the ETL in the class Points and the type geometry field is OPoint.

The enhanced spatial functions for OrientDB 2.2 reminded me of this passage in “Silences and Secrecy: The Hidden Agenda of Cartography in Early Modern Europe:”

Some of the most clear-cut cases of an increasing state concern with control and restriction of map knowledge are associated with military or strategic considerations. In Europe in the sixteenth and seventeenth centuries hardly a year passed without some war being fought. Maps were an object of military intelligence; statesmen and princes collected maps to plan, or, later, to commemorate battles; military textbooks advocated the use of maps. Strategic reasons for keeping map knowledge a secret included the need for confidentiality about the offensive and defensive operations of state armies, the wish to disguise the thrust of external colonization, and the need to stifle opposition within domestic populations when developing administrative and judicial systems as well as the more obvious need to conceal detailed knowledge about fortifications. (reprinted in: The New Nature of Maps: Essays in the History of Cartography, by J.B. Harley: Paul Laxton, John Hopkins, 2001. page 89)

I say “reminded me,” better to say increased my puzzling over the widespread access to geographic data that once upon a time had military value.

Is it the case that “ordinary maps,” maps of streets, restaurants, hotels, etc., aren’t normally imbued (merged?) with enough other information to make them “dangerous?”

If that’s true, the lack of commonly available “dangerous maps” is a disadvantage to emergency and security planners.

You can’t plan for the unknown.

Or to paraphrase Dibert: “Ignorance is not a reliable planning guide.”

How would you cure the ignorance of “ordinary” maps?

PS: While hunting for the quote, I ran across The Power of Maps by Denis Wood; with John Fels. Which has been up-dated: Rethinking the power of maps by Denis Wood; with John Fels and John Krygier. I am now re-reading the first edition and awaiting for the updated version to arrive.

Neither book is a guide to making “dangerous” maps but may awaken in you a sense of the power of maps and map making.

Dodging the Morality Police

Friday, March 25th, 2016

This location-based app helps young Iranians avoid ‘morality police’ by Aleks Buczkowski.

From the post:

Many young Iranians are pretty liberated guys. They like to party and wear fancy clothes but they happened to live in a country where it’s prohibited. There is special police force dedicated to ensuring Iranians follow strict rules on clothing and conduct, called the Gasht-e-Ershad (or Guidance Patrol, commonly known as the “morality police”). Part of their activities include setting up checkpoints around cities and randomly inspecting vehicles driving by.

Now there is a way to avoid the Ershad controls. An anonymous team of Iranian developers have come up with a crowdsource app that allow users marking risky spots on the city map to help others avoid it. Something like Waze but for a much different purpose.

The Gershad app is pretty simple and easy to use. Users can mark where they encounter the “morality police”. The data is added to a database and visualised on a map. The more reports in one place, the bolder the warning on the map. When the number decreases, the alert will fade gradually from the map. Simple as it is.

Sounds quite adaptable to tracking police, FBI agents, narcs, etc. in modern urban environments.

Over time, with enough reports, patterns for police patrols would emerge from the data.

Enjoy!

Searching for Geolocated Posts On YouTube

Sunday, January 3rd, 2016

Searching for Geolocated Posts On YouTube (video) by First Draft News.

Easily the most information filled 1 minutes and 18 seconds of the holiday season!

Illustrates searching for geolocated post to YouTube, despite YouTube not offering that option!

New tool in development may help!

Visit: http://youtube.github.io/geo-search-tool/search.html

Both the video and site are worth a visit!

Don’t forget to check out First Draft News as well!

World Factbook 2015 (paper, online, downloadable)

Wednesday, June 24th, 2015

World Factbook 2015 (GPO)

From the webpage:

The Central Intelligence Agency’s World Factbook provides brief information on the history, geography, people, government, economy, communications, transportation, military, and transnational issues for 267 countries and regions around world.

The CIA’s World Factbook also contains several appendices and maps of major world regions, which are located at the very end of the publication. The appendices cover abbreviations, international organizations and groups, selected international environmental agreements, weights and measures, cross-reference lists of country and hydrographic data codes, and geographic names.

For maps, it provides a country map for each country entry and a total of 12 regional reference maps that display the physical features and political boundaries of each world region. It also includes a pull-out Flags of the World, a Physical Map of the World, a Political Map of the World, and a Standard Time Zones of the World map.

Who should read The World Factbook? It is a great one-stop reference for anyone looking for an expansive body of international data on world statistics, and has been a must-have publication for:

  • US Government officials and diplomats
  • News organizations and researchers
  • Corporations and geographers
  • Teachers, professors, librarians, and students
  • Anyone who travels abroad or who is interested in foreign countries

The print version is $89.00 (U.S.), is 923 pages long and weighs in at 5.75 lb. in paperback.

A convenient and frequently updated alternative is the online CIA World Factbook.

I can’t compare the two versions because I am not going to spend $89.00 for an arm wrecker. 😉

You can also download a copy of the HTML version.

I downloaded and unzipped the file, only to find that the last update was in June, 2014.

That may be updated soon or it may not. I really don’t know.

If you just need background information that is unlikely to change or you want to avoid surveillance on what countries you look at and for how long, download the 2014 HTML version or pony up for the 2015 paper version.

Understanding Map Projections

Thursday, May 28th, 2015

Understanding Map Projections by Tiago Veloso.

From the post:

Few subjects are so controversial – or at least, misunderstood- in cartography as map projections, especially if you’re taking your first steps in this field. And that’s simply because every flat map misrepresents the surface of the Earth in some way. So, in this matter, your work in map-mapping is basically to choose the best projection that suits your needs and reduces the distortion of the most important features you are trying to show/highlight.

But it’s not because you don’t have enough literature about it. There are actually a bunch of great resources and articles that will help you choose the correct projection for your map, so we decided to bring together a quick reference list.

Hope you enjoy it!

I rather like the remark:

…reduces the distortion of the most important features you are trying to show/highlight.

In part because I read it as a concession that all projections are distortions, including those that suit our particular purposes.

I would argue that all maps are at their inception distortions. They never represent every detail of what is being mapped and that implies a process of selective omission. Someone will consider what was omitted important, but it was less important than some other detail to the map maker.

Would the equivalent of projections for topic maps be choice of associations between topics or choices of subjects? Or both?

I lean towards the choice of associations and subjects because graphical rendering of associations creates impressions of the existence and strengths of relationships. Subjects because they are the anchors of the associations.

Speaking of distortion, I would consider any topic map about George H. W. Bush that doesn’t list his war crimes and members of his administration who were also guilty of war crimes as incomplete. There are other opinions on that topic (or at least so I am told).

Suggestions on how to spot “tells” of omission? What can be left out of a map that clues you in that something is missing? Varies from subject to subject but even a rough list would be helpful.

Imagery Processing Pipeline Launches!

Tuesday, April 21st, 2015

Imagery Processing Pipeline Launches!

From the post:

Our imagery processing pipeline is live! You can search the Landsat 8 imagery catalog, filter by date and cloud coverage, then select any image. The image is instantly processed, assembling bands and correcting colors, and loaded into our API. Within minutes you will have an email with a link to the API end point that can be loaded into any web or mobile application.

Our goal is to make it fast for anyone to find imagery for a news story after a disaster, easy for any planner to get the the most recent view of their city, and any developer to pull in thousands of square KM of processed imagery for their precision agriculture app. All directly using our API

There are two ways to get started: via the imagery browser fetch.astrodigital.com, or directly via the the Search and Publish APIs. All API documentation is on astrodigital.com/api. You can either use the API to programmatically pull imagery though the pipeline or build your own UI on top of the API, just like we did.

The API provides direct access to more than 300TB of satellite imagery from Landsat 8. Early next year we’ll make our own imagery available once our own Landmapper constellation is fully commissioned.

Hit us up @astrodigitalgeo or sign up at astrodigital.com to follow as we build. Huge thanks to our partners at Development Seed who is leading our development and for the infinitively scalable API from Mapbox.

If you are interested in Earth images, you really need to check this out!

I haven’t tried the API but did get a link to an image of my city and surrounding area.

Definitely worth a long look!

Wiki New Zealand

Tuesday, February 24th, 2015

Wiki New Zealand

From the about page:

It’s time to democratise data. Data is a language in which few are literate, and the resulting constraints at an individual and societal level are similar to those experienced when the proportion of the population able to read was small. When people require intermediaries before digesting information, the capacity for generating insights is reduced.

To democratise data we need to put users at the centre of our models, we need to design our systems and processes for users of data, and we need to realise that everyone can be a user. We will all win when everyone can make evidence-based decisions.

Wiki New Zealand is a charity devoted to getting people to use data about New Zealand.

We do this by pulling together New Zealand’s public sector, private sector and academic data in one place and making it easy for people to use in simple graphical form for free through this website.

We believe that informed decisions are better decisions. There is a lot of data about New Zealand available online today, but it is too difficult to access and too hard to use. We think that providing usable, clear, digestible and unbiased information will help you make better decisions, and will lead to better outcomes for you, for your community and for New Zealand.

We also believe that by working together we can build the most comprehensive, useful and accurate representation of New Zealand’s situation and performance: the “wiki” part of the name means “collaborative website”. Our site is open and free to use for everyone. Soon, anyone will be able to upload data and make graphs and submit them through our auditing process. We are really passionate about engaging with domain and data experts on their speciality areas.

We will not tell you what to think. We present topics from multiple angles, in wider contexts and over time. All our data is presented in charts that are designed to be compared easily with each other and constructed with as little bias as possible. Our job is to present data on a wide range of subjects relevant to you. Your job is to draw your own conclusions, develop your own opinions and make your decisions.

Whether you want to make a business decision based on how big your market is, fact-check a newspaper story, put together a school project, resolve an argument, build an app based on clean public licensed data, or just get to know this country better, we have made this for you.

Isn’t New Zealand a post-apocalypse destination? Thinking however great it may be now, the neighborhood is going down when all the post-apocalypse folks arrive. Something on the order of Mr. Rogers Neighborhood to Max Max Beyond Thunderdome. 😉

Hopefully, if there is an apocalypse, it will happen quickly enough to prevent a large influx of undesirables into New Zealand.

I first saw this in a tweet by Neil Saunders.

Geojournalism.org

Saturday, February 7th, 2015

Geojournalism.org

From the webpage:

Geojournalism.org provides online resources and training for journalists, designers and developers to dive into the world of data visualization using geographic data.

From the about page:

Geojournalism.org is made for:

Journalists

Reporters, editors and other professionals involved on the noble mission of producing relevant news for their audiences can use Geojournalism.org to produce multimedia stories or simple maps and data visualization to help creating context for complex environmental issues

Developers

Programmers and geeks using a wide variety of languages and tools can drink on the vast knowledge of our contributors. Some of our tutorials explore open source libraries to make maps, infographics or simply deal with large geographical datasets

Designers

Graphic designers and experts on data visualizations find in the Geojournalism.org platform a large amount of resources and tips. They can, for example, improve their knowledge on the right options for coloring maps or how to set up simple charts to depict issues such as deforestation and climate change

It is one thing to have an idea or even a story and quite another to communicate it effectively to a large audience. Geojournalism is designed as a community site that will help you communicate geophysical data to a non-technical audience.

I think it is clear that most governments are shy about accurate and timely communication with their citizens. Are you going to be one of those who fills in the gaps? Geojournalism.org is definitely a site you will be needing.

MrGeo (MapReduce Geo)

Wednesday, January 21st, 2015

MrGeo (MapReduce Geo)

From the webpage:

MrGeo was developed at the National Geospatial-Intelligence Agency (NGA) in collaboration with DigitalGlobe. The government has “unlimited rights” and is releasing this software to increase the impact of government investments by providing developers with the opportunity to take things in new directions. The software use, modification, and distribution rights are stipulated within the Apache 2.0 license.

MrGeo (MapReduce Geo) is a geospatial toolkit designed to provide raster-based geospatial capabilities that can be performed at scale. MrGeo is built upon the Hadoop ecosystem to leverage the storage and processing of hundreds of commodity computers. Functionally, MrGeo stores large raster datasets as a collection of individual tiles stored in Hadoop to enable large-scale data and analytic services. The co-location of data and analytics offers the advantage of minimizing the movement of data in favor of bringing the computation to the data; a more favorable compute method for Geospatial Big Data. This framework has enabled the servicing of terabyte scale raster databases and performed terrain analytics on databases exceeding hundreds of gigabytes in size.

The use cases sound interesting:

Exemplar MrGeo Use Cases:

  • Raster Storage and Provisioning: MrGeo has been used to store, index, tile, and pyramid multi-terabyte scale image databases. Once stored, this data is made available through simple Tiled Map Services (TMS) and or Web Mapping Services (WMS).
  • Large Scale Batch Processing and Serving: MrGeo has been used to pre-compute global 1 ArcSecond (nominally 30 meters) elevation data (300+ GB) into derivative raster products : slope, aspect, relative elevation, terrain shaded relief (collectively terabytes in size)
  • Global Computation of Cost Distance: Given all pub locations in OpenStreetMap, compute 2 hour drive times from each location. The full resolution is 1 ArcSecond (30 meters nominally)
  • I wonder if you started war gaming attacks on well known cities and posting maps on how the attacks could develop if that would be covered under free speech? Assuming your intent was to educate the general populace about areas that are more dangerous than others in case of a major incident.

    I first saw this in a tweet by Marin Dimitrov.

    Getty Thesaurus of Geographic Names (TGN)

    Friday, August 22nd, 2014

    Getty Thesaurus of Geographic Names Released as Linked Open Data by James Cuno.

    From the post:

    We’re delighted to announce that the Getty Research Institute has released the Getty Thesaurus of Geographic Names (TGN)® as Linked Open Data. This represents an important step in the Getty’s ongoing work to make our knowledge resources freely available to all.

    Following the release of the Art & Architecture Thesaurus (AAT)® in February, TGN is now the second of the four Getty vocabularies to be made entirely free to download, share, and modify. Both data sets are available for download at vocab.getty.edu under an Open Data Commons Attribution License (ODC BY 1.0).

    What Is TGN?

    The Getty Thesaurus of Geographic Names is a resource of over 2,000,000 names of current and historical places, including cities, archaeological sites, nations, and physical features. It focuses mainly on places relevant to art, architecture, archaeology, art conservation, and related fields.

    TGN is powerful for humanities research because of its linkages to the three other Getty vocabularies—the Union List of Artist Names, the Art & Architecture Thesaurus, and the Cultural Objects Name Authority. Together the vocabularies provide a suite of research resources covering a vast range of places, makers, objects, and artistic concepts. The work of three decades, the Getty vocabularies are living resources that continue to grow and improve.

    Because they serve as standard references for cataloguing, the Getty vocabularies are also the conduits through which data published by museums, archives, libraries, and other cultural institutions can find and connect to each other.

    A resource where you could loose some serious time!

    Try this entry for London.

    Or Paris.

    Bear in mind the data that underlies this rich display is now available for free downloading.

    Map Distortion!

    Monday, June 9th, 2014

    Mercator: Extreme by Drew Roos.

    The link takes you to a display setting the pole to Atlanta, GA (near my present location).

    You should search for a location near you for the maximum impact of the display. Intellectually I have known about map distortion but seeing it for your location, that’s something different.

    Highly interactive and strongly recommended!

    Makes me wonder about visual displays of other map distortions. Not just well known map projections but policy distortions as well.

    Take for example a map that sizes countries by the amount of aid for the United States divided by their population.

    Are there any map artists in the audience?

    I first saw this in a tweet by Lincoln Mullen.

    Twitter User Targeting Data

    Sunday, May 11th, 2014

    Geotagging One Hundred Million Twitter Accounts with Total Variation Minimization by Ryan Compton, David Jurgens, and, David Allen.

    Abstract:

    Geographically annotated social media is extremely valuable for modern information retrieval. However, when researchers can only access publicly-visible data, one quickly finds that social media users rarely publish location information. In this work, we provide a method which can geolocate the overwhelming majority of active Twitter users, independent of their location sharing preferences, using only publicly-visible Twitter data.

    Our method infers an unknown user’s location by examining their friend’s locations. We frame the geotagging problem as an optimization over a social network with a total variation-based objective and provide a scalable and distributed algorithm for its solution. Furthermore, we show how a robust estimate of the geographic dispersion of each user’s ego network can be used as a per-user accuracy measure, allowing us to discard poor location inferences and control the overall error of our approach.

    Leave-many-out evaluation shows that our method is able to infer location for 101,846,236 Twitter users at a median error of 6.33 km, allowing us to geotag roughly 89\% of public tweets.

    If 6.33 km sounds like a lot of error, check out NUKEMAP by Alex Wellerstein.

    GeoCanvas

    Saturday, April 5th, 2014

    Synthicity Releases 3D Spatial Data Visualization Tool, GeoCanvas by Dean Meyers.

    From the post:

    Synthicity has released a free public beta version of GeoCanvas, its 3D spatial data visualization tool. The software provides a streamlined toolset for exploring geographic data, lowering the barrier to learning and using geographic information systems.

    GeoCanvas is not limited to visualizing parcels in cities. By supporting data formats such as the widely available shapefile for spatial geometry and text files for attribute data, it opens the possibility of rapid 3D spatial data visualization for a wide range of uses and users. The software is expected to be a great addition to the toolkits of students, researchers, and practitioners in fields as diverse as data science, geography, planning, real estate analysis, and market research. A set of video tutorials explaining the basic concepts and a range of examples have been made available to showcase the possibilities.

    The public beta version of GeoCanvas is available as a free download from www.synthicity.com.

    Well, rats! I haven’t installed a VM with Windows 7/8 or Max OS X 10.8 or later.

    Sounds great!

    Comments from actual experience?

    Introducing Google Maps Gallery…

    Thursday, February 27th, 2014

    Introducing Google Maps Gallery: Unlocking the World’s Maps by Jordan Breckenridge.

    From the post:

    Governments, nonprofits and businesses have some of the most valuable mapping data in the world, but it’s often locked away and not accessible to the public. With the goal of making this information more readily available to the world, today we’re launching Google Maps Gallery, a new way for organizations to share and publish their maps online via Google Maps Engine.

    Google Map Gallery

    Maps Gallery works like an interactive, digital atlas where anyone can search for and find rich, compelling maps. Maps included in the Gallery can be viewed in Google Earth and are discoverable through major search engines, making it seamless for citizens and stakeholders to access diverse mapping data, such as locations of municipal construction projects, historic city plans, population statistics, deforestation changes and up-to-date emergency evacuation routes. Organizations using Maps Gallery can communicate critical information, build awareness and inform the public at-large.

    A great site as you would expect from Google.

    I happened upon US Schools with GreatSchools Ratings. Created by GreatSchools.org.

    There has been a rash of 1950’s style legislative efforts this year in the United States, seeking to permit business to discriminate on the basis of their religious beliefs. Recalling the days when stores sported “We Reserve the Right to Refuse Service to Anyone” signs.

    I remember those signs and how they were used.

    With that in mind, scroll around the GreatSchools Rating may and tell me what you think the demographics of non-rated schools look like?

    That’s what I thought too.

    Build your own [Secure] Google Maps…

    Tuesday, February 11th, 2014

    Build your own Google Maps (and more) with GeoServer on OpenShift by Steven Citron-Pousty.

    From the post:

    Greetings Shifters! Today we are going to continue in our spatial series and bring up Geoserver on OpenShift and connect it to our PostGIS database. By the end of the post you will have your own map tile server OR KML (to show on Google Earth) or remote GIS server.

    The team at Geoserver has put together a nice short explanation of the geoserver and then a really detailed list. If you want commercial support, Boundless will give you a commercial release and/or support for all your corporate needs. Today though I am only going to focus on the FOSS bits.

    From the GeoServer site:

    GeoServer allows you to display your spatial information to the world. Implementing the Web Map Service (WMS) standard, GeoServer can create maps in a variety of output formats. OpenLayers, a free mapping library, is integrated into GeoServer, making map generation quick and easy. GeoServer is built on Geotools, an open source Java GIS toolkit.

    There is much more to GeoServer than nicely styled maps, though. GeoServer also conforms to the Web Feature Service (WFS) standard, which permits the actual sharing and editing of the data that is used to generate the maps. Others can incorporate your data into their websites and applications, freeing your data and permitting greater transparency.

    I added “[Secure]” to the title, assuming that you will not hand over data to the NSA about yourself or your maps. I can’t say that for everyone that offers mapping services on the WWW.

    Depending on how much security you need, certainly develop on OpenShift but I would deploy on shielded and physically secure hardware. Depends on your appetite for risk.

    Geocode the world…

    Thursday, October 10th, 2013

    Geocode the world with the new Data Science Toolkit by Pete Warden.

    From the post:

    I’ve published a new version of the Data Science Toolkit, which includes David Blackman’s awesome TwoFishes city-level geocoder. Largely based on data from the Geonames project, the biggest improvement is that the Google-style geocoder now handles millions of places around the world in hundreds of languages:

    Who or what do you want to locate? 😉

    Kindred Britain

    Monday, August 26th, 2013

    Kindred Britian by Nicholas Jenkins, Elijah Meeks and Scott Murray.

    From the website:

    Kindred Britain is a network of nearly 30,000 individuals — many of them iconic figures in British culture — connected through family relationships of blood, marriage, or affiliation. It is a vision of the nation’s history as a giant family affair.

    A quite remarkable resource.

    Family relationships connecting people, a person’s relationship to geographic locations and a host of other associated details for 30,000 people await you!

    From the help page:

    ESSAYS

    Originating Kindred Britain by Nicholas Jenkins

    Developing Kindred Britain by Elijah Meeks and Karl Grossner

    Designing Kindred Britain by Scott Murray

    Kindred Britain: Statistics by Elijah Meeks

    GENERAL INFORMATION

    User’s Guide by Hannah Abalos and Nicholas Jenkins

    FAQs

    Glossary by Hannah Abalos and Emma Townley-Smith

    Acknowledgements

    Terms of Use

    If you notice a problem with the site or have a question or copyright concern, please contact us at kindredbritain@stanford.edu

    An acronym that may puzzle you: ODNB – Oxford Dictionary of National Biography.

    In Developing Kindred Britain you will learn Kindred Britain has no provision for reader annotation or contribution of content.

    Given a choice between the rich presentation and capabilities of Kindred Britain, which required several technical innovations and less capabilities but reader annotation, I would always choose the former over the latter.

    You should forward the link to Kindred Britain to anyone working on robust exploration and display of data, academic or otherwise.

    Visualizing Web Scale Geographic Data…

    Wednesday, July 10th, 2013

    Visualizing Web Scale Geographic Data in the Browser in Real Time: A Meta Tutorial by Sean Murphy.

    From the post:

    Visualizing geographic data is a task many of us face in our jobs as data scientists. Often, we must visualize vast amounts of data (tens of thousands to millions of data points) and we need to do so in the browser in real time to ensure the widest-possible audience for our efforts and we often want to do this leveraging free and/or open software.

    Luckily for us, Google offered a series of fascinating talks at this year’s (2013) IO that show one particular way of solving this problem. Even better, Google discusses all aspects of this problem: from cleaning the data at scale using legacy C++ code to providing low latency yet web-scale data storage and, finally, to rendering efficiently in the browser. Not surprisingly, Google’s approach highly leverages **alot** of Google’s technology stack but we won’t hold that against them.

    (…)

    Sean sets the background for two presentations:

    All the Ships in the World: Visualizing Data with Google Cloud and Maps (36 minutes)

    and,

    Google Maps + HTML5 + Spatial Data Visualization: A Love Story (60 minutes) (source code: https://github.com/brendankenny)

    Both are well worth your time.

    Building A Visual Planetary Time Machine

    Wednesday, June 12th, 2013

    Building A Visual Planetary Time Machine by by Randy Sargent, Google/Carnegie Mellon University; Matt Hancher and Eric Nguyen, Google; and Illah Nourbakhsh, Carnegie Mellon University.

    From the post:

    When a societal or scientific issue is highly contested, visual evidence can cut to the core of the debate in a way that words alone cannot — communicating complicated ideas that can be understood by experts and non-experts alike. After all, it took the invention of the optical telescope to overturn the idea that the heavens revolved around the earth.

    Last month, Google announced a zoomable and explorable time-lapse view of our planet. This time-lapse Earth enables you explore the last 29 years of our planet’s history — from the global scale to the local scale, all across the planet. We hope this new visual dataset will ground debates, encourage discovery, and shift perspectives about some of today’s pressing global issues.

    This project is a collaboration between Google’s Earth Engine team, Carnegie Mellon University’s CREATE Lab, and TIME Magazine — using nearly a petabyte of historical record from USGS’s and NASA’s Landsat satellites. And in this post, we’d like to give a little insight into the process required to build this time-lapse view of our planet.

    Great imaging and a benchmark to compare future progress in this area.

    Within three to five (3-5) years, this should be doable as senior CS project. Graduate students and advanced hackers will be using higher resolution “spy” satellite images.

    From five to eight (5-8) years, software packages appear for the average consumer, processing on the local “grid.”

    From eight to ten (8-10) years, mostly due to the long product cycle, appears in MS Office XXI. 😉

    If not sooner!

    JQVMAP

    Saturday, June 8th, 2013

    JQVMAP

    From the webpage:

    JQVMap is a jQuery plugin that renders Vector Maps. It uses resizable Scalable Vector Graphics (SVG) for modern browsers like Firefox, Safari, Chrome, Opera and Internet Explorer 9. Legacy support for older versions of Internet Explorer 6-8 is provided via VML.

    Whatever your source of data, cellphone location data, user observation, etc., rendering it to a geographic display may be useful.

    CLAVIN [Geotagging – Some Proofing Required]

    Sunday, May 26th, 2013

    CLAVIN

    From the webpage:

    CLAVIN (*Cartographic Location And Vicinity INdexer*) is an open source software package for document geotagging and geoparsing that employs context-based geographic entity resolution. It combines a variety of open source tools with natural language processing techniques to extract location names from unstructured text documents and resolve them against gazetteer records. Importantly, CLAVIN does not simply “look up” location names; rather, it uses intelligent heuristics in an attempt to identify precisely which “Springfield” (for example) was intended by the author, based on the context of the document. CLAVIN also employs fuzzy search to handle incorrectly-spelled location names, and it recognizes alternative names (e.g., “Ivory Coast” and “Côte d’Ivoire”) as referring to the same geographic entity. By enriching text documents with structured geo data, CLAVIN enables hierarchical geospatial search and advanced geospatial analytics on unstructured data.

    See http://clavin.bericotechnologies.com/ for an online demo, videos and other materials.

    Your mileage may vary.

    I used a quote from today’s New York Times (Rockets Hit Hezbollah Stronghold in Lebanon):

    An ongoing battle in the Syrian town of Qusair on the Lebanese border has laid bare Hezbollah’s growing role in the Syrian conflict. The Iranian-backed militia and Syrian troops launched an offensive against the town last weekend. After dozens of Hezbollah fighters were killed in Qusair over the past week and buried in large funerals in Lebanon, Hezbollah could no longer play down its involvement.

    Col. Abdul-Jabbar al-Aqidi, commander of the Syrian rebels’ Military Council in Aleppo, appeared in a video this week while apparently en route to Qusair, in which he threatened to strike in Beirut’s southern suburbs in retaliation for Hezbollah’s involvement in Syria.

    “We used to say before, ‘We are coming Bashar.’ Now we say, ‘We are coming Bashar and we are coming Hassan Nasrallah,'” he said, in reference to Hezbollah’s leader.

    “We will strike at your strongholds in Dahiyeh, God willing,” he said, using the Lebanese name for Hezbollah’s power center in southern Beirut. The video was still online on Youtube on Sunday.

    Hezbollah lawmaker Ali Ammar said the incident targeted coexistence between the Lebanese and claimed the U.S. and Israel want to return Lebanon to the years of civil war. “They want to throw Lebanon backward into the traps of civil wars that we left behind,” he told reporters. “We will not go backward.”

    The results from CLAVIN:

    Locations Extracted and Resolved From Text

    ID Name Lat, Lon Country Code #
    272103 Lebanon 33.83333, 35.83333 LB 3
    6951366 Lebanese 44.49123, 26.0877 RO 3
    276781 Beirut 33.88894, 35.49442 LB 2
    162037 Dahiyeh 38.19023, 57.00984 TM 1
    6252001 U.S. 39.76, -98.5 US 1
    103089 Qusair 25.91667, 40.45 SA 1
    163843 Syria 35, 38 SY 1
    163843 Syrian 35, 38 SY 1
    294640 Israel 31.5, 34.75 IL 1
    170062 Aleppo 36.25, 37.5 SY 1

    (The highlight added to show incorrect resolutions.)

    FYI:

    RO = Romania

    SA = Saudia Arabia

    TM = Turkmenistan

    Plus “Qusair” appears twice in the quoted text.

    For the ten locations mentioned a seventy (70%) percent accuracy rate.

    Better than the average American but proofing is still an essential step in editorial workflow.

    I first saw this in Pete Warden’s Five short links.

    Geography of hate against gays, races, and the disabled

    Wednesday, May 15th, 2013

    Geography of hate against gays, races, and the disabled by Nathan Yau.

    Hate Map

    Nathan reports on the work of Floating Sheep who relied on 150,000 tags to create this map.

    More details at Nathan’s site but as Nathan says, read the FAQ before you get too torqued about the map.

    If nothing else, this should be a good lesson in the choices made collecting and mapping “objective” data (the tweets) and what questions you should ask about that process.

    I found it interesting that the sea coast along the Gulf of Mexico seemed to have less hate.

    How would you defend the choices you make when making a topic map?

    Some information, that is important to someone will have to be left out. Was that out of religious, political, social or ethnic bias?

    You can’t avoid that sort of question but you can be comfortable with your own answers should it arise.

    My stock response is:

    “The paying client is happy with the map. Become a paying client and you can be map happy too.”

    The Map Myth of Sandy Island

    Saturday, May 11th, 2013

    The Map Myth of Sandy Island by Rebecca Maxwell.

    From the post:

    Sandy Island has long appeared on maps dating back to the early twentieth century. This island was supposedly located in the Pacific Ocean northwest of Australia in the Coral Sea. It first appeared on an edition of a British admiralty map back in 1908 proving that Sandy Island had been discovered by the French in 1876. Even modern maps, like the General Bathymetic Chart of the Oceans (the British Oceanopgraphic Dat Centre issued an errata about Sandy Island) and Google Earth, show the presence of an island at its coordinates. Sandy Island is roughly the size of Manhattan; it is about three miles wide and fifteen miles long. However, there is only one problem. The island does not actually exist.

    Back in October 2012, an Australian research ship undiscovered the island. The ship, called the Southern Surveyor, was led by Maria Seton, a scientist from the University of Sydney. The purpose of the twenty-five-day expedition was to gather information about tectonic activity, map the sea floor, and gather rock samples from the bottom. The scientific data that they had, including the General Bathymetic Chart of the Oceans, indicated the presence of Sandy Island halfway between Australia and the island of New Caledonia, a French possession. The crew began to get suspicious, however, when the chart from the ship’s master only showed open water. Plus, Google Earth only showed a dark blob where it should have been.

    When the ship arrived at Sandy Island’s supposed coordinates, they found nothing but ocean a mile deep. One of the ship’s crewmembers, Steven Micklethwaite, said that they all had a good laugh at Google’s expense as they sailed through the island. The crew was quick to make their findings known. The story originally appeared in the Sydney Morning Herald and prompted a large amount of controversy. Cartographers were the most puzzled of all. Many wondered whether the island had ever existed or if it had been eroded away by the ocean waves over the years. Others wondered if the island mysteriously disappeared into the ocean like the legendary city of Atlantis. An “obituary” for Sandy Island, reporting the findings, was published in Eos, Transactions of the Geophysical Union in April of 2013.

    Rebecca details the discovered/undiscovered history of Sandy Island in rich detail.

    It’s a great story and you should treat yourself by reading it.

    My only disagreement with Rebecca comes when she writes:

    Maps are continually changing and modern maps still contain a human element that is vulnerable to mistakes.

    On the contrary, maps, even modern ones, are wholly human constructs.

    Not just the mistakes but the degree of accuracy, the implicit territorial or political claims, what is interesting enough to record, etc., are all human choices in production.

    To say nothing of humans on the side of reading/interpretation as well.

    If there were no sentient creatures to read it, would a map have any meaning?

    Largest Coffee Table Book

    Wednesday, May 8th, 2013

    Largest Atlas in the World Created using ArcGIS by Caitlin Dempsey.

    From the post:

    Earth Platinum, the largest atlas ever printed, was released in February 2012 by Millennium House, Australia. Only 31 copies of the 330 pound, leather-bound book exist and each are priced at $100,000. The book measures 6ft by 9ft and has been recognized by Chris Sheedy of the Guinness Book of World Records as the largest atlas in existence. The book contains 128 pages and requires at least two hands, or in some case multiple people, to turn the pages.

    Earth Platinum has surpassed the previous holder of the world record for largest atlas, the famous Klencke Atlas (which measures about 5′ 9″ by 6′ 3″ when opened). The Klencke Atlas is housed in the Antiquarian Mapping Division of the British Library in London and held the title for largest atlas worldwide from 1660 until the publication of Earth Platinum. Published as a one-off over 350 years ago, the Klencke Atlas is reported to contain all geographical knowledge of that time, just as Earth Platinum does today.

    Amazon doesn’t have it listed so I can’t say if you get a discount and/or free shipping or both. 😉

    Interesting but only as a publishing oddity.

    I would rather have a digital version that is a geographic interface into a general knowledge topic map.

    gvSIG

    Saturday, March 30th, 2013

    gvSIG

    I encountered the gvSIG site while tracking down the latest release of i3Geo.

    From its mission statement:

    The gvSIG project was born in 2004 within a project that consisted in a full migration of the information technology systems of the Regional Ministry of Infrastructure and Transport of Valencia (Spain), henceforth CIT, to free software. Initially, It was born with some objectives according to CIT needs. These objectives were expanded rapidly because of two reasons principally: on the one hand, the nature of free software, which greatly enables the expansion of technology, knowledge, and lays down the bases on which to establish a community, and, on the other hand, a project vision embodied in some guidelines and a plan appropriate to implement it.

    Some of the software projects you will find at gvSIG are:

    gvSIG Desktop

    gvSIG is a Geographic Information System (GIS), that is, a desktop application designed for capturing, storing, handling, analyzing and deploying any kind of referenced geographic information in order to solve complex management and planning problems. gvSIG is known for having a user-friendly interface, being able to access the most common formats, both vector and raster ones. It features a wide range of tools for working with geographic-like information (query tools, layout creation, geoprocessing, networks, etc.), which turns gvSIG into the ideal tool for users working in the land realm.

    gvSIG Mobile

    gvSIG Mobile is a Geographic Information System (GIS) aimed at mobile devices, ideal for projects that capture and update data in the field. It’s known for having a user-friendly interface, being able to access the most common formats and a wide range of GIS and GPS tools which are ideal for working with geographic information.

    gvSIG Mobile aims at broadening gvSIG Desktop execution platforms to a range of mobile devices, in order to give an answer to the needings of a growing number of mobile solutions users, who wish to use a GIS on different types of devices.

    So far, gvSIG Mobile is a Geographic Information System, as well as a Spatial Data Infrastructures client for mobile devices. Such a client is also the first one licensed under open source.

    I3Geo

    i3Geo is an application for the development of interactive web maps. It integrates several open source applications into a single development platform, mainly Mapserver and OpenLayers. Developed in PHP and Javascript, it has functionalities that allows the user to have better control over the map output, allowing to modify the legend of layers, to apply filters, to perform analysis, etc.

    i3Geo is completely customizable and can be tailor to the different users using the interactive map. Furthermore, the spatial data is organized in a catalogue that offers online access services such as WMS, WFS, KML or the download of files.

    i3Geo was developed by the Ministry of the Environment of Brazil and it is actually part of the Brazilian Public Software Portal.

    gvSIG Educa

    What is gvSIG Educa?

    “If I can’t picture it, I can’t understand it (A. Einstein)”

    gvSIG Educa is a customization of the gvSIG Desktop Open Source GIS, adapted as a tool for the education of issues that have a geographic component.

    The aim of gvSIG Educa is to provide educators with a tool that helps students to analyse and understand space, and which can be adapted to different levels or education systems.

    gvSIG Educa is not only useful for the teaching of geographic material, but can also be used for learning any subject that contains a spatial component such as history, economics, natural science, sociology…

    gvSIG Educa facilitates learning by letting students interact with the information, by adding a spatial component to the study of the material, and by facilitating the assimilation of concepts through visual tools such as thematic maps.

    gvSIG Educa provides analysis tools that help to understand spatial relationships.

    Definitely a site to visit if you are interested in open source GIS software and/or projects.

    i3Geo

    Saturday, March 30th, 2013

    i3Geo

    From the homepage:

    i3Geo is an application for the development of interactive web maps. It integrates several open source applications into a single development platform, mainly Mapserver and OpenLayers. Developed in PHP and Javascript, it has functionalities that allows the user to have better control over the map output, allowing to modify the legend of layers, to apply filters, to perform analysis, etc.

    i3Geo is completely customizable and can be tailor to the different users using the interactive map. Furthermore, the spatial data is organized in a catalogue that offers online access services such as WMS, WFS, KML or the download of files.

    i3Geo was developed by the Ministry of the Environment of Brazil and it is actually part of the Brazilian Public Software Portal.

    I followed an announcement about i3Geo 4.7 being available when the line “…an application for the development of interactive web maps,” caught my eye.

    Features include:

    • Basic display: fix zoom, zoom by rectangle, panning, etc.
    • Advanced display: locator by attribute, zoom to point, zoom by geographical area, zoom by selection, zoom to layer
    • Integrated display: Wikipedia, GoogleMaps, Panoramio and Confluence
    • Integration with the OpenLayers, GoogleMaps and GoogleEarth APIs
    • Loading of WMS, KML, GeoRSS, shapefile, GPX and CSV layers
    • Management of independent databases
    • Layer catalog management system
    • Management of layers in maps: Change of the layers order, opacity change, title change, filters, thematic classification, legend and symbology changing
    • Analysis tools: buffers, regular grids, points distribution analysis, layer intersection, centroid calculation, etc.
    • Digitalization: vector editing that allows to create new geometries or edit xisting data.
    • Superposition of existing data at the data of the Google Maps and GoogleEarth catalogs.

    Unless you want to re-invent mapping software, this could be quite useful for location relevant topic map data.

    I first saw this at New final version of i3Geo available: i3Geo 4.7.

    User evaluation of automatically generated keywords and toponyms… [of semantic gaps]

    Tuesday, January 22nd, 2013

    User evaluation of automatically generated keywords and toponyms for geo-referenced images by Frank O. Ostermann, Martin Tomko, Ross Purves. (Ostermann, F. O., Tomko, M. and Purves, R. (2013), User evaluation of automatically generated keywords and toponyms for geo-referenced images. J. Am. Soc. Inf. Sci.. doi: 10.1002/asi.22738)

    Abstract:

    This article presents the results of a user evaluation of automatically generated concept keywords and place names (toponyms) for geo-referenced images. Automatically annotating images is becoming indispensable for effective information retrieval, since the number of geo-referenced images available online is growing, yet many images are insufficiently tagged or captioned to be efficiently searchable by standard information retrieval procedures. The Tripod project developed original methods for automatically annotating geo-referenced images by generating representations of the likely visible footprint of a geo-referenced image, and using this footprint to query spatial databases and web resources. These queries return raw lists of potential keywords and toponyms, which are subsequently filtered and ranked. This article reports on user experiments designed to evaluate the quality of the generated annotations. The experiments combined quantitative and qualitative approaches: To retrieve a large number of responses, participants rated the annotations in standardized online questionnaires that showed an image and its corresponding keywords. In addition, several focus groups provided rich qualitative information in open discussions. The results of the evaluation show that currently the annotation method performs better on rural images than on urban ones. Further, for each image at least one suitable keyword could be generated. The integration of heterogeneous data sources resulted in some images having a high level of noise in the form of obviously wrong or spurious keywords. The article discusses the evaluation itself and methods to improve the automatic generation of annotations.

    An echo of Steve Newcomb’s semantic impedance appears at:

    Despite many advances since Smeulders et al.’s (2002) classic paper that set out challenges in content-based image retrieval, the quality of both nonspecialist text-based and content-based image retrieval still appears to lag behind the quality of specialist text retrieval, and the semantic gap, identified by Smeulders et al. as a fundamental issue in content-based image retrieval, remains to be bridged. Smeulders defined the semantic gap as

    the lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data have for a user in a given situation. (p. 1353)

    In fact, text-based systems that attempt to index images based on text thought to be relevant to an image, for example, by using image captions, tags, or text found near an image in a document, suffer from an identical problem. Since text is being used as a proxy by an individual in annotating image content, those querying a system may or may not have similar worldviews or conceptualizations as the annotator. (emphasis added)

    That last sentence could have come out of a topic map book.

    Curious what you make of the author’s claim that spatial locations provide an “external context” that bridges the “semantic gap?”

    If we all use the same map of spatial locations, are you surprised by the lack of a “semantic gap?”

    Maps in R: Plotting data points on a map

    Tuesday, January 15th, 2013

    Maps in R: Plotting data points on a map by Max Marchi.

    From the post:

    In the introductory post of this series I showed how to plot empty maps in R.

    Today I’ll begin to show how to add data to R maps. The topic of this post is the visualization of data points on a map.

    Max continues this series with datasets from airports in Europe and demonstrates how to map the airports to geographic locations. He also represents the airports with icons that correspond to their traffic statistics.

    Useful principles for any data set with events that can be plotted against geographic locations.

    Parades, patrols, convoys, that sort of thing.