Archive for the ‘NOAA’ Category

NOAA weather data – Valuing Open Data – Guessing – History Repeats

Sunday, April 26th, 2015

Tech titans ready their clouds for NOAA weather data by Greg Otto.

From the post:

It’s fitting that the 20 terabytes of data the National Oceanic and Atmospheric Administration produces every day will now live in the cloud.

The Commerce Department took a step Tuesday to make NOAA data more accessible as Commerce Secretary Penny Pritzker announced a collaboration among some of the country’s top tech companies to give the public a range of environmental, weather and climate data to access and explore.

Amazon Web Services, Google, IBM, Microsoft and the Open Cloud Consortium have entered into a cooperative research and development agreement with the Commerce Department that will push NOAA data into the companies’ respective cloud platforms to increase the quantity of and speed at which the data becomes publicly available.

“The Commerce Department’s data collection literally reaches from the depths of the ocean to the surface of the sun,” Pritzker said during a Monday keynote address at the American Meteorological Society’s Washington Forum. “This announcement is another example of our ongoing commitment to providing a broad foundation for economic growth and opportunity to America’s businesses by transforming the department’s data capabilities and supporting a data-enabled economy.”

According to Commerce, the data used could come from a variety of sources: Doppler radar, weather satellites, buoy networks, tide gauges, and ships and aircraft. Commerce expects this data to launch new products and services that could benefit consumer goods, transportation, health care and energy utilities.

The original press release has this cheery note on the likely economic impact of this data:

So what does this mean to the economy? According to a 2013 McKinsey Global Institute Report, open data could add more than $3 trillion in total value annually to the education, transportation, consumer products, electricity, oil and gas, healthcare, and consumer finance sectors worldwide. If more of this data could be efficiently released, organizations will be able to develop new and innovative products and services to help us better understand our planet and keep communities resilient from extreme events.

Ah, yes, that would be the Open data: Unlocking innovation and performance with liquid information, on which the summary page says:

Open data can help unlock $3 trillion to $5 trillion in economic value annually across seven sectors.

But you need to read the full report (PDF) in order to find footnote 3 on “economic value:”

3. Throughout this report we express value in terms of annual economic surplus in 2013 US dollars, not the discounted value of future cash flows; this valuation represents estimates based on initiatives where open data are necessary but not sufficient for realizing value. Often, value is achieved by combining analysis of open and proprietary information to identify ways to improve business or government practices. Given the interdependence of these factors, we did not attempt to estimate open data’s relative contribution; rather, our estimates represent the total value created.

That is a disclosure that the estimate of $3 to $5 trillion is a guess and/or speculation.

Odd how the guess/speculation disclosure drops out of the Commerce Department press release and when it gets to Greg’s story it reads:

open data could add more than $3 trillion in total value annually to the education, transportation, consumer products, electricity, oil and gas, healthcare, and consumer finance sectors worldwide.

From guess/speculation to no mention to fact, all in the short space of three publications.

Does the valuing of open data remind you of:

virginia-ad

(Image from: http://civics.sites.unc.edu/files/2012/06/EarlyAmericanSettlements1.pdf)

The date of 1609 is important. Wikipedia has an article on Virginia, 1609-1610, titled, Starving Time. That year, only sixty (60) out of five hundred (500) colonists survived.

Does “Excellent Fruites by Planting” sound a lot like “new and innovative products and services?”

It does to me.

I first saw this in a tweet by Kirk Borne.

NOAA Moves to Unleash “Big Data”…

Tuesday, February 25th, 2014

NOAA Moves to Unleash “Big Data” and Calls Upon American Companies to Help by Kathryn Sullivan, Ph.D., Acting Undersecretary of Commerce for Oceans and Atmosphere and Acting NOAA Administrator.

RFI: Deadline March 24, 2014.

From the post:

From the surface of the sun to the depths of the ocean floor, the National Oceanic and Atmospheric Administration (NOAA), part of the Department of Commerce, works to keep citizens informed about the changing environment around them. Our vast network of radars, satellites, buoys, ships, aircraft, tide gauges, and supercomputers keeps tabs on the condition of our planet’s health and provides critical data that are used to predict changes in climate, weather, oceans, and coastlines. As we continue to witness changes on this dynamic planet we call home, the demand for NOAA’s data is only increasing.

Quite simply, NOAA is the quintessential big data agency. Each day, NOAA collects, analyzes, and generates over 20 terabytes of data – twice the amount of data than what is in the United States Library of Congress’ entire printed collection. However, only a small percentage is easily accessible to the public.

NOAA is not the only Commerce agency with a treasure trove of valuable information. The economic and demographic statistics from the Census Bureau, for example, inform business decisions every day. According to a 2013 McKinsey Global Institute Report, open data could add more than $3 trillion in total value annually to the education, transportation, consumer products, electricity, oil and gas, health care, and consumer finance sectors worldwide. That is why U.S. Secretary of Commerce Penny Pritzker has made unleashing the power of Commerce data one of the top priorities of the Department’s “Open for Business Agenda.”

All of that to lead up to:

That’s why we have released a Request for Information (RFI) to help us explore the feasibility of this concept and the range of possibilities to accomplish our goal. At no cost to taxpayers, this RFI calls upon the talents of America’s best minds to help us find the data and IT delivery solutions they need and should have.

This was released on February 21, 2014, so at best, potential responders had a maximum of thirty-two (32) days to respond to an RFI which describes the need and data sets in the broadest possible terms.

The “…no cost to taxpayers…” is particularly ironic, since anyone re-marketing the data to the public isn’t going to do so for free. Some public projects may but not the commercial vendors.

A better strategy would be for NOAA to release 10% of each distinct data set collected over the past two years at a cloud download location along with its documentation. Indicate how much data exists for each data set, the project, contact details.

Let real users and commercial vendors rummage through the 10% data to see what is of interest, how it can be processed, etc.

If NOAA wants real innovation, stop trying to manage it.

Managed innovation gets you Booz Allen type results. Is that what you want?