Archive for the ‘SciDB’ Category

SciDB Version 12.3

Thursday, April 19th, 2012

SciDB Version 12.3

From the email notice:

Highlights of this release include:

  • more compact storage
  • vectorized expression evaluation
  • improvements to grand, grouped and window aggregates
  • support for non-integer dimensions within most major operators, including joins
  • transactional storage engine with error detection and rollback

Internal benchmarks comparing this release with the prior releases show disk usage reduced by 25%-50% and queries that use vectorized expression evaluation sped up by 4-10X.

SciDB Community meeting Oct 18th [2011]

Tuesday, October 11th, 2011

SciDB Community meeting Oct 18th [2011]

From Marilyn Matz:

Join us for updates about the SciDB project at a SciDB community meeting:

Oct 18, 2011, 5:15 PM – 6:00 PM,
SLAC, Research Office Building, bldg number 48
2575 Sand Hill Road, Menlo Park, CA

You are welcome to come even if you have not registered for XLDB.

At the community meeting we will review current performance, preview the contents of the upcoming release, hear about the EPICS project’s use of SciDB, and talk about community work-in-progress on HDF5 and FITs loaders and in-situ access.

This meeting is great opportunity to share your experiences using the software, tell us what you would like to see in future releases, and find out how to participate.

EPICS, just in case you are unfamiliar with the project.

EPICS is a set of Open Source software tools, libraries and applications developed collaboratively and used worldwide to create distributed soft real-time control systems for scientific instruments such as a particle accelerators, telescopes and other large scientific experiments.

SciDB 0.75 release

Tuesday, January 18th, 2011

SciDB 0.75 has just been released.

New features from the release notice:

  • A user query language based on SQL to augment the current functional language
  • Storage management improvements for both dense and sparse data
  • Dynamic query compilation and pipelining for faster execution
  • User-defined types and functions

R1.0 is targeted for May, 2011.

BTW, you will have to register to download the release.

SciDB – Numeric Array Database (NAD)

Saturday, September 25th, 2010

SciDB announced its first source-code release Open Letter to the SciDB Community on 24 September 2010.

In Overview of SciDB, Large Scale Array Storage, Processing and Analysis, the SciDB team says scientific data differs from business data because:

  1. scientific analysis typically requires mathematically and algorithmically sophisticated data processing methods
  2. data generated by modern scientific instruments is extremely large

I don’t find those convincing.

The article also claims: “…scientific data has a necessary and implicit ordering; for each element or data value there are other values left, right, up, down, next, previous, or adjacent to it.”

The content of such arrays is always numeric data and you can talk about numeric array databases.

I find the overall approach refreshing because it isn’t aiming for a general solution to all data issues.

Instead, a solution for numeric data in an array.

Now if we can just get past the search for a general semantic solution.