Accelerating SQL Database Operations on a GPU with CUDA by Peter Bakkum and Kevin Skadron.
Abstract:
Prior work has shown dramatic acceleration for various database operations on GPUs, but only using primitives that are not part of conventional database languages such as SQL. This paper implements a subset of the SQLite command processor directly on the GPU. This dramatically reduces the eff ort required to achieve GPU acceleration by avoiding the need for database programmers to use new programming languages such as CUDA or modify their programs to use non-SQL libraries.
This paper focuses on accelerating SELECT queries and describes the considerations in an efficient GPU implementation of the SQLite command processor. Results on an NVIDIA Tesla C1060 achieve speedups of 20-70X depending on the size of the result set.
Important lessons to be learned from this paper:
- Don’t invent new languages for the average user to learn.
- Avoid the need to modify existing programs
- Write against common software
Remember that 75% of the BI market is still using spreadsheets. For all sorts of data but numeric data in particular.
I don’t have any experience with importing files into Excel but I assume there is a macro language that can used to create import processes.
Curious if there has been any work on creating import macros for Excel that incorporate merging as part of those imports?
That would:
- Not be a new language for users to learn.
- Avoid modification of existing programs (or data)
- Be written against common software
I am not sure about the requirements for merging numeric data but that should make the exploration process all the more enjoyable.