Archive for the ‘Time’ Category

KairosDB

Saturday, April 6th, 2013

KairosDB

From the webpage:

KairosDB is a fast distributed scalable time series database written primarily for Cassandra but works with HBase as well.

It is a rewrite of the original OpenTSDB project started at Stumble Upon. Many thanks go out to the original authors for laying the groundwork and direction for this great product. See a list of changes here.

Because it is written on top of Cassandra (or HBase) it is very fast and scalable. With a single node we are able to capture 40,000 points of data per second.

Why do you need a time series database? The quick answer is so you can be data driven in your IT decisions. With KairosDB you can use it to track the number of hits on your web server and compare that with the load average on your MySQL database.

Getting Started

Metrics

KairosDB stores metrics. Each metric consists of a name, data points (measurements), and tags. Tags are used to classify the metric.

Metrics can be submitted to KairosDB via telnet protocol or a REST API.

Metrics can be queried using a REST API. Aggregators can be used to manipulate the data as it is returned. This allows downsampling, summing, averaging, etc.

Do be aware that values must be either longs or doubles.

If your data can be mapped into metric space, KairosDB may be quite useful.

The intersection of time series data with non-metric data or events awaits a different solution.

I first saw this at Alex Popescu’s Kairosdb – Fast Scalable Time Series Database.

Posted in KairosDB, Time, Time Series | No Comments »

Davy Suvee on FluxGraph – Towards a time aware graph built on Datomic

Saturday, February 2nd, 2013

Davy Suvee on FluxGraph – Towards a time aware graph built on Datomic by René Pickhardt.

From the post:

Davy really nicely introduced the problem of looking at a snapshot of a data base. This problem obviously exists for any data base technology. You have a lot of timestamped records but running a query as if you fired it a couple of month ago is always a difficult challange.

With FluxGraph a solution to this is introduced.

How I understood him in the talk he introduces new versions of a vertex or an edge everytime it gets updated, added or removed. So far I am wondering about scaling and runtime. This approach seems like a lot of overhead to me. Later during Q & A I began to have the feeling that he has a more efficient way of storing this information so I really have to get in touch with davy to rediscuss the internals.

FluxGraph anyway provides a very clean API to access these temporal information.

FluxGraph at GitHub.

Time is an obvious issue in any business or medical context.

But also important when the news hounds ask: “Who knew what when?”

And there you may have personal relationships, meetings, communications, etc.

Posted in Datomic, FluxGraph, Time | No Comments »

Futures in literature from the past

Saturday, November 24th, 2012

Futures in literature from the past by Nathan Yau.

Another very graphic post that merits your attention. In part because of the visualization and Nathan’s suggestions about it. How would you recast the data?

But in a topic map context, how would you represent past projections about the future, both when the future is the present, but also against other projected futures?

I ask because the “Dark Ages” weren’t called that at the time. And in fact, they were a fairly lively time of invention and innovation.

The term was coined in the Renaissance to distinguish their “enlightened” civilization from the “dark” times between them and the fall of the Roman Empire.

It is an old trick but none the less effective for being an old one.

Recent political elections offered a number of examples that will be recognized as such in the fullness of time.

Posted in Graphics, Time, Timelines, Visualization | No Comments »

Windows into Relational Events: Data Structures for Contiguous Subsequences of Edges

Friday, September 28th, 2012

Windows into Relational Events: Data Structures for Contiguous Subsequences of Edges by Michael J. Bannister, Christopher DuBois, David Eppstein, Padhraic Smyth.

Abstract:

We consider the problem of analyzing social network data sets in which the edges of the network have timestamps, and we wish to analyze the subgraphs formed from edges in contiguous subintervals of these timestamps. We provide data structures for these problems that use near-linear preprocessing time, linear space, and sublogarithmic query time to handle queries that ask for the number of connected components, number of components that contain cycles, number of vertices whose degree equals or is at most some predetermined value, number of vertices that can be reached from a starting set of vertices by time-increasing paths, and related queries.

Among other interesting questions, raises the issue of what time span of connections constitutes a network of interest? More than being “dynamic.” A definitional issue for the social network in question.

If you are working with social networks, a must read.

PS: You probably need to read: Relational events vs graphs, a posting by David Eppstein.

David details several different terms for “relational event data,” and says there are probably others they did not find. (Topic maps anyone?)

Posted in Graphs, Networks, Social Media, Social Networks, Time, Time Series | No Comments »

Wrinkling Time

Monday, July 23rd, 2012

The post by Dan Brickley that I mentioned earlier today, Dilbert schematics, made me start thinking about more complex time scenarios than serial assignment of cubicles.

Like Hermione Granger and Harry Potter’s adventure in the Prisoner of Azkaban.

For those of you who are vague on the story, Hermione uses a “Time-Turner” to go back in time several hours. As a result, she and Harry must avoid being seen by themselves (and others). Works quite well in the story but what if I wanted to model that narrative in a topic map?

Some issues/questions that occurred to me:

Harry and Hermione are the same subjects they were during the prior time interval. Or are they?

Does a linear notion of time mean they are different subjects?

How would I model their interactions with others? Such as Buckbeak? Who interacted with both versions (for lack of a better term) of Harry?

Is there a time line running parallel to the “original” time line?

Just curious, what happens if the Time-Turner fails and Harry and Hermoine don’t return to the present, ever? That is their “current” present is forever 3 hours behind their “real” present.

What other time issues, either in literature or elsewhere seem difficult to model to you?

Posted in Modeling, Time, Timelines, Topic Maps | No Comments »

Basic Time Series with Cassandra

Thursday, June 21st, 2012

Basic Time Series with Cassandra

From the post:

One of the most common use cases for Cassandra is tracking time-series data. Server log files, usage, sensor data, SIP packets, stuff that changes over time. For the most part this is a straight forward process but given that Cassandra has real-world limitations on how much data can or should be in a row, there are a few details to consider.

As it says in the title, “basic” time series, the post concludes with:

Indexing and Aggregation

Indexing and aggregation of time-series data is a more complicated topic as they are highly application dependent. Various new and upcoming features of Cassandra also change the best practices for how things like aggregation are done so I won’t go into that. For more details, hit #cassandra on irc.freenode and ask around. There is usually somebody there to help.

But why would you collect time-series data if you weren’t going to index and/or aggregate it?

Anyone care to suggest “best practices?”

Posted in Cassandra, Time, Time Series | No Comments »

Timeline Maps

Wednesday, April 11th, 2012

Timeline Maps

From the post:

Mapping time has long been an interest of cartographers. Visualizing historical events in a timeline or chart or diagram is an effective way to show the rise and fall of empires and states, religious history, and important human and natural occurrences. We have over 100 examples in the Rumsey Map Collection, ranging in date from 1770 to 1967. We highlight a few below.

Sebastian Adams’ 1881 Synchronological Chart of Universal History is 23 feet long and shows 5,885 years of history, from 4004 B.C. to 1881 A.D. It is the longest timeline we have seen. The recently published Cartographies of Time calls it “nineteenth-century America’s surpassing achievement in complexity and synthetic power.” In the key to the map, Adams states that timeline maps enable learning and comprehension “through the eye to the mind.”

Below is a close up detail of a very small part of the chart: (click on the title or the image to open up the full chart)

Stunning visuals.

Our present day narratives aren’t any less arrogant than those of the 19th century but the distance is great enough for us to laugh at their presumption. Which unlike our own, isn’t “true.” ;-)

Worth all the time you can spend with the maps. Likely to provoke insights into how you have viewed “history” as well as how you view current “events.”

Posted in Mapping, Maps, Time, Timelines | No Comments »

Perception and Action: An Introduction to Clojure’s Time Model

Monday, April 18th, 2011

Perception and Action: An Introduction to Clojure’s Time Model

Summary:

Stuart Halloway discusses how we use a total control time model, proposing a different one that represents the world more accurately helping to solve some of the concurrency and parallelism problem.

To tempt you into watching this video, consider the following slide:

identity

  • continuity over time
    • built by minds
  • sameness across a series of perceptions
  • not a name, but can be named
  • can be composite

I will be posting other material from this presentation (as well as watching the video more than once).

(BTW, I saw the reference to this presentation in a tweet from Alex Popescu, myNoSQL.)

Posted in Clojure, Subject Identity, Time | No Comments »

Era of the Interest Graph

Tuesday, March 15th, 2011

Era of the Interest Graph

From the blog:

Social media is maturing as are the people embracing its most engaging tools and networks. Perhaps most notably, is the maturation of relationships and how we are expanding our horizons when it comes to connecting to one another. What started as the social graph, the network of people we knew and connected to in social networks, is now spawning new branches that resemble how we interact in real life.

This is the era of the interest graph – the expansion and contraction of social networks around common interests and events. Interest graphs represent a potential goldmine for brands seeking insight and inspiration to design more meaningful products and services as well as new marketing campaigns that better target potential stakeholders.

While many companies are learning to listen to the conversations related to their brands and competitors, many are simply documenting activity and mentions as a reporting function and in some cases, as part of conversational workflow. However, there’s more to Twitter intelligence than tracking conversations.

We’re now looking beyond the social graph as we move into focused networks that share more than just a relationship.

What struck me about this post was the sense that the graph was a non-stable construct.

Whereas most of the topic maps I have seen are not only stable, but their subjects are as well.

Which is fine for some areas of information, but not all.

A dynamic topic map seems to have different requirements than one that is a fixed editorial product, or at least it seems so to me.

Rather than versioning, for example, a dynamic topic map should have a tracking mechanism to show what information was available at any point in time.

So that say a physician relying upon a dynamic topic map for drug warning information can establish that a warning was or was not available at the time he prescribed a medication.

Oh, that’s not commonly possible even with static topic maps is it?

Hmmm, will have to give some thought to that issue.

It may just be the maps I have looked at but there is a timeless nature to them.

Much like governments, whatever is the case has always been the case. And if you remember differently, well, you are just wrong. If not subversive.

Posted in Graphs, Time, Topic Maps, Versioning | No Comments »