Another Word For It Patrick Durusau on Topic Maps and Semantic Diversity

NoSQL experimentations with Activiti: A (very simple) Neo4J prototype

From the post:

I’ve got this itch for a long time now to try and see how easy it is to write a BPM engine on a graph database such as Neo4J. After all, the data model fits perfectly, as business processes are graphs and executions of those processes basically boil down to keeping pointers to where you are in that graph. It just feels as a very natural fit.

So I spend some time implementing a prototype, which you can find on https://github.com/jbarrez/activiti-neo4j

The prototype contains some unit tests which execute simple BPMN 2.0 processes. I tried to be as close as possible to the Activiti concepts of services, commands, executions, JavaDelegates, etc. Currently covered:

If BPMN is unfamiliar:

A standard Business Process Model and Notation (BPMN) will provide businesses with the capability of understanding their internal business procedures in a graphical notation and will give organizations the ability to communicate these procedures in a standard manner. Furthermore, the graphical notation will facilitate the understanding of the performance collaborations and business transactions between the organizations. This will ensure that businesses will understand themselves and participants in their business and will enable organizations to adjust to new internal and B2B business circumstances quickly.

BPMN homepage, includes links to version 2.0 and other materials.

Processes, business and otherwise, seem like naturals to model as graphs.

On Importing Data into Neo4j (Blog Series) by Michael Hunger.

From the post:

Being able to run interesting queries against a graph database like Neo4j requires the data to be in there in the first place. As many users have questions in this area, I thought a series on importing data into Neo4j would be helpful to get started. This series covers both importing small and moderate data volumes for examples and demonstrations, but also large scale data ingestion.

For operations where massive amounts of data flow in or out of a Neo4j database, the interaction with the available APIs should be more considerate than with your usual, ad-hoc, local graph queries.

This blog series will discuss several ways of importing data into Neo4j and the considerations you should make when choosing one or the other. There is a dedicated page about importing data on neo4j.org which will help you getting started but needs feedback to improve.

Basically Neo4j offers several APIs to import data. Preferably the Cypher query language should be used as it is easiest to use from any programming language. The Neo4j Server’s REST API is not suited for massive data import, only the batch-operation endpoint and the Cypher REST and transactional HTTP (from Neo4j 2.0) endpoints are of interest there. Neo4j’s Java Core APIs provide a way of avoiding network overhead and driving data import directly from a programmatic source of data and also allow to drop down to the lowest API levels for high speed ingestion.

Great overview of importing data into Neo4j with the promise of more post to follow on importing data into Neo4j 2.0.

I first saw this at Alex Popescu’s On Importing Data into Neo4j.

Finding relationships in Trademark Data by Matt Overstreet.

From the post:

At the recent National Day of Civic hacking here at OSC we dug into a few ways to find relationships between Trademarks files with the USPTO.

If you’ve ever played with the US trademark data you’ll know that it’s both plentiful and scarce. There are lot’s of trademark fillings, each with the minimum possible data to make them uniquely identifiable.

That’s great for streamlined government and citizen anonymity, but no fun for finding the relationships between filings. We needed to suss out more information about the graph of trademarks. That’s when we Eric and Wes tripped over the translations included in many of the patent filings. We wondered if the term space for these translations might be smaller and more consistent then the space defined by the actual trademarks. Translations were less likely to play games with spelling or grammar the way one might with the actual mark.

Some Hacking with the data and Neo4j resulted in an intriguing dataset that we are still unpacking. Want to play with the data? Neo4J loaded with data is at this url: http://rosetta.bloom.sh:7474/webadmin/

Curious to know what you make of the theory:

Translations were less likely to play games with spelling or grammar the way one might with the actual mark.

I’m not sure that is a useful assumption:

Marks consisting of or including foreign words or terms from common, modern languages are translated into English to determine genericness, descriptiveness, likelihood of confusion, and other similar issues. See Palm Bay ,396 F.3d at 1377, 73 USPQ2d at 1696. With respect to likelihood of confusion, “[i]t is well established that foreign words or terms are not entitled to be registered if the English language equivalent has been previously used on or registered for products which might reasonably be assumed to come from the same source.” Mary Kay Cosmetics, Inc. v. Dorian Fragrances, Ltd. , 180 USPQ 406, 407 (TTAB 1973).[Examination Guide 1-08]

Use of the English translation to judge “…genericness, descriptiveness, likelihood of confusion, and other similar issues.”, lends an incentive for “…play[ing] games with spelling or grammar….”

If you are interested in the data set, you may find the resources at: Trademarks Home useful.

Caution: Legal terminology may not have the semantics you expect.

Neo4j 1.9.1 Released! by Jim Webber.

From the post:

It’s been a while since I was last let loose on the Neo4j blog, and I’ve marked my return with some good news. This week marks the release of Neo4j 1.9.1, numerically at least just a maintenance release in the 1.9 series.

However under the covers the engineering team has been working away developing more safety features for high-availability clustered deployments, squashing a couple of bugs, and improving SSL support for chained certificates and adding streaming support for paged traversals in the REST API.

If you’re a 1.9 user, you’re strongly recommended to upgrade to 1.9.1 and new users should proceed directly to 1.9.1. You won’t need any store upgrades going from 1.9 to 1.9.1 so it’s an easy upgrade.

Download at the usual place and happy graphing!

Unless you are otherwise occupied, this weekend sounds like the time to upgrade!

Putting All Species In A Graph Database

From the post:

Stephen Smith, an ecology and evolutionary biology professor at the University of Michigan, is going to explain how Neo4j and other digital technologies are assisting in constructing the tree of life. Starting at 10:00 PDT (19:00 CEST), he will also discuss other aspects of the interface of biology with next generation technologies.

“Our project is building the tools with which scientists in the community can continually improve the tree of life as we gather new information. Neo4j allows us to not only store trees in their native graph form, but also allows us to map trees to the same structure, the graph. So in fact, we are facilitating the construction of the graph of life,” says Smith.

Neo4j approached the Open Tree of Life team to present a webinar because it is a project that utilizes the Neo4j graph database to represent the interconnectedness of biological data. The company considers the project a great example of how a graph database can better model the natural world.

The online lecture is intended for a broad audience including beginner computer programmers, advanced hackers, data scientists, natural scientists, and anyone interested in the cross-section of science and technology, especially data modeling. Over 150 people have already registered online.

The registration form: [Registration]

This should be fun.

Get your questions ready!

My questions:

What happens when our understanding of the tree of life changes?

How do we preserve the “old” and “new” understandings of the tree of life?

Can we compare those understandings?

New Milestone Release Neo4j 2.0.0-M03 by Michael Hunger.

From the post:

The latest M03 milestone release of Neo4j 2.0 is as you expected all about improvements to Cypher. This blog post also discusses some changes made in the last milestone (M02) which we didn’t fully cover.

MERGE

Cypher now contains a MERGE clause which is pretty big: It will be replacing CREATE UNIQUE as it also takes indexes and labels into accounts and can even be used for single node creation. MERGE either matches the graph and returns what is there (one or more results) or if it doesn’t find anything it creates the path given. So after the MERGE operation completes, Neo4j guarantees that the declared pattern is there.

We also added additional clauses to the MERGE statement which allow you to create or update properties as a function of whether the node was matched or created. Please note that — as patterns can contain multiple named nodes and relationships — you will have to specify the element for which you want to trigger an update operation upon creation or match.

MERGE (keanu:Person { name:'Keanu Reeves' })
ON CREATE keanu SET keanu.created = timestamp()
ON MATCH  keanu SET keanu.lastSeen = timestamp()
RETURN keanu

We put MERGE out to mainly collect feedback on the syntax and usage, there are still some caveats, like not grabbing locks for unique creation so you might end up with duplicate nodes for now. That will all be fixed by the final release.

Going along with MERGE, MATCH now also supports single node patterns, both with and without labels.

(…)

MERGE is definitely something to investigate in this milestone release.

You need to also take a close look at labels.

What issues, if any, do you see with the label mechanism?

I see several but will cover them early next week. Work up your list (if any) to see if we reach similar conclusions.

Neo4j 1.9 General Availability Announcement! by Philip Rathle.

From the post:

After over a year of R&D, five milestone releases, and two release candidates, we are happy to release Neo4j 1.9 today! It is available for download effective immediately. And the latest source code is available, as always, on Github.

The 1.9 release adds primarily three things:

• Auto-Clustering, which makes Neo4j Enterprise clustering more robust & easier to administer, with fewer moving parts
• Cypher language improvements make the language more functionally powerful and more performant, and
• New welcome pages make learning easier for new users

Reloading my Beergraph – using an in-graph-alcohol-percentage-index by Rik Van Bruggen.

From the post:

As you may remember, I created a little beer graph some time ago to experiment and have fun with beer, and graphs. And yes, I have been having LOTS of fun with it – using it to explain graph concepts to lots of not-so-technical folks, like myself. Many people liked it, and even more people had some questions about it – started thinking in graphs, basically. Which is way more than what I ever hoped for – so that’s great!

One of the questions that people always asked me was about the model. Why did I model things the way I did? Are there no other ways to model this domain? What would be the *best* way to model it? All of these questions have somewhat vague answers, because as a rule, there is no *one way* to model a graph. The data does not determine the model – it’s the QUERY that will drive the modelling decisions.

One of the things that spurred the discussion was – probably not coincidentally – the AlcoholPercentage. Many people were expecting that to be a *property* of the Beerbrand – but instead in my beergraph, I had “pulled it out”. The main reason at the time was more coincidence than anything else, but when you think of it – it’s actually a fantastic thing to “pull things out” and normalise the data model much further than you probably would in a relational model. By making the alcoholpercentage a node of its own, it allowed me to do more interesting queries and pathfinding operations – which led to interesting beer recommendations. Which is what this is all about, right?

(…)

When I read:

All of these questions have somewhat vague answers, because as a rule, there is no *one way* to model a graph. The data does not determine the model – it’s the QUERY that will drive the modelling decisions.

or

…but instead in my beergraph, I had “pulled it out”. The main reason at the time was more coincidence than anything else, but when you think of it – it’s actually a fantastic thing to “pull things out” and normalise the data model much further than you probably would in a relational model.

I don’t feel like I’ve been vague, ever. 😉

Here is my summary of what Rik may have meant:

• “no *one way* to model a graph” -> graphs support multiple models of data
• “The data does not determine the model ” -> may mean you can create any arbitrary model based on any data
• “…the QUERY that will drive the modeling decisions.” -> in topic map terms, what gets represented by a topic (node in a graph) is what you want to talk about (query)
• “…pulled it out…”/”…pull things out…” -> represent a subject with a node (graph) or topic (topic maps).
• “…normlise the data model much further…” -> The distinction from database normalization isn’t clear, may just be filler.

Clarity in writing reduces unnecessary vagueness.

Knowledge Bases in Neo4j by Max De Marzi.

From the post:

From the second we are born we are collecting a wealth of knowledge about the world. This knowledge is accumulated and interrelated inside our brains and it represents what we know. If we could export this knowledge and give it to a computer, it would look like ConceptNet. ConceptNet is a semantic network that…

…is built from nodes representing concepts, in the form of words or short phrases of natural language, and labeled relationships between them. These are the kinds of things computers need to know to search for information better, answer questions, and understand people’s goals.

I wrote a little ruby script to import ConceptNet5 into Neo4j and it gives us a nice graph (243MB) to work with. ConceptNet5 as presented in csv files is actually a hypergraph, with a reason for the concept:

Max gives a script to remove the reasons for concepts (the hypergraph part of ConceptNet5) as duplicate content.

It does make the graph smaller, but only at the expense of information loss.

Think of it as the Benghazi emails with all the duplicate prose removed along with who said it.

If that fits your requirements, ok, but I doubt it would fit in any environment that requires information auditing.

Inserting data into Neo4j with Neo4j-Shell and Cypher by Alireza Rezaei Mahdiraji.

Tip on how to insert large data sets into Neo4j using Cypher and the Neo4j shell.

Alireza comments:

For a file of 64M, I commit after each 500 node/relationship commands and it works just fine. I tried it with 1000 and I got the same error as above.

Comparable commit numbers from other graph databases?

Match Making with NEO4J by Max De Marzi.

From the post:

In the “Matches are the new Hotness” blog post, I showed how to connect a person to a job via a location and skills. We’re going to look at a variation on the theme today by matching people to other people by what they want in a potential mate. We’re gonna use Neo4j to bring the love.

There are a ton of opinions on what’s wrong with current dating sites. I don’t claim to know how to fix them, I’m just giving what may be a piece of the puzzle. We could try to match people on the things they have in common, but the saying “opposites attract” exists for a reason. We often don’t want mirrors of ourselves, but rather to supplement some perceived deficiency. However complete opposites may result in exciting relationships, but may not be long-lasting. Some kind of happy middle ground is probably best.

This should come with a warning:

Don’t try this at home!

😉

Romantic advice, even from close friends, is fraught with peril. The professionals are getting paid for the risk.

Still, graphing high school/college romantic relationships could interest young people in computing and graphs.

Max has a great pic at this post. I had forgotten how beautiful she was.

Almost There: Neo4j 1.9-RC1! by Philip Rathle.

From the post:

Today is Leonhard Euler’s birthday, and we’re celebrating by announcing a first Release Candidate for Neo4j 1.9, now available for download! This release includes a number of incremental changes from the last Milestone (1.9-M05). This release candidate includes the last set of features we’d love our community to try out, as we prepare Neo4j 1.9 for General Availability (GA).

Philip also reports changes since the last milestone (1.9-M05).

I’m curious if the final Neo4j 1.9 release is going to be benchmarked against earlier releases?

Or released with benchmarks at all?

While researching the new labels feature in Neo4j 2.0.0-M01, I ran across the following statement in the documentation:

Note

null is not a valid property value. Nulls can instead be modeled by the absence of a key.

(Properties 3.3)

The question is asked in the comments:

Bryan Watson • a year ago −
What is implied by the absence of a key (“null”)?
(1) the key is relevant but the value is unknown? (spouse of a customer)
…

and answered:

Andrés Taylor Bryan Watson • a year ago −
In Neo4j, the absence of a key can mean all three options. Is that problematic in a particular concrete case, or are you wondering in the general case?

and,

Andrés Taylor Bryan Watson • a year ago −
You are correct. Neo4j is an unstructured database, which moves some of the responsibilities to the application. This is one of the things that the application has to take care of.

Problematic for topic maps because a role in an association may be known but the player of that role unknown.

Moreover, what happens if there are multiple “nulls?”

An application could have a “schema” for a node type that makes it possible to spot missing keys, but that seems like a long way to go for a “null.”

I don’t find “unstructured database” to be a persuasive argument for moving responsibilities to an application.

Databases, unstructured or otherwise, should be able to deal robustly with the various cases of “null.”

Nodes are people, too by Philip Rathle.

From the post:

Today we are releasing Milestone Release Neo4j 2.0.0-M01 of the Neo4j 2.0 series which we expect to be generally available (GA) in the next couple months. This release is significant in that it is the first time since the inception of Neo4j thirteen years ago that we are making a change to the property graph model. Specifically, we will be adding a new construct: labels.

We’ve completed a first cut at a significant addition to the data model: one that we believe nearly every graph will benefit from. Because this is a major change, it merits feedback, and we are opening the code up now for early comment. Please therefore consider 2.0 to be an experimental release. This first milestone is intended to solicit your input. In addition to the new technology being work-in-progress, some of the new terminology is also work-in-progress. We look forward to making 2.0 a better release together, with your feedback. Please tell us how you’d like to use these changes. We can’t wait to hear what you think.

Read the post for the introduction to “labels” for nodes.

Suggest you run 1.8/9 along side the experimental release 2.0.0-M01.

Something about the modeling of person by adding a property to the node strikes me as odd.

Rather than creating a “person” node with an edge to the original node.

Can’t put my finger on it but will be playing with it this week.

What other features would you like to see?

I’m thinking scope on properties would be high on my list.

Permission Resolution With Ne4j — Part 3 by Max De Marzi.

From the post:

Let’s add a couple of performance tests to the mix. We learned about Gatling in a previous blog post, we’re going to use it here again. The first test will randomly choose users and documents (from the graph we created in part 2) and write the results to a file, the second test will re-use the results of the first one and run consistently so we can change hardware, change Neo4j parameters, tune the JVM, etc. and see how they affect our performance.

Interesting post on testing a graph database to the point of:

How well (lousy) would this fair on a relational database?

A 10 Million row table joined to itself 100 times…omgwtfbbq.

Maybe we should ask Facebook?

Facebook releases Linkbench MySQL benchmark

In case you don’t know, MySQL is a relational database.

Last I heard the Facebook graph is one (1) billion users+.

Relational database technology is managing their graph fairly well.

What do you think?