Another Word For It Patrick Durusau on Topic Maps and Semantic Diversity

July 18, 2014

Artificial Intelligence | Natural Language Processing

Filed under: Artificial Intelligence,CS Lectures,Natural Language Processing — Patrick Durusau @ 4:26 pm

Artificial Intelligence | Natural Language Processing by Christopher Manning.

From the webpage:

This course is designed to introduce students to the fundamental concepts and ideas in natural language processing (NLP), and to get them up to speed with current research in the area. It develops an in-depth understanding of both the algorithms available for the processing of linguistic information and the underlying computational properties of natural languages. Wordlevel, syntactic, and semantic processing from both a linguistic and an algorithmic perspective are considered. The focus is on modern quantitative techniques in NLP: using large corpora, statistical models for acquisition, disambiguation, and parsing. Also, it examines and constructs representative systems.

Lectures with notes.

If you are new to natural language processing, it would be hard to point at a better starting point.

Enjoy!

June 24, 2014

‘A Perfect and Beautiful Machine’:…

Filed under: Artificial Intelligence,Evoluntionary — Patrick Durusau @ 4:28 pm

‘A Perfect and Beautiful Machine’: What Darwin’s Theory of Evolution Reveals About Artificial Intelligence by Daniel C. Dennett.

From the post:

…
All things in the universe, from the most exalted (“man”) to the most humble (the ant, the pebble, the raindrop) were creations of a still more exalted thing, God, an omnipotent and omniscient intelligent creator — who bore a striking resemblance to the second-most exalted thing. Call this the trickle-down theory of creation. Darwin replaced it with the bubble-up theory of creation. One of Darwin’s nineteenth-century critics, Robert Beverly MacKenzie, put it vividly:

In the theory with which we have to deal, Absolute Ignorance is the artificer; so that we may enunciate as the fundamental principle of the whole system, that, in order to make a perfect and beautiful machine, it is not requisite to know how to make it. This proposition will be found, on careful examination, to express, in condensed form, the essential purport of the Theory, and to express in a few words all Mr. Darwin’s meaning; who, by a strange inversion of reasoning, seems to think Absolute Ignorance fully qualified to take the place of Absolute Wisdom in all the achievements of creative skill.

It was, indeed, a strange inversion of reasoning. To this day many people cannot get their heads around the unsettling idea that a purposeless, mindless process can crank away through the eons, generating ever more subtle, efficient, and complex organisms without having the slightest whiff of understanding of what it is doing.

Turing’s idea was a similar — in fact remarkably similar — strange inversion of reasoning. The Pre-Turing world was one in which computers were people, who had to understand mathematics in order to do their jobs. Turing realized that this was just not necessary: you could take the tasks they performed and squeeze out the last tiny smidgens of understanding, leaving nothing but brute, mechanical actions. In order to be a perfect and beautiful computing machine, it is not requisite to know what arithmetic is.

What Darwin and Turing had both discovered, in their different ways, was the existence of competence without comprehension. This inverted the deeply plausible assumption that comprehension is in fact the source of all advanced competence. Why, after all, do we insist on sending our children to school, and why do we frown on the old-fashioned methods of rote learning? We expect our children’s growing competence to flow from their growing comprehension. The motto of modern education might be: “Comprehend in order to be competent.” For us members of H. sapiens, this is almost always the right way to look at, and strive for, competence. I suspect that this much-loved principle of education is one of the primary motivators of skepticism about both evolution and its cousin in Turing’s world, artificial intelligence. The very idea that mindless mechanicity can generate human-level — or divine level! — competence strikes many as philistine, repugnant, an insult to our minds, and the mind of God.
….

“…competence without comprehension….” I rather like that!

Is that what we are observing in crowd-sourcing?

The essay is well worth your time and consideration.

May 23, 2014

Learning Everything About Anything (sort of)

Filed under: Artificial Intelligence,Machine Learning — Patrick Durusau @ 2:52 pm

Meet the algorithm that can learn “everything about anything” by Dennis Harris.

From the post:

One of the more interesting projects is a system called LEVAN, which is short for Learn EVerything about ANything and was created by a group of researchers out of the Allen Institute for Artificial Intelligence and the University of Washington. One of them, Carlos Guestrin, is also co-founder and CEO of a data science startup called GraphLab. What’s really interesting about LEVAN is that it’s neither human-supervised nor unsupervised (like many deep learning systems), but what its creators call “webly supervised.”

(image omitted)

What that means, essentially, is that LEVAN uses the web to learn everything it needs to know. It scours Google Books Ngrams to learn common phrases associated with a particular concept, then searches for those phrases in web image repositories such as Google Images, Bing and Flickr. For example, LEVAN now knows that “heavyweight boxing,” “boxing ring” and “ali boxing” are all part of the larger concept of “boxing,” and it knows what each one looks like.

When I said “sort of” in the title I didn’t mean any disrespect for LEVAN. On the contrary, the researchers limiting LEVAN to Google Book Ngrams and images is a brilliant move. That limits LEVAN to the semantic debris that can be found in public image repositories but depending upon your requirements, that may be more than sufficient.

The other upside is that despite a pending patent, sigh, the source code is available for research/academic purposes.

What data sets make useful limits for your AI/machine learning algorithm? Your application need not understand intercepted phone conversations, Barbara Walters, or popular music, if those are not in your requirements. Simplifying your AI problem may be the first step towards solving it.

March 7, 2014

Building fast Bayesian computing machines…

Filed under: Artificial Intelligence,Bayesian Data Analysis,Precision — Patrick Durusau @ 11:41 am

Building fast Bayesian computing machines out of intentionally stochastic, digital parts by Vikash Mansinghka and Eric Jonas.

Abstract:

The brain interprets ambiguous sensory information faster and more reliably than modern computers, using neurons that are slower and less reliable than logic gates. But Bayesian inference, which underpins many computational models of perception and cognition, appears computationally challenging even given modern transistor speeds and energy budgets. The computational principles and structures needed to narrow this gap are unknown. Here we show how to build fast Bayesian computing machines using intentionally stochastic, digital parts, narrowing this efficiency gap by multiple orders of magnitude. We find that by connecting stochastic digital components according to simple mathematical rules, one can build massively parallel, low precision circuits that solve Bayesian inference problems and are compatible with the Poisson firing statistics of cortical neurons. We evaluate circuits for depth and motion perception, perceptual learning and causal reasoning, each performing inference over 10,000+ latent variables in real time – a 1,000x speed advantage over commodity microprocessors. These results suggest a new role for randomness in the engineering and reverse-engineering of intelligent computation.

Ironic that the greater precision and repeatability of our digital computers may be choices that are holding back advancements in Bayesian digital computing machines.

I have written before about the RDF ecosystem being over complex and precise for use by everyday users.

We should strive to capture semantics as understood by scientists, researchers, students, and others. Less precise than professional semantics but precise enough to make it usable?

I first saw this in a tweet by Stefano Bertolo.

March 6, 2014

…Why Watson Can’t Get a Job

Filed under: Artificial Intelligence — Patrick Durusau @ 10:48 am

IBM’s Artificial Intelligence Problem, or Why Watson Can’t Get a Job by Drake Bennett.

From the post:

What if we built a super-smart artificial brain and no one cared? IBM (IBM) is facing that possibility. According to the Wall Street Journal, the company is having a hard time making money off of its Jeopardy-winning supercomputer, Watson. The company has always claimed that Watson was more than a publicity stunt, that it had revolutionary real-world applications in health care, investing, and other realms. IBM Chief Executive Officer Virginia Rometty has promised that Watson will generate $10 billion in annual revenue within 10 years, but according to the Journal, as of last October Watson was far behind projections, only bringing in $100 million.

The Journal article focuses on difficulties and costs in “training” Watson to master the particulars of various businesses—at the M.D. Anderson Cancer Center, at Citigroup (C), at the health insurer WellPoint (WLP). But there may also be another issue: the sort of intelligence Watson possesses might not be a particularly good fit for some of the jobs IBM is looking at.
….

A very good summary of the issues around getting Watson some paying work.

My take away was that you can replace people in a complex situation, like medical diagnosis, but if and only if, you are willing to accept degraded results.

How degraded remains to be seen. I can say I would not want to be the medical malpractice carrier for Watson.

Which makes me wonder about the general trend of replacing people with machines. There are many tasks that machines can perform, but if and only if you are willing to accept degraded results.

For example, I am sure you have seen the machine learning sites that promise you too can analyze data like a pro! No training, expensive data scientists, etc. Just plug your data in and go.

I don’t doubt that you can “plug you data in and go” but I also have little doubt about the quality of results that you obtain.

After all, we (people in general) are the creators of computers, the data you want to process, the algorithms you will use, why it is important to exclude people from your process?

Cheaper? If results are all that count, casino dice are about $12.00 for five (5), even cheaper than online machine learning services. Just roll the dice and fill in the numbers you need.

February 16, 2014

Hofstadter on Watson and Siri – “absolutely vacuous”

Filed under: Artificial Intelligence — Patrick Durusau @ 4:02 pm

Why Watson and Siri Are Not Real AI by William Herkewitz.

Hofstadter’s first response in the interview:

Well, artificial intelligence is a slippery term. It could refer to just getting machines to do things that seem intelligent on the surface, such as playing chess well or translating from one language to another on a superficial level—things that are impressive if you don’t look at the details. In that sense, we’ve already created what some people call artificial intelligence. But if you mean a machine that has real intelligence, that is thinking—that’s inaccurate. Watson is basically a text search algorithm connected to a database just like Google search. It doesn’t understand what it’s reading. In fact, read is the wrong word. It’s not reading anything because it’s not comprehending anything. Watson is finding text without having a clue as to what the text means. In that sense, there’s no intelligence there. It’s clever, it’s impressive, but it’s absolutely vacuous. (emphasis added)

You may remember Douglas Hofstadter as the author of Gödel, Escher, Bach

If Hofstadter’s point is that no mechanical device is “intelligent,” from a cuneiform tablet to a codex or even a digital computer such as Watson, I am in full agreement. A mechanical device can do nor more or less than it has been engineered to do.

What is curious about Watson is that what usefulness it displays, at least at playing Jeopardy, comes from analysis of prior responses of human players.

But most “AI” efforts don’t ask for a stream of human judgments but rather try to capture with algorithms the important points to remember.

Curious isnt it? The failure to ask a large audience of known intelligence users for their opinions, to be captured in an electronic form for further use.

And why stop with a large audience? Why not ask every researcher who submits a publication a series of questions about their paper and related work?

Reasoning (sorry) that the more intelligence you put into a mechanical storage device the more intelligence you maybe able to extract.

Present practices almost sound like discrimination against intelligent users in favor of using mechanical approaches.

I guess that depends on whether using mechanical devices or getting a useful result is the goal.

I first saw this in Stephen Arnold’s Getting a Failing Grade in Artificial Intelligence: Watson and Siri.

December 17, 2013

Cross-categorization of legal concepts…

Filed under: Artificial Intelligence,Law,Legal Informatics,Ontology — Patrick Durusau @ 3:21 pm

Cross-categorization of legal concepts across boundaries of legal systems: in consideration of inferential links by Fumiko Kano Glückstad, Tue Herlau, Mikkel N. Schmidt, Morten Mørup.

Abstract:

This work contrasts Giovanni Sartor’s view of inferential semantics of legal concepts (Sartor in Artif Intell Law 17:217–251, 2009) with a probabilistic model of theory formation (Kemp et al. in Cognition 114:165–196, 2010). The work further explores possibilities of implementing Kemp’s probabilistic model of theory formation in the context of mapping legal concepts between two individual legal systems. For implementing the legal concept mapping, we propose a cross-categorization approach that combines three mathematical models: the Bayesian Model of Generalization (BMG; Tenenbaum and Griffiths in Behav Brain Sci 4:629–640, 2001), the probabilistic model of theory formation, i.e., the Infinite Relational Model (IRM) first introduced by Kemp et al. (The twenty-first national conference on artificial intelligence, 2006, Cognition 114:165–196, 2010) and its extended model, i.e., the normal-IRM (n-IRM) proposed by Herlau et al. (IEEE International Workshop on Machine Learning for Signal Processing, 2012). We apply our cross-categorization approach to datasets where legal concepts related to educational systems are respectively defined by the Japanese- and the Danish authorities according to the International Standard Classification of Education. The main contribution of this work is the proposal of a conceptual framework of the cross-categorization approach that, inspired by Sartor (Artif Intell Law 17:217–251, 2009), attempts to explain reasoner’s inferential mechanisms.

From the introduction:

An ontology is traditionally considered as a means for standardizing knowledge represented by different parties involved in communications (Gruber 1992; Masolo et al. 2003; Declerck et al. 2010). Kemp et al. (2010) also points out that some scholars (Block 1986; Field 1977; Quilian 1968) have argued the importance of knowledge structuring, i.e., ontologies, where concepts are organized into systems of relations and the meaning of a concept partly depends on its relationships to other concepts. However, real human to human communication cannot be absolutely characterized by such standardized representations of knowledge. In Kemp et al. (2010), two challenging issues are raised against such idea of systems of concepts. First, as Fodor and Lepore (1992) originally pointed out, it is beyond comprehension that the meaning of any concept can be defined within a standardized single conceptual system. It is unrealistic that two individuals with different beliefs have common concepts. This issue has also been discussed in semiotics (Peirce 2010; Durst-Andersen 2011) and in cognitive pragmatics (Sperber and Wilson 1986). For example, Sperber and Wilson (1986) discuss how mental representations are constructed diversely under different environmental and cognitive conditions. A second point which Kemp et al. (2010) specifically address in their framework is the concept acquisition problem. According to Kemp et al. (2010; see also: Hempel (1985), Woodfield (1987)):

if the meaning of each concept depends on its role within a system of concepts, it is difficult to see how a learner might break into the system and acquire the concepts that it contains. (Kemp et al. 2010)

Interestingly, the similar issue is also discussed by legal information scientists. Sartor (2009) argues that:

legal concepts are typically encountered in the context of legal norms, and the issue of determining their content cannot be separated from the issue of identifying and interpreting the norms in which they occur, and of using such norms in legal inference. (Sartor 2009)

This argument implies that if two individuals who are respectively belonging to two different societies having different legal systems, they might interpret a legal term differently, since the norms in which the two individuals belong are not identical. The argument also implies that these two individuals must have difficulties in learning a concept contained in the other party’s legal system without interpreting the norms in which the concept occurs.

These arguments motivate us to contrast the theoretical work presented by Sartor (2009) with the probabilistic model of theory formation by Kemp et al. (2010) in the context of mapping legal concepts between two individual legal systems. Although Sartor’s view addresses the inferential mechanisms within a single legal system, we argue that his view is applicable in a situation where a concept learner (reasoner) is, based on the norms belonging to his or her own legal system, going to interpret and adapt a new concept introduced from another legal system. In Sartor (2009), the meaning of a legal term results from the set of inferential links. The inferential links are defined based on the theory of Ross (1957) as:

  1. the links stating what conditions determine the qualification Q (Q-conditioning links), and
  2. the links connecting further properties to possession of the qualification Q (Q-conditioned links.) (Sartor 2009)

These definitions can be seen as causes and effects in Kemp et al. (2010). If a reasoner is learning a new legal concept in his or her own legal system, the reasoner is supposed to seek causes and effects identified in the new concept that are common to the concepts which the reasoner already knows. This way, common-causes and common-effects existing within a concept system, i.e., underlying relationships among domain concepts, are identified by a reasoner. The probabilistic model in Kemp et al. (2010) is supposed to learn these underlying relationships among domain concepts and identify a system of legal concepts from a view where a reasoner acquires new concepts in contrast to the concepts already known by the reasoner.

Pardon the long quote but the paper is pay-per-view.

I haven’t started to run down all the references but this is an interesting piece of work.

I was most impressed by the partial echoing of the topic map paradigm that: “meaning of each concept depends on its role within a system of concepts….”

True, a topic map can capture only “surface” facts and relationships between those facts but that merits a comment on a topic map instance and not topic maps in general.

Noting that you also shouldn’t pay for more topic map than you need. If all you need is a flat mapping between DHS and say the CIA, then doing nor more than mapping terms is sufficient. If you need a maintainable and robust mapping, different techniques would be called for. Both results would be topic maps, but one of them would be far more useful.

One of the principal sources relied upon by the authors’ is: The Nature of Legal Concepts: Inferential Nodes or Ontological Categories? by Giovanni Sartor.

I don’t see any difficulty with Sartor’s rules of inference, any more than saying if a topic has X property (occurrence in TMDM speak), then of necessity it must have property E, F, and G.

Where I would urge caution is with the notion that properties of a legal concept spring from a legal text alone. Or even from a legal ontology. In part because two people in the same legal system can read the same legal text and/or use the same legal ontology and expect to see different properties for a legal concept.

Consider the text of Paradise Lost by John Milton. If Stanley Fish, a noted Milton scholar, were to assign properties to the concepts in Book 1, his list of properties would be quite different from my list of properties. Same words, same text, but very different property lists.

To refine what I said about the topic map paradigm a bit earlier, it should read: “meaning of each concept depends on its role within a system of concepts [and the view of its hearer/reader]….”

The hearer/reader being the paramount consideration. Without a hearer/reader, there is no concept or system of concepts or properties of either one for comparison.

When topics are merged, there is a collecting of properties, some of which you may recognize and some of which I may recognize, as identifying some concept or subject.

No guarantees but better than repeating your term for a concept over and over again, each time in a louder voice. 😉

December 7, 2013

The Society of the Mind

Filed under: Artificial Intelligence,Machine Learning — Patrick Durusau @ 2:40 pm

The Society of the Mind by Marvin Minsky.

From the Prologue:

This book tries to explain how minds work. How can intelligence emerge from nonintelligence? To answer that, we’ll show that you can build a mind from many little parts, each mindless by itself.

I’ll call Society of Mind this scheme in which each mind is made of many smaller processes. These we’ll call agents. Each mental agent by itself can only do some simple thing that needs no mind or thought at all. Yet when we join these agents in societies — in certain very special ways — this leads to true intelligence.

There’s nothing very technical in this book. It, too, is a society — of many small ideas. Each by itself is only common sense, yet when we join enough of them we can explain the strangest mysteries of mind. One trouble is that these ideas have lots of cross-connections. My explanations rarely go in neat, straight lines from start to end. I wish I could have lined them up so that you could climb straight to the top, by mental stair-steps, one by one. Instead they’re tied in tangled webs.

Perhaps the fault is actually mine, for failing to find a tidy base of neatly ordered principles. But I’m inclined to lay the blame upon the nature of the mind: much of its power seems to stem from just the messy ways its agents cross-connect. If so, that complication can’t be helped; it’s only what we must expect from evolution’s countless tricks.

What can we do when things are hard to describe? We start by sketching out the roughest shapes to serve as scaffolds for the rest; it doesn’t matter very much if some of those forms turn out partially wrong. Next, draw details to give these skeletons more lifelike flesh. Last, in the final filling-in, discard whichever first ideas no longer fit.

That’s what we do in real life, with puzzles that seem very hard. It’s much the same for shattered pots as for the cogs of great machines. Until you’ve seen some of the rest, you can’t make sense of any part.

All 270 essays in 30 chapters of Minsky’s 1988 book by the same name.

To be read critically.

It is dated but a good representative of a time in artificial intelligence.

I first saw this in Nat Torkington’s Five Short Links for 6 December 2013.

November 15, 2013

High Accuracy Metadata and Machine Learning: A librarian’s success

Filed under: Artificial Intelligence,Clustering,Indexing,Machine Learning,Metadata — Patrick Durusau @ 7:36 pm

High Accuracy Metadata and Machine Learning: A librarian’s success by Ashleigh Faith.

Description:

Taxonomy is a field that spans both LIS and SIS studies. Ashleigh Faith will be presenting how a librarian can use information science to set up a taxonomy and machine learning process for complex content. Faith created a taxonomy, based in engineering mobility and science terminology, from scratch. Developing a cohesive taxonomy that would also facilitate automatic indexing on content for an engineering database that reaches more than 218,000 documents (and growing) across eight different content types was a challenge. The nature of scientific content makes automatic indexing difficult because it is considered complex –or outside the established standards of taxonomy. Faith discusses the process used to establish a taxonomy to capture content and create the bedrock in which the indexing software could be trained. Using traditionally linguistic techniques, Faith improved the database taxonomy metadata assignment accuracy to 89% accuracy, well above the typically accepted 75% accuracy rate of automatic indexing, and established a repeatable process that was also implemented successfully on NASA Tech Brief content, NATO Terminology Directives, and DOD content. Learn from concrete examples, lessons learned from a librarians perspective, and how to duplicate the process.

Slides: http://prezi.com/jr61rhjoqotb/?utm_campaign=share&utm_medium=copy

Just working from the slides, this was a presentation to see!

In a nutshell: Ahsleigh’s approach netted 89% accuracy, as compared to human indexer accuracy of 91% and typical automated indexing at 75%.

Good illustration of the content finding rules:

Rule 1: If you don’t want to find content, don’t hire a librarian.

Rule 2: If you do want to find content, hire a librarian.

Clear enough?

November 8, 2013

ParLearning 2014

Filed under: Artificial Intelligence,Conferences,GPU,Machine Learning,Parallel Programming — Patrick Durusau @ 4:38 pm

ParLearning 2014 The 3rd International Workshop on Parallel and Distributed Computing for Large Scale Machine Learning and Big Data Analytics.

Dates:

Workshop Paper Due: December 30, 2013
Author Notification: February 14, 2014
Camera-ready Paper Due: March 14, 2014
Workshop: May 23, 2014 Phoenix, AZ, USA

From the webpage:

Data-driven computing needs no introduction today. The case for using data for strategic advantages is exemplified by web search engines, online translation tools and many more examples. The past decade has seen 1) the emergence of multicore architectures and accelerators as GPGPUs, 2) widespread adoption of distributed computing via the map-reduce/hadoop eco-system and 3) democratization of the infrastructure for processing massive datasets ranging into petabytes by cloud computing. The complexity of the technological stack has grown to an extent where it is imperative to provide frameworks to abstract away the system architecture and orchestration of components for massive-scale processing. However, the growth in volume and heterogeneity in data seems to outpace the growth in computing power. A “collect everything” culture stimulated by cheap storage and ubiquitous sensing capabilities contribute to increasing the noise-to-signal ratio in all collected data. Thus, as soon as the data hits the processing infrastructure, determining the value of information, finding its rightful place in a knowledge representation and determining subsequent actions are of paramount importance. To use this data deluge to our advantage, a convergence between the field of Parallel and Distributed Computing and the interdisciplinary science of Artificial Intelligence seems critical. From application domains of national importance as cyber-security, health-care or smart-grid to providing real-time situational awareness via natural interface based smartphones, the fundamental AI tasks of Learning and Inference need to be enabled for large-scale computing across this broad spectrum of application domains.

Many of the prominent algorithms for learning and inference are notorious for their complexity. Adopting parallel and distributed computing appears as an obvious path forward, but the mileage varies depending on how amenable the algorithms are to parallel processing and secondly, the availability of rapid prototyping capabilities with low cost of entry. The first issue represents a wider gap as we continue to think in a sequential paradigm. The second issue is increasingly recognized at the level of programming models, and building robust libraries for various machine-learning and inferencing tasks will be a natural progression. As an example, scalable versions of many prominent graph algorithms written for distributed shared memory architectures or clusters look distinctly different from the textbook versions that generations of programmers have grown with. This reformulation is difficult to accomplish for an interdisciplinary field like Artificial Intelligence for the sheer breadth of the knowledge spectrum involved. The primary motivation of the proposed workshop is to invite leading minds from AI and Parallel & Distributed Computing communities for identifying research areas that require most convergence and assess their impact on the broader technical landscape.

Taking full advantage of parallel processing remains a distant goal. This workshop looks like a good concrete step towards that goal.

October 29, 2013

Graph Triumphalism

Filed under: Artificial Intelligence,Graphs — Patrick Durusau @ 6:20 pm

The Next Battle Ground for the Titans of Tech by Charles Silver.

From the post:

To win this galactic battle for dominance of Web 3.0, the victorious titan must find a way to move the entire tech world off of relational databases — which have been the foundation of computing since the 1970s — and onto graph databases, the key to semantic computing. The reason: Relational databases, though revolutionary way back when, are not up to the job of managing today’s Big Data. There are two huge, insurmountable issues preventing this:

  • Data integration. Relational databases (basically, all that stuff in silos) are finicky. They come in many forms, from many sources, and don’t play well with others. While search engines can find data containing specific keywords, they can’t do much of anything with it.
  • Intelligent “thinking.” While it’s impossible for computers to reason or form concepts using relational databases, they can do exactly that with linked data in graph databases. Semantic search engines can connect related data, forming a big picture out of small pieces, Star Trek-like.

This is exactly what users want and need. Consumers, marketers, advertisers, researchers, defense experts, financiers, medical researchers, astrophysicists, everyone who uses search engines (that’s everyone) wants to type in questions and get clear, accurate, complete answers, fast, that relate to them. If they’re shopping (for insurance, red shoes, DIY drones), they want where-to-get-it resources, ratings and more. Quite a wish list. Yet chunks of it are already happening.

I really like graph databases. I really do.

But to say relational databases = silos, with the implication that graph databases != silos, is just wrong.

Relational or graph databases (or any other kind of information system) will look like a silo if you don’t know the semantics of its structure and the data inside.

Technology doesn’t make silos, users who don’t disclose/document the semantics of data structure and data create silos.

Some technologies make it easier to disclose semantics than others but it is always a users choice that is responsible for the creation of a data silo.

And no, graphs don’t make it possible for computers to “…reason or form concepts….” That’s just silly.

Law of Conservation of Intelligence: You can’t obtain more intelligence from an system than was designed into it.

PS: I know, I’m cheating because I did not define “intelligence.” At least I am aware I didn’t define it. 😉

July 9, 2013

AAAI – Weblogs and Social Media

Filed under: Artificial Intelligence,Blogs,Social Media,Tweets — Patrick Durusau @ 12:34 pm

Seventh International AAAI Conference on Weblogs and Social Media

Abstracts and papers from the Seventh International AAAI Conference on Weblogs and Social Media.

Much to consider:

Frontmatter: Six (6) entries.

Full Papers: Sixty-nine (69) entries.

Poster Papers: Eighteen (18) entries.

Demonstration Papers: Five (5) entries.

Computational Personality Recognition: Ten (10) entries.

Social Computing for Workforce 2.0: Seven (7) entries.

Social Media Visualization: Four (4) entries.

When the City Meets the Citizen: Nine (9) entries.

Be aware that the links for tutorials and workshops only give you the abstracts describing the tutorials and workshops.

There is the obligatory “blind men and the elephant” paper:

Blind Men and the Elephant: Detecting Evolving Groups in Social News

Abstract:

We propose an automated and unsupervised methodology for a novel summarization of group behavior based on content preference. We show that graph theoretical community evolution (based on similarity of user preference for content) is effective in indexing these dynamics. Combined with text analysis that targets automatically-identified representative content for each community, our method produces a novel multi-layered representation of evolving group behavior. We demonstrate this methodology in the context of political discourse on a social news site with data that spans more than four years and find coexisting political leanings over extended periods and a disruptive external event that lead to a significant reorganization of existing patterns. Finally, where there exists no ground truth, we propose a new evaluation approach by using entropy measures as evidence of coherence along the evolution path of these groups. This methodology is valuable to designers and managers of online forums in need of granular analytics of user activity, as well as to researchers in social and political sciences who wish to extend their inquiries to large-scale data available on the web.

It is a great paper but commits a common error when it notes:

Like the parable of Blind Men and the Elephant2, these techniques provide us with disjoint, specific pieces of information.

Yes, the parable is oft told to make a point about partial knowledge, but the careful observer will ask:

How are we different from the blind men trying to determine the nature of an elephant?

Aren’t we also blind men trying to determine the nature of blind men who are examining an elephant?

And so on?

Not that being blind men should keep us from having opinions, but it should may us wary of how deeply we are attached to them.

Not only are there elephants all the way down, there are blind men before, with (including ourselves) and around us.

June 2, 2013

Notable presentations at Technion TCE conference 2013: RevMiner & Boom

Filed under: Artificial Intelligence,Recommendation — Patrick Durusau @ 9:48 am

Notable presentations at Technion TCE conference 2013: RevMiner & Boom by Danny Bickson.

Danny has uncovered two papers to start your week:

http://turing.cs.washington.edu/papers/uist12-huang.pdf (RevMiner)

http://turing.cs.washington.edu/papers/kdd12-ritter.pdf (Twitter data mining)

Danny also describes Boom, for which I found this YouTube video:

See Danny’s post for more comments, etc.

May 22, 2013

Introduction to Artificial Intelligence (Berkeley CS188.1x)

Filed under: Artificial Intelligence,Programming — Patrick Durusau @ 2:17 pm

Introduction to Artificial Intelligence (Berkeley CS188.1x)

The schedule for CS188.2x hasn’t been announced, yet.

In the meantime, you can register for CS188.1x and peruse the videos, exercises, etc. while you wait for the second part of the course.

From the description:

CS188.1x is a new online adaptation of the first half of UC Berkeley’s CS188: Introduction to Artificial Intelligence. The on-campus version of this upper division computer science course draws about 600 Berkeley students each year.

Artificial intelligence is already all around you, from web search to video games. AI methods plan your driving directions, filter your spam, and focus your cameras on faces. AI lets you guide your phone with your voice and read foreign newspapers in English. Beyond today’s applications, AI is at the core of many new technologies that will shape our future. From self-driving cars to household robots, advancements in AI help transform science fiction into real systems.

CS188.1x focuses on Behavior from Computation. It will introduce the basic ideas and techniques underlying the design of intelligent computer systems. A specific emphasis will be on the statistical and decision–theoretic modeling paradigm. By the end of this course, you will have built autonomous agents that efficiently make decisions in stochastic and in adversarial settings. CS188.2x (to follow CS188.1x, precise date to be determined) will cover Reasoning and Learning. With this additional machinery your agents will be able to draw inferences in uncertain environments and optimize actions for arbitrary reward structures. Your machine learning algorithms will classify handwritten digits and photographs. The techniques you learn in CS188x apply to a wide variety of artificial intelligence problems and will serve as the foundation for further study in any application area you choose to pursue.

May 16, 2013

Linguists Circle the Wagons, or Disagreement != Danger

Filed under: Artificial Intelligence,Linguistics,Natural Language Processing — Patrick Durusau @ 2:47 pm

Pullum’s NLP Lament: More Sleight of Hand Than Fact by Christopher Phipps.

From the post:

My first reading of both of Pullum’s recent NLP posts (one and two) interpreted them to be hostile, an attack on a whole field (see my first response here). Upon closer reading, I see Pullum chooses his words carefully and it is less of an attack and more of a lament. He laments that the high-minded goals of early NLP (to create machines that process language like humans do) has not been reached, and more to the point, that commercial pressures have distracted the field from pursuing those original goals, hence they are now neglected. And he’s right about this to some extent.

But, he’s also taking the commonly used term “natural language processing” and insisting that it NOT refer to what 99% of people who use the term use it for, but rather only a very narrow interpretation consisting of something like “computer systems that mimic human language processing.” This is fundamentally unfair.

In the 1980s I was convinced that computers would soon be able to simulate the basics of what (I hope) you are doing right now: processing sentences and determining their meanings.

I feel Pullum is moving the goal posts on us when he says “there is, to my knowledge, no available system for unaided machine answering of free-form questions via general syntactic and semantic analysis” [my emphasis]. Pullum’s agenda appears to be to create a straw-man NLP world where NLP techniques are only admirable if they mimic human processing. And this is unfair for two reasons.

If there is unfairness in this discussion, it is the insistence by Christopher Phipps (and others) that Pullum has invented “…a straw-man NLP world where NLP techniques are only admirable if they mimic human processing.”

On the contrary, it was 1949 when Warren Weaver first proposed computers as the solution to world-wide translation problems. Weaver’s was not the only optimistic projection of language processing by computers. Those have continued up to and including the Semantic Web.

Yes, NLP practitioners such as Christopher Phipps use NLP in a more precise sense than Pullum. And NLP as defined by Phipps has too many achievements to easily list.

Neither one of those statements takes anything away from Pullum’s point that Google found a “sweet spot” between machine processing and human intelligence for search purposes.

What other insights Pullum has to offer may be obscured by the “…circle the wagons…” attitude from linguists.

Disagreement != Danger.

May 3, 2013

Deep learning made easy

Filed under: Artificial Intelligence,Deep Learning,Machine Learning,Sparse Data — Patrick Durusau @ 1:06 pm

Deep learning made easy by Zygmunt Zając.

From the post:

As usual, there’s an interesting competition at Kaggle: The Black Box. It’s connected to ICML 2013 Workshop on Challenges in Representation Learning, held by the deep learning guys from Montreal.

There are a couple benchmarks for this competition and the best one is unusually hard to beat – only less than a fourth of those taking part managed to do so. We’re among them. Here’s how.

The key ingredient in our success is a recently developed secret Stanford technology for deep unsupervised learning, called sparse filtering. Actually, it’s not secret. It’s available at Github, and has one or two very appealling properties. Let us explain.

The main idea of deep unsupervised learning, as we understand it, is feature extraction. One of the most common applications are in multimedia. The reason for that is that multimedia tasks, for example object recognition, are easy for humans, but difficult for the computers*.

Geoff Hinton from Toronto talks about two ends of spectrum in machine learning: one is statistics and getting rid of noise, the other one – AI, or the things that humans are good at but computers are not. Deep learning proponents say that deep, that is, layered, architectures, are the way to solve AI kind of problems.

The idea might have something to do with an inspiration from how the brain works. Each layer is supposed to extract higher-level features, and these features are supposed to be more useful for the task at hand.

Rather say layered architectures are observed to mimic human results.

Just as a shovel mimics and exceeds a human hand for digging.

But you would not say operation of a shovel gives us insight into the operation of a human hand.

Or would you?

March 20, 2013

Large-Scale Learning with Less… [Less Precision Viable?]

Filed under: Algorithms,Artificial Intelligence,Machine Learning — Patrick Durusau @ 4:32 pm

Large-Scale Learning with Less RAM via Randomization by Daniel Golovin, D. Sculley, H. Brendan McMahan, Michael Young.

Abstract:

We reduce the memory footprint of popular large-scale online learning methods by projecting our weight vector onto a coarse discrete set using randomized rounding. Compared to standard 32-bit float encodings, this reduces RAM usage by more than 50% during training and by up to 95% when making predictions from a fixed model, with almost no loss in accuracy. We also show that randomized counting can be used to implement per-coordinate learning rates, improving model quality with little additional RAM. We prove these memory-saving methods achieve regret guarantees similar to their exact variants. Empirical evaluation confirms excellent performance, dominating standard approaches across memory versus accuracy tradeoffs.

I mention this in part because topic map authoring can be assisted by the results of machine learning.

It is also a data point for the proposition that unlike their human masters, machines are too precise.

Perhaps it is the case that the vagueness of human reasoning has significant advantages over the disk grinding precision of our machines.

The question then becomes: How do we capture vagueness in a system where every point is either 0 or 1?

Not probabilistic because that can be expressed but vagueness, which I experience as something different.

Suggestions?

PS: Perhaps that is what makes artificial intelligence artificial. It is too precise. 😉

I first saw this in a tweet by Stefano Bertolo.

March 19, 2013

AI Algorithms, Data Structures, and Idioms…

Filed under: Algorithms,Artificial Intelligence,Data Structures,Java,Lisp,Prolog — Patrick Durusau @ 10:51 am

AI Algorithms, Data Structures, and Idioms in Prolog, Lisp and Java by George F. Luger and William A. Stubblefield.

From the introduction:

Writing a book about designing and implementing representations and search algorithms in Prolog, Lisp, and Java presents the authors with a number of exciting opportunities.

The first opportunity is the chance to compare three languages that give very different expression to the many ideas that have shaped the evolution of programming languages as a whole. These core ideas, which also support modern AI technology, include functional programming, list processing, predicate logic, declarative representation, dynamic binding, meta-linguistic abstraction, strong-typing, meta-circular definition, and object-oriented design and programming. Lisp and Prolog are, of course, widely recognized for their contributions to the evolution, theory, and practice of programming language design. Java, the youngest of this trio, is both an example of how the ideas pioneered in these earlier languages have shaped modern applicative programming, as well as a powerful tool for delivering AI applications on personal computers, local networks, and the world wide web.

Where could you go wrong with comparing Prolog, Lisp and Java?

Either for the intellectual exercise or because you want a better understanding of AI, a resource to enjoy!

Easy 6502

Filed under: Artificial Intelligence,Programming — Patrick Durusau @ 5:18 am

Easy 6502 by Nick Morgan.

From the webpage:

In this tiny ebook I’m going to show you how to get started writing 6502 assembly language. The 6502 processor was massive in the seventies and eighties, powering famous computers like the BBC Micro, Atari 2600, Commodore 64, Apple II, and the Nintendo Entertainment System. Bender in Futurama has a 6502 processor for a brain. Even the Terminator was programmed in 6502.

So, why would you want to learn 6502? It’s a dead language isn’t it? Well, so’s Latin. And they still teach that. Q.E.D.

(Actually, I’ve been reliably informed that 6502 processors are still being produced by Western Design Center, so clearly 6502 isn’t a dead language! Who knew?)

Seriously though, I think it’s valuable to have an understanding of assembly language. Assembly language is the lowest level of abstraction in computers – the point at which the code is still readable. Assembly language translates directly to the bytes that are executed by your computer’s processor. If you understand how it works, you’ve basically become a computer magician.

Then why 6502? Why not a useful assembly language, like x86? Well, I don’t think learning x86 is useful. I don’t think you’ll ever have to write assembly language in your day job – this is purely an academic exercise, something to expand your mind and your thinking. 6502 was originally written in a different age, a time when the majority of developers were writing assembly directly, rather than in these new-fangled high-level programming languages. So, it was designed to be written by humans. More modern assembly languages are meant to written by compilers, so let’s leave it to them. Plus, 6502 is fun. Nobody ever called x86 fun.

A useful reminder about the nature of processing in computers.

Whatever a high level language may imply to you, for your computer, it’s just instructions.

February 22, 2013

“…the flawed man versus machine dichotomy”

Filed under: Artificial Intelligence,BigData — Patrick Durusau @ 6:50 am

The backlash against Big Data has started

Kaiser Fung critiques a recent criticism of big data saying:

Andrew Gelman has a beef with David Brooks over his New York Times column called “What Data Can’t Do”. (link) I will get to Brooks’s critique soon–my overall feeling is, he created a bunch of sound bites, and could have benefited from interviewing people like Andrew and myself, who are skeptical of Big Data claims but not maniacally dismissive.

The biggest issue with Brooks’s column is the incessant use of the flawed man versus machine dichotomy. He warns: “It’s foolish to swap the amazing machine in your skull for the crude machine on your desk.” The machine he has in his mind is the science-fictional, self-sufficient, intelligent computer, as opposed to the algorithmic, dumb-and-dumber computer as it exists today and for the last many decades. A more appropriate analogy of today’s computer (and of the foreseeable future) is a machine that the human brain creates to automate mechanical, repetitious tasks at scale. This machine cannot function without human piloting so it’s man versus man-plus-machine, not man versus machine. (emphasis added)

I would have to plead guilty to falling into that “…flawed man versus machine dichotomy.”

And why not?

When machinery gives absurd answers, such as matching children to wanted terrorists and their human counterparts, blindly accept the conclusion, there is cause for concern.

Kaiser concludes:

Brooks made a really great point at the end of the piece, which I will paraphrase: any useful data is cooked. “The end result looks disinterested, but, in reality, there are value choices all the way through, from construction to interpretation.” Instead of thinking about this as cause for concern, we should celebrate these “value choices” because they make the data more useful.

This brings me back to Gelman’s reaction in which he differentiates between good analysis and bad analysis. Except for the simplest problems, any good analysis uses cooked data but an analysis using cooked data could be good or bad.

Perhaps my criticism should be of people who conceal their “value choices” amidst machinery.

There may be disinterested machines, but only the the absence of people and their input.

Yes?

February 11, 2013

Label propagation in GraphChi

Filed under: Artificial Intelligence,Classifier,GraphChi,Graphs,Machine Learning — Patrick Durusau @ 4:12 pm

Label propagation in GraphChi by Danny Bickson.

From the post:

A few days ago I got a request from Jidong, from the Chinese Renren company to implement label propagation in GraphChi. The algorithm is very simple described here: Zhu, Xiaojin, and Zoubin Ghahramani. Learning from labeled and unlabeled data with label propagation. Technical Report CMU-CALD-02-107, Carnegie Mellon University, 2002.

The basic idea is that we start with a group of users that we have some information about the categories they are interested in. Following the weights in the social network, we propagate the label probabilities from the user seed node (the ones we have label information about) into the general social network population. After several iterations, the algorithm converges and the output is labels for the unknown nodes.

I assume there is more unlabeled data for topic maps than labeled data.

Depending upon your requirements, this could prove to be a useful technique for completing those unlabeled nodes.

February 2, 2013

Simulating the European Commission

Filed under: Artificial Intelligence,EU — Patrick Durusau @ 3:07 pm

Did you see Gary Marcus’ “We are not yet ready to simulate the brain,” last Thursday’s Financial Times?

Gary writes:

The 10-year €1.19bn project to simulate the entire human brain, announced on Monday by the European Commission is, at about a sixth of the cost of the Large Hadron Collider, the biggest neuroscience project undertaken. It is an important, but flawed, step to a better understanding of the organ’s workings.

His analysis is telling but he misses the true goal of the project even as he writes:

Even so, it could foster a great deal of useful science. The crucial question is how the money will be spent. Much of the infrastructure developed will serve a vast number of projects, and the funding will support more than 250 scientists from more than 80 institutions, each with his or her own research agenda. A great many, such as Yadin Dudai (who specialises in memory), Seth Grant (who studies the genetics and evolution of neural function) and Stanislas Dehaene (who works on the brain basis of mathematics and consciousness), are stellar.

Supporting researchers, +1! Building the infrastructure of drones, managers, auditors, meeting coordinators and the like for this project, -1!

Every field of research could benefit from the funding that will now be diverted into “infrastructure” that exists only to be “infrastructure” (read employment).

My counter proposal is to simulate the EU commission using Steven Santy’s online “Magic Eight Ball.”

Put the question: Should project [name] be funded? to the Magic Eight Ball as many times as there are EU votes on projects and sum the answers.

Would avoid some of the “infrastructure” expenses and result in equivalent funding decisions.

If that sounds harsh, recall EU provincialism funds only EU-based research. As though scientific research and discovery depends upon nationality or geographic location. In that regard, the EU is like Alabama, only larger.

January 26, 2013

Human Computation and Crowdsourcing

Filed under: Artificial Intelligence,Crowd Sourcing,Human Computation,Machine Learning — Patrick Durusau @ 1:42 pm

Announcing HCOMP 2013 – Conference on Human Computation and Crowdsourcing by Eric Horvitz.

From the conference website:

Where

Palm Springs, California
Venue information coming soon

When

November 7-9, 2013

Important Dates

All deadlines are 5pm Pacific time unless otherwise noted.

Papers

Submission deadline: May 1, 2013
Author rebuttal period: June 21-28
Notification: July 16, 2013
Camera Ready: September 4, 2013

Workshops & Tutorials

Proposal deadline: May 10, 2013
Notification: July 16, 2013
Camera Ready: September 4, 2013

Posters & Demonstrations

Submission deadline: July 25, 2013
Notification: August 26, 2013
Camera Ready: September 4, 2013

From the post:

Announcing HCOMP 2013, the Conference on Human Computation and Crowdsourcing, Palm Springs, November 7-9, 2013. Paper submission deadline is May 1, 2013. Thanks to the HCOMP community for bringing HCOMP to life as a full conference, following on the successful workshop series.

The First AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2013) will be held November 7-9, 2013 in Palm Springs, California, USA. The conference was created by researchers from diverse fields to serve as a key focal point and scholarly venue for the review and presentation of the highest quality work on principles, studies, and applications of human computation. The conference is aimed at promoting the scientific exchange of advances in human computation and crowdsourcing among researchers, engineers, and practitioners across a spectrum of disciplines. Papers submissions are due May 1, 2013 with author notification on July 16, 2013. Workshop and tutorial proposals are due May 10, 2013. Posters & demonstrations submissions are due July 25, 2013.

I suppose it had to happen.

Instead of asking adding machines for their opinions, someone would decide to ask the creators of adding machines for theirs.

I first saw this at: New AAAI Conference on Human Computation and Crowdsourcing by Shar Steed.

October 31, 2012

Artificial Intelligence – Fall 2012 – CMU

Filed under: Artificial Intelligence,CS Lectures — Patrick Durusau @ 4:25 pm

Artificial Intelligence – Fall 2012 – CMU by Emma Brunskill and Ariel Procaccia.

From the course overview:

Topics:

This course is about the theory and practice of Artificial Intelligence. We will study modern techniques for computers to represent task-relevant information and make intelligent (i.e. satisfying or optimal) decisions towards the achievement of goals. The search and problem solving methods are applicable throughout a large range of industrial, civil, medical, financial, robotic, and information systems. We will investigate questions about AI systems such as: how to represent knowledge, how to effectively generate appropriate sequences of actions and how to search among alternatives to find optimal or near-optimal solutions. We will also explore how to deal with uncertainty in the world, how to learn from experience, and how to learn decision rules from data. We expect that by the end of the course students will have a thorough understanding of the algorithmic foundations of AI, how probability and AI are closely interrelated, and how automated agents learn. We also expect students to acquire a strong appreciation of the big-picture aspects of developing fully autonomous intelligent agents. Other lectures will introduce additional aspects of AI, including unsupervised and on-line learning, autonomous robotics, and economic/game-theoretic decision making.

Learning Objectives

By the end of the course, students should be able to:

  1. Identify the type of an AI problem (search, inference, decision making under uncertainty, game theory, etc).
  2. Formulate the problem as a particular type. (Example: define a state space for a search problem)
  3. Compare the difficulty of different versions of AI problems, in terms of computational complexity and the efficiency of existing algorithms.
  4. Implement, evaluate and compare the performance of various AI algorithms. Evaluation could include empirical demonstration or theoretical proofs.

Textbook:

It is helpful, but not required, to have Artificial Intelligence: A Modern Approach / Russel and Norvig.

Judging from the materials on the website, this is a very good course.

October 21, 2012

7 John McCarthy Papers in 7 weeks – Prologue

Filed under: Artificial Intelligence,CS Lectures,Lisp — Patrick Durusau @ 6:28 pm

7 John McCarthy Papers in 7 weeks – Prologue by Carin Meier.

From the post:

In the spirit of Seven Languages in Seven Weeks, I have decided to embark on a quest. But instead of focusing on expanding my mindset with different programming languages, I am focusing on trying to get into the mindset of John McCarthy, father of LISP and AI, by reading and thinking about seven of his papers.

See Carin’s blog for progress so far.

I first saw this at John D. Cooks’s The Endeavor

How would you react to something similar for topic maps?

October 10, 2012

Artificial Intelligence and Machine Learning [Mid-week present]

Filed under: Artificial Intelligence,Machine Learning — Patrick Durusau @ 4:20 pm

Artificial Intelligence and Machine Learning (Research at Google)

I assume you have been good so far this week so time for a mid-week present!

As of today, a list of two hundred and forty-nine publications in artificial intelligence and machine learning from Google Research!

From the webpage:

Much of our work on language, speech, translation, and visual processing relies on Machine Learning and AI. In all of those tasks and many others, we gather large volumes of direct or indirect evidence of relationships of interest, and we apply learning algorithms to generalize from that evidence to new cases of interest. Machine Learning at Google raises deep scientific and engineering challenges. Contrary to much of current theory and practice, the statistics of the data we observe shifts very rapidly, the features of interest change as well, and the volume of data often precludes the use of standard single-machine training algorithms. When learning systems are placed at the core of interactive services in a rapidly changing and sometimes adversarial environment, statistical models need to be combined with ideas from control and game theory, for example when using learning in auction algorithms.

Research at Google is at the forefront of innovation in Machine Learning with one of the most active groups working on virtually all aspects of learning, theory as well as applications, and a strong academic presence through technical talks and publications in major conferences and journals.

Don’t neglect your “real” work but either find a paper relevant to your “real” work or read one during lunch or on break.

You will be glad you did!

September 3, 2012

Google at UAI 2012

Filed under: Artificial Intelligence — Patrick Durusau @ 2:23 pm

Google at UAI 2012 by Kevin Murphy.

From the post:

The conference on Uncertainty in Artificial Intelligence (UAI) is one of the premier venues for research related to probabilistic models and reasoning under uncertainty. This year’s conference (the 28th) set several new records: the largest number of submissions (304 papers, last year 285), the largest number of participants (216, last year 191), the largest number of tutorials (4, last year 3), and the largest number of workshops (4, last year 1). We interpret this as a sign that the conference is growing, perhaps as part of the larger trend of increasing interest in machine learning and data analysis.

There were many interesting presentations. A couple of my favorites included:

  • “Video In Sentences Out,” by Andrei Barbu et al. This demonstrated an impressive system that is able to create grammatically correct sentences describing the objects and actions occurring in a variety of different videos.
  • “Exploiting Compositionality to Explore a Large Space of Model Structures,” by Roger Grosse et al. This paper (which won the Best Student Paper Award) proposed a way to view many different latent variable models for matrix decomposition – including PCA, ICA, NMF, Co-Clustering, etc. – as special cases of a general grammar. The paper then showed ways to automatically select the right kind of model for a dataset by performing greedy search over grammar productions, combined with Bayesian inference for model fitting.

You can find other individual papers at: Schedule UAI 2012.

Or you can grab the entire proceedings. (972 page PDF file)

Either way, you will find numerous items for exploration and conversation.

August 29, 2012

DARPA Seeking Unconventional Processors for ISR Data Analysis [Analog Computing By Another Name]

Filed under: Analog Computing,Artificial Intelligence — Patrick Durusau @ 2:40 pm

DARPA Seeking Unconventional Processors for ISR Data Analysis by Erwin Gianchandani.

From the post:

Earlier this month, the Defense Advanced Research Projects Agency (DARPA) announced a new initiative that aims “to break the status quo of digital processing” by investigating new ways of “non-digital” computation that are “fundamentally different from current digital processors and the power and speed limitations associated with them.” Called Unconventional Processing of Signals for Intelligent Data Exploitation, or UPSIDE, the initiative specifically seeks “a new, ultra-low power processing method [that] may enable faster, mission-critical analysis of [intelligence, surveillance, and reconnaissance (ISR)] data.”

According to the DARPA announcement (after the jump):

Instead of traditional complementary metal-oxide-semiconductor (CMOS)-based electronics, UPSIDE envisions arrays of physics-based devices (nanoscale oscillators may be one example) performing the processing. These arrays would self-organize and adapt to inputs, meaning that they will not need to be programmed as digital processors are. Unlike traditional digital processors that operate by executing specific instructions to compute, it is envisioned that the UPSIDE arrays will rely on a higher level computational element based on probabilistic inference embedded within a digital system.

Probabilistic inference is the fundamental computational model for the UPSIDE program. An inference process uses energy minimization to determine a probability distribution to find the object that is the most likely interpretation of the sensor data. It can be implemented directly in approximate precision by traditional semiconductors as well as by new kinds of emerging devices.

DARPA program manager Dan Hammerstrom noted:

“Redefining the fundamental computation as inference could unlock processing speeds and power efficiency for visual data sets that are not currently possible. DARPA hopes that this type of technology will not only yield faster video and image analysis, but also lend itself to being scaled for increasingly smaller platforms.

“Leveraging the physics of devices to perform computations is not a new idea, but it is one that has never been fully realized. However, digital processors can no longer keep up with the requirements of the Defense mission. We are reaching a critical mass in terms of our understanding of the required algorithms, of probabilistic inference and its role in sensor data processing, and the sophistication of new kinds of emerging devices. At DARPA, we believe that the time has come to fund the development of systems based on these ideas and take computational capabilities to the next level.”

How much “…not a new idea, but it is one that has never been fully realized[?]”

If you search for “analog computing,” you will get a good idea of how old and how useful a concept it has been.

You can jump to the Wikipedia article, Analog Computer or take a brief tour with the Analog Computer Manual.

Please post a note if you experiment with analog computing and subject identity processing.

Or if you decide that models for chemical reactions in the human brain should be analog ones and not digital.

July 5, 2012

JMLR – Journal of Machine Learning Research

Filed under: Artificial Intelligence,Machine Learning — Patrick Durusau @ 10:37 am

JMLR – Journal of Machine Learning Research

From the webpage:

The Journal of Machine Learning Research (JMLR) provides an international forum for the electronic and paper publication of high-quality scholarly articles in all areas of machine learning. All published papers are freely available online.

Starts with volume 1 in October of 2000 and continues to the present.

Special topics that call out articles from different issues and special issues are also listed.

A first rate collection of machine learning research.

May 28, 2012

Streaming Analytics: with sparse distributed representations

Filed under: Artificial Intelligence,Grok - Numenta,High Order Sequence Memory,Machine Learning,Neuroinformatics,Sparse Distributed Representation SDR — Patrick Durusau @ 10:22 am

Streaming Analytics: with sparse distributed representations by Jeff Hawkins.

Abstract:

Sparse distributed representations appear to be the means by which brains encode information. They have several advantageous properties including the ability to encode semantic meaning. We have created a distributed memory system for learning sequences of sparse distribute representations. In addition we have created a means of encoding structured and unstructured data into sparse distributed representations. The resulting memory system learns in an on-line fashion making it suitable for high velocity data streams. We are currently applying it to commercially valuable data streams for prediction, classification, and anomaly detection In this talk I will describe this distributed memory system and illustrate how it can be used to build models and make predictions from data streams.

Slides: http://www.numenta.com/htm-overview/05-08-2012-Berkeley.pdf

Looking forward to learning more about “sparse distributed representation (SDR).”

Not certain about Jeff’s claim that matching across SDRs = semantic similarity.

Design of the SDR determines the meaning of each bit and consequently of matching.

Which feeds back into the encoders that produce the SDRs.

Other resources:

The core paper: Hierarchical Temporal Memory including HTM Cortical Learning Algorithms. Check the FAQ link if you need the paper in Chinese, Japanese, Korean, Portuguese, Russian, or Spanish. (unverified translations)

Grok – Frequently Asked Questions

A very good FAQ that goes a long way to explaining the capabilities and limitations (currently) of Grok. “Unstructured text” for example isn’t appropriate input into Grok.

Jeff Hawkins and Sandra Blakeslee co-authored On Intelligence in 2004. The FAQ describes the current work as an extension of “On Intelligence.”

BTW, if you think you have heard the name Jeff Hawkins before, you have. Inventor of the Palm Pilot among other things.

« Newer Posts — Older Posts »

Powered by WordPress