Modern Science and the Bayesian-Frequentist Controversy is a post by John Myles White, citing an essay by Bradley Efron, Modern Science and the Bayesian-Frequentist Controversy.
Topic maps will be using statistical means to manage large data sets so I commend the posting and essay to you for review.
Lars Marius Garshol walks through finding duplicate records in data records.
As Lars notes, there are commercial products for this same task but I think this is a useful exercise.
Isn’t that hard to imagine the creation of test data sets with a variety of conditions to underscore lessons about detecting duplicate records.
I suspect such training data may already be available.
Will have to see what I can find and post about it.
*****
PS: Lars is primary editor of the TMDM, working on TMCL and several other parts of the topic maps standard.
Which Automatic Differentiation Tool for C/C++?
OK, not immediately obvious why this is relevant to topic maps.
Nor is Bob Carpenter’s references:
I’ve been playing with all sorts of fun new toys at the new job at Columbia and learning lots of new algorithms. In particular, I’m coming to grips with Hamiltonian (or hybrid) Monte Carlo, which isn’t as complicated as the physics-based motivations may suggest (see the discussion in David MacKay’s book and then move to the more detailed explanation in Christopher Bishop’s book).
particularly useful.
I suspect the two book references are:
but I haven’t asked. In part to illustrate the problem of resolving any entity reference. Both authors have authored other books touching on the same subjects so my guesses may or may not be correct.
Oh, relevance to topic maps. The technique automatic differentiation is used in Hamiltonian Monte Carlo methods for the generation of gradients. Still not helpful? Isn’t to me either.
Ah, what about Bayesian models in IR? That made the light go on!
I will be discussing ways to show more immediate relevance to topic maps, at least for some posts, in post #1000.
It isn’t as far away as you might think.
Modeling Social Annotation: A Bayesian Approach Authors: Anon Plangprasopchok, Kristina Lerman
Abstract:
Collaborative tagging systems, such as Delicious, CiteULike, and others, allow users to annotate resources, for example, Web pages or scientific papers, with descriptive labels called tags. The social annotations contributed by thousands of users can potentially be used to infer categorical knowledge, classify documents, or recommend new relevant information. Traditional text inference methods do not make the best use of social annotation, since they do not take into account variations in individual users’ perspectives and vocabulary. In a previous work, we introduced a simple probabilistic model that takes the interests of individual annotators into account in order to find hidden topics of annotated resources. Unfortunately, that approach had one major shortcoming: the number of topics and interests must be specified a priori. To address this drawback, we extend the model to a fully Bayesian framework, which offers a way to automatically estimate these numbers. In particular, the model allows the number of interests and topics to change as suggested by the structure of the data. We evaluate the proposed model in detail on the synthetic and real-world data by comparing its performance to Latent Dirichlet Allocation on the topic extraction task. For the latter evaluation, we apply the model to infer topics of Web resources from social annotations obtained from Delicious in order to discover new resources similar to a specified one. Our empirical results demonstrate that the proposed model is a promising method for exploiting social knowledge contained in user-generated annotations.
Questions:
Inductive Logic Programming: Theory and Methods Authors: Stephen Muggleton, Luc De Raedt
Abstract:
Inductive Logic Programming (ILP) is a new discipline which investigates the inductive construction of first-order clausal theories from examples and background knowledge. We survey the most important theories and methods of this new eld. Firstly, various problem specifications of ILP are formalised in semantic settings for ILP, yielding a “model-theory” for ILP. Secondly, a generic ILP algorithm is presented. Thirdly, the inference rules and corresponding operators used in ILP are presented, resulting in a “proof-theory” for ILP. Fourthly, since inductive inference does not produce statements which are assured to follow from what is given, inductive inferences require an alternative form of justification. This can take the form of either probabilistic support or logical constraints on the hypothesis language. Information compression techniques used within ILP are presented within a unifying Bayesian approach to confirmation and corroboration of hypotheses. Also, different ways to constrain the hypothesis language, or specify the declarative bias are presented. Fifthly, some advanced topics in ILP are addressed. These include aspects of computational learning theory as applied to ILP, and the issue of predicate invention. Finally, we survey some applications and implementations of ILP. ILP applications fall under two different categories: firstly scientific discovery and knowledge acquisition, and secondly programming assistants.
A good survey of Inductive Logic Programming (ILP) if a bit dated. Feel free to suggest more recent surveys of the area.
As I mentioned under Mining Travel Resources on the Web Using L-Wrappers, the notion of interpretative domains is quite interesting.
I suspect, but cannot prove (at least at this point), that most useful mappings exist between closely related interpretative domains.
Closely related interpretative domains being composed of identifications of a subject that I will quickly recognize as alternative identifications.
Showing me a mapping that includes a Martian identification of my subject, which is not a closely related interpretative domain is unlikely to be useful, at least to me. (I can’t speak for any potential Martians.)
Dirichlet Processes: Tutorial and Practical Course Author: Yee Whye Teh
Slides
Paper
Abstract:
The Bayesian approach allows for a coherent framework for dealing with uncertainty in machine learning. By integrating out parameters, Bayesian models do not suffer from overfitting, thus it is conceivable to consider models with infinite numbers of parameters, aka Bayesian nonparametric models. An example of such models is the Gaussian process, which is a distribution over functions used in regression and classification problems. Another example is the Dirichlet process, which is a distribution over distributions. Dirichlet processes are used in density estimation, clustering, and nonparametric relaxations of parametric models. It has been gaining popularity in both the statistics and machine learning communities, due to its computational tractability and modelling flexibility.
In the tutorial I shall introduce Dirichlet processes, and describe different representations of Dirichlet processes, including the Blackwell-MacQueen? urn scheme, Chinese restaurant processes, and the stick-breaking construction. I shall also go through various extensions of Dirichlet processes, and applications in machine learning, natural language processing, machine vision, computational biology and beyond.
In the practical course I shall describe inference algorithms for Dirichlet processes based on Markov chain Monte Carlo sampling, and we shall implement a Dirichlet process mixture model, hopefully applying it to discovering clusters of NIPS papers and authors.
With the last two posts, that is almost 8 hours of video for streaming to your new phone or other personal device.
That should get you past even a Christmas day sports marathon at your in-laws house (or your own should they be visiting).
Bayesian inference and Gaussian processes Authors: Carl Edward Rasmussen
Quite useful as the presenter concludes with disambiguating terminology used differently in the field. Same terms used to mean different things, different terms to mean the same thing. Hmmm, that sounds really familiar. 😉
Start with this lecture before Dirichlet Processes: Tutorial and Practical Course
BTW, if this seems a bit AI-ish, consider it to be the reverse of supervised classification (person helps machine), that is machine helps person, but the person should say when answer is correct.
Graphical Models Author: Zoubin Ghahramani
Abstract:
An introduction to directed and undirected probabilistic graphical models, including inference (belief propagation and the junction tree algorithm), parameter learning and structure learning, variational approximations, and approximate inference.
- Introduction to graphical models: (directed, undirected and factor graphs; conditional independence; d-separation; plate notation)
- Inference and propagation algorithms: (belief propagation; factor graph propagation; forward-backward and Kalman smoothing; the junction tree algorithm)
- Learning parameters and structure: maximum likelihood and Bayesian parameter learning for complete and incomplete data; EM; Dirichlet distributions; score-based structure learning; Bayesian structural EM; brief comments on causality and on learning undirected models)
- Approximate Inference: (Laplace approximation; BIC; variational Bayesian EM; variational message passing; VB for model selection)
- Bayesian information retrieval using sets of items: (Bayesian Sets; Applications)
- Foundations of Bayesian inference: (Cox Theorem; Dutch Book Theorem; Asymptotic consensus and certainty; choosing priors; limitations)
Start with this lecture before Dirichlet Processes: Tutorial and Practical Course
Philippe Cudré-Mauroux Video, Slides from SOKS: Self-Organising Knowledge Systems, Amsterdam, 29 April 2010
Abstract:
Emergent semantics refers to a set of principles and techniques analyzing the evolution of decentralized semantic structures in large scale distributed information systems. Emergent semantics approaches model the semantics of a distributed system as an ensemble of relationships between syntactic structures.
They consider both the representation of semantics and the discovery of the proper interpretation of symbols as the result of a self-organizing process performed by distributed agents exchanging symbols and having utilities dependent on the proper interpretation of the symbols. This is a complex systems perspective on the problem of dealing with semantics.
A “must see” presentation!
More comments/questions to follow.
*****
Apologies but content/postings will be slow starting today, for a few days. Diagnostic on left hand has me doing hunt-and-peck with my right.
Machine Learning and Data Mining with R
Announcement of course notes and slides, plus live classes in San Francisco, January 2012, courtesy of the Revolutions blog from Revolution Analytics.
Check the post for details and links.
Powered by WordPress