Another Word For It Patrick Durusau on Topic Maps and Semantic Diversity

February 22, 2019

Interpretable Machine Learning

Filed under: Machine Learning — Patrick Durusau @ 5:10 pm

Interpretatable Machine Learning: A Guide for Making Black Box Models Explainable by Christoph Molnar.

From the introduction:

Machine learning has great potential for improving products, processes and research. But computers usually do not explain their predictions which is a barrier to the adoption of machine learning. This book is about making machine learning models and their decisions interpretable.
After exploring the concepts of interpretability, you will learn about simple, interpretable models such as decision trees, decision rules and linear regression. Later chapters focus on general model-agnostic methods for interpreting black box models like feature importance and accumulated local effects and explaining individual predictions with Shapley values and LIME.
All interpretation methods are explained in depth and discussed critically. How do they work under the hood? What are their strengths and weaknesses? How can their outputs be interpreted? This book will enable you to select and correctly apply the interpretation method that is most suitable for your machine learning project.
The book focuses on machine learning models for tabular data (also called relational or structured data) and less on computer vision and natural language processing tasks. Reading the book is recommended for machine learning practitioners, data scientists, statisticians, and anyone else interested in making machine learning models interpretable.

I can see two immediate uses for this book.

First, as Molnar states in the introduction, you can peirce the veil around machine learning and be able to explain why your model has reached a particular result. Think of it as transparency in machine learning.

Second, after peircing the veil around machine learning you can choose the model or nudge a model, into the direction of a result specified by management. Or having gotten a desired result, you can train a more obscure technique to replicate it. Think of it as opacity in machine learning.

Enjoy!

January 5, 2019

Papers With Code [Machine Learning]

Filed under: Machine Learning,Programming — Patrick Durusau @ 10:00 pm

Papers With Code by Zaur Fataliyev

From the webpage:

This work is in continuous progress and update. We are adding new PWC everyday! Tweet me @fvzaur.

Use this thread to request us your favorite conference to be added to our watchlist and to PWC list.

A truly remarkable collection of papers with code for machine learning.

Is this one of the first sites you hit in the morning?

November 26, 2018

Big Brother’s Machine Learning Courses (free) [Fire Prediction As Weapon]

Filed under: Machine Learning — Patrick Durusau @ 11:49 am

Amazon’s own ‘Machine Learning University’ now available to all developers by Dr. Matt Wood.

From the post:

Today, I’m excited to share that, for the first time, the same machine learning courses used to train engineers at Amazon are now available to all developers through AWS.

We’ve been using machine learning across Amazon for more than 20 years. With thousands of engineers focused on machine learning across the company, there are very few Amazon retail pages, products, fulfillment technologies, stores which haven’t been improved through the use of machine learning in one way or another. Many AWS customers share this enthusiasm, and our mission has been to take machine learning from something which had previously been only available to the largest, most well-funded technology companies, and put it in the hands of every developer. Thanks to services such as Amazon SageMaker, Amazon Rekognition, Amazon Comprehend, Amazon Transcribe, Amazon Polly, Amazon Translate, and Amazon Lex, tens of thousands of developers are already on their way to building more intelligent applications through machine learning.

Regardless of where they are in their machine learning journey, one question I hear frequently from customers is: “how can we accelerate the growth of machine learning skills in our teams?” These courses, available as part of a new AWS Training and Certification Machine Learning offering, are now part of my answer.

There are more than 30 self-service, self-paced digital courses with more than 45 hours of courses, videos, and labs for four key groups: developers, data scientists, data platform engineers, and business professionals. Each course starts with the fundamentals, and builds on those through real-world examples and labs, allowing developers to explore machine learning through some fun problems we have had to solve at Amazon. These include predicting gift wrapping eligibility, optimizing delivery routes, or predicting entertainment award nominations using data from IMDb (an Amazon subsidiary). Coursework helps consolidate best practices, and demonstrates how to get started on a range of AWS machine learning services, including Amazon SageMaker, AWS DeepLens, Amazon Rekognition, Amazon Lex, Amazon Polly, and Amazon Comprehend.

Machine learning from one of our digital big brothers at any rate.

The classes are tuned to the capabilities and features of AWS machine learning services but that’s a feature and not a bug.

AWS machine learning services are essential to anyone who doesn’t have the on-call capabilities of the CIA or NSA. Even with AWS, you won’t match the shere capacity of government computing environments, but you have one thing they don’t have, your insight into a problem set.

Let’s say that with enough insight and funds to pay for AWS services, you will be competitive against government agencies.

Wood continues:

To help developers demonstrate their knowledge (and to help employers hire more efficiently), we are also announcing the new “AWS Certified Machine Learning – Specialty” certification. Customers can take the exam now (and at half price for a limited time). Customers at re:Invent can sit for the exam this week at our Training and Certification exam sessions.

The digital courses are now available at no charge at aws.training/machinelearning and you only pay for the services you use in labs and exams during your training.

Fire is a weapon rarely exploited well by counter-government forces. Consider the use of AWS machine learning services to resolve the trade-off between the areas most likely to burn and those where a burn would be the most damaging (by some criteria). Climate change presents opportunities for unconventional insurgent techniques. Will you be ready to recognize and/or seize them?

November 21, 2018

Stanford AI Lab (SAIL) Blog (Attn: All Hats)

Filed under: Artificial Intelligence,Hacking,Machine Learning — Patrick Durusau @ 3:45 pm

Stanford AI Lab (SAIL) Blog

From the Hello World post:

We are excited to launch the Stanford AI Lab (SAIL) Blog, where we hope to share our research, high-level discussions on AI and machine learning, and updates with the general public. SAIL has 18 faculty and 16 affiliated faculty, with hundreds of students working in diverse fields that span natural language processing, robotics, computer vision, bioinformatics, and more. Our vision is to make that work accessible to an audience beyond the academic and technical community.

Whether you are a White, Black, Grey, or Customer, hat, start watching the Stanford AI Lab (SAIL) Blog.

Like a Customer hat, AI (artificial intelligence) knows no preset side, only its purpose as set by others.

If that sounds harsh, remember that it has been preset sides that force otherwise decent people (in some cases) to support the starvation of millions in Yemen or the murder of children in Palestine.

Or to say it differently, laws are only advisory opinions on the morality of any given act.

November 15, 2018

The Unlearned Lesson Of Amazon’s automated hiring tool

Filed under: Artificial Intelligence,Diversity,Machine Learning — Patrick Durusau @ 1:57 pm

Gender, Race and Power: Outlining a New AI Research Agenda.

From the post:


AI systems — which Google and others are rapidly developing and deploying in sensitive social and political domains — can mirror, amplify, and obscure the very issues of inequality and discrimination that Google workers are protesting against. Over the past year, researchers and journalists have highlighted numerous examples where AI systems exhibited biases, including on the basis of race, class, gender, and sexuality.

We saw a dramatic example of these problems in recent news of Amazon’s automated hiring tool. In order to “learn” to differentiate between “good” and “bad” job candidates, it was trained on a massive corpus of of (sic) data documenting the company’s past hiring decisions. The result was, perhaps unsurprisingly, a hiring tool that discriminated against women, even demoting CVs that contained the word ‘women’ or ‘women’s’. Amazon engineers tried to fix the problem, adjusting the algorithm in the attempt to mitigate its biased preferences, but ultimately scrapped the project, concluding that it was unsalvageable.

From the Amazon automated hiring tool and other examples, the AI Now Institute draws this conclusion:


It’s time for research on gender and race in AI to move beyond considering whether AI systems meet narrow technical definitions of ‘fairness.’ We need to ask deeper, more complex questions: Who is in the room when these technologies are created, and which assumptions and worldviews are embedded in this process? How does our identity shape our experiences of AI systems? In what ways do these systems formalize, classify, and amplify rigid and problematic definitions of gender and race? We share some examples of important studies that tackle these questions below — and we have new research publications coming out to contribute to this literature.

AI New misses the most obvious lesson from the Amazon automated hiring tool experience:

In the face of an AI algorithm that discriminates, we don’t know how to cure its discrimination.

Predicting or curing discrimination from an algorithm alone lies beyond our ken.

The creation of reference datasets for testing AI algorithms, however, enables testing and comparison of algorithms. With concrete results that could be used to reduce discrimination in fact.

Actual hiring and other databases are private for good reasons but wholly artificial reference databases would have no such concerns.

Since we don’t understand discrimination in humans, I caution against a quixotic search for its causes in algorithms. Keep or discard algorithms based on their discrimination in practice. Something we have been shown to be capable of spotting.

PS: Not all discrimination is unethical or immoral. If a position requires a law degree, it is “discrimination” to eliminate all applicants without one, but that’s allowable discrimination.

October 30, 2018

My favorite AI newsletters…

Filed under: Artificial Intelligence,Machine Learning — Patrick Durusau @ 7:10 pm

My favorite AI newsletters, run by people working in the field by Rosie Campbell.

Campbell lists her top five (5) AI newsletters. That’s a manageable number, at least if I discontinue other newsletters that fill my inbox.

Not that my current newsletter subscriptions aren’t valuable, but I’m not the web archive for those mailings and if I lack the time to read them, what’s the point?

It’s not Spring so I need to do some Fall cleaning of my newsletter subscriptions.

Any additions to those suggested by Campbell?

August 2, 2018

Learning Math for Machine Learning [for building products/conducting academic research]

Filed under: Machine Learning,Mathematics — Patrick Durusau @ 10:09 am

Learning Math for Machine Learning by Vincent Chen.

From the post:

It’s not entirely clear what level of mathematics is necessary to get started in machine learning, especially for those who didn’t study math or statistics in school.

In this piece, my goal is to suggest the mathematical background necessary to build products or conduct academic research in machine learning. These suggestions are derived from conversations with machine learning engineers, researchers, and educators, as well as my own experiences in both machine learning research and industry roles.

To frame the math prerequisites, I first propose different mindsets and strategies for approaching your math education outside of traditional classroom settings. Then, I outline the specific backgrounds necessary for different kinds of machine learning work, as these subjects range from high school-level statistics and calculus to the latest developments in probabilistic graphical models (PGMs). By the end of the post, my hope is that you’ll have a sense of the math education you’ll need to be effective in your machine learning work, whatever that may be!

I headlined:

…my goal is to suggest the mathematical background necessary to build products or conduct academic research in machine learning.

because the amount of math you need for machine learning depends on your use of machine learning tools.

If you intend to “build products or conduct academic research in machine learning,” then Chen’s post is as good a place to start as any. And knowing more math is always a good thing. If for no other reason than to challenge “machine learning” others try to foist off on you.

However, there are existing machine learning tools which come with their own documentation and lore about their use in a wide variety of situations.

I always applaud deeper understanding of vulnerabilities or code, but it isn’t necessary that you re-write every, most, some tools from scratch to be effective in using machine learning.

While learning the math of machine learning at your own pace, I suggest:

  1. Define the goal of your machine learning. Recommendation? Recognition?
  2. Define the subject area and likely inputs for your goal.
  3. Search for the use of your tool (if you already have one) and experience reports.
  4. Test and compare your results to industry reports in the same area.

My list assumes you already understand the goals of your client. Except in rare cases, machine learning is a means to reach those goals, not a goal itself.

June 7, 2018

Are AI Psychopaths Cost Effective?

Filed under: Artificial Intelligence,Machine Learning,Reddit — Patrick Durusau @ 3:34 pm

Norman, World’s first psychopath AI

From the webpage:


We present you Norman, world’s first psychopath AI. Norman is born from the fact that the data that is used to teach a machine learning algorithm can significantly influence its behavior. So when people talk about AI algorithms being biased and unfair, the culprit is often not the algorithm itself, but the biased data that was fed to it. The same method can see very different things in an image, even sick things, if trained on the wrong (or, the right!) data set. Norman suffered from extended exposure to the darkest corners of Reddit, and represents a case study on the dangers of Artificial Intelligence gone wrong when biased data is used in machine learning algorithms.

Norman is an AI that is trained to perform image captioning; a popular deep learning method of generating a textual description of an image. We trained Norman on image captions from an infamous subreddit (the name is redacted due to its graphic content) that is dedicated to document and observe the disturbing reality of death. Then, we compared Norman’s responses with a standard image captioning neural network (trained on MSCOCO dataset) on Rorschach inkblots; a test that is used to detect underlying thought disorders.

Note: Due to the ethical concerns, we only introduced bias in terms of image captions from the subreddit which are later matched with randomly generated inkblots (therefore, no image of a real person dying was utilized in this experiment).

I have written to the authors to ask for more details about their training process, the “…which are later matched with randomly generated inkblots…” seeming especially opaque to me.

While waiting for that answer, we should ask whether training psychopath AIs is cost effective?

Compare the limited MIT-Norman with the PeopleFuckingDying Reddit with 690,761 readers.

That’s a single Reddit. Many Reddits count psychopaths among their members. To say nothing of Twitter trolls and other social media where psychopaths gather.

New psychopaths appear on social media every day, without the ethics-limited training provided to MIT-Norman. Is this really a cost effective approach to developing psychopaths?

The MIT-Norman project has great graphics but Hitchcock demonstrated over and over again, simple scenes can be packed with heart pounding terror.

May 3, 2018

Not All AI Uses Are Serious: Generating Pusheen with AI

Filed under: Artificial Intelligence,Humor,Machine Learning — Patrick Durusau @ 8:22 pm

Generating Pusheen with AI by Zack Nado.

From the post:

I made a machine learning program that generates new (sometimes novel!) Pusheen pictures!

(I don’t claim any ownership of Pusheen or anything Pusheen related, which is trademarked by Pusheen Corp.) It’s no secret that my girlfriend and I both are huge fans of Pusheen the cat, which many people know from the cute Facebook sticker sets. So, for her birthday I set out to try to create a machine learning program to create cat pictures for her to enjoy! To set some expectations for this post, I only did this for a fun project and didn’t really know what I expected to get out of it given that in all the data available there are really only a handful of unique poses and scenes. Also, you really only need a roughly oval shaped gray blob with eyes to look like Pusheen, so the bar wasn’t that high. That being said I am happy with the outcome and think it produces interesting and (usually) realistic poses and positions.

Given the revolving door at the White House in Washington, DC, you will never be sort of material for generating facial or full body images for posting on social media.

None of them will be as attractive as Pusheen the cat, but, consider your starting point.

Enjoy!

PS: You will be laughing too much to notice you are learning AI skills. Does that suggest a possible approach to an introduction to AI?

April 26, 2018

Ethics and Law in AI ML

Filed under: Artificial Intelligence,Ethics,Machine Learning — Patrick Durusau @ 3:50 pm

Ethics and Law in AI ML

Data Science Renee, author of Women in Data Science (Twitter list with over 1400 members), has created a Twitter list focused on ethics and law in AI/ML.

When discussions of ethics for data scientists come up, remember that many players, corporations, governments, military organizations, spy agencies, abide by no code of ethics or laws. Adjust your ethics expectations accordingly.

February 10, 2018

MIT Intelligence Quest

Filed under: Artificial Intelligence,Machine Learning — Patrick Durusau @ 8:36 pm

MIT Intelligence Quest

From the webpage:

The MIT Intelligence Quest will advance the science and engineering of both human and machine intelligence. Launched on February 1, 2018, MIT IQ seeks to discover the foundations of human intelligence and drive the development of technological tools that can positively influence virtually every aspect of society.

The Institute’s culture of collaboration will encourage life scientists, computer scientists, social scientists, and engineers to join forces to investigate the societal implications of their work as they pursue hard problems lying beyond the current horizon of intelligence research. By uniting diverse fields and capitalizing on what they can teach each other, we seek to answer the deepest questions about intelligence.

We are setting out to answer two big questions: How does human intelligence work, in engineering terms? And how can we use that deep grasp of human intelligence to build wiser and more useful machines, to the benefit of society?

Drawing on MIT’s deep strengths and signature values, culture, and history, MIT IQ promises to make important contributions to understanding the nature of intelligence, and to harnessing it to make a better world.

The most refreshing aspect of the MIT Intelligence Quest page is that it ends a contact form.

That’s right, a contact form.

Unlike the ill-fated EU brain project that had pre-chosen approaches and had a roadmap for replicating a human brain. Are they still consuming funds with meetings, hotel rooms, etc.?

You know my mis-givings about creating intelligence in the absence of understanding our own.

On the other hand, mimicking how human intelligence works in bounded situations is a far more tractable problem.

Not too tractable but tractable enough to yield useful results.

February 7, 2018

The Matrix Calculus You Need For Deep Learning

Filed under: Deep Learning,Machine Learning,Mathematics — Patrick Durusau @ 9:22 pm

The Matrix Calculus You Need For Deep Learning by Terence Parr, Jeremy Howard.

Abstract:

This paper is an attempt to explain all the matrix calculus you need in order to understand the training of deep neural networks. We assume no math knowledge beyond what you learned in calculus 1, and provide links to help you refresh the necessary math where needed. Note that you do not need to understand this material before you start learning to train and use deep learning in practice; rather, this material is for those who are already familiar with the basics of neural networks, and wish to deepen their understanding of the underlying math. Don’t worry if you get stuck at some point along the way—just go back and reread the previous section, and try writing down and working through some examples. And if you’re still stuck, we’re happy to answer your questions in the Theory category at forums.fast.ai. Note: There is a reference section at the end of the paper summarizing all the key matrix calculus rules and terminology discussed here.

Here’s a recommendation for reading the paper:

(We teach in University of San Francisco’s MS in Data Science program and have other nefarious projects underway. You might know Terence as the creator of the ANTLR parser generator. For more material, see Jeremy’s fast.ai courses and University of San Francisco’s Data Institute in-person version of the deep learning course.

Apologies to Jeremy but I recognize ANTLR more quickly than I do Jeremy’s fast.ai courses. (Need to fix that.)

The paper runs thirty-three pages and as the authors say, most of it is unnecessary unless you want to understand what’s happening under the hood with deep learning.

Think of it as the difference between knowing how to drive a sports car and being able to work on a sports car.

With the latter set of skills, you can:

  • tweak your sports car for maximum performance
  • tweak someone else’s sports car for less performance
  • detect someone tweaking your sports car

Read the paper, master the paper.

No test, just real world consequences that separate the prepared from the unprepared.

February 6, 2018

Finally! A Main Stream Use for Deep Learning!

Filed under: Deep Learning,Humor,Machine Learning — Patrick Durusau @ 7:45 pm

Using deep learning to generate offensive license plates by Jonathan Nolis.

From the post:

If you’ve been on the internet for long enough you’ve seen quality content generated by deep learning algorithms. This includes algorithms trained on band names, video game titles, and Pokémon. As a data scientist who wants to keep up with modern tends in the field, I figured there would be no better way to learn how to use deep learning myself than to find a fun topic to generate text for. After having the desire to do this, I waited for a year before I found just the right data set to do it,

I happened to stumble on a list of banned license plates in Arizona. This list contains all of the personalized license plates that people requested but were denied by the Arizona Motor Vehicle Division. This dataset contained over 30,000 license plates which makes a great set of text for a deep learning algorithm. I included the data as text in my GitHub repository so other people can use it if they so choose. Unfortunately the data is from 2012, but I have an active Public Records Request to the state of Arizona for an updated list. I highly recommend you look through it, it’s very funny.

What a great idea! Not only are you learning deep learning but you are being offensive at the same time. A double-dipper!

A script for banging against your state license registration is left as an exercise for the reader.

A password generator using phonetics to spell offensive phrases for c-suite users would be nice.

January 18, 2018

For Some Definition of “Read” and “Answer” – MS Clickbait

Filed under: Artificial Intelligence,Machine Learning,Microsoft — Patrick Durusau @ 11:37 am

Microsoft creates AI that can read a document and answer questions about it as well as a person by Allison Linn.

From the post:

It’s a major milestone in the push to have search engines such as Bing and intelligent assistants such as Cortana interact with people and provide information in more natural ways, much like people communicate with each other.

A team at Microsoft Research Asia reached the human parity milestone using the Stanford Question Answering Dataset, known among researchers as SQuAD. It’s a machine reading comprehension dataset that is made up of questions about a set of Wikipedia articles.

According to the SQuAD leaderboard, on Jan. 3, Microsoft submitted a model that reached the score of 82.650 on the exact match portion. The human performance on the same set of questions and answers is 82.304. On Jan. 5, researchers with the Chinese e-commerce company Alibaba submitted a score of 82.440, also about the same as a human.

With machine reading comprehension, researchers say computers also would be able to quickly parse through information found in books and documents and provide people with the information they need most in an easily understandable way.

That would let drivers more easily find the answer they need in a dense car manual, saving time and effort in tense or difficult situations.

These tools also could let doctors, lawyers and other experts more quickly get through the drudgery of things like reading through large documents for specific medical findings or rarified legal precedent. The technology would augment their work and leave them with more time to apply the knowledge to focus on treating patients or formulating legal opinions.

Wait, wait! If you read the details about SQuAD, you realize how far Microsoft (or anyone else) is from “…reading through large documents for specific medical findings or rarified legal precedent….”

What is the SQuAD test?

Stanford Question Answering Dataset (SQuAD) is a new reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage. With 100,000+ question-answer pairs on 500+ articles, SQuAD is significantly larger than previous reading comprehension datasets.

Not to take anything away from Microsoft Research Asia or the creators of SQuAD, but “…the answer to every question is a segment of text, or span, from the corresponding reading passage.” is a long way from synthesizing an answer from a long legal document.

The first hurdle is asking a question that can be scored against every “…segment of text, or span…” such that a relevant snippet of text can be found.

The second hurdle is the process of scoring snippets of text in order to retrieve the most useful one. That’s a mechanical process, not one that depends on the semantics of the underlying question or text.

There are other hurdles but those two suffice to show there is no “reading and answering questions” in the same sense we would apply to any human reader.

Click-bait headlines don’t serve the cause of advocating more AI research. On the contrary, a close reading of alleged progress leads to disappointment.

December 27, 2017

Tutorial on Deep Generative Models (slides and video)

Filed under: Artificial Intelligence,Deep Learning,Machine Learning — Patrick Durusau @ 10:55 am

Slides for: Tutorial on Deep Generative Models by Shakir Mohamed and Danilo Rezende.

Abstract:

This tutorial will be a review of recent advances in deep generative models. Generative models have a long history at UAI and recent methods have combined the generality of probabilistic reasoning with the scalability of deep learning to develop learning algorithms that have been applied to a wide variety of problems giving state-of-the-art results in image generation, text-to-speech synthesis, and image captioning, amongst many others. Advances in deep generative models are at the forefront of deep learning research because of the promise they offer for allowing data-efficient learning, and for model-based reinforcement learning. At the end of this tutorial, audience member will have a full understanding of the latest advances in generative modelling covering three of the active types of models: Markov models, latent variable models and implicit models, and how these models can be scaled to high dimensional data. The tutorial will expose many questions that remain in this area, and for which thereremains a great deal of opportunity from members of the UAI community.

Deep sledding on the latest developments in deep generative models (August 2017 presentation) that ends with a bibliography starting on slide 84 of 96.

Depending on how much time has passed since the tutorial, try searching the topics as they are covered, keep a bibliography of your finds and compare it to that of the authors.

December 24, 2017

Adversarial Learning Market Opportunity

The Pentagon’s New Artificial Intelligence Is Already Hunting Terrorists by Marcus Weisgerber.

From the post:

Earlier this month at an undisclosed location in the Middle East, computers using special algorithms helped intelligence analysts identify objects in a video feed from a small ScanEagle drone over the battlefield.

A few days into the trials, the computer identified objects – people, cars, types of building – correctly about 60 percent of the time. Just over a week on the job – and a handful of on-the-fly software updates later – the machine’s accuracy improved to around 80 percent. Next month, when its creators send the technology back to war with more software and hardware updates, they believe it will become even more accurate.

It’s an early win for a small team of just 12 people who started working on the project in April. Over the next year, they plan to expand the project to help automate the analysis of video feeds coming from large drones – and that’s just the beginning.

“What we’re setting the stage for is a future of human-machine teaming,” said Air Force Lt. Gen. John N.T. “Jack” Shanahan, director for defense intelligence for warfighter support, the Pentagon general who is overseeing the effort. Shanahan believes the concept will revolutionize the way the military fights.

So you will recognize Air Force Lt. Gen. John N.T. “Jack” Shanahan (Nvidia conference):

From the Nvidia conference:

Don’t change the culture. Unleash the culture.

That was the message one young officer gave Lt. General John “Jack” Shanahan — the Pentagon’s director for defense for warfighter support — who is hustling to put artificial intelligence and machine learning to work for the U.S. Defense Department.

Highlighting the growing role AI is playing in security, intelligence and defense, Shanahan spoke Wednesday during a keynote address about his team’s use of GPU-driven deep learning at our GPU Technology Conference in Washington.

Shanahan leads Project Maven, an effort launched in April to put machine learning and AI to work, starting with efforts to turn the countless hours of aerial video surveillance collected by the U.S. military into actionable intelligence.

There are at least two market opportunity for adversarial learning. The most obvious one is testing a competitor’s algorithm so it performs less well than yours on “… people, cars, types of building….”

The less obvious market requires US sales of AI-enabled weapon systems to its client states. Client states have an interest in verifying the quality of AI-enabled weapon systems, not to mention non-client states who will be interested in defeating such systems.

For any of those markets, weaponizing adversarial learning and developing a reputation for the same can’t start too soon. Is your anti-AI research department hiring?

December 20, 2017

Is it a vehicle? A helicopter? No, it’s a rifle! Messing with Machine Learning

Filed under: Classifier,Image Recognition,Image Understanding,Machine Learning — Patrick Durusau @ 8:14 pm

Partial Information Attacks on Real-world AI

From the post:

We’ve developed a query-efficient approach for finding adversarial examples for black-box machine learning classifiers. We can even produce adversarial examples in the partial information black-box setting, where the attacker only gets access to “scores” for a small number of likely classes, as is the case with commercial services such as Google Cloud Vision (GCV).

The post is a quick read (est. 2 minutes) with references but you really need to see:

Query-efficient Black-box Adversarial Examples by Andrew Ilyas, Logan Engstrom, Anish Athalye, Jessy Lin.

Abstract:

Current neural network-based image classifiers are susceptible to adversarial examples, even in the black-box setting, where the attacker is limited to query access without access to gradients. Previous methods — substitute networks and coordinate-based finite-difference methods — are either unreliable or query-inefficient, making these methods impractical for certain problems.

We introduce a new method for reliably generating adversarial examples under more restricted, practical black-box threat models. First, we apply natural evolution strategies to perform black-box attacks using two to three orders of magnitude fewer queries than previous methods. Second, we introduce a new algorithm to perform targeted adversarial attacks in the partial-information setting, where the attacker only has access to a limited number of target classes. Using these techniques, we successfully perform the first targeted adversarial attack against a commercially deployed machine learning system, the Google Cloud Vision API, in the partial information setting.

The paper contains this example:

How does it go? Seeing is believing!

Defeating image classifiers will be an exploding market for jewel merchants, bankers, diplomats, and others with reasons to avoid being captured by modern image classification systems.

December 19, 2017

Was that Stevie Nicks or Tacotron 2.0? ML Singing in 2018

Filed under: Machine Learning,Music,Neural Networks — Patrick Durusau @ 7:15 pm

[S]amim @samim tweeted:

In 2018, machine learning based singing vocal synthesisers will go mainstream. It will transform the music industry beyond recognition.

With these two links:

Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions by Jonathan Shen, et al.

Abstract:

This paper describes Tacotron 2, a neural network architecture for speech synthesis directly from text. The system is composed of a recurrent sequence-to-sequence feature prediction network that maps character embeddings to mel-scale spectrograms, followed by a modified WaveNet model acting as a vocoder to synthesize timedomain waveforms from those spectrograms. Our model achieves a mean opinion score (MOS) of 4.53 comparable to a MOS of 4.58 for professionally recorded speech. To validate our design choices, we present ablation studies of key components of our system and evaluate the impact of using mel spectrograms as the input to WaveNet instead of linguistic, duration, and F0 features. We further demonstrate that using a compact acoustic intermediate representation enables significant simplification of the WaveNet architecture.

and,

Audio samples from “Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions”

Try the samples before dismissing the prediction of machine learning singing in 2018.

I have a different question:

What is in your test set for ML singing?

Among my top picks, Stevie Nicks, Janis Joplin, and of course, Grace Slick.

December 16, 2017

Statistics vs. Machine Learning Dictionary (flat text vs. topic map)

Filed under: Dictionary,Machine Learning,Statistics,Topic Maps — Patrick Durusau @ 10:43 am

Data science terminology (UBC Master of Data Science)

From the webpage:

About this document

This document is intended to help students navigate the large amount of jargon, terminology, and acronyms encountered in the MDS program and beyond. There is also an accompanying blog post.

Stat-ML dictionary

This section covers terms that have different meanings in different contexts, specifically statistics vs. machine learning (ML).
… (emphasis in original)

Gasp! You don’t mean that the same words have different meanings in machine learning and statistics!

Even more shocking, some words/acronyms, have the same meaning!

Never fear, a human reader can use this document to distinguish the usages.

Automated processors, not so much.

If these terms were treated as occurrences of topics, where the topics had the respective scopes of statistics and machine-learning, then for any scoped document, an enhanced view with the correct definition for the unsteady reader could be supplied.

Static markup of legacy documents is not required as annotations can be added as a document is streamed to a reader. Opening the potential, of course, for different annotations depending upon the skill and interest of the reader.

If for each term/subject, more properties than the scope of statistics or machine-learning or both were supplied, users of the topic map could search on those properties to match terms not included here. Such as which type of bias (in statistics) does bias mean in your paper? A casually written Wikipedia article reports twelve and with refinement, the number could be higher.

Flat text is far easier to write than a topic map but tasks every reader with re-discovering the distinctions already known to the author of the document.

Imagine your office, department, agency’s vocabulary and its definitions captured and then used to annotate internal or external documentation for your staff.

Instead of very new staffer asking (hopefully), what do we mean by (your common term), the definition appears with a mouse-over in a document.

Are you capturing the soft knowledge of your staff?

December 12, 2017

IJCAI – Proceedings 1969-2016 Treasure Trove of AI Papers

Filed under: Artificial Intelligence,Machine Learning — Patrick Durusau @ 8:44 pm

IJCAI – Proceedings 1969-2016

From the about page:

International Joint Conferences on Artificial Intelligence is a non-profit corporation founded in California, in 1969 for scientific and educational purposes, including dissemination of information on Artificial Intelligence at conferences in which cutting-edge scientific results are presented and through dissemination of materials presented at these meetings in form of Proceedings, books, video recordings, and other educational materials. IJCAI conferences present premier international gatherings of AI researchers and practitioners. IJCAI conferences were held biennially in odd-numbered years since 1969. They are sponsored jointly by International Joint Conferences on Artificial Intelligence Organization (IJCAI), and the national AI societie(s) of the host nation(s).

While looking for a paper on automatic concept formulation for Jack Park, I found this archive of prior International Joint Conferences on Artificial Intelligence proceedings.

The latest proceedings, 2016, runs six volumes and approximately 4276 pages.

Enjoy!

December 7, 2017

CatBoost: Yandex’s machine learning algorithm (here be Russians)

Filed under: CERN,Machine Learning — Patrick Durusau @ 3:08 pm

CatBoost: Yandex’s machine learning algorithm is available free of charge Victoria Zavyalova.

From the post:

Russia’s Internet giant Yandex has launched CatBoost, an open source machine learning service. The algorithm has already been integrated by the European Organization for Nuclear Research to analyze data from the Large Hadron Collider, the world’s most sophisticated experimental facility.

Machine learning helps make decisions by analyzing data and can be used in many different areas, including music choice and facial recognition. Yandex, one of Russia’s leading tech companies, has made its advanced machine learning algorithm, CatBoost, available free of charge for developers around the globe.

“This is the first Russian machine learning technology that’s an open source,” said Mikhail Bilenko, Yandex’s head of machine intelligence and research.

I called out the Russian origin of the CatBoost algorithm, not because I have any nationalistic tendencies but you can find frothing paranoids in U.S. government agencies and their familiars who do. In those cases, avoid CatBoost.

If you work in saner environments, or need to use categorical data (read not converted to numbers), give CatBoost a close look!

Enjoy!

December 6, 2017

AlphaZero: Mastering Unambiguous, Low-Dimensional Data

Filed under: Ambiguity,Artificial Intelligence,High Dimensionality,Machine Learning — Patrick Durusau @ 8:57 pm

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm by David Silver, et al.

Abstract:

The game of chess is the most widely-studied domain in the history of artificial intelligence. The strongest programs are based on a combination of sophisticated search techniques, domain-specific adaptations, and handcrafted evaluation functions that have been refined by human experts over several decades. In contrast, the AlphaGo Zero program recently achieved superhuman performance in the game of Go, by tabula rasa reinforcement learning from games of self-play. In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

The achievements by the AlphaZero team and their algorithm merit joyous celebration.

Joyous celebration recognizing AlphaZero masters unambiguous, low-dimensional data governed by deterministic rules that define the outcomes for any state, more quickly and completely than any human.

Chess, Shogi and Go appear complex to humans due to the large number of potential outcomes. But every outcome is the result of the application of deterministic rules to unambiguous, low-dimensional data. Something that AlphaZero excels at doing.

What hasn’t been shown is equivalent performance on ambiguous, high-dimensional data, governed by partially (if that) known rules, for a limited set of sub-cases. For those cases, well, you need a human being.

That’s not to take anything away from the AlphaZero team, but to recognize the strengths of AlphaZero and to avoid its application where it is weak.

November 25, 2017

23 Deep Learning Papers To Get You Started — Part 1 (Reading Slowly)

Filed under: Artificial Intelligence,Deep Learning,Machine Learning — Patrick Durusau @ 9:36 pm

23 Deep Learning Papers To Get You Started — Part 1 by Rupak Kr. Thakur.

Deep Learning has probably been the single-most discussed topic in the academia and industry in recent times. Today, it is no longer exclusive to an elite group of scientists. Its widespread applications warrants that people from all disciplines have an understanding of the underlying concepts, so as to be able to better apply these techniques in their field of work. As a result of which, MOOCs, certifications and bootcamps have flourished. People have generally preferred the hands-on learning experiences. However, there is a considerable population who still give in to the charm of learning the subject the traditional way — through research papers.

Reading research papers can be pretty time-consuming, especially since there are hordes of publications available nowadays, as Andrew Ng said at an AI conference, recently, along with encouraging people to use the existing research output to build truly transformative solutions across industries.

In this series of blog posts, I’ll try to condense the learnings from some really important papers into 15–20 min reads, without missing out on any key formulas or explanations. The blog posts are written, keeping in mind the people, who want to learn basic concepts and applications of deep learning, but can’t spend too much time scouring through the vast literature available. Each part of the blog will broadly cater to a theme and will introduce related key papers, along with suggesting some great papers for additional reading.

In the first part, we’ll explore papers related to CNNs — an important network architecture in deep learning. Let’s get started!

The start of what promises to be a great series on deep learning!

While the posts will extract the concepts and important points of the papers, I suggest you download the papers and map the summaries back to the papers themselves.

It will be good practice on reading original research, not to mention re-enforcing what you have learned from the posts.

In my reading, I will be looking for ways to influence deep learning towards one answer or another.

Whatever they may say about “facts” in public, no sane client asks for advice without an opinion on the range of acceptable answers.

Imagine you found ISIS content on Twitter has no measurable impact on ISIS recruiting. Would any intelligence agency would ask you for deep learning services again?

November 14, 2017

Hackers! 90% of Federal IT Managers Aiming for Their Own Feet!

Filed under: Artificial Intelligence,Cybersecurity,Government,Machine Learning,Security — Patrick Durusau @ 2:58 pm

The Federal Cyber AI IQ Test November 14, 2017 reports:


Most Powerful Applications:

  • 90% of Feds say AI could help prepare agencies for real-world cyber attack scenarios and 87% say it would improve the efficiency of the Federal cyber security workforce
  • 91% say their agency could utilize AI to monitor human activity and deter insider threats, including detecting suspicious elements and large amounts of data being downloaded, and analyzing risky user behavior
  • (emphasis in original)

One sure conclusion from this report, 90% of Feds don’t know AIs mistake turtles for rifles, 90% of the time. The adversarial example literature is full of such cases and getting more robust by the day.

The trap federal IT managers have fallen into is a familiar one. To solve an entirely human problem, a shortage of qualified labor, they want mechanize the required task, even if it means a lower qualify end result. Human problems are solved poorly, if at all, by mechanized solutions.

Opposed by lowest common denominator AI systems, hackers will be all but running the mints as cybersecurity AI systems spread across the federal government. “Ghost” federal installations will appear on agency records for confirmation of FedEx/UPS shipments. The possibilities are endless.

If you are a state or local government or even a federal IT manager, letting hackers run wild isn’t a foregone conclusion.

You could pattern your compensation packages after West Coast start-ups, along with similar perks. Expensive but do you want an OMB type data leak on your record?

November 9, 2017

Is That a Turtle in Your Pocket or Are You Just Glad To See Me?

Filed under: Image Recognition,Machine Learning — Patrick Durusau @ 10:14 am

Apologies to Mae West for spoiling her famous line from Sexette:

Is that a gun in your pocket, or are you just glad to see me?

Seems appropriate since Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok have created a 3-D turtle that is mistaken by neural networks as a rifle.

You can find the details in: Synthesizing Robust Adversarial Examples.

Abstract:

Neural network-based classifiers parallel or exceed human-level accuracy on many common tasks and are used in practical systems. Yet, neural networks are susceptible to adversarial examples, carefully perturbed inputs that cause networks to misbehave in arbitrarily chosen ways. When generated with standard methods, these examples do not consistently fool a classifier in the physical world due to viewpoint shifts, camera noise, and other natural transformations. Adversarial examples generated using standard techniques require complete control over direct input to the classifier, which is impossible in many real-world systems.

We introduce the first method for constructing real-world 3D objects that consistently fool a neural network across a wide distribution of angles and viewpoints. We present a general-purpose algorithm for generating adversarial examples that are robust across any chosen distribution of transformations. We demonstrate its application in two dimensions, producing adversarial images that are robust to noise, distortion, and affine transformation. Finally, we apply the algorithm to produce arbitrary physical 3D-printed adversarial objects, demonstrating that our approach works end-to-end in the real world. Our results show that adversarial examples are a practical concern for real-world systems.

All in good fun until you remember neural networks feed classification decisions to humans who make fire/no fire decisions and soon, fire/no fire decisions will be made by autonomous systems. Errors in classification decisions such as turtle vs. rifle will have deadly results.

What are the stakes in your neural net classification system? How easily can it be fooled by adversaries?

Metasploit for Machine Learning: Deep-Pwning

Filed under: Cybersecurity,Machine Learning,Security — Patrick Durusau @ 8:46 am

Metasploit for Machine Learning: Deep-Pwning

From the post:

Deep-pwning is a lightweight framework for experimenting with machine learning models with the goal of evaluating their robustness against a motivated adversary.

Note that deep-pwning in its current state is no where close to maturity or completion. It is meant to be experimented with, expanded upon, and extended by you. Only then can we help it truly become the goto penetration testing toolkit for statistical machine learning models.

Metasploit for Machine Learning: Background

Researchers have found that it is surprisingly trivial to trick a machine learning model (classifier, clusterer, regressor etc.) into making an objectively wrong decisions. This field of research is called Adversarial Machine Learning. It is not hyperbole to claim that any motivated attacker can bypass any machine learning system, given enough information and time. However, this issue is often overlooked when architects and engineers design and build machine learning systems. The consequences are worrying when these systems are put into use in critical scenarios, such as in the medical, transportation, financial, or security-related fields.

Hence, when one is evaluating the efficacy of applications using machine learning, their malleability in an adversarial setting should be measured alongside the system’s precision and recall.

(emphasis in original)

As motivation for a deep dive into machine learning, looming reliance on machine learning to compensate for a shortage of cybersecurity defender talent is hard to beat. (Why Machine Learning will Boost Cyber Security Defenses amid Talent Shortfall)

Reducing cybersecurity to the level of machine learning is nearly as inviting as use of an older, less secure version of MINIX by Intel. If you are going to take advantage of a Berkeley software license, at least get the best stuff. Yes?

Machine learning is of growing importance, but since classifiers can be fooled into identifying a 3-D turtle as a rifle, it hasn’t reached human levels of robustness.

Or to put that differently, when was the last time you identified a turtle as a rifle?

Turtle vs. rifle is a distinction few of us would miss in language, even without additional properties, as in a topic map. But thinking of their properties or characteristics, maybe a fruitful way to understand why they can be confused.

Or even planning for their confusion and communicating that plan to others.

October 12, 2017

Cheap Tracking of Public Officials/Police

Filed under: Image Recognition,Machine Learning,Privacy — Patrick Durusau @ 2:12 pm

The use of license plate readers by law enforcement and others is on the rise. Such readers record the location of your license plate at a particular time and place. They also relieve public bodies of large sums of money.

How I replicated an $86 million project in 57 lines of code by Tait Brown details how he used open source software to create a “…good enough…” license plate reader for far less than the ticket price of $86 million.

Brown has an amusing (read unrealistic) good Samaritan scenario for his less expensive/more extensive surveillance system:


While it’s easy to get caught up in the Orwellian nature of an “always on” network of license plate snitchers, there are many positive applications of this technology. Imagine a passive system scanning fellow motorists for an abductors car that automatically alerts authorities and family members to their current location and direction.

The Teslas vehicles are already brimming with cameras and sensors with the ability to receive OTA updates — imagine turning them into a virtual fleet of good samaritans. Ubers and Lyft drivers could also be outfitted with these devices to dramatically increase the coverage area.

Using open source technology and existing components, it seems possible to offer a solution that provides a much higher rate of return — for an investment much less than $86M.

The better use of Brown’s less expensive/more extensive surveillance system is tracking police and public official cars. Invite them to the gold fish bowl they have created for all the rest of us.

A great public data resource for testing testimony about the presence/absence of police officers at crime scenes, protests, long rides to the police station and public officials consorting with co-conspirators.

ACLU calls for government to monitor itself, reflect an unhealthy confidence in governmental integrity. Only a close watch on government by citizens enables governmental integrity.

October 2, 2017

Machine Translation and Automated Analysis of Cuneiform Languages

Filed under: Cuneiform,Language,Machine Learning,Translation — Patrick Durusau @ 8:46 pm

Machine Translation and Automated Analysis of Cuneiform Languages

From the webpage:

The MTAAC project develops and applies new computerized methods to translate and analyze the contents of some 67,000 highly standardized administrative documents from southern Mesopotamia (ancient Iraq) from the 21st century BC. Our methodology, which combines machine learning with statistical and neural machine translation technologies, can then be applied to other ancient languages. This methodology, the translations, and the historical, social and economic data extracted from them, will be offered to the public in open access.

A recently funded (March 2017) project that strikes a number of resonances with me!

“Open access” and cuneiform isn’t an unheard of combination but many remember when access to cuneiform primary materials was a matter of whim and caprice. There are dark pockets where such practices continue but projects like MTAAC are hard on their heels.

The use of machine learning and automated analysis have the potential, when all extant cuneiform texts (multiple projects such as this one) are available, to provide a firm basis for grammars, lexicons, translations.

Do read: Machine Translation and Automated Analysis of the Sumerian Language by Émilie Pagé-Perron, Maria Sukhareva, Ilya Khait, Christian Chiarcos, for more details about the project.

There’s more to data science than taking advantage of sex-starved neurotics with under five second attention spans and twitchy mouse fingers.

August 7, 2017

BuzzFeed News Searches For Hidden Spy Planes

Filed under: Journalism,Machine Learning,News,Reporting — Patrick Durusau @ 8:52 pm

BuzzFeed News Trained A Computer To Search For Hidden Spy Planes. This Is What We Found.

From the post:

Data and R code for the analysis supporting this August 7, 2017 BuzzFeed News post on identifying potential surveillance aircraft. Supporting files are in this GitHub repository.

Awesome! This is what data journalism is about!

While Musk and others are wringing their hands over AI, BuzzFeed uses machine learning to out government spy planes. How cool is that?

So, what are some of the headlines from The New York Times today?

  1. Scientists Fear Trump Will Dismiss Climate Change Report
  2. What Music Do Americans Love the Most? 50 Detailed Fan Maps
  3. Partisan C.I.A. Chief Heartens Trump and Worries the Agency
  4. North Korea Warns U.S. of Retaliation Over Sanctions
  5. Industries Are Left in the Lurch by Trump’s Stalled Trade Plans
  6. White House Won’t Say Who Is on Its Deregulation Teams
  7. Wells Fargo Faces New Inquiry Over Insurance Refunds
  8. Take the Generic, Patients Are Told. Until They Are Not.
  9. $78,000 of Debt for a Harvard Theater Degree
  10. Investigators in Israel Turn Up the Heat on Netanyahu

Four out of ten stories are about our accidental president (1, 3, 5, 6) The other six (2, 4, 7, 8, 9, 10), offer no actionable information.

Not a word about government spy planes.

Why isn’t The New York Times pressing the government hard?

Or perhaps the easier question: Why are you still reading The New York Times?

August 5, 2017

Overlap – Attacking on Machine Learning Models

Filed under: Machine Learning,XML — Patrick Durusau @ 4:48 pm

Robust Physical-World Attacks on Machine Learning Models by Ivan Evtimov, et al.

Abstract:

Deep neural network-based classifiers are known to be vulnerable to adversarial examples that can fool them into misclassifying their input through the addition of small-magnitude perturbations. However, recent studies have demonstrated that such adversarial examples are not very effective in the physical world–they either completely fail to cause misclassification or only work in restricted cases where a relatively complex image is perturbed and printed on paper. In this paper we propose a new attack algorithm–Robust Physical Perturbations (RP2)– that generates perturbations by taking images under different conditions into account. Our algorithm can create spatially-constrained perturbations that mimic vandalism or art to reduce the likelihood of detection by a casual observer. We show that adversarial examples generated by RP2 achieve high success rates under various conditions for real road sign recognition by using an evaluation methodology that captures physical world conditions. We physically realized and evaluated two attacks, one that causes a Stop sign to be misclassified as a Speed Limit sign in 100% of the testing conditions, and one that causes a Right Turn sign to be misclassified as either a Stop or Added Lane sign in 100% of the testing conditions.

I was struck by the image used for this paper in a tweet:

I recognized this as an “overlapping” markup problem before discovering the authors were attacking machine learning models. On overlapping markup, see: Towards the unification of formats for overlapping markup by Paolo Marinelli, Fabio Vitali, Stefano Zacchiroli, or more recently, It’s more than just overlap: Text As Graph – Refining our notion of what text really is—this time for sure! by Ronald Haentjens Dekker and David J. Birnbaum.

From the conclusion:


In this paper, we introduced Robust Physical Perturbations (RP2), an algorithm that generates robust, physically realizable adversarial perturbations. Previous algorithms assume that the inputs of DNNs can be modified digitally to achieve misclassification, but such an assumption is infeasible, as an attacker with control over DNN inputs can simply replace it with an input of his choice. Therefore, adversarial attack algorithms must apply perturbations physically, and in doing so, need to account for new challenges such as a changing viewpoint due to distances, camera angles, different lighting conditions, and occlusion of the sign. Furthermore, fabrication of a perturbation introduces a new source of error due to a limited color gamut in printers.

We use RP2 to create two types of perturbations: subtle perturbations, which are small, undetectable changes to the entire sign, and camouflage perturbations, which are visible perturbations in the shape of graffiti or art. When the Stop sign was overlayed with a print out, subtle perturbations fooled the classifier 100% of the time under different physical conditions. When only the perturbations were added to the sign, the classifier was fooled by camouflage graffiti and art perturbations 66.7% and 100% of the time respectively under different physical conditions. Finally, when an untargeted poster-printed camouflage perturbation was overlayed on a Right Turn sign, the classifier was fooled 100% of the time. In future work, we plan to test our algorithm further by varying some of the other conditions we did not consider in this paper, such as sign occlusion.

Excellent work but my question: Is the inability of the classifier to recognize overlapping images similar to the issues encountered as overlapping markup?

To be sure overlapping markup is in part an artifice of unimaginative XML rules, since overlapping texts are far more common than non-overlapping texts. Especially when talking about critical editions or even differing analysis of the same text.

But beyond syntax, there is the subtlety of treating separate “layers” or stacks of a text as separate and yet tracking the relationship between two or more such stacks, when arbitrary additions or deletions can occur in any of them. Additions and deletions that must be accounted for across all layers/stacks.

I don’t have a solution to offer but pose the question of layers of recognition in hopes that machine learning models can capitalize on the lessons learned about a very similar problem with overlapping markup.

Older Posts »

Powered by WordPress