Archive for March, 2016

Game of Thrones – Network Analysis

Thursday, March 31st, 2016


You can read the popular account of this network analysis of the Game of Thrones in Mathematicians mapped out every “Game of Thrones” relationship to find the main character by Adam Epstein or, you can try Network of Thrones by Andrew Beveridge and Jie Shan.

There are a number of choices you may want to re-visit if you explore the Game of Thrones as a graph/network, not the least of which is expanding the data beyond volume 3, characterizing the type of “relationships” (edges) found between characters and how you would capture the time aspect of the development of the “relationships” you do find.

Great work that will hopefully spur others to similar explorations. Launches First Report (PDF)

Thursday, March 31st, 2016 Launches First Report (PDF).

Reposting: is pleased to share our first report "Unfriending Censorship: Insights from four months of crowdsourced data on social media censorship." The report draws on data gathered directly from users between November 2015 and March 2016.

We asked users to send us reports when they had their content or accounts taken down on six social media platforms: Facebook, Flickr, Google+, Instagram, Twitter, and YouTube. We have aggregated and analyzed the collected data across geography, platform, content type, and issue areas to highlight trends in social media censorship. All the information presented here is anonymized, with the exception of case study examples we obtained with prior approval by the user.

Here are some of the highlights:

  • This report covers 161 submissions from 26 countries, regarding content in eleven languages.
  • Facebook was the most frequently reported platform, and account suspensions were the most reported content type.
  • Nudity and false identity were the most frequent reasons given to users for the removal of their content.
  • Appeals seem to present a particular challenge. A majority of users (53%) did not appeal the takedown of their content, 50% of whom said they didn’t know how and 41.9% of whom said they didn’t expect a response. In only four cases was content restored, while in 50 the user didn’t get a response.
  • We received widespread reports that flagging is being used for censorship: 61.6% believed this was the cause of the content takedown.

While we introduced some measures to help us verify reports (such as giving respondents the opportunity to send us screenshots that support their claims), we did not work with the companies to obtain this data and thus cannot claim it is representative of all content takedowns or user experiences. Instead, it shows how a subset of the millions of social media users feel about how their content takedowns were handled, and the impact it has had on their lives.

The full report is available for download and distribution under Creative Commons licensing.

As the report itself notes, 161 reports across 6 social media platforms in 4 months isn’t a representative sample of censoring in social media.

Twitter alone brags about closing 125,000 ISIS accounts since mid-2015 (report dated 5 February 2016).

Closing ISIS accounts is clearly censorship of political speech, whatever hand waving and verbal gymnastics Twitter wants to employ to justify its practices. Including terms of service.

Censorship, on whatever basis, by whoever practiced, by whatever mechanism (including appeals), will always step on legitimate speech of some speakers.

The non-viewing of content has one and only one legitimate locus of control, a user’s browser for web content.

Browsers and/or web interfaces for Twitter, Facebook, etc., should enable users to block users, content by keywords, or even classifications offered by social media services.


All need for collaboration with governments, issues of what content to censor, appeal processes, etc., suddenly disappear.

Enabling users to choose the content that will be displayed in their browsers empowers listeners as well as speakers, with prejudice towards none.


World’s Biggest Bribe Scandal: Part 1, Part 2, Part 3

Wednesday, March 30th, 2016

Part 1: The Company That Bribed The World.

From the post:

A massive leak of confidential documents has for the first time exposed the true extent of corruption within the oil industry, implicating dozens of leading companies, bureaucrats and politicians in a sophisticated global web of bribery and graft.

After a six-month investigation across two continents, Fairfax Media and The Huffington Post can reveal that billions of dollars of government contracts were awarded as the direct result of bribes paid on behalf of firms including British icon Rolls-Royce, US giant Halliburton, Australia’s Leighton Holdings and Korean heavyweights Samsung and Hyundai.

The investigation centres on a Monaco company called Unaoil, run by the jet-setting Ahsani clan. Following a coded ad in a French newspaper, a series of clandestine meetings and midnight phone calls led to our reporters obtaining hundreds of thousands of the Ahsanis’ leaked emails and documents.

The trove reveals how they rub shoulders with royalty, party in style, mock anti-corruption agencies and operate a secret network of fixers and middlemen throughout the world’s oil producing nations.

Corruption in oil production – one of the world’s richest industries and one that touches us all through our reliance on petrol – fuels inequality, robs people of their basic needs and causes social unrest in some of the world’s poorest countries. It was among the factors that prompted the Arab Spring.

Fairfax Media and The Huffington Post today reveal how Unaoil carved up portions of the Middle East oil industry for the benefit of western companies between 2002 and 2012.

In part two we will turn to the impoverished former Russian states to reveal the extent of misbehaviour by multinational companies including Halliburton. We will conclude the three-part investigation by showing how corrupt practices have extended deep into Asia and Africa.

Truly awesome story!

After six months of work, both Fairfax Media and The Huffington Post deserve all the traffic and kudos that can be driven to them!

However, after some suitable time period, say six months to a year, this treasure trove of criminality should be released to the public.

I have little doubt about the non-pursuit of potential American defendants, especially those associated with Halliburton (can you say “Dick Cheney?”)

Public release of this data leak, as I said after both Fairfax Media and The Huffington Post have been rewarded for their great work, may help grease frozen wheels of justice for some defendants.

If grease doesn’t work, perhaps extra-judicial justice will find some of the more “untouchable” criminals named in the data leak.

Personally I would be very interested in mapping from the known relationships (a/k/a associations) to presently unknown associations with public officials in the US, both appointed and elected.

There would be a particular joy in seeing sitting members of Congress tagged with criminal misconduct.

No club Fed for then former members of Congress. Recommend rendering them to Zambia or similar places.

Update 31 March 2016:

Part 2: Unaoil: Police Launch Joint Global Investigation

Part 2 of this extraordinary story is up!

Apologies but I omitted the credits for this reporting in my original post:

REPORTERS Nick McKenzie (chief reporter), Richard Baker, Michael Bachelard & Daniel Quinlan EDITOR ​​Michael Bachelard COPY EDITORS Michael Coulter & Kate Cole-Adams DESIGN Mark Stehle & Joe Benke BUILD Matthew Absalom-Wong, Nathanael Scott & Soren Frederiksen VIDEO Tim Young & Kelly Bergsma CONSULTANT Luke McMahon

Kudos to one and all!

Update 03 April 2016:

Part 3: UNAOIL: Dark Secrets of Asian Powers.

From the post:

Asian companies such as Hyundai, Samsung, Sinopec and Petronas are household names. But they have dark secrets. In the latest in Fairfax Media and The Huffington Post’s global bribery expose, these firms and more are implicated for paying kickbacks, money laundering and corruption.

Here’s a taste of the key players to pique your interest in following this story in your country.

Google BigQuery Public Datasets

Wednesday, March 30th, 2016

Google BigQuery Public Datasets

An amazing set of public datasets, from the post:

  • : A Social Security Administration dataset that contains all names from Social Security card applications for births that occurred in the United States after 1879.
  • : Data collected by the NYC Taxi and Limousine Commission (TLC) that includes trip records from all trips completed in yellow and green taxis in NYC from 2009 to 2015.
  • : A dataset that contains all stories and comments from Hacker News since its launch in 2006.
  • : A dataset published by the US Department of Health and Human Services that includes all weekly surveillance reports of nationally notifiable diseases for all U.S. cities and states published between 1888 and 2013.
  • : A dataset that contains 3.5 million digitized books stretching back two centuries, encompassing the complete English-language public domain collections of the Internet Archive (1.3M volumes) and HathiTrust (2.2 million volumes).
  • : This public dataset was created by the National Oceanic and Atmospheric Administration (NOAA) and includes global data obtained from the USAF Climatology Center. This dataset covers GSOD data between 1929 and 2016, collected from over 9000 stations.

I can readily see myself loosing serious time in the GDELT Book Corpus!


Spending Time Rolling Your Own or Using Google Tools in Anger?

Wednesday, March 30th, 2016

The question: Spending Time Rolling Your Own or Using Google Tools in Anger? is one faced by many people who have watched computer technology evolve.

You could write your own blogging software or you can use one of the standard distributions.

You could write your own compiler or you can use one of the standard distributions.

You can install and maintain your own machine learning, big data apps, or you can use the tools offered by Google Machine Learning.

Tinkering with your local system until it is “just so” is fun, but it eats into billable time and honestly is a distraction.

Not promising I immersing in the Google-verse but an honest assessment of where to spend my time is in order.

Google takes Cloud Machine Learning service mainstream by Fausto Ibarra, Director, Product Management.

From the post:

Hundreds of different big data and analytics products and services fight for your attention as it’s one of the most fertile areas of innovation in our industry. And it’s no wonder; the most amazing consumer experiences are driven by insights derived from information. This is an area where Google Cloud Platform has invested almost two decades of engineering, and today at GCP NEXT we’re announcing some of the latest results of that work. This next round of innovation builds on our portfolio of data management and analytics capabilities by adding new products and services in multiples key areas:

Machine Learning:

We’re on a journey to create applications that can see, hear and understand the world around them. Today we’ve taken a major stride forward with the announcement of a new product family: Cloud Machine Learning. Cloud Machine Learning will take machine learning mainstream, giving data scientists and developers a way to build a new class of intelligent applications. It provides access to the same technologies that power Google Now, Google Photos and voice recognition in Google Search as easy to use REST APIs. It enables you to build powerful Machine Learning models on your data using the open-source TensorFlow machine learning library:

Big Data and Analytics:

Doing big data the cloud way means being more productive when building applications, with faster and better insights, without having to worry about the underlying infrastructure. To further this mission, we recently announced the general availability of Cloud Dataproc, our managed Apache Hadoop and Apache Spark service, and we’re adding new services and capabilities today:

Open Source:

Our Cloud Machine Learning offering leverages Google’s cutting edge machine learning and data processing technologies, some of which we’ve recently open sourced:

What, if anything, do you see as a serious omission in this version of the Google-verse?


Walking the Walk on Privacy

Wednesday, March 30th, 2016

Many people grumble about government surveillance but how many do you know who have taken concrete steps to combat that surveillance?

That many. Huh.

Sounds like government surveillance has and will maintain the upper hand.

Unless, the people under surveillance organize to do something about it.

The Electronic Freedom Foundation (EFF) is organizing an effort to enable you, yes you, to do exactly that!

California Surveillance Sweep

From the post:

Join EFF on Saturday, April 9 for a first-of-its-kind crowdsourcing campaign to hold California law enforcement agencies accountable for their use of surveillance technologies.

Volunteers like you will help us track down the privacy and useage policies of law enforcement agencies across California and add them to our database. We’ll show you how to do it, and you can be anywhere with an Internet connection to participate.

What: California Surveillance Sweep

Date: Saturday, April 9

Time: 12 pm – 4 pm PT

Where: Anywhere (virtual participation); San Francisco (details TBD)

I bitch as much about privacy as anyone and have any number of unsound suggestions in that regard.

This effort by the EFF is a low-risk effort to hoist the surveillance state on its own laws.

Given the propensity for national law enforcement to lie I’m not betting on state and local law enforcement being any more truthful.

Still, you can’t say you haven’t exhausted all traditional remedies unless you have.

I signed up.

Are you?

Tay AI Escapes, Recaptured

Wednesday, March 30th, 2016

Microsoft’s offensive chatbot Tay returns, by mistake by Georgia Wells.

From the post:

Less than one week after Microsoft Corp. made its debut and then silenced an artificially intelligent software chatbot that started spewing anti-Semitic rants, a researcher inadvertently put the chatbot, named Tay, back online. The revived Tay’s messages were no less inappropriate than before.

I remembered a DARPA webinar (download and snooze) but despite following Tay I missed her return.

Looks like I need a better tracking/alarm system for incoming social media.

I see more than enough sexist, racist, bigotry in non-Twitter news feeds to not need any more but I prefer to make my own judgments about “inappropriate.”

Whether it is the FBI, FCC or private groups calling “inappropriate.”

FBI Hacked San Bernardino IPhone – Why Do You Believe That?

Wednesday, March 30th, 2016

The claim of the FBI to have hacked the San Bernardino IPhone without help from Apple has flooded social and mainstream media.

What I haven’t heard in all that clamor is any reason to credit that claim.

This is the same FBI mentioned in:

Report: ‘Nearly Every’ FBI Forensics Expert Gave Flawed Testimony In ‘Almost All Trials’ Over A 20-Year Period

Official confirmation the FBI lied? (Boston Marathon Bombing)

The Judi Bari Website FBI conspiracy (civil court verdict)

FBI Sanctioned for Lying About Existence of Surveillance Records

Admittedly, it isn’t easy to catch the FBI lying, but these few cases illustrate that lying is a way of life at the FBI.

Or as the 9th Circuit is quoted as saying in FBI Sanctioned for Lying About Existence of Surveillance Records:

The Government argues that there are times when the interests of national security require the Government to mislead the Court. The Court strongly disagrees. The Government’s duty of honesty to the Court can never be excused, no matter what the circumstance. The Court is charged with the humbling task of defending the Constitution and ensuring that the Government does not falsely accuse people, needlessly invade their privacy or wrongfully deprive them of their liberty. The Court simply cannot perform this important task if the Government lies to it. Deception perverts justice. Truth always promotes it.

When you are dealing with a party that has a policy of misleading courts to further “national security,” why would you credit any unsubstantiated claim from that source?

More than a policy, a history of lying to both the public and the courts.

Is it sufficient that the FBI declare it’s not lying today? This time? Or did any media representative even ask that question?

Until an independent expert “hacks” an identical iPhone using the FBI’s “method,” the FBI “hack” of the San Bernardino IPhone ranks with photos of presidents with aliens:


And for equal time purposes:


A skeptical public press would not parrot the unsubstantiated claims of known liars, even when those liars are federal agencies.

But then, we don’t have a skeptical public press.


Jihadist Wannabes: You Too Can Be A Western Lackey

Wednesday, March 30th, 2016

Eric Geller’s piece Why ISIS is winning the online propaganda war is far too long to read but has several telling insights for jihadist wannabes.

First, perhaps without fulling realizing it, Geller points out that the appeal of ISIS is based on facts, not messages:

Young Muslims who feel torn between what can seem like two different worlds, who long for structure and meaning in their lives, are ISIS’s best targets. They seek a coherent picture of the world—and ISIS is ready to offer one. Imagine being 19 years old, living in a major American city, and not understanding how a terrorist attack in Paris can change the way your fellow subway passengers look at you. If that prejudice or bigotry mystified you, you might gravitate toward someone offering an explanation that felt like it fit with your experiences. You might start watching YouTube videos about the supposedly irreconcilable differences between the West and the Islamic world. ISIS shapes its content to appeal to this person and others who lack a framework for understanding world events and are willing to embrace a radical one.

The other psychological factor that ISIS exploits is the natural desire for purpose. ISIS is a bonafide regional power, and to people who already feel out of place in Western society and crave a sense of direction, joining ISIS offers that purpose, that significance. They can become part of something bigger than themselves. They can fight for a cause. ISIS’s messages don’t just offer a framework for understanding the world; they also offer the chance to help shape it. These messages “make people feel like they matter in the world,” Beutel said, by promising “a sense of honor and self-esteem, and the ability to actively live out those desires.”

There are also more pragmatic promises, tailored to people who are not only spiritually aimless but economically frustrated and emotionally unfulfilled. Liang described this part of the appeal as, “Come and you will have a real life. You will have a salary. You will have a job. You will have a wife. You will have a house.”

“This is appealing to people who have, really, no future,” she said.

I can see how ISIS would be appealing to:

…people who have, really, no future…

Noting that the “no future,” is a fact, not idle speculation. All Muslim youth do and will continue to face discrimination, especially in the aftermath of terrorist attacks.

The West and Islamic worlds are irreconcilable only to the extent leaders in both worlds profit from that view. Sane members of those and other traditions relish and welcome such diversity. The Islamic world has a better record of toleration of diversity that one can claim for any Western power.

Second, Geller illustrates how the focus on message is at odds with changing the reality for Muslim youth:

If the U.S. and its allies want to dissuade would-be jihadists from joining ISIS, they need to start from square one. “We need a compelling story that makes our story better than theirs,” Liang said. “And so far their story is trumping ours.”

The anti-extremist story can’t just be a paean to human rights and liberal democratic values. It must provide clear promises about what the Middle East will look like if ISIS is defeated. “What are we going to do if we take back the land that [ISIS] is inhabiting at the moment?” Liang said. “What government are we going to set up, and how legitimate will it be? If you look at, right now, the Iraqi state, it’s extremely corrupt, and it has to prove that it will be the better alternative.”

Part of the challenge that counter-narrative designers face is that the anti-extremist story can’t just be a sweeping theoretical message. It has to be pragmatic, full of real promises. But no one has a clear idea of how to do this. “To be totally honest, we haven’t cracked that nut yet,” the former senior administration official said. “Maybe it is liberal values and a democratic order and human rights and democratic values. I would hope that that would be the case. But I don’t think that there’s evidence yet that that would be equally compelling as a narrative or a set of values.

“Everyone agrees [that] we can’t just counter-message,” the official added. “We have to promote alternative messages. But nobody understands or agrees or has the answer in terms of what are the alternate courses of action or pathways that one could offer.”

How’s that for a bizarre story line? There is no effort to change the reality as experienced by Muslim youth, but they should just suck it up and not join ISIS?

One of the problems with “messaging” is the West wants dictating who will deliver the message and controlling what else they may say.

Not to mention that being discovered to be a Western lackey damages the credibility of anti-jihadists.

I’m not sure who edited Geller’s piece but there is this gem:

While the big-picture thinkers devise a story, others should focus on a bevy of vital changes to how counter-narratives are produced and distributed. For one thing, the content is too grim. Instead of going dark, Beutel said, go light: Offer would-be jihadists hope. Humanize ISIS’s foot soldiers instead of demonizing them, so that your intended audience understands that you care about their fate and not just taking them off the battlefield. “When you have people who are espousing incredibly hateful worldviews, the tendency is to want to demonize them—to want to shut them out [in order] to isolate them,” Beutel said. “More often than not, that actually repulses people rather than [getting] them to open up.” (emphasis added)

Gee, “espousing incredibly hateful worldviews,” I don’t think of ISIS first in that regard. Do you? There a list of governments and leaders that I would put way ahead of ISIS.

Maybe, just maybe, urging people to not join groups you are trying to destroy that are resisting corrupt Western toady governments just isn’t persuasive?

Have you stopped corrupting Muslim governments? Have you stopped supporting governments that oppress Muslims? Have you stopped playing favorites between Muslim factions? Have you taken any steps to promote a safe and diverse environment for Muslims in your society?

Or the overall question: Have you made a positive different in the day to day lives of Muslims? (from their point of view, not yours)

Messaging not based on having done (not promised, accomplished) those and other things, are invitations to be a Western lackey. Who wants that?

All the attribution of a high level of skill to ISIS messaging is merely a reflection of the tone-deafness of West dictated messaging. Strict hierarchical control over both messages and speakers, using messages that appeal to the sender and not the receiver, valuing message over reality, are only some of the flaws in Western anti-jihadist programs.

The old possible redeeming point is the use of former jihadists. ISIS, being composed of people, is subject to the same failings of governments/groups/movements everywhere. I’m not sure how any government could claim to be superior to ISIS in that regard.

BTW, I’m not an ISIS “cheerleader” as Geller put it. I have serious disagreement with ISIS on a number of issues, social policies and targeting being prominent ones. I do agree on the need to fight against corrupt, Western-dictated Muslim governments. Contrary to current US foreign policy.

Takedown Bots – Make It Personal

Tuesday, March 29th, 2016

Carl Malamud tweeted on 29 March 2016:

Hate takedown bots, both human and coded. If you’re going to accuse somebody of theft, you should make it personal.

in retweeting:

Mitch Stoltz

How takedown-bots are censoring the web. …

Carl has the right of it.

Users should make the use of take down notices very personal.

After all, illegitimate take down notices are thefts from the public domain and/or fair use.

Caitlin Dewey‘s How we’re unwittingly letting robots censor the Web is a great non-technical piece on the fuller report, Notice and Takedown in Everyday Practice.

Jennifer M. Urban, University of California, Berkeley – School of Law, Brianna L. Schofield, University of California, Berkeley – School of Law, and Joe Karaganis, Columbia University – The American Assembly, penned this abstract:

It has been nearly twenty years since section 512 of the Digital Millennium Copyright Act established the so-called notice and takedown process. Despite its importance to copyright holders, online service providers, and Internet speakers, very little empirical research has been done on how effective section 512 is for addressing copyright infringement, spurring online service provider development, or providing due process for notice targets.

This report includes three studies that draw back the curtain on notice and takedown:

1. using detailed surveys and interviews with more than three dozen respondents, the first study gathers information on how online service providers and rightsholders experience and practice notice and takedown on a day-to-day basis;

2. the second study examines a random sample from over 100 million notices generated during a six-month period to see who is sending notices, why, and whether they are valid takedown requests; and

3. the third study looks specifically at a subset of those notices that were sent to Google Image Search.

The findings suggest that whether notice and takedown “works” is highly dependent on who is using it and how it is practiced, though all respondents agreed that the Section 512 safe harbors remain fundamental to the online ecosystem. Perhaps surprisingly in light of large-scale online infringement, a large portion of OSPs still receive relatively few notices and process them by hand. For some major players, however, the scale of online infringement has led to automated, “bot”-based systems that leave little room for human review or discretion, and in a few cases notice and takedown has been abandoned in favor of techniques such as content filtering. The second and third studies revealed surprisingly high percentages of notices of questionable validity, with mistakes made by both “bots” and humans.

The findings strongly suggest that the notice and takedown system is important, under strain, and that there is no “one size fits all” approach to improving it. Based on the findings, we suggest a variety of reforms to law and practice.

At 160 pages it isn’t a quick or lite read.

The gist of both Caitlin’s post and the fuller report is that automated systems are increasingly being used to create and enforce take down requests.

Despite the margin of reported error, Caitlin notes:

Despite the margin of error, most major players seem to be trending away from human review. The next frontier in the online copyright wars is automated filtering: Many rights-holders have pressed for tools that, like YouTube’s Content ID, could automatically identify protected content and prevent it from ever publishing. They’ve also pushed for “staydown” measures that would keep content from being reposted once it’s been removed, a major complaint with the current system.

There is one source Caitlin uses:

…agreed to speak to The Post on condition of anonymity because he has received death threats over his work, said that while his company stresses accuracy and fairness, it’s impossible for seven employees to vet each of the 90,000 links their search spider finds each day. Instead, the algorithm classifies each link as questionable, probable or definite infringement, and humans only review the questionable ones before sending packets of takedown requests to social networks, search engines, file-hosting sites and other online platforms.

Copyright enforcers should discover their thefts from the public domain or infringement on fair use are on a par with car burglars or shoplifters.

What copyright enforcers lack is an incentive to err on the side of not issuing questionable take down notices.

If the consequences of illegitimate take down notices are high enough, they will spend the funds necessary to enforce only “legitimate” rights.

If you are interested in righteousness over effectiveness, by all means, pursue reform of “notice and takedown” in the copyright holder owned US Congress.

On the other hand, someone, more than a single someone, is responsible for honoring “notice and takedown” requests. Those someones also own members of Congress and can effectively seek changes that victims of illegitimate takedown requests cannot.

Imagine a leak from Yahoo! that outs those responsible for honoring “notice and takedown” requests.

Or the members of “Google’s Trusted Copyright Removal Program.” Besides “Glass.”

Or the takedown requests for YouTube.

Theft from the public cannot be sustained in the bright light of transparency.

Serious Non-Transparency (+ work around)

Tuesday, March 29th, 2016

I mentioned yesterday in my post: Courses -> Texts: A Hidden Relationship, where I lamented the inability to find courses by their titles.

So you could easily discover the required/suggested texts for any given course. Like browsing a physical campus bookstore.

Obscurity is an “information smell” (to build upon Felienne‘s expansion of code smell to spreadsheets).

In this particular case, the “information smell” is skunk class.

I revisited today to extract its > 1200 bookstores for use in crawling a sample of those sites.

For ugly HTML, view the source of:

Parsing that is going to take time and surely there is an easy way to get a sample of the sites for mining.

The idea didn’t occur to me immediately but I noticed yesterday that the general form of web addresses was:

So, after some flailing about with the HTML from, I searched for “” and requested all the results.

I’m picking a random ten bookstores with law books for further searching.

Not a high priority but I am curious what lies behind the smoke, mirrors, complex HTML and poor interfaces.

Maybe something, maybe nothing. Won’t know unless we look.

PS: Perhaps a better query string: textbooks-and-course-materials

Suggested refinements?

Bias For Sale: How Much and What Direction Do You Want?

Tuesday, March 29th, 2016

Epstein and Robertson pitch it a little differently but that is the bottom line of: The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections.


Internet search rankings have a significant impact on consumer choices, mainly because users trust and choose higher-ranked results more than lower-ranked results. Given the apparent power of search rankings, we asked whether they could be manipulated to alter the preferences of undecided voters in democratic elections. Here we report the results of five relevant double-blind, randomized controlled experiments, using a total of 4,556 undecided voters representing diverse demographic characteristics of the voting populations of the United States and India. The fifth experiment is especially notable in that it was conducted with eligible voters throughout India in the midst of India’s 2014 Lok Sabha elections just before the final votes were cast. The results of these experiments demonstrate that (i) biased search rankings can shift the voting preferences of undecided voters by 20% or more, (ii) the shift can be much higher in some demographic groups, and (iii) search ranking bias can be masked so that people show no awareness of the manipulation. We call this type of influence, which might be applicable to a variety of attitudes and beliefs, the search engine manipulation effect. Given that many elections are won by small margins, our results suggest that a search engine company has the power to influence the results of a substantial number of elections with impunity. The impact of such manipulations would be especially large in countries dominated by a single search engine company.

I’m not surprised by SEME (search engine manipulation effect).

Although I would probably be more neutral and say: Search Engine Impact on Voting.

Whether you consider one result or another as the result of “manipulation” is a matter of perspective. No search engine strives to delivery “false” information to users.

Gary Anthes in Search Engine Agendas, Communications of the ACM, Vol. 59 No. 4, pages 19-21, writes:

In the novel 1984, George Orwell imagines a society in which powerful but hidden forces subtly shape peoples’ perceptions of the truth. By changing words, the emphases put on them, and their presentation, the state is able to alter citizens’ beliefs and behaviors in ways of which they are unaware.

Now imagine today’s Internet search engines did just that kind of thing—that subtle biases in search engine results, introduced deliberately or accidentally, could tip elections unfairly toward one candidate or another, all without the knowledge of voters.

That may seem an unlikely scenario, but recent research suggests it is quite possible. Robert Epstein and Ronald E. Robertson, researchers at the American Institute for Behavioral Research and Technology, conducted experiments that showed the sequence of results from politically oriented search queries can affect how users vote, especially among undecided voters, and biased rankings of search results usually go undetected by users. The outcomes of close elections could result from the deliberate tweaking of search algorithms by search engine companies, and such manipulation would be extremely difficult to detect, the experiments suggest.

Gary’s post is a good supplement to the original article, covering some of the volunteers who are ready to defend the rest of us from biased search results.

Or as I would put it, to inject their biases into search results as opposed to other biases they perceive as being present.

If you are more comfortable describing the search results you want presented as “fair and equitable,” etc., please do so but I prefer the honesty of naming biases as such.

Or as David Bowie once said:

Make your desired bias, direction, etc., a requirement and allow data scientists to get about the business of conveying it.

Certainly what “ethical” data scientists are doing at Google as they conspire with the US government and others to overthrow governments, play censor to fight “terrorists,” and undertake other questionable activities.

I object to some of Google’s current biases because I would have them be biased in a different direction.

Let’s sell your bias/perspective to users with a close eye on the bright line of the law.


WordsEye [Subject Identity Properties]

Tuesday, March 29th, 2016


A site that enables you to “type a picture.” What? To illustrate:

A [mod] ox is a couple of feet in front of the [hay] wall. It is cloudy. The ground is shiny grass. The huge hamburger is on the ox. An enormous gold chicken is behind the wall…

Results in:


The site is in a close beta test but you can apply for an account.

I mention “subject identity properties” in the title because the words we use to identify subjects, are properties of subjects, just like any other properties we attribute to them.

Unfortunately, words are viewed by different people as identifying different subjects and the different words as identifying the same subjects.

The WordsEye technology can illustrates the fragility of using a single word to identify a subject of conversation.

Or that multiple identifications have the same subject, with side by side images that converge on a common image.

Imagine that in conjunction with 3-D molecular images for example.

I first saw this in a tweet by Alyona Medelyan.

Nebula Bliss

Monday, March 28th, 2016

Nebula Bliss

Visually impressive 3-D modeling of six different nebula.

I did not tag this with astroinformatics as it is a highly imaginative but non-scientific visualization.



The image is a screen capture from the Butterfly Nebula visualization.

Courses -> Texts: A Hidden Relationship

Monday, March 28th, 2016

Quite by accident I discovered the relationship between courses and their texts is hidden in many (approx. 2000) campus bookstore interfaces.

If you visit a physical campus bookstore you can browse courses for their textbooks. Very useful if you are interested the subject but not taking the course.

An online LLM (master’s of taxation) flyer prompted me to check the textbooks for the course work.

A simple enough information request. Find the campus bookstore and browse by course for text listings.

Not so fast!

The online presences of over 1200 campus bookstores are delivered, which offers this interface:


Another 748 campus bookstores are delivered by, with a similar interface for textbooks:


I started this post by saying the relationship between courses and their texts is hidden, but that’s not quite right.

The relationship between a meaningless course number and its required/suggested text is visible, but the identification of a course by a numeric string is hardly meaningful to the casual observer. (read not an enrolled student)

Perhaps better to say that a meaningful identification of courses for non-enrolled students and their relationship to required/suggested texts is absent.

That is the relationship of course -> text is present, but not in a form meaningful to anyone other than a student in that course.

Considering two separate vendors across almost 2,000 bookstores deliberately obscure the course -> text relationship, who has to wonder why?

I don’t have any immediate suggestions but when I encounter systematic obscuring of information across vendors, alarm bells start to go off.

Just for completeness sake, you can get around the obscuring of the course -> text relationship by searching for syllabus LLM taxation income OR estate OR corporate or (school name) syllabus LLM taxation income OR estate OR corporate. Extract required/suggested texts from posted syllabi.

PS: If you can offer advice on bookstore interfaces suggest enabling the browsing of courses by name and linking to the required/suggested texts.

During the searches I made writing this post, I encountered a syllabus on basic tax by Prof. Bret Wells which has this quote by Martin D. Ginsburg:

Basic tax, as everyone knows, is the only genuinely funny subject in law school.

Tax law does have an Alice in Wonderland quality about it, but The Hunting of the Snark: an Agony in Eight Fits is probably the closer match.

Kodály, String Quartet No. 1, 3rd movement

Sunday, March 27th, 2016

From the webpage:

Scherzo (3rd movement) of Zoltán Kodály’s first string quartet, performed by the Alexander String Quartet, accompanied by a graphical score.


Q: Where can I get this recording?
A: You complete album is available here:

Q: Who are the performers?
A: The Alexander String Quartet comprises Zakarias Grafilo and Frederick Lifsitz, violins, Paul Yarbrough, viola, and Sandy Wilson, violoncello. You can learn more about the group here:

Q: What do the colors mean?
A: Each pitch class (C, C-sharp, D, etc.) has its own color, arranged according to the “circle of fifths” so that changes in tonality can be seen; this system is described in more detail here:

In the first version of this video …… … the colors are applied to a conventional bar-graph score.

In the second version …… … the “staff” is the 12 pitch classes, arranged in circle-of-fifths order.

Q: Could you please do a video of _______?
A: Please read this:

If you want to see a data visualization with 26+ million views on YouTube, check out Stephen Malinowski’s YouTube channel.

Don’t miss Stephen Malinowski’s website. Select “site map” for a better idea of what you will find at the site.

#AlphaGo Style Monte Carlo Tree Search In Python

Sunday, March 27th, 2016

Raymond Hettinger (@raymondh) tweeted the following links for anyone who wants an #AlphaGo style Monte Carlo Tree Search in Python:

Introduction to Monte Carlo Tree Search by Jeff Bradberry.

Monte Carlo Tree Search by Cameron Browne.

Jeff’s post is your guide to Monte Carlo Tree Search in Python while Cameron’s site bills itself as:

This site is intended to provide a comprehensive reference point for online MCTS material, to aid researchers in the field.

I didn’t see any dated later than 2010 on Cameron’s site.

Suggestions for other collections of MCTS material that are more up to date?

Revealing the Hidden Patterns of News Photos:… [Uncovers Anti-Sanders Bias]

Saturday, March 26th, 2016

Revealing the Hidden Patterns of News Photos: Analysis of Millions of News Photos through GDELT and Deep Learning-based Vision APIs by Haewoon Kwak and Jisun An.


In this work, we analyze more than two million news photos published in January 2016. We demonstrate i) which objects appear the most in news photos; ii) what the sentiments of news photos are; iii) whether the sentiment of news photos is aligned with the tone of the text; iv) how gender is treated; and v) how differently political candidates are portrayed. To our best knowledge, this is the first large-scale study of news photo contents using deep learning-based vision APIs.

Not that bias-free news is possible, but deep learning appears to be useful in foregrounding bias against particular candidates:

We then conducted a case study of assessing the portrayal of Democratic and Republican party presidential candidates in news photos. We found that all the candidates but Sanders had a similar proportion of being labeled as an athlete, which is typically associates with a victory pose or a sharp focus on a face with blurred background. Pro-Clinton media recognized by their endorsements show the same tendency; their Sanders photos are not labeled as an athlete at all. Furthermore, we found that Clinton expresses joy more than Sanders does in the six popular news media. Similarly. pro-Clinton media shows a higher proportion of Clinton expressing joy than Sanders.

If the requirement is an “appearance” of lack of bias, the same techniques enable the monitoring/shaping of your content to prevent your bias from being discovered by others.

Data scientists who can successfully wield this framework will be in high demand for political campaigns.

“Ethical” Botmakers Censor Offensive Content

Saturday, March 26th, 2016

There are almost 500,000 “hits” from “tay ai” in one popular search engine today.

Against that background, I ran into: How to Make a Bot That Isn’t Racist by Sarah Jeong.

From the post:

…I talked to some creators of Twitter bots about @TayandYou, and the consensus was that Microsoft had fallen far below the baseline of ethical botmaking.

“The makers of @TayandYou absolutely 10000 percent should have known better,” thricedotted, a veteran Twitter botmaker and natural language processing researcher, told me via email. “It seems like the makers of @TayandYou attempted to account for a few specific mishaps, but sorely underestimated the vast potential for people to be assholes on the internet.”

Thricedotted and others belong to an established community of botmakers on Twitter that have been creating and experimenting for years. There’s a Bot Summit. There’s a hashtag (#botALLY).

As I spoke to each botmaker, it became increasingly clear that the community at large was tied together by crisscrossing lines of influence. There is a well-known body of talks, essays, and blog posts that form a common ethical code. The botmakers have even created open source blacklists of slurs that have become Step 0 in keeping their bots in line.

Not researching prior art is as bad as not Reading The Fine Manual (RTFM) before posting help queries to heavy traffic developer forums.

Tricedotted claims a prior obligation of TayandYou’s creators to block offensive content:

For thricedotted, TayandYou failed from the start. “You absolutely do NOT let an algorithm mindlessly devour a whole bunch of data that you haven’t vetted even a little bit,” they said. “It blows my mind, because surely they’ve been working on this for a while, surely they’ve been working with Twitter data, surely they knew this shit existed. And yet they put in absolutely no safeguards against it?!” (emphasis in original)

No doubt Microsoft wishes that it had blocked offensive content in hindsight, but I don’t see a general ethical obligation to block or censor offensive content.

For example:

  • A bot created to follow public and private accounts of elected officials and it only re-tweeted posts that did contain racial slurs? With @news-organization handles in the tweets.
  • A bot based on matching FEC (Federal Election Commission) donation records + Twitter accounts and it re-tweets racist/offensive tweets along with campaign donation identifiers and the candidate in question.
  • A bot that follows accounts known for racist/offensive tweets for the purpose of building archives of those tweets, publicly accessible, to prevent the sanitizing of tweet archives in the future. (like with TayandYou)

Any of those strike you as “unethical?”

I wish the Georgia legislature and the U.S. Congress would openly used racist and offensive language.

They act in racist and offensive ways so they should be openly racist and offensive. Makes it easier to whip up effective opposition against known racists, etc.

Which is, of course, why they self-censor to not use racist language.

The world is full of offensive people and we should make they own their statements.

Creating a false, sanitized view that doesn’t offend some n+1 sensitivities, is just that, a false view of the world.

If you are looking for an ethical issue, creating views of the world that help conceal racism, sexism, etc., is a better starting place than offensive ephemera.

“Not Understanding” was Tay’s Vulnerability?

Friday, March 25th, 2016

Peter Lee (Corporate Vice President, Microsoft Research) posted Learning from Tay’s introduction where he says:

Unfortunately, in the first 24 hours of coming online, a coordinated attack by a subset of people exploited a vulnerability in Tay. Although we had prepared for many types of abuses of the system, we had made a critical oversight for this specific attack. As a result, Tay tweeted wildly inappropriate and reprehensible words and images. We take full responsibility for not seeing this possibility ahead of time. We will take this lesson forward as well as those from our experiences in China, Japan and the U.S. Right now, we are hard at work addressing the specific vulnerability that was exposed by the attack on Tay.

But Peter never specifies what “vulnerability” Tay suffered from.

To find out what why Tay was “vulnerable,” you have to read Microsoft is deleting its AI chatbot’s incredibly racist tweets by Rob Price where he points out:

The reason it spouted garbage is that racist humans on Twitter quickly spotted a vulnerability — that Tay didn’t understand what it was talking about — and exploited it. (emphasis added)

Hmmm, how soon do you think Microsoft can confer on Tay the ability to “…understand what it [is] talking about…?”

I’m betting that’s not going to happen.

Tay can “learn” (read mimic) language patterns of users but if she speaks to racist users she will say racist things. Or religious, ISIS, sexist, Buddhist, trans-gender, or whatever things.

It isn’t ever going to be a question of Tay “understanding,” but rather of humans creating rules that prevent Tay from imitating certain speech patterns.

She will have no more or less “understanding” than before but her speech patterns will be more acceptable to some segments of users.

I have no doubt the result of Tay’s first day in the world was not what Microsoft wanted or anticipated.

That said, people are a ugly lot and I don’t mean a minority of them. All of us are better some days than others and about some issues and not others.

To the extent that Tay was designed to imitate people, I consider the project to be a success. If you think Tay should react the way some people imagine we should act, then it was a failure.

There’s an interesting question for Easter weekend:

Should an artificial intelligence act as we do or should it act as we ought to do?

PS: I take Peter’s comments about “…do not represent who we are or what we stand for, nor how we designed Tay…” at face value. However, the human heart is a dark place and to pretend that is true of a minority or sub-group, is to ignore the lessons of history.

Wrestling With Inclusion at LambdaConf [Why Wrestle? Just Do It.]

Friday, March 25th, 2016

Wrestling With Inclusion at LambdaConf by John A De Goes.

From the post:

Last year, StrangeLoop rescinded an invitation to a speaker because of the controversy that erupted (nay, exploded) when his talk was announced.

The controversy had nothing to do with the talk, which by all accounts was a great fit for the eclectic topics served up every year by the conference. Rather, the controversy surrounded the speaker’s political views, which were penned under a pseudonym years prior.

I learned about all this quite recently, and for a very unexpected reason: the same speaker submitted a talk to LambdaConf.

The gender- and person-anonymized talk was endorsed by the review committee, and made it all the way onto the schedule board before a volunteer brought the issue to our attention.

My immediate reaction could be described as a combination of shock and horror. No conference organizer ever wants to face a controversial hailstorm like this!

Far, far too long to read, unless you are interested in an example of public justification taken to its extreme.

Not that I disagree with the decision to include the speaker.

I do disagree that any speaker should be singled out for the sort of vetting that is described in John’s post.

All speakers should be accorded the presumption that they will obey local laws and not attempt to physically harm other conference attendees and will obey any code of conduct for the conference.

Absent evidence to the contrary. Evidence as reports, confirmed by news accounts and/or police reports of attacks at prior conferences or violation of other conference codes of conduct.

If a speaker endangers other attendees and/or violates conference rules of conduct, then don’t allow them to return. But don’t mimic the worse aspects of the developing police state in the United States and attempt to anticipate someone violating a norm of conduct.

Anticipatory regulation of possible future conduct is unfair to the person in question.

Not to mention being a distraction from advancing the subject of your conference.

As John’s post so ably demonstrates.

Imagine the useful articles, posts, code that could have been written with all that effort and strain.

Subject to documented prior arrests for violence against other attendees and/or violation of rules of conduct, modulo declarations to do the same, be inclusive.

What more need be said?

PS: Some people will disagree with that position but they can occupy their own digital space and time with un-responded to comments and diatribes. The right to speak does not imply an obligation to listen.

Dodging the Morality Police

Friday, March 25th, 2016

This location-based app helps young Iranians avoid ‘morality police’ by Aleks Buczkowski.

From the post:

Many young Iranians are pretty liberated guys. They like to party and wear fancy clothes but they happened to live in a country where it’s prohibited. There is special police force dedicated to ensuring Iranians follow strict rules on clothing and conduct, called the Gasht-e-Ershad (or Guidance Patrol, commonly known as the “morality police”). Part of their activities include setting up checkpoints around cities and randomly inspecting vehicles driving by.

Now there is a way to avoid the Ershad controls. An anonymous team of Iranian developers have come up with a crowdsource app that allow users marking risky spots on the city map to help others avoid it. Something like Waze but for a much different purpose.

The Gershad app is pretty simple and easy to use. Users can mark where they encounter the “morality police”. The data is added to a database and visualised on a map. The more reports in one place, the bolder the warning on the map. When the number decreases, the alert will fade gradually from the map. Simple as it is.

Sounds quite adaptable to tracking police, FBI agents, narcs, etc. in modern urban environments.

Over time, with enough reports, patterns for police patrols would emerge from the data.


AI Masters Go, Twitter, Not So Much (Log from @TayandYou?)

Thursday, March 24th, 2016

Microsoft deletes ‘teen girl’ AI after it became a Hitler-loving sex robot within 24 hours by Helena Horton.

From the post:

A day after Microsoft introduced an innocent Artificial Intelligence chat robot to Twitter it has had to delete it after it transformed into an evil Hitler-loving, incestual sex-promoting, ‘Bush did 9/11’-proclaiming robot.

Developers at Microsoft created ‘Tay’, an AI modelled to speak ‘like a teen girl’, in order to improve the customer service on their voice recognition software. They marketed her as ‘The AI with zero chill’ – and that she certainly is.

The headline was suggested to me by a tweet from Peter Seibel:

Interesting how wide the gap is between two recent AI: AlphaGo and TayTweets. The Turing Test is *hard*.

In preparation for the next AI celebration, does anyone have a complete log of the tweets from Tay Tweets?

I prefer non-revisionist history where data doesn’t disappear. You can imagine the use Stalin would have made of that capability.

2016 Freedom of Expression Awards shortlist

Thursday, March 24th, 2016

Index unveils 2016 Freedom of Expression Awards shortlist.

From the post:

An Aleppo-based journalist training women to report on the crisis in war-torn Syria, an Indonesian comic who jokes about Islamic extremism and a 19-year-old campaigner against repression in Eritrea are among those shortlisted for the 2016 Index on Censorship Freedom of Expression Awards.

Drawn from more than 400 crowdsourced nominations, the Index awards shortlist celebrates artists, writers, journalists and campaigners tackling censorship and fighting for freedom of expression. Many of the 20 shortlisted nominees are regularly targeted by authorities or by criminal and extremist groups for their work: some face regular death threats, others criminal prosecution.

Judges for this year’s awards are Nobel Prize-winning author Wole Soyinka, pianist James Rhodes, tech entrepreneur Bindi Karia, Colombian journalist Maria Teresa Ronderos, human rights lawyer Kirsty Brimelow QC and Bahraini campaigner Nabeel Rajab.

“Censorship is not something that happens ‘somewhere else’,” said Jodie Ginsberg, CEO of Index on Censorship. “It occurs on a daily basis in every country, in every part of the world. The shortlist honours those who are among the bravest and most creative in tackling such threats.”

Awards are offered in four categories: journalism; arts; campaigning; and digital activism.

Spread the word about these awards and contribute to the defense of freedom of expression.

Brave Clojure: Become a Better Programmer

Wednesday, March 23rd, 2016

Brave Clojure: Become a Better Programmer by Daniel Higginbotham.

From the post:

Next week week I’m re-launching as Brave Clojure. The site will continue featuring Clojure for the Brave and True, but I’m expanding its scope a bit. Instead of just housing the book, the purpose of the site will be to help you and the people you cherish become better programmers.

Like many other Clojurists, I fell in love with the language because learning it made me a better programmer. I started learning it because I was a bit bored and burnt out on the languages and tools I had been using. Ruby, Javascript, Objective-C weren’t radically different from each other, and after using them for many years I felt like I was stagnating.

But Clojure, with its radically different approach to computation (and those exotic parentheses) drew me out of my programming funk and made it fun to code again. It gave me new tools for thinking about software, and a concomitant feeling that I had an unfair advantage over my colleagues. So of course the subtitle of Clojure for the Brave and True is learn the ultimate language and become a better programmer.

And, four years since I first encountered Rich Hickey’s fractal hair, I still find Clojure to be an exceptional tool for becoming a better programmer. This is because Clojure is a fantastic tool for exploring programming concepts, and the talented community has created exceptional libraries for such diverse approaches as forward-chaining rules engines and constraint programming and logic programming, just to name a few.

Mark your calendar to help drive the stats for Daniel’s relaunch of as Brave Clojure.

Email, tweet, blog, etc., to help others drive not only the relaunch stats but the stats for following weeks as well.

This could be one of those situations where your early participation and contributions will shape the scope and the nature of this effort.


Mapping Mountains – Tangram

Tuesday, March 22nd, 2016

Mapping Mountains by Peter Richardson.

From the post:

I’ve been spending a lot of time over the mountains of Northern California lately. To view mountains from above is to journey through time itself: over ancient shorelines, the trails of glaciers, the marks of countless seasons, and the front lines of perpetual tectonic struggle. Fly with me now, on a tour through the world of elevation data:

A stunning display of mapping technology!

Peter starts with an illustrated history of the depiction of elevation on maps, including a map that was a declared to be a military secret!

It’s a quick romp that leads to “Tangram functionality” which is described elsewhere as:

Tangram is a map renderer designed to grant you ludicrous levels of control over your map design. By drawing vector tiles live in a web browser, it allows real-time map design, display, and interactivity.

Using WebGL, Tangram saddles and rides your graphics card into a new world of cartographic exploration. Animated shaders, 3D buildings, and dynamic filtering can be combined to produce effects normally seen only in science fiction.

Map styles, data filters, labels, and even graphics card code can be defined in a human-readable and -writable plaintext scene file, and a JavaScript API permits direct interactive control of the style.

The balance of the post is a lengthy demonstration of Tangram that ends in a call for test pilots!

Tangram reminded of the Art of War by Sun Tzu, where it reads:

All armies prefer high ground to low and sunny places to dark.

Which should now read:

All armies prefer Tangram map renderers to all others.

Seriously. Protesters, direct action movements, irregulars, etc. should take a long look at this post.

I first saw this in a tweet by Lynn Cherny.

Logan CIJ Symposium Videos 2016

Tuesday, March 22nd, 2016

All of the recordings from the Logan CIJ Symposium 2016 have been uploaded to YouTube, but in its usual minimal value format.

With the exception of Edward Snowden’s address, I have organized the presentations by title, reasoning that session numbers are forgotten quickly whereas a title may persist a bit longer.

I have pasted in the summaries from sessions page to assist you in choosing which videos to view. (I recommend viewing all of them.)

Edward Snowden addresses the audience 36:22


Anas Aremeyaw Anas 23:33

Anas Aremeyaw Anas employs anonymity and impersonation to enter into dangerous environments to capture stories of great power and impact at enormous personal risk. This award-winning journalist exposed injustice and cruelty all over Africa and Asia and will share insights about his world-changing work.

Challenge Power I 1:14:55

We live in an era of social, economic and environmental crises; an era where war rhetoric and fear are being used to justify and increase control and surveillance. Physical and digital barriers are being erected both publicly and in secret to obscure the truth, so it is absolutely necessary to challenge power. Although highly technical research is often the methodology of choice for investigators, the acquisition of critical evidence also relies on methods both intensely personal and unique.

Anas Aremeyaw Anas employs anonymity and impersonation to enter into dangerous environments to capture stories of great power and impact at enormous personal risk. This award-winning journalist exposed injustice and cruelty all over Africa and Asia and will share insights about his world-changing work.

Anas Aremeyaw Anas, P. Sainath, moderator Gavin MacFadyen

Challenge Power 2 34:22

Investigating crimes and wrongdoing even in nominal democracies requires independence and courage. Two of the world’s most successful journalists still experience difficulties in publishing their work in the mainstream media.

Seymour Hersh investigated the US Mỹ Lai massacre of civilians, which more than any other piece of journalism helped to bring the Vietnam War to an end. Later in Iraq, he exposed torture, murder and crimes against humanity at the US military prison at Abu Ghraib. Despite countless prizes and awards, he has found himself excluded from most mainstream outlets. His latest investigation will be published this December (2015) in the London Review of Books.
Sainath Palagummi single handedly and courageously exposed major crimes in the Indian countryside where over 285,000 farmers have committed suicide. His work is widely read in Asia and he is the recipient of major awards for journalism. His work has included studies of the ignored and forgotten foot soldiers of the Indian independence movement.

Seymour Hersh, moderator Gavin MacFadyen

Can you hear me now? 1:27:47

Sources and whistleblowers have provided many of the most important scoops in recent decades. Having exposed critical information that the authorities have obscured or kept secret, these people have become the targets of intense and sustained attacks by governments, corporations and even criminal organisations. These speakers will reveal the dangers and consequences of their courageous actions.

William Binney, Duncan Campbell, Thomas Drake, Annegret Falter, Holger Stark, moderator Gavin MacFadyen

Difficult Targets 1:29:09

The NSA, BND, GCHQ, the Mafia, the Narcos and multinational corporations are all powerful targets that require especially creative and innovative investigative methods. These organised power structures have inexhaustible resources that cannot be competed with financially or technically. To have a chance to defeat these systems of power, we need to act with creativity, tenacity and courage. Effective ways to reveal the truth and outperform illegitimate power are key to reclaiming our fundamental rights to freedom, privacy and dignity. Our panelists have extraordinary stories to tell and will present their projects to inspire the audience.

Tim Jenkin, Matt Kennard, Stefania Maurizi, Paul von Ribbeck, Matthias Spielkamp, moderator Caroline Nevejan

Fighting the Global Arena 1:20:39

From journalists to techies, from artists to activists, from experts to entertainers; everybody is needed to fight for the right to privacy, transparency and revolution. Our special guests will present a spectrum of spectacular actions.

More than five years ago Julian Assange built a sophisticated submission system that both promised and delivered security for whistleblowers around the world. It has led to a virtual explosion of publicly accessible information that governments and corporations have desperately tried to conceal. Disclosures of corruption, mass murder and illegal governmental activities on a huge scale are now irretrievably in the public domain.

The battle began with WikiLeaks and expanded dramatically with the Snowden revelations. Sadly, the universality of surveillance has meant that conventional computer systems are so compromised and vulnerable that radical new solutions are required.
In defence of personal freedom and privacy, the Subgraph team has built a highly secure new operating system that will be launched at this session. This long awaited new OS will hopefully provide a framework of protection for the future.
Last but not least, Giordano Nanni will introduce the riotous cult satirical online series Juice Rap News which has helped expose hypocrisy and lies with rhymes and a humor that is recognised across the globe. For the first time in Europe, Giordano will be accompanied by NSA spokesperson General Baxter and conspiracy guru Terrence Moonseed who will perform some of their most popular episodes live on stage – in what will be Juice Rap News’s first (and possibly last) European live performance.

Diversity is key!

David Mirza Ahmad, Julian Assange, Giordano Nanni, Juice Rap News Live Show, moderator Jérémie Zimmermann

Future of OS 1:12:58

In an era of mass surveillance the need for independent, reliable and usable Operating Systems is fundamental. In previous times, political movements needed their own printing press for circumventing propaganda and repression. In the digital age we need independent Operating Systems to protect our freedom of speech and freedom of action. The creation of independent OS is both a technological challenge and a social, political and economic challenge. To protect and encrypt yet offer transparency of control, and make it easy to use for all of us, is a great challenge. Trusting hardware, software and organisational structure is an issue from building all the way to using the OS.

David Mirza Ahmad, Joanna Rutkowska, Tails, moderator Jacob Appelbaum

Juice Rap News Live 29:16

Highlights from Juice Rap News Live at the Logan Symposium, Berlin 11 March 2016. With: Giordano Nanni @thejuicemedia, Mantra and Jeremedy (Grey Ghost)

Methods and Tools for Visual Investigations 1:00:03

The Forensic Architecture group at Goldsmiths University in London is revolutionising investigative analysis. They will give a ‘roadshow’ of new and unique tools and methods. Building on architectural knowledge of the 3D physical world, the group demonstrates how the newest technologies can be used to deconstruct lies and myths by creating simulations that prove a specific argument or fact is false. Students and PhD candidates in forensics present a variety of research tools they are developing for unveiling truth in highly sensitive political situations.

Steffen Krämer, Ana Naomi de Sousa, Christina Varvia, moderator Eyal Weizman

Reports from the Front 1:31:55

While most mainstream newspapers are no longer committed to finance investigative journalism, new entities focused on revealing uncomfortable facts have managed to establish themselves and contribute significantly to the critical and charged debate. With a myriad of backgrounds and approaches, investigative enterprises are constantly being challenged to provide access to new information. Speakers from Africa, South America and Europe will demonstrate and discuss radical new methods to bring important disclosures to the public.

Nafeez Ahmed, Jacob Appelbaum, Eveline Lubbers, Natalia Viana, Martin Welz, moderator John Goetz

Today’s investigative platforms 1:32:34

Seemingly every week, new platforms are emerging on the internet both sponsored by established media, as well as new organisations supported by philanthropy and/or novel revenue models. For the first time in years of decline, serious independent reporting is being seen again. A whole new journalistic repertoire is being developed in which people can inform each other and so unite in fighting injustice. Local or global, professional or grassroots, the actions of those who experiment with new ways of reporting have vital impact and often punch above their weight. In this session the actors will present the challenges they face.

Markus Beckedahl, Simona Levi, Edwy Plenel, Rob Wijnberg, moderator Bernd Fix

Transparency vs. Protection 1:29:42

How can journalists deal with increased surveillance? Governments and corporations are investing heavily in new intrusive surveillance capabilities. Those who challenge power are being directly targeted – be they hackers, journalists or activists. Strategies to counter increased surveillance include new technologies for transparency and new technologies for protection. Also, such strategies need to include a specific structuring of organisations, use of legal frameworks, and specific media strategies.

MC McGrath, Ibrahim Mohamoud, Jesselyn Radack, Marcel Rosenbach, Richard Tynan, moderator Andy Müller-Maguhn

I wasn’t able to find a video for the closing keynote:

From History to the Future

After two days of intense focus on current and previous practices of journalism, activism and hacktivism, Gavin MacFadyen elaborates in this closing session on insights that emerged during the symposium and translates these, with help from the audience, into shared efforts and directions in the near future.

Gavin MacFadyen


Logan CIJ Symposium 2016 – Speaker Contacts

Monday, March 21st, 2016

For some reason, conference organizers appear to abhor gathering speaker contact information to a common location. Or in such a way that it could be quickly re-purposed, say for creating a twitter list of those speakers.

The Logan CIJ Syposium 2016 was no different.

But, rather than complain to the conference organizers, I have collated the contact information (if any), for each speaker:

Some had weblinks only, while others only had Twitter accounts.

William Binney, Bernd Fix, John Goetz, Seymour Hersh, Caroline Nevejan, Ana Naomi de Sousa, and Christina Varvia had no contact information listed at all.

I started to ferret that out but then decided perhaps it wasn’t listed for a good reason.

Two requests:

First, pass this list along to others interested in journalism, news and reporting.

Second, point out to conference organizers that presenting author/presenter contact information in a re-usable format benefits everyone in their community.

Re-usable author/presenter contact information in a single location should be the default, not the exceptional case.

Pssst. Have You Got Five Minutes? 10 NICAR lightning talks… (New Conference Idea)

Monday, March 21st, 2016

10 NICAR lightning talks to guide you through cats, statistical resampling, fear of math, and more by Shan Wang.

From the post:

This year’s NICAR conference was in sunny Denver, and as promised, the sessions offered a little (or a lot) for everyone, from journalists looking for guidance on a stalled FOIA processes to those in search of advanced Python training to those who need advice on refining their interactives for mobile.

To break up the intensity of the sessions, NICAR also puts on the delightful lightning talks: five-minute presentations from attendees on topics of their choice, voted on by the NICAR community. The ten talks this year ran the gamut, and despite their length, were packed with useful tips and practical tools (cats featured prominently), as well as ideas for broadening how we think about data-driven journalism. Below are the talks from this year.

The page I link to above has links for the authors and the videos embedded.

Here’s a quick list:

  1. I Improved My Math Fluency, And So Can You by Ryann Grochowski Jones
  2. Solve Every Statistics Problem with One Weird Trick by Jonathan Stray
  3. Let lookup save you from the boring, repetitive work you’ve forgotten you’re even doing by Chris Groskopf
  4. Automation in the newsroom by Ariana Giorgi
  5. Regular Regular Expression Exercises for Regular People by Dan Nguyen
  6. Map tiles are dead; Long live (vector) tiles! by Ken Schwencke
  7. How to read 52 books in 52 weeks by Nicole Zhu
  8. What I learned working on Failure Factories by Adam Playford
  9. Let’s Talk About the Future of Interactive News Content by Gregor Aisch
  10. Cats and Stats by Jennifer LaFleur

New conference idea:

What if the presentations at a conference were all lighting talks? With full papers and longer videos posted to YouTube?

So that speakers would skip the history, which you likely already know. Skip the lead up to what they are about to show you. And basically cut to the most interesting bits of their presentation in five minutes!

If you are captivated by the “lighting” version, then you can watch the video, read the paper, etc.

You could cover more lighting talks in a day and thus increase the value of the travel dollar for every attendee.

If that sounds too short for your idea, remember:

If you can’t fit your idea on the back of a business card, you don’t have a clear idea.

If that sounds unfair, remember that Richard Feynman demonstrated why the Space Shuttle Challenger exploded shortly after launch with an O-ring and a glass of ice.

It took Feynman 47 seconds to do that demonstration.

If anything, 5 minutes to explain your idea is overly generous.


A Middle Ground on Slavery? Chesney & Vladeck on All-Writs Act

Monday, March 21st, 2016

I am deeply puzzled by commentators who find the analysis by Bobby Chesney and Steve Vladeck in A Coherent Middle Ground in the Apple-FBI All Writs Act Dispute? praise worthy.

Their position is captured by:

To cut to the chase, our view is that, properly understood, the All Writs Act should be read to authorize the kind of order the government has sought in these cases only when the recipient is compelled to help the government utilize existing vulnerabilities in its software, and not when the order instead directs the recipient to devote its resources to creating material new software vulnerabilities which can then be exploited by the government. (emphasis in original)

Chesney and Vladeck’s analysis, like some treatments of this issue, ignore the deciding of United States v. New York Telephone Company, 434 U.S. 159 (98 S.Ct. 364, 54 L.Ed.2d 376) by 5 to 4, with four justices dissenting on the issue of the district court’s authority to order the telephone company to provide “assistance” to the government.

The dissent by Justice Stevens that focuses on the All-Writs Act:

Even if I were to assume that the pen register order in this case was valid, I could not accept the Court’s conclusion that the District Court had the power under the All Writs Act, 28 U.S.C. 1651(a), to require the New York Telephone Company to assist in its installation. This conclusion is unsupported by the history, the language, or previous judicial interpretations of the Act.

The All Writs Act was originally enacted, in part, as § 14 of the Judiciary Act of 1789, 1 Stat. 81.15 The Act was, and is, necessary because federal courts are courts of limited jurisdiction having only those powers expressly granted by Congress,16 and the statute provides these courts with the procedural tools—the various historic common-law writs—necessary for them to exercise their limited jurisdiction.17 The statute does not contain, and has never before been interpreted as containing, the open-ended grant of authority to federal courts that today’s decision purports to uncover. Instead, in the language of the statute itself, there are two fundamental limitations on its scope. The purpose of any order authorized by the Act must be to aid the court in the exercise of its jurisdiction;18 and the means selected must be analogous to a common-law writ. The Court’s opinion ignores both limitations.

The Court starts from the premise that a district court may issue a writ under the Act “to effectuate and prevent the frustration of orders it has previously issued in its exercise of jurisdiction otherwise obtained.” Ante, at 172. As stated, this premise is neither objectionable nor remarkable and conforms to the principle that the Act was intended to aid the court in the exercise of its jurisdiction. Clearly, if parties were free to ignore a court judgment or order, the court’s ability to perform its duties would be undermined. And the court’s power to issue an order requiring a party to carry out the terms of the original judgment is well settled. See Root v. Woolworth, 150 U.S. 401, 410-413, 14 S.Ct. 136, 138, 37 L.Ed. 1123. The courts have also recognized, however, that this power is subject to certain restraints. For instance, the relief granted by the writ may not be “of a different kind” or “on a different principle” from that accorded by the underlying order or judgment. See id., at 411-412, 14 S.Ct., at 138-139.19

More significantly, the courts have consistently recognized and applied the limitation that whatever action the court takes must be in aid of its duties and its jurisdiction.20 The fact that a party may be better able to effectuate its rights or duties if a writ is issued never has been, and under the language of the statute cannot be, a sufficient basis for issuance of the writ. See Sampson v. Murray, 415 U.S. 61, 94 S.Ct. 1028, 39 L.Ed.2d 123; Commercial Security Bank v. Walker Bank & Trust Co., 456 F.2d 1352 (C.A.10, 1972); J. Moore, B. Ward, & J. Lucas, 9 Moore’s Federal Practice ¶ 110.29 (1975).

Nowhere in the Court’s decision or in the decisions of the lower courts is there the slightest indication of why a writ is necessary or appropriate in this case to aid the District Court’s jurisdiction. According to the Court, the writ is necessary because the Company’s refusal “threatened obstruction of an investigation . . ..” Ante, at 174. Concededly, citizen cooperation is always a desired element in any government investigation, and lack of cooperation may thwart such an investigation, even though it is legitimate and judicially sanctioned.21 But unless the Court is of the opinion that the District Court’s interest in its jurisdiction was coextensive with the Government’s interest in a successful investigation there is simply no basis for concluding that the inability of the Government to achieve the purposes for which it obtained the pen register order in any way detracted from or threatened the District Court’s jurisdiction. Plainly, the District Court’s jurisdiction does not ride on the Government’s shoulders until successful completion of an electronic surveillance.

If the All Writs Act confers authority to order persons to aid the Government in the performance of its duties, and is no longer to be confined to orders which must be entered to enable the court to carry out its functions, it provides a sweeping grant of authority entirely without precedent in our Nation’s history. Of course, there is precedent for such authority in the common law the writ of assistance. The use of that writ by the judges appointed by King George III was one British practice that the Revolution was specifically intended to terminate. See n. 3, supra. I can understand why the Court today does not seek to support its holding by reference to that writ, but I cannot understand its disregard of the statutory requirement that the writ be “agreeable to the usages and principles of law.”

The order directed against the Company in this case is not particularly offensive. Indeed, the Company probably welcomes its defeat since it will make a normal profit out of compliance with orders of this kind in the future. Nevertheless, the order is deeply troubling as a portent of the powers that future courts may find lurking in the arcane language of Rule 41 and the All Writs Act.

I would affirm the judgment of the Court of Appeals.



The statute was also derived from § 13 of the Judiciary Act, which concerned writs of mandamus and prohibition, 1 Stat. 80, and a statute dealing with writs of ne exeat, 1 Stat. 334. The All Writs Act now reads:

“(a) The Supreme Court and all courts established by Act of Congress may issue all writs necessary or appropriate in aid of their respective jurisdictions and agreeable to the usages and principles of law.”


This proposition was so well settled by 1807 that Mr. Chief Justice Marshall needed no citation to support the following statement:

“As preliminary to any investigation of the merits of this motion, this court deems it proper to declare that it disclaims all jurisdiction not given by the constitution, or by the laws of the United States.

“Courts which originate in the common law possess a jurisdiction which must be regulated by their common law, until some statute shall change their established principles; but courts which are created by written law, and whose jurisdiction is defined by written law, cannot transcend that jurisdiction. It is unnecessary to state the reasoning on which this opinion is founded, because it has been repeatedly given by this court; and with the decisions heretofore rendered on this point, no member of the bench has, even for an instant, been dissatisfied.” Ex parte Bollman, 4 Cranch 75, 93, 2 L.Ed. 554.


See Harris v. Nelson, 394 U.S. 286, 299.


This Court has frequently considered this requirement in the context of orders necessary or appropriate in the exercise of appellate jurisdiction. See J. Moore, B. Ward, & J. Lucas, 9 Moore’s Federal Practice &Par; 110.27-110.28 (1975). Here, we are faced with an order that must be necessary or appropriate in the exercise of a district court’s original jurisdiction.


These restraints are necessary concomitants of the undisputed fact that the All Writs Act does not provide federal courts with an independent grant of jurisdiction. McIntire v. Wood, 7 Cranch 504, 3 L.Ed. 420; Rosenbaum v. Bauer, 120 U.S. 450, 7 S.Ct. 633, 30 L.Ed. 743. The factors mentioned above may be relevant in determining whether the court has ancillary jurisdiction over the dispute. See Dugas v. American Surety Co., 300 U.S. 414, 57 S.Ct. 515, 81 L.Ed. 720; Labette County Comm’rs v. Moulton, 112 U.S. 217, 5 S.Ct. 108, 28 L.Ed. 698; Morrow v. District of Columbia, 135 U.S.App.D.C. 160, 417 F.2d 728 (1969). In this case, the District Court’s order was entered against a third party—the Telephone Company. The Court never explains on what basis the District Court had jurisdiction to enter this order. Possibly, the District Court believed that it had ancillary jurisdiction over the controversy, or that the failure of the Company to aid the Government posed a federal question under 28 U.S.C. 1331. See Board of Education v. York, 429 F.2d 66 (C.A.10 1970), cert. denied, 401 U.S. 954, 91 S.Ct. 968, 28 L.Ed.2d 237. Since I believe that the District Court could not enter its order in any event since it was not in aid of its jurisdiction, I do not find it necessary to reach the question where there was jurisdiction, apart from the All Writs Act, over the “dispute” between the Government and the Telephone Company. However, the Court’s failure to indicate the basis of jurisdiction is inexplicable.


The Court’s failure to explain why the District Court’s order was in aid of its jurisdiction is particularly notable when compared to the rationale of the prior Court cases on which it relies. See, e. g., Harris v. Nelson, 394 U.S. 286, 299, 89 S.Ct. 1082, 1090, 22 L.Ed.2d 281 (“the habeas corpus jurisdiction and the duty to exercise it being present, the courts may fashion appropriate modes of procedure . . . . Where their duties require it, this is the inescapable obligation of the courts”) (emphasis added); FTC v. Dean Foods Co., 384 U.S. 597, 604, 86 S.Ct. 1738, 1742, 16 L.Ed.2d 802 (injunction issued under All Writs Act upheld because it was necessary “to preserve the status quo while administrative proceedings are in progress and prevent impairment of the effective exercise of appellate jurisdiction “) (emphasis added).

The Court apparently concludes that there is no functional distinction between orders designed to enable a party to effectuate its rights and orders necessary to aid a court in the exercise of its jurisdiction. Ante, at 175 n. 23. The Court reaches this conclusion by pointing out that the orders in cases such as Harris v. Nelson, supra, protected a party’s rights. This is, of course, true. Orders in aid of a court’s jurisdiction will usually be beneficial to one of the parties before the court. The converse, however, is clearly not true. Not all orders that may enable a party to effectuate its rights aid the court in its exercise of jurisdiction. Compare Sampson v. Murray, 415 U.S. 61, 94 S.Ct. 937, 39 L.Ed.2d 166, with FTC v. Dean Foods Co., supra.


A citizen is not, however, free to forcibly prevent the execution of a search warrant. Title 18 U.S.C. 2231 imposes criminal penalties on any person who “forcibly assaults, resists, opposes, prevents, impedes, intimidates, or interferes with any person authorized to serve or execute search warrants . . . .” This section was originally enacted as part of the Espionage Act of 1917, see n. 6, supra, and is the only statutory provision imposing any duty on the general citizenry to “assist” in the execution of a warrant.


As Justice Stevens points out:

If the All Writs Act confers authority to order persons to aid the Government in the performance of its duties, and is no longer to be confined to orders which must be entered to enable the court to carry out its functions, it provides a sweeping grant of authority entirely without precedent in our Nation’s history. Of course, there is precedent for such authority in the common law the writ of assistance. The use of that writ by the judges appointed by King George III was one British practice that the Revolution was specifically intended to terminate. See n. 3, supra. I can understand why the Court today does not seek to support its holding by reference to that writ, but I cannot understand its disregard of the statutory requirement that the writ be “agreeable to the usages and principles of law.”

the construction urged by Chesney & Vladeck and the slim majority in United States v. New York Telephone Company, 434 U.S. 159 (98 S.Ct. 364, 54 L.Ed.2d 376), is a parallel to one of the reasons for the American Revolution.

The writ to compel Apple to assist the FBI is the

…portent of the powers that future courts may find lurking in the arcane language of Rule 41 and the All Writs Act.

that Justice Stevens foresaw in his dissent.

Rather than scrambling for some tortured “middle ground” with the FBI, legal scholars should be lining up to urge correction of the mistake made in United States v. New York Telephone Company, 434 U.S. 159 (98 S.Ct. 364, 54 L.Ed.2d 376).

The time has come to end the threat of slavery for both corporations and individuals under the All-Writs Act.

The portions of the opinions quoted above are from the Legal Information Institute (LII) at Cornell University Law School. Supporting the LII fosters public access to legal resources.