Google Helps Spread Fake News [Fake News & Ad Revenue – Testing]

December 10th, 2016

Google changed its search algorithm and that made it more vulnerable to the spread of fake news by Hannah Roberts.

From the post:

Google’s search algorithm has been changed over the last year to increasingly reward search results based on how likely you are to click on them, multiple sources tell Business Insider.

As a result, fake news now often outranks accurate reports on higher quality websites.

The problem is so acute that Google’s autocomplete suggestions now actually predict that you are searching for fake news even when you might not be, as Business Insider noted on December 5.

Hannah does a great job of setting for the evidence and opinions on the algorithm change but best summarizes it when she says:


The changes to the algorithm now move links up Google’s search results page if Google detects that more people are clicking on them, search experts tell Business Insider.

Just in case you don’t know:

more clicks != credible/useful search results

But it is true:

more clicks = more usage/ad revenue

Google and Facebook find “fake news” profitable. Both will make a great show of suppressing outlying “fake news,” but not so much as to impact profits.

There’s a data science “fake news” project:

Track the suppression of “fake news” by Google and Facebook against the performance of their ad revenue.

Hypotheses: When suppression of “fake news” impinges on ad revenue for more than two consecutive hours, dial back on suppression mechanisms. (ditto for 4, 6, 12 and 24 hour cycles)

Odds on Google and Facebook being transparent regard to suppression of “fake news” and ad revenue to make the results of testing that hypotheses verifiable?

;-)

The Koch Brothers are Attacking Libraries – FYI – Funding Appeal

December 10th, 2016

EveryLibrary has a funding appeal you need to seriously consider.

The Koch Brothers are Attacking Libraries

From the post:

We are continuing to see the Koch Brothers Super PAC, Americans for Prosperity go after libraries. This last election cycle was the fifth clear example of their involvement in the agenda to defund libraries. We need your help to fight back. When the Koch Brothers and AFP puts out an anti-tax and anti-library attack, they do it with direct mail and robocalls – and they always do it late in the campaign. We need the resources to confront these anti-tax forces before they can start in the next town. Help us stop them with a one time donation today or a $5-10 monthly donation.
… (emphasis in original)

I won’t repeat the crimes committed against libraries by the Koch Brothers and their Super PAC, Americans for Prosperity, here, they are too sickening. The EveryLibrary post has a sub-set of their offenses described.

Be sure to check out the EveryLibrary site and their journal, The Political Librarian.

From their What We Do page:

EveryLibrary is the first and only national organization dedicated to building voter support for libraries. We are chartered “to promote public, school, and college libraries, including by advocating in support of public funding for libraries and building public awareness of public funding initiatives”. Our primary work is to support local public libraries when they have a referendum or measure on the ballot. We do this in three ways: by training library staff, trustees, and volunteers to plan and run effective Information Only campaigns; by assisting local Vote Yes committees on planning and executing Get Out the Vote work for their library’s measure; and by speaking directly to the public about the value and relevance of libraries and librarians. Our focus on activating voters on Election Day is unique in the library advocacy ecosystem. This is reflected in the training and coaching we do for campaigns.

If you have ever fantasized about saving the Library at Alexandria or opposing the sack of Rome by the Vandals and the Visigoths, now is your chance to do more than fantasize.

Libraries are islands of knowledge under siege by the modern analogues of the barbarians that plunged the world into centuries of darkness.

Will you piss and moan on Facebook, Twitter, etc. about the crumbling defenses of libraries or will you take your place on the ramparts?

Yes?

Data Science and Protests During the Age of Trump [How To Brick A School Bus…]

December 9th, 2016

Pre-inauguration suppression of free speech/protests is underway for the Trump regime. (CNN link as subject identifier for Donald J. Trump, even though it fails to mention he looks like a cheeto in a suit.)

Women’s March on Washington barred from Lincoln Memorial by Amber Jamieson and Jessica Glenza.

From the post:


For the thousands hoping to echo the civil rights and anti-Vietnam rallies at Lincoln Memorial by joining the women’s march on Washington the day after Donald Trump’s inauguration: time to readjust your expectations.

The Women’s March won’t be held at the Lincoln Memorial.

That’s because the National Park Service, on behalf of the Presidential Inauguration Committee, filed documents securing large swaths of the national mall and Pennsylvania Avenue, the Washington Monument and the Lincoln Memorial for the inauguration festivities. None of these spots will be open for protesters.

The NPS filed a “massive omnibus blocking permit” for many of Washington DC’s most famous political locations for days and weeks before and after the inauguration on 20 January, said Mara Verheyden-Hilliard, a constitutional rights litigator and the executive director of the Partnership for Civil Justice Fund.

I contacted Amber Jamieson for more details on the permits and she forwarded two links (thanks Amber!):

Press Conference: Mass Protests Will Go Forward During Inauguration, which had the second link she forwarded:

PresidentialInauguralCommittee12052016.pdf, the permit requests made by the National Park Service on behalf of the Presidential Inaugural Committee.

Start with where protests are “permitted” to see what has been lost.

A grim read but 36 CFR 7.96 says in part:


3 (i) White House area. No permit may be issued authorizing demonstrations in the White House area, except for the White House sidewalk, Lafayette Park and the Ellipse. No permit may be issued authorizing special events, except for the Ellipse, and except for annual commemorative wreath-laying ceremonies relating to the statutes in Lafayette Park.

(emphasis added, material hosted by the Legal Information Institute (LII))

Summary: In White House area, protesters have only three places for permits to protest:

  • White House sidewalk
  • Lafayette Park
  • Ellipse

White House sidewalk / Lafayette Park (except North-East Quadrant) – Application 16-0289

Dates:

Set-up dates starting 11/1/2016 6:00 am ending 1/19/2017
Activity dates starting 1/20/2017 ending 1/20/2017
Break-down dates starting 1/21/2017 ending 3/1/2017 11:59 pm

Closes:


All of Lafayette Park except for its northeast quadrant pursuant to 36 CFR 7.96 (g)(4)(iii)(A). The initial areas of Lafayette Park and the White House Sidewalk that will be needed for construction set-up, and which will to be closed to ensure public safety, is detailed in the attached map. The attached map depicts the center portion of the White House Sidewalk as well as a portion of the southern oval of Lafayette Park. The other remaining areas in Lafayette Park and the White House Sidewalk that will be needed for construction set-up, will be closed as construction set-up progresses into these other areas, which will also then be delineated by fencing and sign age to ensure public safety.

Two of the three possible protest sites in the White House closed by Application 16-0289.

Ellipse – Application 17-0001

Dates:

Set-up dates starting 01/6/2017 6:00 am ending 1/19/2017
Activity dates starting 1/20/2017 ending 1/20/2017
Break-down dates starting 1/21/2017 ending 2/17/2017 11:59 pm

These dates are at variance with those for the White House sidewalk and Lafayette Park (shorter).

Closes:

Ellipse, a fitty-two acre park, as depicted by Google Maps:

ellipse-460

Plans for the Ellipse?


Purpose of Activity: In connection with the Presidential Inaugural Ceremonies, this application is for use of the Ellipse by PIC, in the event that PIC seeks its use for Inaugural ceremonies and any necessary staging, which is expected to be:

A) In the event that PIC seeks the use of the Ellipse for pre- and/or post- Inaugural ceremonies, the area will be used for staging the event(s), staging of media to cover and/or broadcast the event, and if possible for ticketed and/or public viewing; and/or ­

B) In the event that PIC seeks the use of the Ellipse for the Inaugural ceremony and Inaugural parade staging, the area will be used to stage the various parade elements, for media to cover and/or broadcast the event, and if possible for ticketed and/or public viewing.

The PIC has no plans to use the Ellipse but has reserved it no doubt to deny its use to others.

Those two applications close three out of three protest sites in the White House area. The PIC went even further to reach out and close off other potential protest sites.

Other permits granted to the PIC include:

Misc. Areas – Application 16-0357

Ten (10) misc. areas identified by attached maps for PIC activities.

Arguably legitimate since the camp followers, sycophants and purveyors of “false news” need somewhere to be during the festivities.

National Mall -> Trump Mall – Application 17-0002

The National Mall will become Trump Mall for the following dates:

Set-up dates starting 01/6/2017 6:00 am ending 1/19/2017
Activity dates starting 1/20/2017 ending 1/20/2017
Break-down dates starting 1/21/2017 ending 1/30/2017 11:59 pm

Closes:


Plan for Proposed Activity: Consistent with NPS regulations at 36 CFR 7.96{g)(4)(iii)(C), this application seeks, in connection with the Presidential Inaugural Ceremonies, the area of the National Mall between 14th – 4th Streets, for the exclusive use of the Joint Task Force Headquarters (JTFHQ) on Inaugural Day for the assembly, staging, security and weather protection of the pre-Inaugural parade components and floats on Inaugural Day between 14th – 7th Streets. It also includes the placement of jumbotrons and sound towers by the Architect of the Capitol or the Joint Congressional Committee on Inaugural Ceremonies so that the Inaugural Ceremony may be observed by the Joint Congressional Committee’s ticketed standing room ticket holders between 4th – 3rd streets and the general public, which will be located on the National Mall between 7th – 4th Streets. Further, a 150-foot by 200-foot area on the National Mall just east of 7th Street, will be for the exclusive use of the Presidential Inaugural Committee for television and radio media broadcasts on Inaugural Day.

In the plans thus far, no mention of the main card or where the ring plus cage will be erected on Trump Mall. (that’s sarcasm, not “fake news”)

Most Other Places – Application 17-0003

If you read 36 CFR 7.96 carefully, you noticed there are places always prohibited to protesters:


(ii) Other park areas. Demonstrations and special events are not allowed in the following other park areas:

(A) The Washington Monument, which means the area enclosed within the inner circle that surrounds the Monument’s base, except for the official annual commemorative Washington birthday ceremony.

(B) The Lincoln Memorial, which means that portion of the park area which is on the same level or above the base of the large marble columns surrounding the structure, and the single series of marble stairs immediately adjacent to and below that level, except for the official annual commemorative Lincoln birthday ceremony.

(C) The Jefferson Memorial, which means the circular portion of the Jefferson Memorial enclosed by the outermost series of columns, and all portions on the same levels or above the base of these columns, except for the official annual commemorative Jefferson birthday ceremony.

(D) The Vietnam Veterans Memorial, except for official annual Memorial Day and Veterans Day commemorative ceremonies.

What about places just outside the already restricted areas?

Dates:

Set-up dates starting 01/6/2017 6:00 am ending 1/19/2017
Activity dates starting 1/20/2017 ending 1/20/2017
Break-down dates starting 1/21/2017 ending 2/10/2017 11:59 pm

Closes:


The Lincoln Memorial area, as more fully detailed as the park area bordered by 23rd Street, Daniel French Drive and Independence Avenue, Henry Bacon Drive and Constitution Avenue, Constitution Avenue between 15th & 23rd Streets, Constitution Gardens to include Area #5 outside of the Vietnam Veteran’s Memorial restricted area, the Lincoln Memorial outside of its restricted area, the Lincoln Memorial Plaza and Reflecting Pool Area, JFK Hockey Field, park area west of Lincoln Memorial between French Drive, Henry Bacon Drive, Parking Lots A, Band C, East and West Potomac Park, Memorial Bridge, Memorial Circle and Memorial Drive, the World War II Memorial. The Washington Monument Grounds as more fully depicted as the park area bounded by 14th & 15th Streets and Madison Drive and Independence Avenue.

Not to use but to prevent its use by others:


Purpose of Activity: In connection with the Presidential Inaugural Ceremonies, this application is for use of the Lincoln Memorial areas and Washington Monument grounds by PIC, in the event that PIC seeks its use for the Inaugural related ceremonies and any necessary staging, which is expected to be:

A) In the event that PIC seeks the use of the Lincoln Memorial areas for a pre-and/or post Inaugural ceremonies, the area will be used for staging the event(s), staging of media to cover and/or broadcast the event, and for ticketed and/or public viewing.

B) In the event that PIC seeks to use the Washington Monument grounds for a public overflow area to view the Inaugural ceremony and/ or parade, the area will be used for the public who will observe the activities through prepositioned jumbotrons and sound towers.

Next Steps

For your amusement, all five applications contain the following question answered No:

Do you have any reason to believe or any information indicating that any individual, group or organization might seek to disrupt the activity for which this application is submitted?

I would venture to say someone hasn’t been listening. ;-)

Among the data science questions raised by this background information are:

  • How best to represent these no free speech and/or no free assembly zones on a map?
  • What data sets do you need to make protesters effective under these restrictions?
  • What questions would you ask of those data sets?
  • How to decide between viral/spontaneous action versus publicly known but lawful conduct, up until the point it becomes unlawful?

If you use any of this information, please credit Amber Jamieson, Jessica Glenza and the Partnership for Civil Justice Fund as the primary sources.

See further news from the Partnership for Civil Justice Fund at: Your Right of Resistance.

Tune in next Monday for: How To Brick A School Bus, Data Science Helps Park It.

PS: “The White House Sidewalk is the sidewalk between East and West Executive Avenues, on the south side Pennsylvania Avenue, N.W.” From OMB Control No. 1024-0021 – Application for a Permit to Conduct a Demonstration or Special Event in Park Areas and a Waiver of Numerical Limitations on Demonstrations for White House Sidewalk and/or Lafayette Park

Comparing Symbolic Deep Learning Frameworks

December 8th, 2016

Deep Learning Part 1: Comparison of Symbolic Deep Learning Frameworks by Anusua Trivedi.

From the post:

This blog series is based on my upcoming talk on re-usability of Deep Learning Models at the Hadoop+Strata World Conference in Singapore. This blog series will be in several parts – where I describe my experiences and go deep into the reasons behind my choices.

Deep learning is an emerging field of research, which has its application across multiple domains. I try to show how transfer learning and fine tuning strategy leads to re-usability of the same Convolution Neural Network model in different disjoint domains. Application of this model across various different domains brings value to using this fine-tuned model.

In this blog (Part1), I describe and compare the commonly used open-source deep learning frameworks. I dive deep into different pros and cons for each framework, and discuss why I chose Theano for my work.

Your mileage may vary but a great starting place!

Pearl Harbor – 1941 – Talking Heads Blamed Germany (Now North Korea, Russia, etc.)

December 7th, 2016

Reporting and punditry that escaped infamy by Michael J. Socolow.

Does this remind you of reporting during 2016:


As the day wore on, real reporting receded, giving way to more speculation. Right-wing commentator Fulton Lewis Jr. told an audience five hours after the attack that he shared the doubts of many American authorities that the Japanese were truly responsible. He “reported” that US military officials weren’t convinced Japanese pilots had the skills to carry out such an impressive raid. The War Department, he said, is “concerned to find out who the pilots of these planes are—whether they are Japanese pilots. There is some doubt as to that, some skepticism whether they may be pilots of some other nationality, perhaps Germans, perhaps Italians,” he explained. The rumor that Germans bombed Pearl Harbor lingered on the airwaves, with NBC reporting, on December 8, that eyewitnesses claimed to have seen Nazi swastikas painted on some of the bombers.

More recent media failures include 9/11 and Hurricane Katrina.

Even more recently, the media has seized upon flights of fancy by “experts” to blame North Korea, Russia, the Islamic State and others for a variety of ills and disasters.

Thoughts on what leads to such media failures time and time again? I can’t think of a single major news event in the last fifty (50) years that wasn’t accompanied by:

…terrible punditry, inaccurate reporting, and ridiculous commentary

to steal Socolow’s closing line.

The failure of the news media reminds me of a discussion with a Hebrew Bible professor over the translation of a verse into English. He conceded that we don’t know the meaning of a particular term but said a translator cannot simply pass over an unknown term, but must translate it. The verse in question is well-known so the committee took refuge in giving the term an incorrect but “traditional” translation.

To what degree does the news media offer “terrible punditry, inaccurate reporting, and ridiculous commentary” because of a requirement that events, people, causes, “make sense?”

That is it is unsatisfying to report a plane crash, stock failure, bombing, without some attempt to outfit the event with an explanation.

I’m not sure if unsatisfying applies to the reporters, the news consuming public, or both.

For my part, I’m incurious about the motives of people who harm other people, assuming that even the “alleged perpetrator” has some insight into their motives. Motive is a complex and difficult subject under the best of circumstances and a sound bite of less than 30 seconds is a long way from being sufficient.

But it leaves the viewer with the false impression they have learned something about an event, people, etc.

One way to avoid “…terrible punditry, inaccurate reporting, and ridiculous commentary…,” unless you are required to respond to a particular event, is to simply ignore reporting for several days or weeks after an event. The more major the event, the longer you should delay.

For example, when 9/11 occurred, I was in San Jose, California at a Unicode conference. In fact, I was working on email waiting for the conference to start.

After the news had spread, the conference organizers asked the attendees what we wanted to do. Given the choice of watching loops of planes crashing into the World Trade Center and uninformed commentary or continuing with the conference, we chose the latter.

Delayed consumption of news of major events won’t improve the quality of the immediate reporting but it may give time for more reasonable voices to emerge. Still waiting on that to happen for 9/11.

Facebook Patents Tool To Think For You

December 7th, 2016

My apologies but Facebook thinks you are too stupid to detect “fake news.” Facebook will compensate for your stupidity with a process submitted for a US patent. For free!

Facebook is patenting a tool that could help automate removal of fake news by Casey Newton.

From the post:

As Facebook works on new tools to stop the spread of misinformation on its network, it’s seeking to patent technology that could be used for that purpose. This month the US Trademark and Patent Office published Facebook’s application for Patent 0350675: “systems and methods to identify objectionable content.” The application, which was filed in June 2015, describes a sophisticated system for identifying inappropriate text and images and removing them from the network.

As described in the application, the primary purpose of the tool is to improve the detection of pornography, hate speech, and bullying. But last month, Zuckerberg highlighted the need for “better technical systems to detect what people will flag as false before they do it themselves.” The patent published Thursday, which is still pending approval, offers some ideas for how such a system could work.

A Facebook spokeswoman said the company often seeks patents for technology that it never implements, and said this patent should not be taken as an indication of the company’s future plans. The spokeswoman declined to comment on whether it was now in use.

The system described in the application is largely consistent with Facebook’s own descriptions of how it currently handles objectionable content. But it also adds a layer of machine learning to make reporting bad posts more efficient, and to help the system learn common markers of objectionable content over time — tools that sound similar to the anticipatory flagging that Zuckerberg says is needed to combat fake news.

If you substitute “user” for “administrator” where it appears in the text, Facebook would be enabling users to police the content they view.

Why Facebook finds users making decisions about the content they view objectionable isn’t clear. Suggestions on that question?

The process doesn’t appear to be either accountable and/or transparent.

If I can’t see the content that is removed by Facebook, how do I make judgments about why it was removed and/or how that compares to content about to be uploaded to Facebook?

Urge Facebook users to demand empowering them to make decisions about the content they view.

Urge Facebook shareholders to pressure management to abandon this quixotic quest to be an internet censor.

Writing a Halite Bot in Clojure [Incomplete Knowledge/Deception Bots?]

December 6th, 2016

Writing a Halite Bot in Clojure by Matt Adereth.

From the post:

Halite is a new AI programming competition that was recently released by Two Sigma and Cornell Tech. It was designed and implemented by two interns at Two Sigma and was run as the annual internal summer programming competition.

While the rules are relatively simple, it proved to be a surprisingly deep challenge. It’s played on a 2D grid and a typical game looks like this:

halite-game-460

Each turn, all players simultaneously issue movement commands to each of their pieces:

  1. Move to an adjacent location and capture it if you are stronger than what’s currently there.
  2. Stay put and build strength based on the production value of your current location.

When two players’ pieces are adjacent to each other, they automatically fight. A much more detailed description is available on the Halite Game Rules page.

Looking at the rules page, it looks like:

  • Bots have accurate knowledge of all positions and values.
  • Deception of bots isn’t permitted.
  • Interesting from a bot programming perspective but lack of knowledge and the ever present danger of deception are integral parts of human games.

    Any bot games that feature both a lack of knowledge and/or deception?

Four Experiments in Handwriting with a Neural Network

December 6th, 2016

Four Experiments in Handwriting with a Neural Network by Shan Carter, David Ha, Ian Johnson, and Chris Olah.

While the handwriting experiments are compelling and entertaining, the author’s have a more profound goal for this activity:


The black box reputation of machine learning models is well deserved, but we believe part of that reputation has been born from the programming context into which they have been locked into. The experience of having an easily inspectable model available in the same programming context as the interactive visualization environment (here, javascript) proved to be very productive for prototyping and exploring new ideas for this post.

As we are able to move them more and more into the same programming context that user interface work is done, we believe we will see richer modes of human-ai interactions flourish. This could have a marked impact on debugging and building models, for sure, but also in how the models are used. Machine learning research typically seeks to mimic and substitute humans, and increasingly it’s able to. What seems less explored is using machine learning to augment humans. This sort of complicated human-machine interaction is best explored when the full capabilities of the model are available in the user interface context.

Setting up a search alert for future work from these authors!

Pattern Overloading

December 6th, 2016

Pattern Overloading by Ramsey Nasser.

From the post:

C-like languages have a problem of overloaded syntax that I noticed while teaching high school students. Consider the following snippets in such a language:

foo(45)

function foo(int x) {

for(int i=0;i < 10; i++) {

if(x > 10) {

case(x) {

A programmer experienced with this family would see

  1. Function invocation
  2. Function definition
  3. Control flow examples

In my experience, new programmers see these constructs as instances of the same idea: name(some-stuff) more-stuff. This is not an unreasonable conclusion to reach. The syntax for each construct is shockingly similar given that their semantics are wildly different.

You won’t be called upon to re-design C but Nasser’s advice:

Syntactic similarity should mirror semantic similarity

Or, to take a quote from the UX world

Similar things should look similar and dissimilar things should look dissimilar

is equally applicable to any syntax that you design.

Attn: “Fake News” Warriors! Where’s The Harm In Terrorist Propaganda?

December 6th, 2016

Facebook, Microsoft, Twitter, and YouTube team up to stop terrorist propaganda by Justin Carissimo.

Justin’s report is true, at least in the sense that Facebook, Microsoft, Twitter, and YouTube are collaborating to censor “terrorist propaganda.”

Justin’s post also propagates the “fake news” that online content from terrorists “…threaten our national security and public safety….”

Really? You would think after all these years of terrorist propaganda, there would be evidence to support that claim.

True enough, potential terrorists can meet online, but “recruitment” is a far different tale than reading online terrorist content. Consider ISIS and the Lonely Young American, a tale told to support the idea of online recruiting, but is one of the better refutations of that danger.

It’s not hard to whistle up alleged social science studies of online “terrorist propaganda” but the impacts of that so-called propaganda, are speculation at best, when not actually fantasies of the authors.

“Fake News” warriors should challenge the harmful terrorist propaganda narrative as well as those that are laughably false (denying climate change for example).

The Great Scone Map…

December 6th, 2016

uk-scone-map-460

I was deeply disappointed to find the “Great Scone Map” represents differing pronunciations of “scone.”

Reading hurriedly, I thought perhaps it was a map of scone recipes. ;-)

Suggestions of maps of biscuit (a small, typically round cake of bread leavened with baking powder, baking soda, or sometimes yeast) recipes?

To avoid confusion over the term “biscuit,” ask it the “biscuit” in question is eaten by the British. If yes, then odds are it not a “biscuit” in the North American sense of the word.

There’s an a/b test for you.

Put a British “biscuit” along side a buttered Popeyes biscuit and see which one is chosen more often.

Eat several Popeyes biscuits before starting to avoid being stuck with British “biscuits.”

Clojure/conj 2016 – Videos – Sorted

December 5th, 2016

Clojure/conf 2016 has posted videos of all presentations (thanks!) to YouTube, which displays them in no particular order.

To help with my viewing and perhaps yours, here are the videos in title order:

  1. Adventures in Understanding Documents – Scott Tuddenham
  2. Audyx.com 40k locs to build the first web – based sonogram – Asher Coren
  3. Barliman: trying the halting problem backwards, blindfolded – William Byrd, Greg Rosenblatt
  4. Becoming Omniscient with Sayid – Bill Piel
  5. Building a powerful Double Entry Accounting system – Lucas Cavalcanti
  6. Building composable abstractions – Eric Normand
  7. Charting the English Language…in pure Clojure – Alexander Mann
  8. Clarifying Rules Engines with Clara Rules – Mike Rodriguez
  9. Clojure at DataStax: The Long Road From Python to Clojure – Nick Bailey
  10. A Clojure DSL for defining CI/CD orchestrations at scale – Rohit Kumar, Viraj Purang
  11. Composing music with clojure.spec – Wojciech Franke
  12. In situ model-based learning in PAMELA – Paul Robertson, Tom Marble
  13. Juggling Patterns and Programs – Steve Miner
  14. Overcoming the Challenges of Mentoring – Kim Crayton
  15. A Peek Inside SAT Solvers – Jon Smock
  16. Powderkeg: teaching Clojure to Spark – Igor Ges, Christophe Grand
  17. Production Rules on Databases – Paula Gearon
  18. Programming What Cannot Be Programmed: Aesthetics and Narrative – D. Schmüdde
  19. Proto REPL, a New Clojure Development and Visualization Tool – Jason Gilman
  20. Simplifying ETL with Clojure and Datomic – Stuart Halloway
  21. Spec-ulation Keynote – Rich Hickey
  22. Spectrum, a library for statically "typing" clojure.spec – Allen Rohner
  23. Using Clojure with C APIs for crypto and more – lvh
  24. WormBase database migration to Datomic on AWS: A case Study – Adam Wright

Enjoy!

Resisting EU Censorship

December 5th, 2016

US tech giants like Facebook could face new EU laws forcing them to tackle hate speech by Arjun Kharpal.

From the post:

U.S. technology giants including Facebook, Twitter, Microsoft, and Google’s YouTube could face new laws forcing them to deal with online hate speech if they don’t tackle the problem themselves, the European Commission warned.

In May, the four U.S. firms unveiled a “code of conduct” drawn up in conjunction with the Commission, the European Union’s executive arm, to take on hate speech on their platforms. It involved a series of commitments including a pledge to review the majority of notifications of suspected illegal hate speech in less than 24 hours and remove or disable access to the content if necessary. Another promise was to provide regular training to staff around hate speech.

But six months on, the Commission is not happy with the progress. EU Justice Commissioner Vera Jourova has commissioned a report, set to be released later this week, which claims that progress in removing offending material has been too slow.

I posted about this ill-fated “code of conduct” under Four Horsemen Of Internet Censorship + One. I pointed out the only robust solution to the “hate speech” problem was to enable users to filter the content they see, as opposed to the EU.

Fast forward 2 internet years (3 months = 1 internet year) and the EU is seeking to increase its censorship powers and not to empower users to regulate the content they consume.

Adding injury to insult, the EU proposes directives that require uncompensated expenditures on the part of its victims, Facebook, Twitter, Microsoft, and Google, to meet criteria that can only be specified user by user.

Why the first refuge of the EU for disagreeable speech is censorship I don’t know. What I do know is any tolerance of EU censorship demands encourages even more outrageous censorship demands.

The usual suspects should push back and push back hard against EU demands for censorship.

Enabling users to filter content means users can shape incoming streams to fit their personal sensitivities and dislikes, without impinging on the rights of others.

Had Facebook, Twitter, Microsoft, and Google started developing shareable content filters when they proposed their foolish “code of conduct” to the EU last May, they would either be available or nearly so by today.

Social media providers should not waste any further time attempting to censor on behalf of the EU or users. Enable users to censor their own content and get out of the censorship business.

There’s no profit in the censorship business. In fact, there is only expense and wasted effort.

PS: The “EU report” in question won’t be released until Wednesday, December 7, 2016 (or so I am told).

VA State Police Paid $585K+ For Cell Site Simulator – Your Price?

December 4th, 2016

Virginia State Police releases cellphone surveillance logs Since May 2015, the VSP have used their DRTbox unit 12 times – 5 of which appeared ineffective by Curtis Waltman.

From the post:

As part of a nationwide FOIA census for cell site simulator surveillance devices, the Virginia State Police responded with new documents detailing their acquisition and use of the DRT 1183C. Made by Digital Receiver Technology of Maryland, the DRT 1183C is a device that is commonly referred to as a DRTbox. It is very similar to other cell site simulators like the Harris Corporation’s Stingray, except that DRTboxes can also intercept voice communication as well as GPS location and other metadata.

Astonishingly unredacted, these documents detail their 2014 purchase, which upgraded their obsolete DRTbox model to the smaller and more powerful 1183C. This cost the VSP $585,265, and came complete with a whole bunch of accessories, including a Chevrolet Suburban outfitted specifically to run the device.

Two questions:

  1. What features are offered by your home-brew cell site simulator?
  2. Estimated price (parts + labor)?

When law enforcement, judges, legislators, etc., join ordinary citizens in the smartphone gold fish bowl, effective privacy will be a goal of vendors.

Data Leakage, Data Breaches and Explosives

December 4th, 2016

Data breach exposed locations of oil-industry explosives, handler credentials by Dell Cameron.

From the post:

A misconfigured storage device discovered by a security researcher in October left exposed thousands of internal files belonging to an explosives-handling company. [The original story is dated 01/12/2016, so this post is almost a year out of date.]

The files, which have since been secured, reportedly included details about facilities in three U.S. states where explosives are stored.

The leaky file repository belonged to Allied-Horizontal Wireline Services (AHWS), a leading wireline company with more than 400 employees and 70 wireline units throughout the United States. (“Wireline” is an industry term that refers to cabling technology used at oil and gas wells.) The company is licensed by the federal government to store and use explosives to complete an oil-drilling process known as “perforation.”

Chris Vickery, a lead security researcher at MacKeeper who notably discovered several misconfigured voter databases this year, found the breach in early October. After verifying the device’s owner, Vickery reached out to an AHWS executive, who quickly moved to secure the company’s data.

Data breach stories are so common that out-dated ones are especially of little interest.

Except, that this date breach story illustrates the problem of data leakage.

From the tone of the post, you are thinking evil-doers need to find companies that use explosives, hack their systems, blackmail their staffs, etc. All of which is serious stuff, not to mention a lot of effort.

But an unknown ATF official leaks the information that makes all that work unnecessary:


There are no federal laws prohibiting Allied-Horizontal Wireline Services from disclosing the location of its explosives, an ATF official said. “Licensees storing explosive materials must notify the authority having jurisdiction for fire safety in the locality where the explosive materials are stored.”

Where are explosives are stored? (United States only) Check with your local fire safety authority.

Thanks ATF!

Pence, Stephanopoulos and False Statements

December 4th, 2016

‘This Week’ Transcript: Vice President-Elect Mike Pence and Gen. David Petraeus, covers President-elect Donald Trump’s tweet:

In addition to winning the electoral college in a landslide, I won the popular vote if you deduct the millions of people who voted illegally.

That portion of the transcript reads as follows (apologies for the long quote but I think you will agree its all relevant):


STEPHANOPOULOS: As I said, President-Elect Trump has been quite active on Twitter, including this week at the beginning of this week, that tweet which I want to show right now, about the popular vote.

And he said, “In addition to winning the electoral college in a landslide, I won the popular vote if you deduct the millions of people who voted illegally.”

That claim is groundless. There’s no evidence to back it up.

Is it responsible for a president-elect to make false statements like that?

PENCE: Well, look, I think four years ago the Pew Research Center found that there were millions of inaccurate voter registrations.

STEPHANOPOULOS: Yes, but the author of this said he — he has said it is not any evidence about what happened in this election or any evidence of voter fraud.

PENCE: I think what, you know, what is — what is historic here is that our president-elect won 30 to 50 states, he won more counties than any candidate on our side since Ronald Reagan.

And the fact that some partisans, who are frustrated with the outcome of the election and disappointed with the outcome of the election, are pointing to the popular vote, I can assure you, if this had been about the popular vote, Donald Trump and I have been campaigning a whole lot more in Illinois and California and New York.

STEPHANOPOULOS: And no one is questioning your victory, certainly I’m not questioning your victory. I’m asking just about that tweet, which I want to say that he said he would have won the popular vote if you deduct the millions of people who voted illegally. That statement is false. Why is it responsible to make it?

PENCE: Well, I think the president-elect wants to call to attention the fact that there has been evidence over many years of…

STEPHANOPOULOS: That’s not what he said.

PENCE: …voter fraud. And expressing that reality Pew Research Center found evidence of that four years ago.

STEPHANPOULOS: That’s not the evidence…

PENCE: …that certainly his right.

But, you know…

STEPHANOPOULOS: It’s his right to make false statements?

PENCE: Well, it’s his right to express his opinion as president-elect of the United States.

I think one of the things that’s refreshing about our president-elect and one of the reasons why I think he made such an incredible connection with people all across this country is because he tells you what’s on his mind.

STEPHANOPOULOS: But why is it refreshing to make false statements?

PENCE: Look, I don’t know that that is a false statement, George, and neither do you. The simple fact is that…

STEPHANOPOULOS: I know there’s no evidence for it.

PENCE: There is evidence, historic evidence from the Pew Research Center of voter fraud that’s taken place. We’re in the process of investigating irregularities in the state of Indiana that were leading up to this election. The fact that voter fraud exists is…

STEPHANPOULOS: But can you provide any evidence — can you provide any evidence to back up that statement?

PENCE; Well, look, I think he’s expressed his opinion on that. And he’s entitled to express his opinion on that. And I think the American people — I think the American people find it very refreshing that they have a president who will tell them what’s on his mind. And I think the connection that he made in the course…

STEPHANOPOULOS: Whether it’s true or not?

PENCE: Well, they’re going to tell them — he’s going to say what he believes to be true and I know that he’s always going to speak in that way as president.
….

Just to be clear, I agree with Stepanopoulos and others who say there is no evidence of millions of illegal votes being cast in the 2016 presidential election.

After reading Stephanopoulos press Pence on this false statement by President-elect Trump, can you recall Stepanopoulos or another other major reporter pressing President Obama on his statements about terrorism, such as:


Tonight I want to talk with you about this tragedy, the broader threat of terrorism and how we can keep our country safe. The FBI is still gathering the facts about what happened in San Bernardino, but here’s what we know. The victims were brutally murdered and injured by one of their co-workers and his wife. So far, we have no evidence that the killers were directed by a terrorist organization overseas or that they were part of a broader conspiracy here at home. But it is clear that the two of them had gone down the dark path of radicalization, embracing a perverted interpretation of Islam that calls for war against America and the West. They had stockpiled assault weapons, ammunition, and pipe bombs.

So this was an act of terrorism designed to kill innocent people. Our nation has been at war with terrorists since Al Qaeda killed nearly 3,000 Americans on 9/11. In the process, we’ve hardened our defenses, from airports, to financial centers, to other critical infrastructure. Intelligence and law enforcement agencies have disrupted countless plots here and overseas and worked around the clock to keep us safe.

Our military and counterterrorism professionals have relentlessly pursued terrorist networks overseas, disrupting safe havens in several different countries, killing Osama Bin Laden, and decimating Al Qaeda’s leadership.

Over the last few years, however, the terrorist threat has evolved into a new phase. As we’ve become better at preventing complex multifaceted attacks like 9/11, terrorists turn to less complicated acts of violence like the mass shootings that are all too common in our society. It is this type of attack that we saw at Fort Hood in 2009, in Chattanooga earlier this year, and now in San Bernardino.

And as groups like ISIL grew stronger amidst the chaos of war in Iraq and then Syria, and as the Internet erases the distance between countries, we see growing efforts by terrorists to poison the minds of people like the Boston Marathon bombers and the San Bernardino killers.

For seven years, I’ve confronted this evolving threat each and every morning in my intelligence briefing, and since the day I took this office, I have authorized U.S. forces to take out terrorists abroad precisely because I know how real the danger is.
Here’s what Obama said in his Sunday night address: An annotated transcript

Really? “…because I know how real the danger is.

Do you recall anyone pressing President Obama on his claims about the danger of terrorism?

If you ever get to pose such a question to President Obama, remind him that 685 American die every day from medial errors, 44,0000 Americans die every 6 months due to excessive alcohol consumption, and that 430 Americans died between 2000 and 2013 due to falling furniture.

Can you think of a single instance when Obama’s flights of fancy about terrorism were challenged as Stephanopoulos did Trump’s delusion about illegal voters?

The media can and should challenge such flights of fancy.

At the same time, they should challenge those favored by other politicians, their editors, fellow journalists and advertisers.

PS: The medical error article: Medical error—the third leading cause of death in the US, BMJ 2016; 353 doi: http://dx.doi.org/10.1136/bmj.i2139 (Published 03 May 2016) Cite this as: BMJ 2016;353:i2139 (The Guardian article, my source, didn’t include a link to the original article.)

Identifying Speech/News Writers

December 2nd, 2016

David Smith’s post: Stylometry: Identifying authors of texts using R details the use of R to distinguish tweets by president-elect Donald Trump from his campaign staff. (Hmmm, sharing a Twitter account password, there’s bad security for you.)

The same techniques may distinguish texts delivered “live” versus those “inserted” into Congressional Record.

What other texts are ripe for distinguishing authors?

From the post:

Few people expect politicians to write every word they utter themselves; reliance on speechwriters and spokepersons is a long-established political practice. Still, it's interesting to know which statements are truly the politician's own words, and which are driven primarily by advisors or influencers.

Recently, David Robinson established a way of figuring out which tweets from Donald Trump's Twitter account came from him personally, as opposed to from campaign staff, whcih he verified by comparing the sentiment of tweets from Android vs iPhone devices. Now, Ali Arsalan Kazmi has used stylometric analysis to investigate the provenance of speeches by the Prime Minister of Pakistan

A small amount of transparency can go a long way.

Email archives anyone?

War and Peace & R

December 2nd, 2016

No, not a post about R versus Python but about R and Tolstoy‘s War and Peace.

Using R to Gain Insights into the Emotional Journeys in War and Peace by Wee Hyong Tok.

From the post:

How do you read a novel in record time, and gain insights into the emotional journey of main characters, as they go through various trials and tribulations, as an exciting story unfolds from chapter to chapter?

I remembered my experiences when I start reading a novel, and I get intrigued by the story, and simply cannot wait to get to the last chapter. I also recall many conversations with friends on some of the interesting novels that I have read awhile back, and somehow have only vague recollection of what happened in a specific chapter. In this post, I’ll work through how we can use R to analyze the English translation of War and Peace.

War and Peace is a novel by Leo Tolstoy, and captures the salient points about Russian history from the period 1805 to 1812. The novel consists of the stories of five families, and captures the trials and tribulations of various characters (e.g. Natasha and Andre). The novel consists of about 1400 pages, and is one of the longest novels that have been written.

We hypothesize that if we can build a dashboard (shown below), this will allow us to gain insights into the emotional journey undertaken by the characters in War and Peace.

Impressive work, even though I would not use it as a short-cut to “read a novel in record time.”

Rather I take this as an alternative way of reading War and Peace, one that can capture insights a casual reader may miss.

Moreover, the techniques demonstrated here could be used with other works of literature, or even non-fictional works.

Imagine conducting this analysis over the reportedly more than 7,000 page full CIA Torture Report, for example.

A heatmap does not connect any dots, but points a user towards places where interesting dots may be found.

Certainly a tool for exploring large releases/leaks of text data.

Enjoy!

PS: Large, tiresome, obscure-on-purpose, government reports to practice on with this method?

OSS-Fuzz: Continuous fuzzing for open source software

December 1st, 2016

Announcing OSS-Fuzz: Continuous fuzzing for open source software

From the post:

We are happy to announce OSS-Fuzz, a new Beta program developed over the past years with the Core Infrastructure Initiative community. This program will provide continuous fuzzing for select core open source software.

Open source software is the backbone of the many apps, sites, services, and networked things that make up “the internet.” It is important that the open source foundation be stable, secure, and reliable, as cracks and weaknesses impact all who build on it.

Recent security stories confirm that errors like buffer overflow and use-after-free can have serious, widespread consequences when they occur in critical open source software. These errors are not only serious, but notoriously difficult to find via routine code audits, even for experienced developers. That’s where fuzz testing comes in. By generating random inputs to a given program, fuzzing triggers and helps uncover errors quickly and thoroughly.

In recent years, several efficient general purpose fuzzing engines have been implemented (e.g. AFL and libFuzzer), and we use them to fuzz various components of the Chrome browser. These fuzzers, when combined with Sanitizers, can help find security vulnerabilities (e.g. buffer overflows, use-after-free, bad casts, integer overflows, etc), stability bugs (e.g. null dereferences, memory leaks, out-of-memory, assertion failures, etc) and sometimes even logical bugs.

OSS-Fuzz’s goal is to make common software infrastructure more secure and stable by combining modern fuzzing techniques with scalable distributed execution. OSS-Fuzz combines various fuzzing engines (initially, libFuzzer) with Sanitizers (initially, AddressSanitizer) and provides a massive distributed execution environment powered by ClusterFuzz.
… (emphasis in original)

Another similarity between open and closed source software.

Closed source software is continuously being fuzzed.

By volunteers.

Yes? ;-)

One starting place for more information: Effective file format fuzzing by Mateusz “j00ru” Jurczyk (Black Hat Europe 2016, London) and his website: http://j00ru.vexillium.org/.

If You Don’t Get A Quantum Computer For Christmas

December 1st, 2016

Learn Quantum Mechanics with Haskell by Scott N. Walck.

Abstract:

To learn quantum mechanics, one must become adept in the use of various mathematical structures that make up the theory; one must also become familiar with some basic laboratory experiments that the theory is designed to explain. The laboratory ideas are naturally expressed in one language, and the theoretical ideas in another. We present a method for learning quantum mechanics that begins with a laboratory language for the description and simulation of simple but essential laboratory experiments, so that students can gain some intuition about the phenomena that a theory of quantum mechanics needs to explain. Then, in parallel with the introduction of the mathematical framework on which quantum mechanics is based, we introduce a calculational language for describing important mathematical objects and operations, allowing students to do calculations in quantum mechanics, including calculations that cannot be done by hand. Finally, we ask students to use the calculational language to implement a simplified version of the laboratory language, bringing together the theoretical and laboratory ideas.

You won’t find a quantum computer under your Christmas tree this year.

But Haskell + Walck will teach you the basics of quantum mechanics.

You may also want to read:

Structure and Interpretation of Quantum Mechanics – a Functional Framework (2003) by Jerzy Karczmarczuk.

You will have to search for it but “Gerald Jay Sussman & Jack Wisdom (2013): Functional Differential Geometry. The MIT Press.” is out on the net somewhere.

Very tough sledding but this snippet from the preface may tempt you into buying a copy:


But the single biggest difference between our treatment and others is that we integrate computer programming into our explanations. By programming a computer to interpret our formulas we soon learn whether or not a formula is correct. If a formula is not clear, it will not be interpretable. If it is wrong, we will get a wrong answer. In either case we are led to improve our program and as a result improve our understanding. We have been teaching advanced classical mechanics at MIT for many years using this strategy. We use precise functional notation and we have students program in a functional language. The students enjoy this approach and we have learned a lot ourselves. It is the experience of writing software for expressing the mathematical content and the insights that we gain from doing it that we feel is revolutionary. We want others to have a similar experience.

If that interests you, check out courses by Sussman at MITOpenCourseware.

Enjoy!

Recycling Old News – NPR Station WMOT

December 1st, 2016

Avoiding “fake” news, NPR station WMOT is recycling “old news.”

Seriously.

Looking for a recent article on combining multiple sources of DNA I found:

Combining The DNA Of Three People Raises Ethical Questions by Rob Stein, Nov. 10, 2014.

combining-dna-460

In a darkened lab in the north of England, a research associate is intensely focused on the microscope in front of her. She carefully maneuvers a long glass tube that she uses to manipulate early human embryos.

“It’s like microsurgery,” says Laura Irving of Newcastle University.

Irving is part of a team of scientists trying to replace defective DNA with healthy DNA. They hope this procedure could one day help women who are carrying genetic disorders have healthy children.

Compare that post to:

Combining The DNA Of Three People Raises Ethical Questions by Rob Stein, 22 hours ago.

combining-dna-460

In a darkened lab in the north of England, a research associate is intensely focused on the microscope in front of her. She carefully maneuvers a long glass tube that she uses to manipulate early human embryos.

“It’s like microsurgery,” says Laura Irving of Newcastle University.

Irving is part of a team of scientists trying to replace defective DNA with healthy DNA. They hope this procedure could one day help women who are carrying genetic disorders have healthy children.

I took a screen shot that includes WMOT and the article title, plus saved the page, just in case through the magic of silent correction, this example of “news” reporting goes away.

At least to me, two year old news isn’t the same as news 22 hours ago.

You?

PS: The loss of credibility by the media has been entirely self-inflicted. See media coverage of the 2016 presidential race for example. Why would anyone trust a news source that was so badly wrong?

Hard work, good journalism, timely reporting, all of those are the elements needed for the media to regain credibility. Credible journalists don’t attempt to suppress “fake news.” Attempts to suppress “fake news” signal a lack of commitment to credible journalism. Credible journalism doesn’t notice “fake news.”

Internet Censor(s) Spotted in Mirror

November 30th, 2016

How to solve Facebook’s fake news problem: experts pitch their ideas by Nicky Woolf.

From the post:

The impact of fake news, propaganda and misinformation has been widely scrutinized since the US election. Fake news actually outperformed real news on Facebook during the final weeks of the election campaign, according to an analysis by Buzzfeed, and even outgoing president Barack Obama has expressed his concerns.

But a growing cadre of technologists, academics and media experts are now beginning the quixotic process of trying to think up solutions to the problem, starting with a rambling 100+ page open Google document set up by Upworthy founder Eli Pariser.

Woolf captures the essential wrongness with the now, 120 pages, of suggestions, quoting Claire Wardle:


“The biggest challenge is who wants to be the arbiter of truth and what truth is,” said Claire Wardle, research director for the Tow Center for Digital Journalism at Columbia University. “The way that people receive information now is increasingly via social networks, so any solution that anybody comes up with, the social networks have to be on board.”

Don’t worry, selecting the arbiter of truth and what truth is won’t be difficult.

The authors of these suggestions see their favorite candidate every day:

mirror-460

So long as they aren’t seeing my image (substitute your name/image) in the mirror, I’m not interested in any censorship proposal.

Personally, even if offered the post of Internet Censor, I would turn it down.

I can’t speak for you but I am unable to be equally impartial to all. Nor do I trust anyone else to be equally impartial.

The “solution” to “fake news,” if you think that is a meaningful term, is more news, not less.

Enable users to easily compare and contrast news sources, if they so choose. Freedom means being free to make mistakes as well as good choices (from some point of view).

Constitution Free Zone [The Only Advantage To Not Living In Hawaii]

November 30th, 2016

Know Your Rights: The Government’s 100-Mile “Border” Zone – Map

From the post:

Many people think that border-related policies impact only people living in border towns like El Paso or San Diego. The reality is that Border Patrol’s interior enforcement operations encroach deep into and across the United States, affecting the majority of Americans.

Roughly two-thirds of the United States’ population, about 200 million people, lives within the 100-mile zone that an outdated federal regulation defines as the border zone—that is, within 100 miles of a U.S. land or coastal border.

Although this zone is not literally “Constitution free”—constitutional protections do still apply—the Border Patrol frequently ignores those protections and runs roughshod over individuals’ civil liberties.

Learn more about the government’s 100-mile border zone.

Read the ACLU factsheet on Custom and Border Protection’s 100-mile zone

constitutionfreezonemap-460

The ACLU map demonstrates there are no locations in Hawaii where the border zone does not reach.

Now you can name the one advantage of living outside of Hawaii, just in case it comes up on Jeopardy.

;-)

In some ways, this map is mis-leading.

The U.S. government runs roughshod over everyone within and without its borders.

Ask the people of Aleppo for tales of the American government. A city rumored to be founded in the 6th millennium BCE, may be about to become the largest graveyard in history.

Be sure to mention that on holiday cards to the Obama White House.

Urgent: Update Your Tor Browser [Today, Yes, Today] + Aside on shallow bugs

November 30th, 2016

Tor Browser 6.0.7 is released

From the webpage:

Tor Browser 6.0.7 is now available from the Tor Browser Project page and also from our distribution directory.

This release features an important security update to Firefox and contains, in addition to that, an update to NoScript (2.9.5.2).

The security flaw responsible for this urgent release is already actively exploited on Windows systems. Even though there is currently, to the best of our knowledge, no similar exploit for OS X or Linux users available the underlying bug affects those platforms as well. Thus we strongly recommend that all users apply the update to their Tor Browser immediately. A restart is required for it to take effect.

Tor Browser users who had set their security slider to “High” are believed to have been safe from this vulnerability.

We will have alpha and hardened Tor Browser updates out shortly. In the meantime, users of these series can mitigate the security flaw in at least two ways:

1) Set the security slider to “High” as this is preventing the exploit from working.
2) Switch to the stable series until updates for alpha and hardened are available, too.

Here is the full changelog since 6.0.6:

  • All Platforms
    • Update Firefox to 45.5.1esr
    • Update NoScript to 2.9.5.2

A reminder from the Tor project that:

many eyes make all bugs shallow

is marketing talk for open source, nothing more.

For more on that theme: Linus’s Law aka “Many Eyes Make All Bugs Shallow” by Jeff Jones.

A little over 10 years old now, predating HeartBleed for example, but still an interesting read.

I am and remain an open source advocate but not on the basis of false claims of bug finding. Open source improves your changes of finding spyware. No guarantees but open source improves your chances.

Why any government or enterprise would run closed source software is a mystery to me. Upload all your work to the NSA on a weekly basis. With uploads you create a reminder of your risk, which is missing with non-open source software.

Hacking Journalists (Of self-protection)

November 30th, 2016

Inside the mind of digital attackers: Part 1 — The connection by Justin Kosslyn.

From the post:

John has a target: name, country, brief context, and maybe the email address or website. John has been given a goal: maybe eavesdropping, taking a website offline, or stealing intellectual property. And John has been given constraints: maybe he cannot risk detection, or he has to act within 24 hours, or he cannot reach out to the state-owned telecommunications company for help.

John is a government-backed digital attacker. He sits in an office building somewhere, at a desk. Maybe this is the job he wanted when he was growing up, or maybe it was a way to pay the bills and stretch his technical muscles. He probably has plans for the weekend.

Let’s say, for the sake of this example, that John’s target is Henry, in the same country as John. John’s goal is to copy all the information on Henry’s computer without being detected. John can get help from other government agencies. There’s no rush.

The first thing to realize is that John, like most people, is a busy guy. He’s not going to do more work than necessary. First, he’ll try to use traditional, straightforward techniques — nothing fancy — and only if those methods fail will he try to be more creative with his attack.

The start of an interesting series from Jigsaw:

A technology incubator at Alphabet that tackles geopolitical problems.

Justin proposes to take us inside the mind of hackers who target journalists.

Understanding the enemy and their likely strategies is a starting place for effective defense/protection.

My only caveat is the description of John as a …government-backed digital attacker….

Could be and increases John’s range of tools but don’t premise any defense on attackers being government-backed.

There are only two types of people in the world:

  1. People who are attacking your system.
  2. People have not yet attacked your system.

Any sane and useful security policy accounts for both.

I’m looking forward to the next installment in this series.

1 Million Compromised Google Accounts – 86 Goolian Infected Apps – In Sort Order

November 30th, 2016

“Gooligan” Android Malware Compromised 1 Million Google Accounts by Bogdan Popa.

From the post:

Security experts at Check Point have discovered a new very aggressive form of Android malware that already compromised no less than 1 million Google accounts and which can infect approximately 74 percent of the Android phones currently on the market.

The firm warns that the malware which they call Gooligan is injected into a total of 86 Android apps that are delivered through third-party marketplaces (you can check the full list of apps in the box at the end of the article). Once installed, these apps root the phone to get full access to the device and then attempt to deploy malicious software which can be used to steal authentication tokens for Google accounts.

This pretty much gives the attackers full control over the targeted Google accounts, and as long as vulnerable phones have Gmail, Google Drive, Google Chrome, YouTube, Google Photos, or any other Google app that can be used with an account, there’s a big chance that the attack is successful.
…(emphasis in original)

You can check to see if your account has been breached: Gooligan Checker.

The article also lists 86 Goolian infected apps, in no particular order. (Rhetorical questions: Why do people make it difficult for readers? What is their payoff?)

To save you from digging through and possibly missing an infected app, here are the 86 Googlian infected apps in dictionary order:

  • แข่งรถสุดโหด
  • Assistive Touch
  • ballSmove_004
  • Battery Monitor
  • Beautiful Alarm
  • Best Wallpapers
  • Billiards
  • Blue Point
  • CakeSweety
  • Calculator
  • Chrono Marker
  • Clean Master
  • Clear
  • com.browser.provider
  • com.example.ddeo
  • com.fabullacop.loudcallernameringtone
  • Compass Lite
  • com.so.itouch
  • Daily Racing
  • Demm
  • Demo
  • Demoad
  • Detecting instrument
  • Dircet Browser
  • Fast Cleaner
  • Fingerprint unlock
  • Flashlight Free
  • Fruit Slots
  • FUNNY DROPS
  • gla.pev.zvh
  • Google
  • GPS
  • GPS Speed
  • Hip Good
  • HotH5Games
  • Hot Photo
  • Html5 Games
  • Kiss Browser
  • KXService
  • Light Advanced
  • Light Browser
  • memory booste
  • memory booster
  • Memory Booster
  • Minibooster
  • Multifunction Flashlight
  • Music Cloud
  • OneKeyLock
  • Pedometer
  • Perfect Cleaner
  • phone booster
  • PornClub
  • PronClub
  • Puzzle Bubble-Pet Paradise
  • QPlay
  • SettingService
  • Sex Cademy
  • Sex Photo
  • Sexy hot wallpaper
  • Shadow Crush
  • Simple Calculator
  • Slots Mania
  • Small Blue Point
  • SmartFolder
  • Smart Touch
  • Snake
  • So Hot
  • StopWatch
  • Swamm Browser
  • System Booster
  • Talking Tom 3
  • TcashDemo
  • Test
  • Touch Beauty
  • tub.ajy.ics
  • UC Mini
  • Virtual
  • Weather
  • Wifi Accelerate
  • WiFi Enhancer
  • Wifi Master
  • Wifi Speed Pro
  • YouTube Downloader
  • youtubeplayer
  • 小白点
  • 清理大师

Visualizing XML Schemas

November 29th, 2016

I don’t have one of the commercial XML packages at the moment and was casting about for a free visualization technique for a large XML schema when I encountered:

schema-visualization-460

I won’t be trying it on my schema until tomorrow but I thought it looked interesting enough to pass along.

Further details: Visualizing Complex Content Models with Spatial Schemas by Joe Pairman.

This looks almost teachable.

Thoughts?

Other “free” visualization tools to suggest?

Gab – Censorship Lite?

November 29th, 2016

I submitted my email today at Gab and got this message:

Done! You’re #1320420 in the waiting list.

Only three rules:

Illegal Pornography

We have a zero tolerance policy against illegal pornography. Such material will be instantly removed and the owning account will be dealt with appropriately per the advice of our legal counsel. We reserve the right to ban accounts that share such material. We may also report the user to local law enforcement per the advice our legal counsel.

Threats and Terrorism

We have a zero tolerance policy for violence and terrorism. Users are not allowed to make threats of, or promote, violence of any kind or promote terrorist organizations or agendas. Such users will be instantly removed and the owning account will be dealt with appropriately per the advice of our legal counsel. We may also report the user to local and/or federal law enforcement per the advice of our legal counsel.

What defines a ‘terrorist organization or agenda’? Any group that is labelled as a terrorist organization by the United Nations and/or United States of America classifies as a terrorist organization on Gab.

Private Information

Users are not allowed to post other’s confidential information, including but not limited to, credit card numbers, street numbers, SSNs, without their expressed authorization.

If Gab is listening, I can get the rules down to one:

Court Ordered Removal

When Gab receives a court order from a court of competent jurisdiction ordering the removal of identified, posted content, at (service address), the posted, identified content will be removed.

Simple, fair, gets Gab and its staff out of the censorship business and provides a transparent remedy.

At no cost to Gab!

What’s there not to like?

Gab should review my posts: Monetizing Hate Speech and False News and Preserving Ad Revenue With Filtering (Hate As Renewal Resource), while it is in closed beta.

Twitter and Facebook can keep spending uncompensated time and effort trying to be universal and fair censors. Gab has the opportunity to reach up and grab those $100 bills flying overhead for filtered news services.

What is the New York Times if not an opinionated and poorly run filter on all the possible information it could report?

Apply that same lesson to social media!

PS: Seriously, before going public, I would go to the one court-based rule on content. There’s no profit and no wins in censoring any content on your own. Someone will always want more or less. Courts get paid to make those decisions.

Check with your lawyers but if you don’t look at any content, you can’t be charged with constructive notice of it. Unless and until someone points it out, then you have to follow DCMA, court orders, etc.

Spies in the Skies [Fostered by Obama, Inherited by Trump]

November 29th, 2016

Spies in the Skies by Peter Aldhous and Charles Seife.

Post in April of 2016, it reads in part:

Each weekday, dozens of U.S. government aircraft take to the skies and slowly circle over American cities. Piloted by agents of the FBI and the Department of Homeland Security (DHS), the planes are fitted with high-resolution video cameras, often working with “augmented reality” software that can superimpose onto the video images everything from street and business names to the owners of individual homes. At least a few planes have carried devices that can track the cell phones of people below. Most of the aircraft are small, flying a mile or so above ground, and many use exhaust mufflers to mute their engines — making them hard to detect by the people they’re spying on.

The government’s airborne surveillance has received little public scrutiny — until now. BuzzFeed News has assembled an unprecedented picture of the operation’s scale and sweep by analyzing aircraft location data collected by the flight-tracking website Flightradar24 from mid-August to the end of December last year, identifying about 200 federal aircraft. Day after day, dozens of these planes circled above cities across the nation.

The FBI and the DHS would not discuss the reasons for individual flights but told BuzzFeed News that their planes are not conducting mass surveillance.

The DHS said that its aircraft were involved with securing the nation’s borders, as well as targeting drug smuggling and human trafficking, and may also be used to support investigations by the FBI and other law enforcement agencies. The FBI said that its planes are only used to target suspects in specific investigations of serious crimes, pointing to a statement issued in June 2015, after reporters and lawmakers started asking questions about FBI surveillance flights.

“It should come as no surprise that the FBI uses planes to follow terrorists, spies, and serious criminals,” said FBI Deputy Director Mark Giuliano, in that statement. “We have an obligation to follow those people who want to hurt our country and its citizens, and we will continue to do so.”

I’m not surprised the FBI follows terrorists, spies, and serious criminals.

What’s problematic is that the FBI follows all of us and then, after the fact, picks out alleged terrorists, spies and serious criminals.

The FBI could just as easily select people on their way to a tryst with a government official’s wife, or to attend an AA meeting, or to attend an unpopular church.

Once collected, the resulting information is subject to any number of uses and abuses.

Aldhous and Seife report the flights drop 70% on the weekend so if you are up to mischief, plan around your weekends.

When writing about the inevitable surveillance excesses under President Trump, give credit to President Obama and his supporters, who built the surveillance state Trump inherited.

Trump, Twitter and Bullying The Press

November 29th, 2016

Jay Smooth tweeted yesterday:

Keep in mind the purpose of this clown show: the President-Elect of the United States is using twitter to single out & bully a journalist.

Attaching an image that contained tweets 5 through 8 from the following list:

  1. “Nobody should be allowed to burn the American flag – if they do, there must be consequences – perhaps loss of citizenship or year in jail!”
  2. “I thought that @CNN would get better after they failed so badly in their support of Hillary Clinton however, since election, they are worse!”
  3. “The Great State of Michigan was just certified as a Trump WIN giving all of our MAKE AMERICA GREAT AGAIN supporters another victory – 306!”
  4. “@CNN is so embarrassed by their total (100%) support of Hillary Clinton, and yet her loss in a landslide, that they don’t know what to do.”
  5. “@sdcritic: @HighonHillcrest @jeffzeleny @CNN There is NO QUESTION THAT #voterfraud did take place, and in favor of #CorruptHillary !”
  6. “@FiIibuster: @jeffzeleny Pathetic – you have no sufficient evidence that Donald Trump did not suffer from voter fraud, shame! Bad reporter.”
  7. ‘”@JoeBowman12: @jeffzeleny just another generic CNN part time wannabe journalist !” @CNN still doesn’t get it. They will never learn!’
  8. “@HighonHillcrest: @jeffzeleny what PROOF do u have DonaldTrump did not suffer from millions of FRAUD votes? Journalist? Do your job! @CNN”
  9. “Just met with General Petraeus–was very impressed!”
  10. “If Cuba is unwilling to make a better deal for the Cuban people, the Cuban/American people and the U.S. as a whole, I will terminate deal.”

Can Trump bully @jeffzeleny if Jeff and the press aren’t listening?

Jeff filters @realDonaldTrump excluding any tweets with @jeffzeleny and subscribes to a similar filter for all journalists twitter handles.

His feed from @realDonaldTrump now reads:

  1. “Nobody should be allowed to burn the American flag – if they do, there must be consequences – perhaps loss of citizenship or year in jail!”
  2. “I thought that @CNN would get better after they failed so badly in their support of Hillary Clinton however, since election, they are worse!”
  3. “The Great State of Michigan was just certified as a Trump WIN giving all of our MAKE AMERICA GREAT AGAIN supporters another victory – 306!”
  4. “@CNN is so embarrassed by their total (100%) support of Hillary Clinton, and yet her loss in a landslide, that they don’t know what to do.”
  5. “Just met with General Petraeus–was very impressed!”
  6. “If Cuba is unwilling to make a better deal for the Cuban people, the Cuban/American people and the U.S. as a whole, I will terminate deal.”

Trump’s tweets still contain enough material for a stand up routine by a comic or the front page of a news paper.

On the other hand, shareable user filters starve Trump (and other bullies) of the ability to be bullies.

Why isn’t Twitter doing something as dead simple as user filters than can be shared?

You would have to ask Twitter that question, I certainly don’t know.