Another Word For It Patrick Durusau on Topic Maps and Semantic Diversity

December 10, 2016

Google Helps Spread Fake News [Fake News & Ad Revenue – Testing]

Filed under: Journalism,News,Reporting,Searching — Patrick Durusau @ 1:31 pm

Google changed its search algorithm and that made it more vulnerable to the spread of fake news by Hannah Roberts.

From the post:

Google’s search algorithm has been changed over the last year to increasingly reward search results based on how likely you are to click on them, multiple sources tell Business Insider.

As a result, fake news now often outranks accurate reports on higher quality websites.

The problem is so acute that Google’s autocomplete suggestions now actually predict that you are searching for fake news even when you might not be, as Business Insider noted on December 5.
…

Hannah does a great job of setting for the evidence and opinions on the algorithm change but best summarizes it when she says:

…
The changes to the algorithm now move links up Google’s search results page if Google detects that more people are clicking on them, search experts tell Business Insider.
…

Just in case you don’t know:

more clicks != credible/useful search results

But it is true:

more clicks = more usage/ad revenue

Google and Facebook find “fake news” profitable. Both will make a great show of suppressing outlying “fake news,” but not so much as to impact profits.

There’s a data science “fake news” project:

Track the suppression of “fake news” by Google and Facebook against the performance of their ad revenue.

Hypotheses: When suppression of “fake news” impinges on ad revenue for more than two consecutive hours, dial back on suppression mechanisms. (ditto for 4, 6, 12 and 24 hour cycles)

Odds on Google and Facebook being transparent regard to suppression of “fake news” and ad revenue to make the results of testing that hypotheses verifiable?

😉

The Koch Brothers are Attacking Libraries – FYI – Funding Appeal

Filed under: Funding,Library,Politics — Patrick Durusau @ 11:09 am

EveryLibrary has a funding appeal you need to seriously consider.

The Koch Brothers are Attacking Libraries

From the post:

We are continuing to see the Koch Brothers Super PAC, Americans for Prosperity go after libraries. This last election cycle was the fifth clear example of their involvement in the agenda to defund libraries. We need your help to fight back. When the Koch Brothers and AFP puts out an anti-tax and anti-library attack, they do it with direct mail and robocalls – and they always do it late in the campaign. We need the resources to confront these anti-tax forces before they can start in the next town. Help us stop them with a one time donation today or a $5-10 monthly donation.
… (emphasis in original)

I won’t repeat the crimes committed against libraries by the Koch Brothers and their Super PAC, Americans for Prosperity, here, they are too sickening. The EveryLibrary post has a sub-set of their offenses described.

Be sure to check out the EveryLibrary site and their journal, The Political Librarian.

From their What We Do page:

EveryLibrary is the first and only national organization dedicated to building voter support for libraries. We are chartered “to promote public, school, and college libraries, including by advocating in support of public funding for libraries and building public awareness of public funding initiatives”. Our primary work is to support local public libraries when they have a referendum or measure on the ballot. We do this in three ways: by training library staff, trustees, and volunteers to plan and run effective Information Only campaigns; by assisting local Vote Yes committees on planning and executing Get Out the Vote work for their library’s measure; and by speaking directly to the public about the value and relevance of libraries and librarians. Our focus on activating voters on Election Day is unique in the library advocacy ecosystem. This is reflected in the training and coaching we do for campaigns.
…

If you have ever fantasized about saving the Library at Alexandria or opposing the sack of Rome by the Vandals and the Visigoths, now is your chance to do more than fantasize.

Libraries are islands of knowledge under siege by the modern analogues of the barbarians that plunged the world into centuries of darkness.

Will you piss and moan on Facebook, Twitter, etc. about the crumbling defenses of libraries or will you take your place on the ramparts?

Yes?

December 9, 2016

Data Science and Protests During the Age of Trump [How To Brick A School Bus…]

Filed under: Censorship,Data Science,Government,Politics,Protests — Patrick Durusau @ 3:48 pm

Pre-inauguration suppression of free speech/protests is underway for the Trump regime. (CNN link as subject identifier for Donald J. Trump, even though it fails to mention he looks like a cheeto in a suit.)

Women’s March on Washington barred from Lincoln Memorial by Amber Jamieson and Jessica Glenza.

From the post:

…
For the thousands hoping to echo the civil rights and anti-Vietnam rallies at Lincoln Memorial by joining the women’s march on Washington the day after Donald Trump’s inauguration: time to readjust your expectations.

The Women’s March won’t be held at the Lincoln Memorial.

That’s because the National Park Service, on behalf of the Presidential Inauguration Committee, filed documents securing large swaths of the national mall and Pennsylvania Avenue, the Washington Monument and the Lincoln Memorial for the inauguration festivities. None of these spots will be open for protesters.

The NPS filed a “massive omnibus blocking permit” for many of Washington DC’s most famous political locations for days and weeks before and after the inauguration on 20 January, said Mara Verheyden-Hilliard, a constitutional rights litigator and the executive director of the Partnership for Civil Justice Fund.
…

ďżź

I contacted Amber Jamieson for more details on the permits and she forwarded two links (thanks Amber!):

Press Conference: Mass Protests Will Go Forward During Inauguration, which had the second link she forwarded:

PresidentialInauguralCommittee12052016.pdf, the permit requests made by the National Park Service on behalf of the Presidential Inaugural Committee.

Start with where protests are “permitted” to see what has been lost.

A grim read but 36 CFR 7.96 says in part:

…
3 (i) White House area. No permit may be issued authorizing demonstrations in the White House area, except for the White House sidewalk, Lafayette Park and the Ellipse. No permit may be issued authorizing special events, except for the Ellipse, and except for annual commemorative wreath-laying ceremonies relating to the statutes in Lafayette Park.
…
(emphasis added, material hosted by the Legal Information Institute (LII))

Summary: In White House area, protesters have only three places for permits to protest:

  • White House sidewalk
  • Lafayette Park
  • Ellipse

White House sidewalk / Lafayette Park (except North-East Quadrant) – Application 16-0289

Dates:

Set-up dates starting 11/1/2016 6:00 am ending 1/19/2017
Activity dates starting 1/20/2017 ending 1/20/2017
Break-down dates starting 1/21/2017 ending 3/1/2017 11:59 pm

Closes:

…
All of Lafayette Park except for its northeast quadrant pursuant to 36 CFR 7.96 (g)(4)(iii)(A). The initial areas of Lafayette Park and the White House Sidewalk that will be needed for construction set-up, and which will to be closed to ensure public safety, is detailed in the attached map. The attached map depicts the center portion of the White House Sidewalk as well as a portion of the southern oval of Lafayette Park. The other remaining areas in Lafayette Park and the White House Sidewalk that will be needed for construction set-up, will be closed as construction set-up progresses into these other areas, which will also then be delineated by fencing and sign age to ensure public safety.
…

Two of the three possible protest sites in the White House closed by Application 16-0289.

Ellipse – Application 17-0001

Dates:

Set-up dates starting 01/6/2017 6:00 am ending 1/19/2017
Activity dates starting 1/20/2017 ending 1/20/2017
Break-down dates starting 1/21/2017 ending 2/17/2017 11:59 pm

These dates are at variance with those for the White House sidewalk and Lafayette Park (shorter).

Closes:

Ellipse, a fitty-two acre park, as depicted by Google Maps:

ellipse-460

Plans for the Ellipse?

…
Purpose of Activity: In connection with the Presidential Inaugural Ceremonies, this application is for use of the Ellipse by PIC, in the event that PIC seeks its use for Inaugural ceremonies and any necessary staging, which is expected to be:

A) In the event that PIC seeks the use of the Ellipse for pre- and/or post- Inaugural ceremonies, the area will be used for staging the event(s), staging of media to cover and/or broadcast the event, and if possible for ticketed and/or public viewing; and/or ­

B) In the event that PIC seeks the use of the Ellipse for the Inaugural ceremony and Inaugural parade staging, the area will be used to stage the various parade elements, for media to cover and/or broadcast the event, and if possible for ticketed and/or public viewing.
…

The PIC has no plans to use the Ellipse but has reserved it no doubt to deny its use to others.

Those two applications close three out of three protest sites in the White House area. The PIC went even further to reach out and close off other potential protest sites.

Other permits granted to the PIC include:

Misc. Areas – Application 16-0357

Ten (10) misc. areas identified by attached maps for PIC activities.

Arguably legitimate since the camp followers, sycophants and purveyors of “false news” need somewhere to be during the festivities.

National Mall -> Trump Mall – Application 17-0002

The National Mall will become Trump Mall for the following dates:

Set-up dates starting 01/6/2017 6:00 am ending 1/19/2017
Activity dates starting 1/20/2017 ending 1/20/2017
Break-down dates starting 1/21/2017 ending 1/30/2017 11:59 pm

Closes:

…
Plan for Proposed Activity: Consistent with NPS regulations at 36 CFR 7.96{g)(4)(iii)(C), this application seeks, in connection with the Presidential Inaugural Ceremonies, the area of the National Mall between 14th – 4th Streets, for the exclusive use of the Joint Task Force Headquarters (JTFHQ) on Inaugural Day for the assembly, staging, security and weather protection of the pre-Inaugural parade components and floats on Inaugural Day between 14th – 7th Streets. It also includes the placement of jumbotrons and sound towers by the Architect of the Capitol or the Joint Congressional Committee on Inaugural Ceremonies so that the Inaugural Ceremony may be observed by the Joint Congressional Committee’s ticketed standing room ticket holders between 4th – 3rd streets and the general public, which will be located on the National Mall between 7th – 4th Streets. Further, a 150-foot by 200-foot area on the National Mall just east of 7th Street, will be for the exclusive use of the Presidential Inaugural Committee for television and radio media broadcasts on Inaugural Day.
…

In the plans thus far, no mention of the main card or where the ring plus cage will be erected on Trump Mall. (that’s sarcasm, not “fake news”)

Most Other Places – Application 17-0003

If you read 36 CFR 7.96 carefully, you noticed there are places always prohibited to protesters:

…
(ii) Other park areas. Demonstrations and special events are not allowed in the following other park areas:

(A) The Washington Monument, which means the area enclosed within the inner circle that surrounds the Monument’s base, except for the official annual commemorative Washington birthday ceremony.

(B) The Lincoln Memorial, which means that portion of the park area which is on the same level or above the base of the large marble columns surrounding the structure, and the single series of marble stairs immediately adjacent to and below that level, except for the official annual commemorative Lincoln birthday ceremony.

(C) The Jefferson Memorial, which means the circular portion of the Jefferson Memorial enclosed by the outermost series of columns, and all portions on the same levels or above the base of these columns, except for the official annual commemorative Jefferson birthday ceremony.

(D) The Vietnam Veterans Memorial, except for official annual Memorial Day and Veterans Day commemorative ceremonies.
…

What about places just outside the already restricted areas?

Dates:

Set-up dates starting 01/6/2017 6:00 am ending 1/19/2017
Activity dates starting 1/20/2017 ending 1/20/2017
Break-down dates starting 1/21/2017 ending 2/10/2017 11:59 pm

Closes:

…
The Lincoln Memorial area, as more fully detailed as the park area bordered by 23rd Street, Daniel French Drive and Independence Avenue, Henry Bacon Drive and Constitution Avenue, Constitution Avenue between 15th & 23rd Streets, Constitution Gardens to include Area #5 outside of the Vietnam Veteran’s Memorial restricted area, the Lincoln Memorial outside of its restricted area, the Lincoln Memorial Plaza and Reflecting Pool Area, JFK Hockey Field, park area west of Lincoln Memorial between French Drive, Henry Bacon Drive, Parking Lots A, Band C, East and West Potomac Park, Memorial Bridge, Memorial Circle and Memorial Drive, the World War II Memorial. The Washington Monument Grounds as more fully depicted as the park area bounded by 14th & 15th Streets and Madison Drive and Independence Avenue.
…

Not to use but to prevent its use by others:

…
Purpose of Activity: In connection with the Presidential Inaugural Ceremonies, this application is for use of the Lincoln Memorial areas and Washington Monument grounds by PIC, in the event that PIC seeks its use for the Inaugural related ceremonies and any necessary staging, which is expected to be:

A) In the event that PIC seeks the use of the Lincoln Memorial areas for a pre-and/or post Inaugural ceremonies, the area will be used for staging the event(s), staging of media to cover and/or broadcast the event, and for ticketed and/or public viewing.

B) In the event that PIC seeks to use the Washington Monument grounds for a public overflow area to view the Inaugural ceremony and/ or parade, the area will be used for the public who will observe the activities through prepositioned jumbotrons and sound towers.
…

Next Steps

For your amusement, all five applications contain the following question answered No:

Do you have any reason to believe or any information indicating that any individual, group or organization might seek to disrupt the activity for which this application is submitted?

I would venture to say someone hasn’t been listening. 😉

Among the data science questions raised by this background information are:

  • How best to represent these no free speech and/or no free assembly zones on a map?
  • What data sets do you need to make protesters effective under these restrictions?
  • What questions would you ask of those data sets?
  • How to decide between viral/spontaneous action versus publicly known but lawful conduct, up until the point it becomes unlawful?

If you use any of this information, please credit Amber Jamieson, Jessica Glenza and the Partnership for Civil Justice Fund as the primary sources.

See further news from the Partnership for Civil Justice Fund at: Your Right of Resistance.

Tune in next Monday for: How To Brick A School Bus, Data Science Helps Park It.

PS: “The White House Sidewalk is the sidewalk between East and West Executive Avenues, on the south side Pennsylvania Avenue, N.W.” From OMB Control No. 1024-0021 – Application for a Permit to Conduct a Demonstration or Special Event in Park Areas and a Waiver of Numerical Limitations on Demonstrations for White House Sidewalk and/or Lafayette Park

December 8, 2016

Comparing Symbolic Deep Learning Frameworks

Filed under: Deep Learning,Machine Learning — Patrick Durusau @ 9:32 pm

Deep Learning Part 1: Comparison of Symbolic Deep Learning Frameworks by Anusua Trivedi.

From the post:

This blog series is based on my upcoming talk on re-usability of Deep Learning Models at the Hadoop+Strata World Conference in Singapore. This blog series will be in several parts – where I describe my experiences and go deep into the reasons behind my choices.

Deep learning is an emerging field of research, which has its application across multiple domains. I try to show how transfer learning and fine tuning strategy leads to re-usability of the same Convolution Neural Network model in different disjoint domains. Application of this model across various different domains brings value to using this fine-tuned model.

In this blog (Part1), I describe and compare the commonly used open-source deep learning frameworks. I dive deep into different pros and cons for each framework, and discuss why I chose Theano for my work.
…

Your mileage may vary but a great starting place!

December 7, 2016

Pearl Harbor – 1941 – Talking Heads Blamed Germany (Now North Korea, Russia, etc.)

Filed under: Journalism,News,Reporting — Patrick Durusau @ 8:37 pm

Reporting and punditry that escaped infamy by Michael J. Socolow.

Does this remind you of reporting during 2016:

…
As the day wore on, real reporting receded, giving way to more speculation. Right-wing commentator Fulton Lewis Jr. told an audience five hours after the attack that he shared the doubts of many American authorities that the Japanese were truly responsible. He “reported” that US military officials weren’t convinced Japanese pilots had the skills to carry out such an impressive raid. The War Department, he said, is “concerned to find out who the pilots of these planes are—whether they are Japanese pilots. There is some doubt as to that, some skepticism whether they may be pilots of some other nationality, perhaps Germans, perhaps Italians,” he explained. The rumor that Germans bombed Pearl Harbor lingered on the airwaves, with NBC reporting, on December 8, that eyewitnesses claimed to have seen Nazi swastikas painted on some of the bombers.
…

More recent media failures include 9/11 and Hurricane Katrina.

Even more recently, the media has seized upon flights of fancy by “experts” to blame North Korea, Russia, the Islamic State and others for a variety of ills and disasters.

Thoughts on what leads to such media failures time and time again? I can’t think of a single major news event in the last fifty (50) years that wasn’t accompanied by:

…terrible punditry, inaccurate reporting, and ridiculous commentary

to steal Socolow’s closing line.

The failure of the news media reminds me of a discussion with a Hebrew Bible professor over the translation of a verse into English. He conceded that we don’t know the meaning of a particular term but said a translator cannot simply pass over an unknown term, but must translate it. The verse in question is well-known so the committee took refuge in giving the term an incorrect but “traditional” translation.

To what degree does the news media offer “terrible punditry, inaccurate reporting, and ridiculous commentary” because of a requirement that events, people, causes, “make sense?”

That is it is unsatisfying to report a plane crash, stock failure, bombing, without some attempt to outfit the event with an explanation.

I’m not sure if unsatisfying applies to the reporters, the news consuming public, or both.

For my part, I’m incurious about the motives of people who harm other people, assuming that even the “alleged perpetrator” has some insight into their motives. Motive is a complex and difficult subject under the best of circumstances and a sound bite of less than 30 seconds is a long way from being sufficient.

But it leaves the viewer with the false impression they have learned something about an event, people, etc.

One way to avoid “…terrible punditry, inaccurate reporting, and ridiculous commentary…,” unless you are required to respond to a particular event, is to simply ignore reporting for several days or weeks after an event. The more major the event, the longer you should delay.

For example, when 9/11 occurred, I was in San Jose, California at a Unicode conference. In fact, I was working on email waiting for the conference to start.

After the news had spread, the conference organizers asked the attendees what we wanted to do. Given the choice of watching loops of planes crashing into the World Trade Center and uninformed commentary or continuing with the conference, we chose the latter.

Delayed consumption of news of major events won’t improve the quality of the immediate reporting but it may give time for more reasonable voices to emerge. Still waiting on that to happen for 9/11.

Facebook Patents Tool To Think For You

Filed under: Censorship,Facebook,Free Speech,News — Patrick Durusau @ 4:40 pm

My apologies but Facebook thinks you are too stupid to detect “fake news.” Facebook will compensate for your stupidity with a process submitted for a US patent. For free!

Facebook is patenting a tool that could help automate removal of fake news by Casey Newton.

From the post:

As Facebook works on new tools to stop the spread of misinformation on its network, it’s seeking to patent technology that could be used for that purpose. This month the US Trademark and Patent Office published Facebook’s application for Patent 0350675: “systems and methods to identify objectionable content.” The application, which was filed in June 2015, describes a sophisticated system for identifying inappropriate text and images and removing them from the network.

As described in the application, the primary purpose of the tool is to improve the detection of pornography, hate speech, and bullying. But last month, Zuckerberg highlighted the need for “better technical systems to detect what people will flag as false before they do it themselves.” The patent published Thursday, which is still pending approval, offers some ideas for how such a system could work.

A Facebook spokeswoman said the company often seeks patents for technology that it never implements, and said this patent should not be taken as an indication of the company’s future plans. The spokeswoman declined to comment on whether it was now in use.

The system described in the application is largely consistent with Facebook’s own descriptions of how it currently handles objectionable content. But it also adds a layer of machine learning to make reporting bad posts more efficient, and to help the system learn common markers of objectionable content over time — tools that sound similar to the anticipatory flagging that Zuckerberg says is needed to combat fake news.
…

If you substitute “user” for “administrator” where it appears in the text, Facebook would be enabling users to police the content they view.

Why Facebook finds users making decisions about the content they view objectionable isn’t clear. Suggestions on that question?

The process doesn’t appear to be either accountable and/or transparent.

If I can’t see the content that is removed by Facebook, how do I make judgments about why it was removed and/or how that compares to content about to be uploaded to Facebook?

Urge Facebook users to demand empowering them to make decisions about the content they view.

Urge Facebook shareholders to pressure management to abandon this quixotic quest to be an internet censor.

December 6, 2016

Writing a Halite Bot in Clojure [Incomplete Knowledge/Deception Bots?]

Filed under: Game Theory,Games,Wargames — Patrick Durusau @ 8:57 pm

Writing a Halite Bot in Clojure by Matt Adereth.

From the post:

Halite is a new AI programming competition that was recently released by Two Sigma and Cornell Tech. It was designed and implemented by two interns at Two Sigma and was run as the annual internal summer programming competition.

While the rules are relatively simple, it proved to be a surprisingly deep challenge. It’s played on a 2D grid and a typical game looks like this:

halite-game-460

Each turn, all players simultaneously issue movement commands to each of their pieces:

  1. Move to an adjacent location and capture it if you are stronger than what’s currently there.
  2. Stay put and build strength based on the production value of your current location.

When two players’ pieces are adjacent to each other, they automatically fight. A much more detailed description is available on the Halite Game Rules page.
…

Looking at the rules page, it looks like:

  • Bots have accurate knowledge of all positions and values.
  • Deception of bots isn’t permitted.

    • Interesting from a bot programming perspective but lack of knowledge and the ever present danger of deception are integral parts of human games.

    Any bot games that feature both a lack of knowledge and/or deception?

Four Experiments in Handwriting with a Neural Network

Filed under: Artificial Intelligence,Human-Computer Interaction Lab (HCIL),Machine Learning,Neural Information Processing,Neural Networks,Visualization — Patrick Durusau @ 8:29 pm

Four Experiments in Handwriting with a Neural Network by Shan Carter, David Ha, Ian Johnson, and Chris Olah.

While the handwriting experiments are compelling and entertaining, the author’s have a more profound goal for this activity:

…
The black box reputation of machine learning models is well deserved, but we believe part of that reputation has been born from the programming context into which they have been locked into. The experience of having an easily inspectable model available in the same programming context as the interactive visualization environment (here, javascript) proved to be very productive for prototyping and exploring new ideas for this post.

As we are able to move them more and more into the same programming context that user interface work is done, we believe we will see richer modes of human-ai interactions flourish. This could have a marked impact on debugging and building models, for sure, but also in how the models are used. Machine learning research typically seeks to mimic and substitute humans, and increasingly it’s able to. What seems less explored is using machine learning to augment humans. This sort of complicated human-machine interaction is best explored when the full capabilities of the model are available in the user interface context.

Setting up a search alert for future work from these authors!

Pattern Overloading

Filed under: Clojure,Language,Language Design,Lisp,Programming — Patrick Durusau @ 8:03 pm

Pattern Overloading by Ramsey Nasser.

From the post:

C-like languages have a problem of overloaded syntax that I noticed while teaching high school students. Consider the following snippets in such a language:

foo(45)

function foo(int x) {

for(int i=0;i < 10; i++) {

if(x > 10) {

case(x) {

A programmer experienced with this family would see

  1. Function invocation
  2. Function definition
  3. Control flow examples

In my experience, new programmers see these constructs as instances of the same idea: name(some-stuff) more-stuff. This is not an unreasonable conclusion to reach. The syntax for each construct is shockingly similar given that their semantics are wildly different.

You won’t be called upon to re-design C but Nasser’s advice:

Syntactic similarity should mirror semantic similarity

Or, to take a quote from the UX world

Similar things should look similar and dissimilar things should look dissimilar

is equally applicable to any syntax that you design.

Attn: “Fake News” Warriors! Where’s The Harm In Terrorist Propaganda?

Filed under: Censorship,Free Speech,Government,Journalism — Patrick Durusau @ 7:34 pm

Facebook, Microsoft, Twitter, and YouTube team up to stop terrorist propaganda by Justin Carissimo.

Justin’s report is true, at least in the sense that Facebook, Microsoft, Twitter, and YouTube are collaborating to censor “terrorist propaganda.”

Justin’s post also propagates the “fake news” that online content from terrorists “…threaten our national security and public safety….”

Really? You would think after all these years of terrorist propaganda, there would be evidence to support that claim.

True enough, potential terrorists can meet online, but “recruitment” is a far different tale than reading online terrorist content. Consider ISIS and the Lonely Young American, a tale told to support the idea of online recruiting, but is one of the better refutations of that danger.

It’s not hard to whistle up alleged social science studies of online “terrorist propaganda” but the impacts of that so-called propaganda, are speculation at best, when not actually fantasies of the authors.

“Fake News” warriors should challenge the harmful terrorist propaganda narrative as well as those that are laughably false (denying climate change for example).

The Great Scone Map…

Filed under: Food,Mapping,Maps — Patrick Durusau @ 5:52 pm

uk-scone-map-460

I was deeply disappointed to find the “Great Scone Map” represents differing pronunciations of “scone.”

Reading hurriedly, I thought perhaps it was a map of scone recipes. 😉

Suggestions of maps of biscuit (a small, typically round cake of bread leavened with baking powder, baking soda, or sometimes yeast) recipes?

To avoid confusion over the term “biscuit,” ask it the “biscuit” in question is eaten by the British. If yes, then odds are it not a “biscuit” in the North American sense of the word.

There’s an a/b test for you.

Put a British “biscuit” along side a buttered Popeyes biscuit and see which one is chosen more often.

Eat several Popeyes biscuits before starting to avoid being stuck with British “biscuits.”

December 5, 2016

Clojure/conj 2016 – Videos – Sorted

Filed under: Clojure,Conferences,Functional Programming,Programming — Patrick Durusau @ 3:08 pm

Clojure/conf 2016 has posted videos of all presentations (thanks!) to YouTube, which displays them in no particular order.

To help with my viewing and perhaps yours, here are the videos in title order:

  1. Adventures in Understanding Documents – Scott Tuddenham
  2. Audyx.com 40k locs to build the first web – based sonogram – Asher Coren
  3. Barliman: trying the halting problem backwards, blindfolded – William Byrd, Greg Rosenblatt
  4. Becoming Omniscient with Sayid – Bill Piel
  5. Building a powerful Double Entry Accounting system – Lucas Cavalcanti
  6. Building composable abstractions – Eric Normand
  7. Charting the English Language…in pure Clojure – Alexander Mann
  8. Clarifying Rules Engines with Clara Rules – Mike Rodriguez
  9. Clojure at DataStax: The Long Road From Python to Clojure – Nick Bailey
  10. A Clojure DSL for defining CI/CD orchestrations at scale – Rohit Kumar, Viraj Purang
  11. Composing music with clojure.spec – Wojciech Franke
  12. In situ model-based learning in PAMELA – Paul Robertson, Tom Marble
  13. Juggling Patterns and Programs – Steve Miner
  14. Overcoming the Challenges of Mentoring – Kim Crayton
  15. A Peek Inside SAT Solvers – Jon Smock
  16. Powderkeg: teaching Clojure to Spark – Igor Ges, Christophe Grand
  17. Production Rules on Databases – Paula Gearon
  18. Programming What Cannot Be Programmed: Aesthetics and Narrative – D. SchmĂźdde
  19. Proto REPL, a New Clojure Development and Visualization Tool – Jason Gilman
  20. Simplifying ETL with Clojure and Datomic – Stuart Halloway
  21. Spec-ulation Keynote – Rich Hickey
  22. Spectrum, a library for statically "typing" clojure.spec – Allen Rohner
  23. Using Clojure with C APIs for crypto and more – lvh
  24. WormBase database migration to Datomic on AWS: A case Study – Adam Wright

Enjoy!

Resisting EU Censorship

Filed under: Censorship,Free Speech,Government — Patrick Durusau @ 10:45 am

US tech giants like Facebook could face new EU laws forcing them to tackle hate speech by Arjun Kharpal.

From the post:

U.S. technology giants including Facebook, Twitter, Microsoft, and Google’s YouTube could face new laws forcing them to deal with online hate speech if they don’t tackle the problem themselves, the European Commission warned.

In May, the four U.S. firms unveiled a “code of conduct” drawn up in conjunction with the Commission, the European Union’s executive arm, to take on hate speech on their platforms. It involved a series of commitments including a pledge to review the majority of notifications of suspected illegal hate speech in less than 24 hours and remove or disable access to the content if necessary. Another promise was to provide regular training to staff around hate speech.

But six months on, the Commission is not happy with the progress. EU Justice Commissioner Vera Jourova has commissioned a report, set to be released later this week, which claims that progress in removing offending material has been too slow.
…

I posted about this ill-fated “code of conduct” under Four Horsemen Of Internet Censorship + One. I pointed out the only robust solution to the “hate speech” problem was to enable users to filter the content they see, as opposed to the EU.

Fast forward 2 internet years (3 months = 1 internet year) and the EU is seeking to increase its censorship powers and not to empower users to regulate the content they consume.

Adding injury to insult, the EU proposes directives that require uncompensated expenditures on the part of its victims, Facebook, Twitter, Microsoft, and Google, to meet criteria that can only be specified user by user.

Why the first refuge of the EU for disagreeable speech is censorship I don’t know. What I do know is any tolerance of EU censorship demands encourages even more outrageous censorship demands.

The usual suspects should push back and push back hard against EU demands for censorship.

Enabling users to filter content means users can shape incoming streams to fit their personal sensitivities and dislikes, without impinging on the rights of others.

Had Facebook, Twitter, Microsoft, and Google started developing shareable content filters when they proposed their foolish “code of conduct” to the EU last May, they would either be available or nearly so by today.

Social media providers should not waste any further time attempting to censor on behalf of the EU or users. Enable users to censor their own content and get out of the censorship business.

There’s no profit in the censorship business. In fact, there is only expense and wasted effort.

PS: The “EU report” in question won’t be released until Wednesday, December 7, 2016 (or so I am told).

December 4, 2016

VA State Police Paid $585K+ For Cell Site Simulator – Your Price?

Filed under: Cybersecurity,Government,Privacy,Security — Patrick Durusau @ 9:48 pm

Virginia State Police releases cellphone surveillance logs Since May 2015, the VSP have used their DRTbox unit 12 times – 5 of which appeared ineffective by Curtis Waltman.

From the post:

As part of a nationwide FOIA census for cell site simulator surveillance devices, the Virginia State Police responded with new documents detailing their acquisition and use of the DRT 1183C. Made by Digital Receiver Technology of Maryland, the DRT 1183C is a device that is commonly referred to as a DRTbox. It is very similar to other cell site simulators like the Harris Corporation’s Stingray, except that DRTboxes can also intercept voice communication as well as GPS location and other metadata.

Astonishingly unredacted, these documents detail their 2014 purchase, which upgraded their obsolete DRTbox model to the smaller and more powerful 1183C. This cost the VSP $585,265, and came complete with a whole bunch of accessories, including a Chevrolet Suburban outfitted specifically to run the device.
…

Two questions:

  1. What features are offered by your home-brew cell site simulator?
  2. Estimated price (parts + labor)?

When law enforcement, judges, legislators, etc., join ordinary citizens in the smartphone gold fish bowl, effective privacy will be a goal of vendors.

Data Leakage, Data Breaches and Explosives

Filed under: Cybersecurity,Security — Patrick Durusau @ 9:14 pm

Data breach exposed locations of oil-industry explosives, handler credentials by Dell Cameron.

From the post:

A misconfigured storage device discovered by a security researcher in October left exposed thousands of internal files belonging to an explosives-handling company. [The original story is dated 01/12/2016, so this post is almost a year out of date.]

The files, which have since been secured, reportedly included details about facilities in three U.S. states where explosives are stored.

The leaky file repository belonged to Allied-Horizontal Wireline Services (AHWS), a leading wireline company with more than 400 employees and 70 wireline units throughout the United States. (“Wireline” is an industry term that refers to cabling technology used at oil and gas wells.) The company is licensed by the federal government to store and use explosives to complete an oil-drilling process known as “perforation.”

Chris Vickery, a lead security researcher at MacKeeper who notably discovered several misconfigured voter databases this year, found the breach in early October. After verifying the device’s owner, Vickery reached out to an AHWS executive, who quickly moved to secure the company’s data.
…

Data breach stories are so common that out-dated ones are especially of little interest.

Except, that this date breach story illustrates the problem of data leakage.

From the tone of the post, you are thinking evil-doers need to find companies that use explosives, hack their systems, blackmail their staffs, etc. All of which is serious stuff, not to mention a lot of effort.

But an unknown ATF official leaks the information that makes all that work unnecessary:

…
There are no federal laws prohibiting Allied-Horizontal Wireline Services from disclosing the location of its explosives, an ATF official said. “Licensees storing explosive materials must notify the authority having jurisdiction for fire safety in the locality where the explosive materials are stored.”
…

Where are explosives are stored? (United States only) Check with your local fire safety authority.

Thanks ATF!

Pence, Stephanopoulos and False Statements

Filed under: Bias,Government,Journalism,News,Reporting — Patrick Durusau @ 8:23 pm

‘This Week’ Transcript: Vice President-Elect Mike Pence and Gen. David Petraeus, covers President-elect Donald Trump’s tweet:

In addition to winning the electoral college in a landslide, I won the popular vote if you deduct the millions of people who voted illegally.

That portion of the transcript reads as follows (apologies for the long quote but I think you will agree its all relevant):

…
STEPHANOPOULOS: As I said, President-Elect Trump has been quite active on Twitter, including this week at the beginning of this week, that tweet which I want to show right now, about the popular vote.

And he said, “In addition to winning the electoral college in a landslide, I won the popular vote if you deduct the millions of people who voted illegally.”

That claim is groundless. There’s no evidence to back it up.

Is it responsible for a president-elect to make false statements like that?

PENCE: Well, look, I think four years ago the Pew Research Center found that there were millions of inaccurate voter registrations.

STEPHANOPOULOS: Yes, but the author of this said he — he has said it is not any evidence about what happened in this election or any evidence of voter fraud.

PENCE: I think what, you know, what is — what is historic here is that our president-elect won 30 to 50 states, he won more counties than any candidate on our side since Ronald Reagan.

And the fact that some partisans, who are frustrated with the outcome of the election and disappointed with the outcome of the election, are pointing to the popular vote, I can assure you, if this had been about the popular vote, Donald Trump and I have been campaigning a whole lot more in Illinois and California and New York.

STEPHANOPOULOS: And no one is questioning your victory, certainly I’m not questioning your victory. I’m asking just about that tweet, which I want to say that he said he would have won the popular vote if you deduct the millions of people who voted illegally. That statement is false. Why is it responsible to make it?

PENCE: Well, I think the president-elect wants to call to attention the fact that there has been evidence over many years of…

STEPHANOPOULOS: That’s not what he said.

PENCE: …voter fraud. And expressing that reality Pew Research Center found evidence of that four years ago.

STEPHANPOULOS: That’s not the evidence…

PENCE: …that certainly his right.

But, you know…

STEPHANOPOULOS: It’s his right to make false statements?

PENCE: Well, it’s his right to express his opinion as president-elect of the United States.

I think one of the things that’s refreshing about our president-elect and one of the reasons why I think he made such an incredible connection with people all across this country is because he tells you what’s on his mind.

STEPHANOPOULOS: But why is it refreshing to make false statements?

PENCE: Look, I don’t know that that is a false statement, George, and neither do you. The simple fact is that…

STEPHANOPOULOS: I know there’s no evidence for it.

PENCE: There is evidence, historic evidence from the Pew Research Center of voter fraud that’s taken place. We’re in the process of investigating irregularities in the state of Indiana that were leading up to this election. The fact that voter fraud exists is…

STEPHANPOULOS: But can you provide any evidence — can you provide any evidence to back up that statement?

PENCE; Well, look, I think he’s expressed his opinion on that. And he’s entitled to express his opinion on that. And I think the American people — I think the American people find it very refreshing that they have a president who will tell them what’s on his mind. And I think the connection that he made in the course…

STEPHANOPOULOS: Whether it’s true or not?

PENCE: Well, they’re going to tell them — he’s going to say what he believes to be true and I know that he’s always going to speak in that way as president.
….

Just to be clear, I agree with Stepanopoulos and others who say there is no evidence of millions of illegal votes being cast in the 2016 presidential election.

After reading Stephanopoulos press Pence on this false statement by President-elect Trump, can you recall Stepanopoulos or another other major reporter pressing President Obama on his statements about terrorism, such as:

…
Tonight I want to talk with you about this tragedy, the broader threat of terrorism and how we can keep our country safe. The FBI is still gathering the facts about what happened in San Bernardino, but here’s what we know. The victims were brutally murdered and injured by one of their co-workers and his wife. So far, we have no evidence that the killers were directed by a terrorist organization overseas or that they were part of a broader conspiracy here at home. But it is clear that the two of them had gone down the dark path of radicalization, embracing a perverted interpretation of Islam that calls for war against America and the West. They had stockpiled assault weapons, ammunition, and pipe bombs.

So this was an act of terrorism designed to kill innocent people. Our nation has been at war with terrorists since Al Qaeda killed nearly 3,000 Americans on 9/11. In the process, we’ve hardened our defenses, from airports, to financial centers, to other critical infrastructure. Intelligence and law enforcement agencies have disrupted countless plots here and overseas and worked around the clock to keep us safe.

Our military and counterterrorism professionals have relentlessly pursued terrorist networks overseas, disrupting safe havens in several different countries, killing Osama Bin Laden, and decimating Al Qaeda’s leadership.

Over the last few years, however, the terrorist threat has evolved into a new phase. As we’ve become better at preventing complex multifaceted attacks like 9/11, terrorists turn to less complicated acts of violence like the mass shootings that are all too common in our society. It is this type of attack that we saw at Fort Hood in 2009, in Chattanooga earlier this year, and now in San Bernardino.

And as groups like ISIL grew stronger amidst the chaos of war in Iraq and then Syria, and as the Internet erases the distance between countries, we see growing efforts by terrorists to poison the minds of people like the Boston Marathon bombers and the San Bernardino killers.

For seven years, I’ve confronted this evolving threat each and every morning in my intelligence briefing, and since the day I took this office, I have authorized U.S. forces to take out terrorists abroad precisely because I know how real the danger is.
… Here’s what Obama said in his Sunday night address: An annotated transcript

Really? “…because I know how real the danger is.”

Do you recall anyone pressing President Obama on his claims about the danger of terrorism?

If you ever get to pose such a question to President Obama, remind him that 685 American die every day from medial errors, 44,0000 Americans die every 6 months due to excessive alcohol consumption, and that 430 Americans died between 2000 and 2013 due to falling furniture.

Can you think of a single instance when Obama’s flights of fancy about terrorism were challenged as Stephanopoulos did Trump’s delusion about illegal voters?

The media can and should challenge such flights of fancy.

At the same time, they should challenge those favored by other politicians, their editors, fellow journalists and advertisers.

PS: The medical error article: Medical error—the third leading cause of death in the US, BMJ 2016; 353 doi: http://dx.doi.org/10.1136/bmj.i2139 (Published 03 May 2016) Cite this as: BMJ 2016;353:i2139 (The Guardian article, my source, didn’t include a link to the original article.)

December 2, 2016

Identifying Speech/News Writers

Filed under: R,Text Analytics,Text Corpus — Patrick Durusau @ 8:44 pm

David Smith’s post: Stylometry: Identifying authors of texts using R details the use of R to distinguish tweets by president-elect Donald Trump from his campaign staff. (Hmmm, sharing a Twitter account password, there’s bad security for you.)

The same techniques may distinguish texts delivered “live” versus those “inserted” into Congressional Record.

What other texts are ripe for distinguishing authors?

From the post:

Few people expect politicians to write every word they utter themselves; reliance on speechwriters and spokepersons is a long-established political practice. Still, it's interesting to know which statements are truly the politician's own words, and which are driven primarily by advisors or influencers.

Recently, David Robinson established a way of figuring out which tweets from Donald Trump's Twitter account came from him personally, as opposed to from campaign staff, whcih he verified by comparing the sentiment of tweets from Android vs iPhone devices. Now, Ali Arsalan Kazmi has used stylometric analysis to investigate the provenance of speeches by the Prime Minister of Pakistan. 

…

A small amount of transparency can go a long way.

Email archives anyone?

War and Peace & R

Filed under: Humanities,Literature,R,Visualization — Patrick Durusau @ 5:13 pm

No, not a post about R versus Python but about R and Tolstoy‘s War and Peace.

Using R to Gain Insights into the Emotional Journeys in War and Peace by Wee Hyong Tok.

From the post:

How do you read a novel in record time, and gain insights into the emotional journey of main characters, as they go through various trials and tribulations, as an exciting story unfolds from chapter to chapter?

I remembered my experiences when I start reading a novel, and I get intrigued by the story, and simply cannot wait to get to the last chapter. I also recall many conversations with friends on some of the interesting novels that I have read awhile back, and somehow have only vague recollection of what happened in a specific chapter. In this post, I’ll work through how we can use R to analyze the English translation of War and Peace.

War and Peace is a novel by Leo Tolstoy, and captures the salient points about Russian history from the period 1805 to 1812. The novel consists of the stories of five families, and captures the trials and tribulations of various characters (e.g. Natasha and Andre). The novel consists of about 1400 pages, and is one of the longest novels that have been written.

We hypothesize that if we can build a dashboard (shown below), this will allow us to gain insights into the emotional journey undertaken by the characters in War and Peace.
…

Impressive work, even though I would not use it as a short-cut to “read a novel in record time.”

Rather I take this as an alternative way of reading War and Peace, one that can capture insights a casual reader may miss.

Moreover, the techniques demonstrated here could be used with other works of literature, or even non-fictional works.

Imagine conducting this analysis over the reportedly more than 7,000 page full CIA Torture Report, for example.

A heatmap does not connect any dots, but points a user towards places where interesting dots may be found.

Certainly a tool for exploring large releases/leaks of text data.

Enjoy!

PS: Large, tiresome, obscure-on-purpose, government reports to practice on with this method?

December 1, 2016

OSS-Fuzz: Continuous fuzzing for open source software

Filed under: Cybersecurity,Programming,Security — Patrick Durusau @ 3:16 pm

Announcing OSS-Fuzz: Continuous fuzzing for open source software

From the post:

We are happy to announce OSS-Fuzz, a new Beta program developed over the past years with the Core Infrastructure Initiative community. This program will provide continuous fuzzing for select core open source software.

Open source software is the backbone of the many apps, sites, services, and networked things that make up “the internet.” It is important that the open source foundation be stable, secure, and reliable, as cracks and weaknesses impact all who build on it.

Recent security stories confirm that errors like buffer overflow and use-after-free can have serious, widespread consequences when they occur in critical open source software. These errors are not only serious, but notoriously difficult to find via routine code audits, even for experienced developers. That’s where fuzz testing comes in. By generating random inputs to a given program, fuzzing triggers and helps uncover errors quickly and thoroughly.

In recent years, several efficient general purpose fuzzing engines have been implemented (e.g. AFL and libFuzzer), and we use them to fuzz various components of the Chrome browser. These fuzzers, when combined with Sanitizers, can help find security vulnerabilities (e.g. buffer overflows, use-after-free, bad casts, integer overflows, etc), stability bugs (e.g. null dereferences, memory leaks, out-of-memory, assertion failures, etc) and sometimes even logical bugs.

OSS-Fuzz’s goal is to make common software infrastructure more secure and stable by combining modern fuzzing techniques with scalable distributed execution. OSS-Fuzz combines various fuzzing engines (initially, libFuzzer) with Sanitizers (initially, AddressSanitizer) and provides a massive distributed execution environment powered by ClusterFuzz.
… (emphasis in original)

Another similarity between open and closed source software.

Closed source software is continuously being fuzzed.

By volunteers.

Yes? 😉

One starting place for more information: Effective file format fuzzing by Mateusz “j00ru” Jurczyk (Black Hat Europe 2016, London) and his website: http://j00ru.vexillium.org/.

If You Don’t Get A Quantum Computer For Christmas

Filed under: Haskell,Physics,Quantum — Patrick Durusau @ 1:38 pm

Learn Quantum Mechanics with Haskell by Scott N. Walck.

Abstract:

To learn quantum mechanics, one must become adept in the use of various mathematical structures that make up the theory; one must also become familiar with some basic laboratory experiments that the theory is designed to explain. The laboratory ideas are naturally expressed in one language, and the theoretical ideas in another. We present a method for learning quantum mechanics that begins with a laboratory language for the description and simulation of simple but essential laboratory experiments, so that students can gain some intuition about the phenomena that a theory of quantum mechanics needs to explain. Then, in parallel with the introduction of the mathematical framework on which quantum mechanics is based, we introduce a calculational language for describing important mathematical objects and operations, allowing students to do calculations in quantum mechanics, including calculations that cannot be done by hand. Finally, we ask students to use the calculational language to implement a simplified version of the laboratory language, bringing together the theoretical and laboratory ideas.

You won’t find a quantum computer under your Christmas tree this year.

But Haskell + Walck will teach you the basics of quantum mechanics.

You may also want to read:

Structure and Interpretation of Quantum Mechanics – a Functional Framework (2003) by Jerzy Karczmarczuk.

You will have to search for it but “Gerald Jay Sussman & Jack Wisdom (2013): Functional Differential Geometry. The MIT Press.” is out on the net somewhere.

Very tough sledding but this snippet from the preface may tempt you into buying a copy:

…
But the single biggest difference between our treatment and others is that we integrate computer programming into our explanations. By programming a computer to interpret our formulas we soon learn whether or not a formula is correct. If a formula is not clear, it will not be interpretable. If it is wrong, we will get a wrong answer. In either case we are led to improve our program and as a result improve our understanding. We have been teaching advanced classical mechanics at MIT for many years using this strategy. We use precise functional notation and we have students program in a functional language. The students enjoy this approach and we have learned a lot ourselves. It is the experience of writing software for expressing the mathematical content and the insights that we gain from doing it that we feel is revolutionary. We want others to have a similar experience.
…

If that interests you, check out courses by Sussman at MITOpenCourseware.

Enjoy!

Recycling Old News – NPR Station WMOT

Filed under: Journalism,News,Reporting — Patrick Durusau @ 12:06 pm

Avoiding “fake” news, NPR station WMOT is recycling “old news.”

Seriously.

Looking for a recent article on combining multiple sources of DNA I found:

Combining The DNA Of Three People Raises Ethical Questions by Rob Stein, Nov. 10, 2014.

combining-dna-460

In a darkened lab in the north of England, a research associate is intensely focused on the microscope in front of her. She carefully maneuvers a long glass tube that she uses to manipulate early human embryos.

“It’s like microsurgery,” says Laura Irving of Newcastle University.

Irving is part of a team of scientists trying to replace defective DNA with healthy DNA. They hope this procedure could one day help women who are carrying genetic disorders have healthy children.
…

Compare that post to:

Combining The DNA Of Three People Raises Ethical Questions by Rob Stein, 22 hours ago.

combining-dna-460

In a darkened lab in the north of England, a research associate is intensely focused on the microscope in front of her. She carefully maneuvers a long glass tube that she uses to manipulate early human embryos.

“It’s like microsurgery,” says Laura Irving of Newcastle University.

Irving is part of a team of scientists trying to replace defective DNA with healthy DNA. They hope this procedure could one day help women who are carrying genetic disorders have healthy children.
…

I took a screen shot that includes WMOT and the article title, plus saved the page, just in case through the magic of silent correction, this example of “news” reporting goes away.

At least to me, two year old news isn’t the same as news 22 hours ago.

You?

PS: The loss of credibility by the media has been entirely self-inflicted. See media coverage of the 2016 presidential race for example. Why would anyone trust a news source that was so badly wrong?

Hard work, good journalism, timely reporting, all of those are the elements needed for the media to regain credibility. Credible journalists don’t attempt to suppress “fake news.” Attempts to suppress “fake news” signal a lack of commitment to credible journalism. Credible journalism doesn’t notice “fake news.”

« Newer Posts

Powered by WordPress