Archive for the ‘Reporting’ Category

.Rddj (data journalism with R)

Wednesday, June 21st, 2017

.Rddj Hand-curated, high quality resources for doing data journalism with R by Timo Grossenbacher.

From the webpage:

The R Project is a great software environment for doing all sorts of data-driven journalism. It can be used for any of the stages of a typical data project: data collection, cleaning, analysis and even (interactive) visualization. And it’s all reproducible and transparent! Sure, it requires a fair amount of scripting, yet…

Do not fear! With this hand-curated (and opinionated) list of resources, you will be guided through the thick jungle of countless R packages, from learning the basics of R’s syntax, to scraping HTML tables, to a guide on how to make your work comprehensible and reproducible.

Now, enjoy your journey.

Some more efforts at persuasion: As I work in the media, I know how a lot of journalists are turned off by everything that doesn’t have a graphical interface with buttons to click on. However, you don’t need to spend days studying programming concepts in order to get started with R, as most operations can be carried out without applying scary things such as loops or conditionals – and, nowadays, high-level abstrations like dplyr make working with data a breeze. My advice if you’re new to data journalism or data processing in general: Better learn R than Excel, ’cause getting to know Excel (and the countless other tools that each do a single thing) doesn’t come for free, either.

This list is (partially) inspired by R for Journalists by Ed Borasky, which is another great resource for getting to know R.

… (emphasis in original)

The topics are familiar:

  • RStudio
  • Syntax and basic R programming
  • Collecting Data (from the Web)
  • Data cleaning and manipulation
  • Text mining / natural language processing
  • Exploratory data analysis and plotting
  • Interactive data visualization
  • Publication-quality graphics
  • Reproducibility
  • Examples of using R in (data) journalism
  • What makes this list of resources different from search results?

    Hand curation.

    How much of a difference?

    Compare the search results of “R” + any of these categories to the resources here.

    Bookmark .Rddj for data journalism and R, then ping me with the hand curated list of resources you are creating.

    Save yourself and the rest of us from search. Thanks!

    Storyzy A.I. Fights Fake Quotes (Ineffective Against Trump White House)

    Wednesday, June 21st, 2017

    In the battle against fake news, Storyzy A.I. fights fake quotes

    From the post:

    The Quote Verifier launched today by Storyzy takes the battle against fake news to a whole new automated level by conveniently flagging fake quotes on social networks and search engines with +50,000 new authentic quotes added daily.

    Storyzy aims to help social networks and search engines by spotting fake quotes. To fulfill this ambition Storyzy developed a tool (currently available in Beta version) that verifies whether a quote is authentic or not by checking if a person truly said that or not.
    … (emphasis in original)

    A tool for your short-list of verification tools to use on a daily basis.

    It’s ineffective against the Trump White House because accurate quotes can still be “false.”

    “Truthful quotes,” as per Trump White House policy, issue only from the President and must reflect what he meant to say. Subject to correction by the President.

    A “truthful quote,” consists of three parts:

    1. Said by the President
    2. Reflects what he meant to say
    3. Includes any subsequent correction by the President (one or more)

    There is a simply solution to avoiding “false” quotes from President Trump:

    Never quote him or his tweets at all.

    Quote his lackeys, familiars and sycophants, but not him.

    The Quartz Directory of Essential Data (Directory of Directories Is More Accurate)

    Saturday, June 17th, 2017

    The Quartz Directory of Essential Data

    From the webpage:

    A curated list of useful datasets published by important sources. Please remember that “important” does not mean “correct.” You should vet these data as you would with any human source.

    Switch to the “Data” tab at the bottom of this spreadsheet and use Find (⌘ + F) to search for datasets on a particular topic.

    Note: Just because data is useful, doesn’t mean it’s easy to use. The point of this directory is to help you find data. If you need help accessing or interpreting one of these datasets, please reach out to your friendly Quartz data editor, Chris.

    Slack: @chris

    A directory of 77 data directories. The breath of organizing topics, health, trade, government, for example, creates a need for repeated data mining by every new user.

    A low/no-friction method for creating more specific and re-usable directories has remained elusive.

    Are You A Serious Reader?

    Saturday, June 17th, 2017

    What does it mean for a journalist today to be a Serious Reader? by Danny Funt.

    From the post:

    BEFORE THE BOOKS ARRIVED, Adam Gopnik, in an effort to be polite, almost contradicted the essential insight of his life. An essayist, critic, and reporter at The New Yorker for the last 31 years, he was asked whether there is an imperative for busy, ambitious journalists to read books seriously—especially with journalism, and not just White House reporting, feeling unusually high-stakes these days—when the doorbell rang in his apartment, a block east of Central Park. He came back with a shipment and said, “It would be,” pausing to think of and lean into the proper word, “brutally unkind and unrealistic to say, Oh, all of you should be reading Stendhal. You’ll be better BuzzFeeders for it.” For the part about the 19th-century French novelist, he switched from his naturally delicate voice to a buffoonish, apparently bookish, baritone.

    Then, as he tore open the packaging of two nonfiction paperbacks (one, obscure research for an assignment on Ernest Hemingway; the other, a new book on Adam Smith, a past essay subject) and sat facing a wall-length bookcase and sliding ladder in his heavenly, all-white living room, Gopnik took that back. His instinct was to avoid sermonizing about books, particularly to colleagues with grueling workloads, because time for books is a privilege of his job. And yet, to achieve such an amazingly prolific life, the truth is he simply read his way here.

    I spoke with a dozen accomplished journalists of various specialties who manage to do their work while reading a phenomenal number of books, about and beyond their latest project. With journalists so fiercely resented after last year’s election for their perceived elitist detachment, it might seem like a bizarre response to double down on something as hermetic as reading—unless you see books as the only way to fully see the world.

    Being well-read is a transcendent achievement similar to training to run 26.2 miles, then showing up for a marathon in New York City and finding 50,000 people there. It is at once superhuman and pedestrian.

    … (emphasis in original)

    A deeply inspirational and instructive essay on serious readers and the benefits that accrue to them. Very much worth two or more slow reads, plus looking up the authors, writers and reporters who are mentioned.

    Earlier this year I began the 2017 Women of Color Reading Challenge. I have not discovered any technical insights into data science or topic maps, but I am gaining, incrementally for sure, a deeper appreciation for how race and gender shapes a point of view.

    Or perhaps more accurately, I am encountering points of view different enough from my own that I recognize them as being different. That in and of itself, the encountering of different views, is one reason I aspire to become a “serious reader.”


    Man Bites Dog Or Shoots Member of Congress – Novelty Rules the News

    Thursday, June 15th, 2017

    The need for “novelty” in a 24 x 7 news cycle, identified by Lewis and Marwick in Megyn Kelly fiasco is one more instance of far right outmaneuvering media comes to the fore in coverage of the recent shooting reported in Capitol Hill shaken by baseball shooting.

    Boiled down to the essentials, James Hodgkinson, 66, of Illinois, who is now dead, wounded “House Majority Whip Steve Scalise (R-La.) and four others in the Washington suburb of Alexandria, Va.,” on June 14, 2017. The medical status of the wounded vary from critical to released.

    That’s all the useful information, aside from identification of the victims, that can be wrung from that story.

    Not terribly useful information, considering Hodgkinson is dead and so not a candidate for a no-fly/sell list.

    But you will read column inch after column inch of non-informative comments by and between special interest groups, “experts,” and even experienced political reporters, on a one-off event.

    A per capita murder rate of 5 per 100,000, works out to 50 murderers per million people. Approximately 136 million people voted in the 2016 election so 50 x 136 means 6800 people who will commit murder this year voted in the 2016 election. (I’m assuming 1 murderer per murder, which isn’t true but it does simplify the calculation.)

    One of those 6800 people (I could have used shootings per capita for an even larger number) shot a member of Congress.

    Will this story, plus or minus hand wringing, accusations, counter-accusations, etc., change your routine tomorrow? Next week? Your plans for this year?

    All I see is novelty and no news.


    PS: Identifying the “novelty” of this story did not require a large research/fact-checking budget. What it did require is a realization that everyone is talking about the shooting of a member of congress means only “everyone is talking about….” Whether that is just a freakish event or genuine news, requires deeper inquiry.

    One nutter shoots a member of Congress, man bites dog, novelty, not news. Organization succeeds in killing 3rd member of Congress, that looks like news. Pattern, behavior, facts, goals, etc.

    The Media and Far Right Trolls – Mutual Reinforcing Exploitation (MRE)

    Thursday, June 15th, 2017

    The Columbia Journalism Review (CJR) normally has great headlines but the editors missed a serious opportunity with: Megyn Kelly fiasco is one more instance of far right outmaneuvering media by Becca Lewis and Alice Marwick.

    Lewis and Marwick capture the essential facts and then lose their key insights in order to portray “the media” (whoever that is) as a victim of far right trolls.

    Indeed, research suggests that even debunking falsehoods can reinforce and amplify them. In addition, if a media outlet declines to cover a story that has widely circulated in the far-right and mainstream conservative press, it is accused of lying and promoting a liberal agenda. Far-right subcultures are able to exploit this, using the media to spread ideas and target potential new recruits.

    A number of factors make the mainstream media susceptible to manipulation from the far-right. The cost-cutting measures instituted by traditional newspapers since the 1990s have resulted in less fact-checking and investigative reporting. At the same time, there is a constant need for novelty to fill a 24/7 news cycle driven by cable networks and social media. Many of those outlets have benefited from the new and increased partisanship in the country, meaning there is now more incentive to address memes and half-truths, even if it’s only to shoot them down.

    Did you catch them? The key insights/phrases?

    1. “…declines to cover a story that has widely circulated…it is accused of lying and promoting a liberal agenda…”
    2. “…less fact-checking and investigative reporting…”
    3. “…constant need for novelty to fill a 24/7 news cycle driven by cable networks and social media…”

    Declining to Cover a Story

    “Far-right subcultures” don’t exploit “the media” with just any stories, they are “…widely circulated…” stories. That is “the media” is being exploited over stories it carries out of fear of losing click-through advertising revenue. If a story is “widely circulated,” it attracts reader interest, page-views, click-throughs and hence, is news.

    Less Fact-Checking and Investigative Reporting

    Lewis and Marwick report the decline in fact-checking and investigative reporting as fact but don’t connect it to “the media” carrying stories promoted by “far-right subcultures.” Even if fact-checking and investigative reporting were available in abundance, for every story, given enough public interest (read “…widely circulated…”), is any editor going to decline a story of wide spread interest? (BTW, who chose to reduce fact-checking and investigative reporting? It wasn’t “far-right subcultures” choosing for “the media.”

    Constant Need for Novelty

    The “…constant need for novelty…” and its relationship to producing income for “the media” is best captured by the following dialogue from Santa Claus (1985)

    How can I tell all the people
    about my something special?
    Advertise. Advertise?
    How do I do that?
    In my line,
    television works best.
    Oh, I know! Those little picture
    box thingies? Can we get on those?
    With enough money, a horse in a
    hoop skirt can get on one of those.

    In the context of Lewis and Marwick, far-right subculture news is the “horse in a hoop skirt” of the dialogue. It’s a “horse in a hoop skirt” that is generating page-views and click-through rates.

    The Missed Headline

    I’m partial to my headline but the CJR aims at a more literary audience, I would suggest:

    The Media and Far Right Trolls – Imitating Alessandro and Napoleone

    Alessandro and Napoleone, currently residents of Hell, are described in Canto 32 of the Inferno (Dante, Ciardi translation) as follows:

    When I had stared about me, I looked down
    and at my feet I saw two clamped together
    so tightly that the hair of their heads had grown

    together, “Who are you,” I said, “who lie
    so tightly breast to breast?” They strained their necks
    and when they had raised their heads as if to reply,

    the tears their eyes had managed to contain
    up to that time gushed out, and the cold froze them
    between the lids, sealing them shut again

    tighter than any clamp grips wood to wood,
    and mad with pain, they fell to butting heads
    like billy-goats in a sudden savage mood.

    “The media” now reports its “butting heads” with “far-right subcultures,” generating more noise, in addition to reports of non-fact-checked but click-stream revenue producing right-wing fantasies.

    Raw FBI Uniform Crime Report (UCR) Files for 2015 (NICAR Database Library)

    Friday, June 9th, 2017

    IRE & NICAR to freely publish unprocessed data by Charles Minshew.

    From the post:

    Inspired by our members, IRE is pleased to announce the first release of raw, unprocessed data from the NICAR Database Library.

    The contents of the FBI’s Uniform Crime Report (UCR) master file for 2015 are now available for free download on our website. The package contains the original fixed-width files, data dictionaries for the tables as well as the FBI’s UCR user guide. We are planning subsequent releases of other raw data that is not readily available online.

    The yearly data from the FBI details arrest and offense numbers for police agencies across the United States. If you download this unprocessed data, expect to do some work to get it in a useable format. The data is fixed-width, across multiple tables, contains many records on a single row that need to be unpacked and in some cases decoded, before being cleaned and imported for use in programs like Excel or your favorite database manager. Not up to the task? We do all of this work in the version of the data that we will soon have for sale in the Database Library.

    I have peeked at the data and documentation files and “raw” is the correct term.

    Think of it as great exercise for when an already cleaned and formatted data set isn’t available.

    More to follow on processing this data set.

    Protecting Sources, Leaks and Journalistic Credibility

    Thursday, June 8th, 2017

    Protecting Your Sources When Releasing Sensitive Documents by Ted Han and Quinn Norton.

    From the post:

    Extraordinary documentation can make for an extraordinary story—and terrible trouble for sources and vulnerable populations if handled without enough care. Recently, the Intercept published a story about a leaked NSA report, posted to DocumentCloud, that alleged Russian hacker involvement in a campaign to phish American election officials. Simultaneously, the FBI arrested a government contractor, Reality Winner, for allegedly leaking documents to an online news outlet. The affidavit partially revealed how Winner was caught leaking by the FBI, including a postmark and physical characteristics of the document that the Intercept posted.

    The Intercept isn’t alone in leaving digital footprints in their article material. In a post called “We Are with John McAfee Right Now, Suckers,” Vice posted a picture of the at-the-time fugitive John McAfee, complete with GPS coordinates pinpointing their source’s location, who was shortly in official custody. In 2014, the New York Times improperly redacted an NSA document from the Snowden trove, revealing the name of an NSA agent.

    The first step with any sensitive material is to consider what will happen when the subjects or public sees that material. It can be hard to pause in the rush of getting a story out, but giving some thought to the nature of the information you’re releasing, what needs to be released, what could be used in unexpected ways, and what could harm people, can prevent real problems.

    Han and Norton cover document metadata, which I omitted in Are Printer Dots The Only Risk? along with some of the physical identifiers I mentioned.

    Plus they have good advice on other identifying aspects of documents, such as content and locations.

    Despite my waiting and calling for a full release of the Panama Papers, is there a credibility aspect to the publication of sensitive documents?

    Another era but had Walter Cronkite said that he read a leaked NSA document and reported the same facts as the Intercept, his report would have been taken as the “facts” contained in that report.

    To what extent is journalism losing credibility because it isn’t asking to be treated as credible? Merely as accurate repeaters of lies prepared and printed elsewhere?

    Verifying Burn Of Source – Who You Gonna Call?

    Tuesday, June 6th, 2017

    WikiLeaks offers $10,000 to get Intercept reporter fired by Joe Uchill.

    From the post:

    WikiLeaks offered a $10,000 bounty Monday aimed at getting a reporter for The Intercept fired, following the arrest of a government contractor who allegedly leaked an NSA report to the site.

    The Justice Department announced earlier Monday that it had arrested Reality Leigh Winner, a 25-year-old government contractor, for leaking classified documents to a news organization. It has been widely reported that Winner allegedly leaked documents from the NSA to The Intercept about Russian attempts to hack U.S. elections officials.

    Investigators were able to find Winner in part, according to a government court filings, because of clues gained when an Intercept reporter showed the leaked report to the government.

    The Intercept article lists four reporters:

    From the affidavit for Reality Leigh Winner’s arrest:

    12. On June I, 2017, the FBI was notified by the U.S. Government Agency that the U.S. Government Agency had been contacted by the News Outlet on May 30, 2017, regarding an upcoming story. The News Outlet informed the U.S. Government Agency that it was in possession of what it believed to be a classified document authored by the U.S. Government Agency. The News Outlet provided the U.S. Government Agency with a copy of this document. Subsequent analysis by the U.S. Government Agency confirmed that the document in the News Outlet’s possession is the intelligence reporting. The intelligence reporting is classified at the Top Secret level, indicating that its unauthorized disclosure could reasonably result in exceptionally grave damage to the national security, and is marked as such. The U.S. Government Agency has since confirmed that the reporting contains information that was classified at that level at the time that the reporting was published on or about May 5, 2017, and that such information currently remains classified at that level.

    13. The U.S. Government Agency examined the document shared by the News Outlet and determined the pages of the intelligence reporting appeared to be folded and/or creased, suggesting they had been printed and hand-carried out of a secured space.

    Why on earth?:

    The News Outlet provided the U.S. Government Agency with a copy of this document.

    How sloppy is that?

    Do you trust the “U.S. Government Agency” given a copy of the document to out the reporter in question?

    Or does this give them a free shot at a good reporter and blackmail evidence on the real culprit?


    How NOT To Leak! (Educational Materials on Leaking?)

    Tuesday, June 6th, 2017

    The Intercept’s Russian hacking report also seems to be a good example of how not to handle leaks by Laura Hazard Owen.

    From the post:

    On Monday afternoon, The Intercept published a bombshell story: “Top-secret NSA report details Russian hacking effort days before 2016 election.” The story — later confirmed by CBS — reveals that “Russian military intelligence executed a cyberattack on at least one U.S. voting software supplier and sent spear-phishing emails to more than 100 local election officials just days before last November’s presidential election, according to a highly classified intelligence report obtained by The Intercept,” and includes PDFs of the NSA’s report.

    The story is a potentially huge one, providing the most evidence we’ve seen thus far that the Russian government attempted to influence the outcome of the U.S. election in ways beyond just spreading misinformation (and Russian president Vladimir Putin had even denied his government’s role in that). But another story is emerging around The Intercept’s story as well: By Monday evening, a 25-year-old federal contractor, Reality Leigh Winner, was charged with leaking the documents (the first criminal leak case under Trump). If Winner was indeed The Intercept’s source, there are questions about whether The Intercept could have done more to protect her — starting with those PDFs it published as part of its story.

    FYI, the Intercept has a huffy denial at the end of Owen’s post. Huffy enough to confirm they screwed up.

    In the rush to publication, the Intercept failed to observe basic information hygiene with regard to the leaked PDFs. Leaked PDFs included printer steganography that enables tracing the printer.

    Numerous other failures, such as the alleged source using their work computer to leak the documents, etc., were also present.

    Enough errors, between the Intercept and its alleged source, to make you think dead pages advising on how to leak properly aren’t enough.

    Suggestions on how to effectively educate people on proper leaking techniques?

    New York Times Mutes Public Editor (And Effective Criticism)

    Thursday, June 1st, 2017

    New York Times public editor Liz Spayd on decision to eliminate her position by Pete Vernon.

    From the post:

    THE DECISION THIS MORNING BY THE NEW YORK TIMES to eliminate the position of public editor touched off a debate over the value of a position established in the wake of the Jayson Blair fabrication scandal to hold the paper’s editors and reporters accountable to industry standards and reader concerns.

    Times Publisher Arthur Sulzberger, Jr. explained the move in a memo to staff: “The responsibility of the public editor—to serve as the reader’s representative—has outgrown that one office.”

    According to Sulzberger, “When our audience has questions or concerns, whether about current events or our coverage decisions, we must answer them ourselves.” To that end, the paper will rely on an expanded comment section and social media feedback, as well as a new “reader center,” which was announced yesterday.

    Relying on social media critiques and angry voices in the comment sections is a curious way of replacing an experienced journalist who could offer nuance and perspective while writing with the institutional backing of the nation’s most influential newspaper. The move comes at a moment when public confidence in the media is at an all-time low. In a time when the value of introspection and transparency is at a premium, cutting a position designed to provide both smacks of self-satisfaction and a misreading of the current media landscape.
    … (emphasis in original)

    Vernon’s post deserves your attention but the adage:

    Everyone’s Responsibility Is No One’s Responsibility

    answers Sulzberger the best.

    Can you name a single reader of the New York Times for who holding reporters and the editorial process of the New York Times (NYT) accountable is their day job??

    That’s the trick isn’t it?

    If it’s not your day job, with resources commensurate to the task and access, how will you “hold” the New York Times accountable?

    Will you post to Facebook or Twitter? Exactly how many people do you think will see/consider your “speaking truth to power?”

    The public editor, publishing in the NYT, had a voice at least as loud as the editors and reporters.

    That was Sulzberger’s real problem with the public editor. He wants the appearance of accountability but not its reality. Critics should be unfunded, isolated, powerless voices that can be easily ignored.

    Sulzberger needs to go down on your list of enemies of journalism and the public in general.

    Data journalists, start tracking NYT contents for your degradation of journalism stories two or three years hence. (I not presuming an outcome of silencing the public editor, that’s a forecast.)

    Data Journalists! Data Gif Tool (Google)

    Monday, May 29th, 2017

    While not hiding its prior salary discrimination against women, Google has created and released a tool for creating data gifs.

    Make your own data gifs with our new tool by Simon Rogers.

    From the post:

    Data visualizations are an essential storytelling tool in journalism, and though they are often intricate, they don’t have to be complex. In fact, with the growth of mobile devices as a primary method of consuming news, data visualizations can be simple images formatted for the device they appear on.

    Enter data gifs.

    (gif omitted)

    These animations can be used for a variety of sophisticated storytelling approaches among data journalists: one example is Lena Groeger, who has become *the* expert in working with data gifs.

    Today we are releasing Data Gif Maker, a tool to help journalists make these visuals, which show share of search interest for two competing topics.

    A good way to get your feet wet with simple data gifs.

    Don’t be surprised that Google does good things for the larger community while engaging in evil conduct.

    Racists sheriffs who used water cannon and dogs on Black children loved their own children and remembered their birthdays. WWII death camps guards attended church. Were kind to small animals.

    People and their organizations are complicated and the reading public is ill-served by shallow reporting of only one aspect or another as the “true” view.

    How Not To Be Wrong

    Thursday, May 25th, 2017

    How Not To Be Wrong by Winny de Jong.

    From the post:

    At the intersection of data and journalism, lots can go wrong. Merely taking precautions might not be enough.

    “It’s very well possible that your story is true but wrong,” New York Times data journalist Robert Gebeloff explained at the European Investigative Journalism Conference & Dataharvest, which was recently held in Mechelen, a city 20 minutes outside of Brussels.

    “When I work on a big story, I want to know everything about the topic.” To make sure he doesn’t miss out, Gebeloff gets all the data sources he can, examines it in all relevant ways and publishes only what he believes to be true.

    The best part of this post is the distillation of Gebeloff’s presentation into a How Not To Be Wrong Checklist.

    De Jong’s checklist is remarkably similar to requirements for replication of experiments in science.

    It would make a great PDF file to share with data scientists in general.

    Leaking Photos Of: “Sophisticated Bomb Parts”

    Wednesday, May 24th, 2017

    Theresa May to tackle Donald Trump over Manchester bombing evidence by Heather Stewart, Robert Booth and Vikram Dodd.

    From the post:

    British officials were infuriated on Wednesday when the New York Times published forensic photographs of sophisticated bomb parts that UK authorities fear could complicate the expanding investigation into the lethal blast in which five further arrests have been made in the UK and two more in Libya.

    See for yourself: Found at the Scene in Manchester: Shrapnel, a Backpack and a Battery by C. J. Chivers.

    Let’s see, remains of a backpack, detonator, metal scrap, battery.

    Do you see any sophisticated bomb parts?

    Sophistication, skill, encryption, etc., are emphasized after terrorist attacks, I assume to excuse the failure of authorities to prevent such attacks.

    That’s more generous than assuming UK authorities are so untrained they consider this a “sophisticated” bomb. Just guessing from the parts, hardly.

    “Click Bait” at The Kicker – Covering Manchester

    Wednesday, May 24th, 2017

    The Kicker: The media’s model for covering terrorist attacks is broken by Pete Vernon.

    From the webpage:

    ON THE LATEST EPISODE of The Kicker, we run through some of the week’s biggest media stories, including a ratings leaderboard shakeup for cable news, a spurious conspiracy that consumed the right-wing media universe, and a new study that says–surprise–journalists drink too much caffeine and alcohol. Then, we move on to the media coverage of the terrorist attack in Manchester, and tackle why we think the industry’s model for covering terror attacks is broken. Finally, CJR’s David Uberti interviews Clara Jeffery, editor in chief of Mother Jones. They discuss the magazine’s novel approach to funding its political coverage as well as the role Mother Jones played in breaking the Trump-Russia story.

    Subscribe via iTunes · Stitcher · RSS Feed · SoundCloud.

    The podcast.

    Leading with the promise of The media’s model for covering terrorist attacks is broken, I listened to The Kicker today.

    If you like podcasts, you will like The Kicker, but it illustrates for me the difficulties associated with podcasts.

    First, the podcast covered five separate stories in a little over thirty minutes. Ranging from cable news ratings, Seth Rich and fake news, the drinking habits of journalists, the media model for terrorist coverage (the story of interest to me), and the role of Mother Jones in the continuing From Russia With Love connection to Donald Trump.

    As “click bait” for the podcast, the media reporting on terrorism segment starts at approximately 8:20 and ends at approximately 16:50, some 8 minutes and 30 seconds of coverage, much shorter than the account concerning Mother Jones (16:49 – 31:14).

    Second, what discussion occurred, included insights such as “…breaking news rooms, larger news rooms, don’t have the privilege of deciding whether to cover a story…?” To be fair, that was followed by discussions of “how to cover stories,” the use of raw/unexplained user video, and the appropriateness of experts discussing politics immediately following such events.

    The point that got dropped in the podcast was Christie Chisholm‘s remark:

    …breaking news rooms, larger news rooms, don’t have the privilege of deciding whether to cover a story…

    Why so?

    I may be reading entirely too much into Christie’s comment, but it implies that some news rooms must fill N minutes of coverage on breaking events, whether there is meaningful content to be delivered or not. Yes?

    If that is the case, that coverage of breaking events requires wall-to-wall coverage for N minutes, then raw, unexplained video, expert opinions with no facts, reporters asking for each others reactions, the spontaneous speculation and condemnations, become easily explainable.

    There is too little content and too much media time available to cover it.

    Building on Christie’s insight, The Kicker could have created a timeline of “facts” with regard to the explosion in Manchester as a way to illustrate when facts became known about the explosion and contrast that with the drone of factless coverage of the event.

    That would have made a rocking podcast and a pointed one at that.

    PS: The podcast did discuss other issues with media coverage of Manchester but the lack of depth and time prevented substantive analysis or proposals. Media coverage of terrorist events certainly merits extended treatment by podcast or otherwise.

    The power of algorithms and how to investigate them (w/ resources)

    Tuesday, May 23rd, 2017

    The power of algorithms and how to investigate them by Katrien Vanherck.

    From the post:

    Most Americans these days get their main news from Google or Facebook, two tools that rely heavily on algorithms. A study in 2015 showed that the way a search engine like Google selects and prioritises search results on political candidates can have an influence on voters’ preferences.

    Similarly, it has been shown that by tweaking the algorithms behind the Facebook newsfeed, the turnout of voters in American elections can be influenced. If Marc Zuckerberg were ever to run for president, he would theoretically have an enormously powerful tool at his disposal. (Note: as recent article in The Guardian investigated the misuse of big data and social media in the context of the Brexit referendum).

    Algorithms are everywhere in our everyday life and are exerting a lot of power in our society. They prioritise, classify, connect and filter information, automatically making decisions on our behalf all the time. But as long as the algorithms remain a ‘black box’, we don’t know exactly how these decisions are made.

    Are these algorithms always fair? Examples of possible racial bias in algorithms include the risk analysis score that is calculated for prisoners that are up for parole or release (white people appear to get more favourable scores more often) and the service quality of Uber in Washington DC (waiting times are shorter in predominantly white neighbourhoods). Maybe such unfair results are not only due to the algorithms, but the lack of transparency remains a concern.

    So what is going on in these algorithms, and how can we make them more accountable?
    … (emphasis in original)

    A great inspirational keynote but short on details for investigation of algorithms.

    Such as failing to mention the algorithms of both Google and Facebook are secret.

    Reverse engineering those from results would be a neat trick.

    Google would be the easier of the two, since you could script searches domain by domain with a list of search terms to build up a data set of its results. That would not result in the algorithm per se but you could detect some of its contours.

    Google has been accused of liberal bias, Who would Google vote for? An analysis of political bias in internet search engine results, bias in favor of Hillary Clinton, Google defends its search engine against charges it favors Clinton, and, bias in favor of the right wing, How Google’s search algorithm spreads false information with a rightwing bias.

    To the extent you identify Hillary Clinton with the rightwing, those results may be expressions of the same bias.

    In any event, you can discern from those studies some likely techniques to use in testing Google search/auto-completion results.

    Facebook is be harder because you don’t have access to or control over the content it is manipulating for delivery. Although by manipulating social media identities, you could test and compare the content that Facebook delivers.

    Breaking News Consumer’s Handbook

    Monday, May 22nd, 2017

    From a tweet by @onthemedia, see their website:

    If you follow #2:

    2. Don’t trust anonymous sources.

    Skip political reports in the New York Times and Washington Post.

    Is there a market for delayed news?

    I ask because I understand there was an explosion in Manchester Arena in England, 10:35 PM their local. Even as I type this, mis-information is flooding social media channels from any number of sources.

    What if there was a news service with a variable delay, say minimum 7 days but maximum of 14 days, that delivered a coherent and summarized version of breaking events?

    As opposed to the click-bait teasers that get shared/forwarded/re-tweeted without anyone reading the mis-information behind the click-bait.

    Global Investigative Journalism Network: Russian Feed

    Sunday, May 21st, 2017

    Global Investigative Journalism Network has added a Twitter feed in Russian: @gijnRu!

    Great way for journalists to learn/reinforce their skills with Russian.

    You can rely on The New York Times or the Washington Post as primary sources for the next 1339 days (as of today, Trump presidency) or you can strike out on your own.

    As an editor, I would tire pretty quickly of “…as reported in NYT/WaPo….”


    Python for Data Journalists: Analyzing Money in Politics

    Friday, May 19th, 2017

    Python for Data Journalists: Analyzing Money in Politics by Knight Center.

    From the webpage:

    Data journalists are the newest rock stars of the newsroom. Using computer programming and data journalism techniques, they have the power to cull through big data to find original and important stories.

    Learn these techniques and some savvy computer programming to produce your own bombshell investigations in the latest massive open online course (MOOC) from the Knight Center, “Python for Data Journalists: Analyzing Money in Politics.”

    Instructor Ben Welsh, editor of the Los Angeles Times Data Desk and co-founder of the California Civic Data Coalition, will show students how to turn big data into great journalism with speed and veracity. The course takes place from June 12 to July 9, 2017, so register now.

    A high priority for your summer because:

    1. You will learn techniques for data analysis
    2. Learning #1 enables you to perform data analysis
    3. Learning #1 enables you to better question data analysis

    I skimmed the post and did not see any coverage of obtaining concealed information.

    Perhaps that will be the subject of a wholly anonymous MOOC. 😉

    Do register! This looks like useful and fun!

    PS: Developing a relationship with a credit bureau or bank staffer should be an early career goal. No one is capable of obtaining “extra” money and just sitting on it forever.

    Media Manipulation and Disinformation Online – Overview

    Thursday, May 18th, 2017

    Media Manipulation and Disinformation Online.

    The quick summary from the webpage:

    “the spread of false or misleading information is having real and negative effects on the public consumption of news.”

    • Internet subcultures take advantage of the current media ecosystem to manipulate news frames, set agendas, and propagate ideas.
    • Far-right groups develop techniques of “attention hacking” to increase the visibility of their ideas through the strategic use of social media, memes, and bots—as well as by targeting journalists, bloggers, and influencers to help spread content.
    • The media’s dependence on social media, analytics and metrics, sensationalism, novelty over newsworthiness, and clickbait makes them vulnerable to such media manipulation.
    • While trolls, white nationalists, Men’s Rights Activists, gamergaters, the “alt-right,” and conspiracy theorists may diverge deeply in their beliefs, they share tactics and converge on common issues.
    • The far-right exploits young men’s rebellion and dislike of “political correctness” to spread white supremacist thought, Islamophobia, and misogyny through irony and knowledge of internet culture.
    • Media manipulation may contribute to decreased trust of mainstream media, increased misinformation, and further radicalization.

    The full report, Media Manipulation and Disinformation Online by Alice Marwick and Rebecca Lewis.

    A useful report but know up front that its concern is very much agenda driven. The following terms occur in the text, alt-right (89), racists (44), white supremacists (30), without treatment of similar groups but of different agendas.

    I think the aforementioned groups are loathsome but when treating media manipulation/disinformation, a broader sampling would be more instructive.

    There are extensive footnotes and a great bibliography if you are interested in reading further.

    As an overview of the issues of media manipulation/disinformation, I don’t think I have seen a better one.

    Suggestions of more detailed case study collections?

    Local News: Willing Buyers/Willing Sellers

    Thursday, May 11th, 2017

    The reference to We Interrupt This Newscast and the quote being from @cjr, was enough to get me to read: Oy, the TRAFFIC. And it’s POURING! Do I hear SIRENS? by Simon Van Zuylen-Wood.

    First things first, We Interrupt This Newscast: How to Improve Local News and Win Ratings, Too, mentioned without linking by Zuylen-Wood, can be found at: (link is to WorldCat which will display copies held at libraries close to you).

    Second, Zuylen-Wood’s definition of the “problem” of local TV news:

    Local TV news has a problem. Broadcasts are dominated by sensationalistic crime stories, weather reports, and human-interest puff pieces. The format—two plasticky news anchors reading from teleprompters—has not meaningfully changed in 40 years. The end product tends to be irrelevant journalism packaged in an increasingly irrelevant way. The problem isn’t that the product is partisan or under-resourced or “fake.” The problem is that it’s lame.

    isn’t shared by local TV news directors:

    “You’d be hard-pressed to find a news director who isn’t saying we need to be [innovating],” Bob Papper says. And yet, none of them really is. “The biggest hindrance to innovation,” he continues, “is the success of TV news. The fact is, it’s doing well, and if anything it may be doing better and better. That’s not an impetus to change.”

    Zuylen-Woods’ definition is not shared by local TV news viewers, who lap up stories of random, non-repeatable events. Generally speaking people are murdered only once, photogenic teenagers die in automobile accidents only once, tree limbs turn toddlers into life long medical cases only occassionally, although TV news does milk those stories for weeks, months, even years.

    Zuylen-Woods is right, local TV news is “lame,” but it’s a product tailored to the taste of willing customers.

    If “willing customers” are buying “irrelevant journalism” (Zuylen-Woods’ term), I don’t see the obligation of the media to create products, one assumes “relevant journalism,” for which there is no market.

    If you do, is it because you are a better judge of what the public should be reading, viewing, discussing?


    Beginning with Plato, if not earlier, prescribing better, more appropriate content for others, censorship, has an unhappy history.

    Tools and Resources to Help Facts Keep Pace with Fake News (June 30, 2017 Deadline)

    Wednesday, May 10th, 2017

    Tools and Resources to Help Facts Keep Pace with Fake News by Oren Levine.

    From the post:

    When fake news moves fast, you need the right tools and resources to help the truth keep pace.

    To inspire you to enter TruthBuzz: The viral fact-checking contest, we have collected some useful tools, along with resources that shed light on fake and misleading news and information, and how it spreads online.

    During our recent TruthBuzz webinar, my fellow contest judges, Aimee Rinehart, Shaheryar Popalzai and I shared several resources and tools that could be useful in helping you craft your TruthBuzz entry. We’ve also rounded those up here:

    Enabling people to decide for themselves what is or is not “fake news,” gets my full support.

    Filtering or suppressing “fake news” requires others to determine fake/not fake and is censorship whatever other label you want to use.

    The resources listed can be helpful and the contest, TruthBuzz: The viral fact-checking contest, does have a $10K, $5K and $2.5K prizes.

    FOIA Data Models for Everyone [If You Are Going To Ask]

    Monday, May 8th, 2017

    FOIA Data Models for Everyone by Jeremy B. Merrill.

    From the post:

    Listen to two FOIA practitioners describe their request strategies and you’ll probably get two very different answers. I know because I’ve done it. As someone with not much of a personal FOIA strategy—besides “wait and hope”—I was surprised that journalists skilled at prying obscure records from the government have wildly different approaches.

    These differences in how to engage with the FOIA process can cover questions that are flashy—to us nerds—like whether to ask for “any and all” documents or to call the officer every week or so. But the idiosyncrasies in journalists’ mental models trickle down even into the little details, like how they keep track of agencies’ contact info.

    When I began an internal FOIA tracker app for the New York Times, I knew I’d have to understand different mental models of the FOIA process in order to represent that process in a database. So, I put out a call to the friendly community of news nerds on Twitter and in the NewsNerdery Slack:

    Tracking your FOIAs with a spreadsheet (or an app) is a best practice. But everyone’s chart is a little different and probably encodes different nuggets of hard-earned wisdom. Care to share the column headers from your spreadsheet?

    Computers don’t know anything about FOIAs. Bless their hearts, but they’re dumb; data modeling is how we imbue computers with little morsels of our human wisdom hidden in row 1 of a spreadsheet. I collated the results—from eight individuals’ spreadsheets and two open-source FOIA tracker apps plus my own, so hopefully a lot of little morsels of wisdom—and analyzed them to see what I might have missed. I want to share the results back to the community.

    A gold mine of curated advice and practices on FOIA data models.

    Old pros and newbies at FOIA requests are going to benefit from Merrill’s post.

    Be sure to ping him @jeremybmerrill to show your appreciation for this summary.

    Tackling “Fake News” (So You Don’t Have To, How Nice)

    Monday, May 8th, 2017

    A Global Guide to Initiatives Tackling “Fake News” by Fergus Bell.

    From the post:

    Here’s a list of initiatives that hope to fix trust in journalism and tackle “fake news”.

    There’s a lot.

    I’ve tried to collect an extensive list of projects, initiatives and tools created to fix trust in journalism and false/fake news and misinformation. This also includes efforts and initiatives around verification. Where possible I’ve also tried to attach where the funding has come from for each initiative.

    A great resource for tracking efforts with the self-appointed goal of:

    Protecting you from “fake news.”

    The arrogance of such efforts is almost palpable. They can recognize “fake news” but millions of benighted souls on the Internet are victims in waiting.

    I have a great deal of sympathy for the efforts to teach readers how to evaluate information, the source of its reporting and consistency with other sources of information.

    However, efforts like that of Google, are an attempt to privilege certain narratives with an imprimatur of truth.

    Skip to the “guides” section of Bell’s post and preserve your own judgment in the face of the hue and cry over “fake news.”

    The New York Times — Glory Days

    Sunday, May 7th, 2017

    Hell hath no fury like The New York Times scorned by Hollywood by Thomas Vinciguerra.

    From the post:

    GOD, IT’S BEEN SAID, makes a lousy playwright. As far as an upcoming film that spotlights the Pentagon Papers is concerned, though, The New York Times is seething not at the Almighty but at the producers.

    In March it was announced that Steven Spielberg would direct The Post, which offers as its backdrop the dramatic story of how the press exposed the federal government’s infamous secret history of the Vietnam War. Liz Hannah, who studied at the American Film Institute, sold her spec script to former Sony co-chair Amy Pascal’s production company last fall. Meryl Streep and Tom Hanks, Variety reported, are “attached to star” as Washington Post publisher Katharine Graham and executive editor Ben Bradlee.

    But it was The New York Times—not the Washington Post—that broke the Pentagon Papers story. It is the Times whose name is on the landmark 1971 Supreme Court case that affirmed the right to publish the classified documents. And it was the Times that won the 1972 Pulitzer Prize for meritorious public service for its labors.

    Nonetheless, as its title implies, the Spielberg project emphasizes the ancillary role of the Post. Not unexpectedly, Times people from back in the day are incensed.
    … (emphasis in original)

    Any mention of The New York Times and the Pentagon Papers brings Glory Days by Bruce Springsteen, E Street Band to mind:

    I had a friend was a big baseball player
    Back in high school
    He could throw that speedball by you
    Make you look like a fool boy
    Saw him the other night at this roadside bar
    I was walking in, he was walking out
    We went back inside sat down had a few drinks
    But all he kept talking about was

    Glory days, well, they’ll pass you by
    Glory days, in the wink of a young girl’s eye
    Glory days, glory days

    Well there’s a girl that lives up the block
    Back in school she could turn all the boy’s heads
    Sometimes on a Friday I’ll stop by
    And have a few drinks after she put her kids to bed
    Her and her husband Bobby well they split up
    I guess it’s two years gone by now
    We just sit around talking about the old times,
    She says when she feels like crying
    She starts laughing thinking about

    Glory days, well, they’ll pass you by
    Glory days, in the wink of a young girl’s eye
    Glory days, glory days

    To be sure, The New York Times (NYT) broke the story, fought for the right to publish up to the Supreme Court, but what has the NYT done for you or journalism lately?

    The NYT, along with others, did publish the Afghan War Diaries, although sanitized as described by Bill Keller:

    We used that month to study the material, try to assess its value and credibility, weigh it against our own reporters’ experience of the war and against other sources, and then tell our readers what it all meant. In doing so, we took great care both to put the information in context and to excise anything that would put lives at risk or jeopardize ongoing military missions.

    What does that mean in practice? Obviously we did not disclose the names of Afghans, except for public officials, who have cooperated with the war effort, either in our articles or in the selection of documents we posted on our own Web site. We did not disclose anything that would compromise intelligence-gathering methods. We erred, if at all, on the side of prudence. For example, when a document reported that a certain aircraft left a certain place at a certain time and arrived at another place at a certain time, we omitted those details on the off chance that an enemy could gain some small tactical advantage by knowing the response time of military aircraft.

    The administration, while strongly condemning WikiLeaks for making these documents public, did not suggest that The Times should not write about them. On the contrary, in our discussions prior to the publication of our articles, White House officials, while challenging some of the conclusions we drew from the material, thanked us for handling the documents with care, and asked us to urge WikiLeaks to withhold information that could cost lives. We did pass along that message.

    Journalists have a role in supporting “…ongoing military missions[?]”

    Pointers to any school of journalism that teaches that role? (Thanks!)

    Worse, Keller describes the NYT consulting with and acting as a surrogate for the US government in urging Wikileaks to withhold information.

    Isn’t withholding information contrary to creating an informed public?

    Crowd-funding opportunity: Francis Ford Coppola directs: From Government Watchdog to Mouthpiece – The New York Times

    Introduction: The New Face of Censorship

    Saturday, May 6th, 2017

    Introduction: The New Face of Censorship by Joel Simon.

    From the post:

    In the days when news was printed on paper, censorship was a crude practice involving government officials with black pens, the seizure of printing presses and raids on newsrooms. The complexity and centralization of broadcasting also made radio and television vulnerable to censorship even when the governments didn’t exercise direct control of the airwaves. After all, frequencies can be withheld; equipment can be confiscated; media owners can be pressured.

    New information technologies–the global, interconnected internet; ubiquitous social media platforms; smart phones with cameras–were supposed to make censorship obsolete. Instead, they have just made it more complicated.

    Does anyone still believe the utopian mantras that information wants to be free and the internet is impossible to censor or control?

    The fact is that while we are awash in information, there are tremendous gaps in our knowledge of the world. The gaps are growing as violent attacks against the media spike, as governments develop new systems of information control, and as the technology that allows information to circulate is co-opted and used to stifle free expression.

    The work of Joel Simon and the Committee to Protect Journalists is invaluable. The challenges, dangers and hazards for journalists around the world are constant and unrelenting.

    I have no doubt about Simon’s account of suppression of journalists. His essay is a must read for everyone who opposes censorship, at least in its obvious forms.

    A more subtle form of censorship is practiced in the United States, self-censorship.

    How many stories on this theme have you read in the last couple of weeks? U.S. spy agency abandons controversial surveillance technique

    Now, how many of those same stories mentioned that the NSA has a long and storied history of lying to the American public, presidents and congress?

    By my count, which wasn’t exhaustive, the total is 0.

    Instead of challenging this absurd account, Reuters reports the NSA reports as though it were true and fails to remind the public it is relying on a habitual liar.

    Show of hands, how many readers think the Reuters staff forgot that the NSA is a hotbed of liars and cheats?

    There is little cause for government censorship of US media outlets. They censor themselves before the government can even ask.

    Support the Committee to Protect Journalists and perhaps their support of journalists facing real censorship will shame US media into growing a spine.

    What’s Your Best Star Wars Line?

    Thursday, May 4th, 2017

    Ben Child has gathered the forty (40) best lines from the Star Wars saga in: May the 4th be with you: the 40 best lines from the Star Wars saga.

    No spoilers here!

    Read Ben’s post and support the Guardian while you are there.

    Seriously, do support the Guardian. They’re a bit conservative for my tastes but still worthy of support.

    Practical Suggestions For Improving Transparency

    Tuesday, May 2nd, 2017

    A crowd wail about Presidents Obama, Trump, opacity, lack of transparency, loss of democracy, freedom of the press, the imminent death of civilization, etc., isn’t going to improve transparency.

    I have two practical suggestions for improving transparency.

    First suggestion: Always re-post, tweet, share stories with links to leaked materials. If the story you read doesn’t have such a link, seek out one that does to re-post, tweet, share.

    Some stories of leaks include a URL to the leaked material, like Hacker leaks Orange is the New Black new season after ransom demands ignored by Sean Gallagher, or NSA-leaking Shadow Brokers just dumped its most damaging release yet by Dan Goodin, both of Ars Technica

    Some stories of the same leaks do not include a URL to the leaked material,The Netflix ‘Orange is the New Black’ Leak Shows TV Piracy Is So 2012 (which does have the best strategy for fighting piracy I have ever read) or, Shadow Brokers leak trove of NSA hacking tools.

    Second suggestion: If you encounter leaked materials, post, tweet and share them as widely as possible. (Translations are always needed.)

    Improving transparency requires only internet access and the initiative to do so.

    Are you game?

    Journalism Is Skepticism as a Service (SaaS)

    Saturday, April 22nd, 2017

    Image from the Fourth Estate Journalism Association.

    I applaud the sentiment and supporting the Fourth Estate is one way to bring it closer to reality.

    At the same time, unless and until The New York Times, National Public Radio, and others start reporting US terrorist attacks (bombings) with the same terminology as so-called “terrorists” in their coverage, “Journalism Is Skepticism as a Service (SaaS)” remains an aspiration, not a reality.

    Tiny Narratives – Upgrade Your Writing

    Thursday, April 13th, 2017

    8 steps to upgrade your everyday news stories with ‘tiny narratives’ by Katia Savchuk.

    From the post:

    BEFORE BETH SCHWARTZAPFEL became a staff writer for The Marshall Project three years ago, she spent a decade as a freelance magazine writer. She got used to spinning 4,000-word narratives for places like Mother Jones and the Boston Review. When she arrived at the nonprofit newsroom, which covers criminal justice, Schwartzapfel found herself tackling an entirely different animal: breaking news and hard-hitting features that put the facts center stage.

    Schwartzapfel considered how she could bring her storytelling chops to these new formats. Her answer was what she calls “tiny narratives”: compact anecdotes, sometimes only a few lines long, scattered throughout a fact-driven article. “I think of them as raisins in oatmeal, or the signs people hold on the sidelines of a marathon. They’re little surprises or jolts of pleasure to remind people of what they’re reading and why it matters,” she explained in a session at the Power of Narrative Conference at Boston University in late March.

    Those nuggets of humanity can help keep readers on the page at a time when news organizations are scrambling for the public’s attention. But it isn’t easy to do well. Injecting narrative elements into a news or investigative story can bring unnecessary clutter or overwhelm the essential facts.

    Here are tips from Schwartzapfel and other speakers at the conference about how to get “tiny narratives” right.
    … (emphasis in original)

    A series of great tips, but if you want more examples of Schwartzapfel’s writing, try Beth Schwartzapfel, Staff Writer.

    I count fifty-five (55) stories.

    More than enough for a Hunter Thompson exercise of re-typing great stories:

    Posted by Brian John Spencer in Hunter S. Thompson – Typing out the work of the best writers.

    Think of Thompson’s approach as developing “muscle and verbal cadence” memory.

    I’m much more likely to try that with Schwartzapfel’s stories than with XQuery, but it would be an interesting exercise in both cases.