Another Word For It Patrick Durusau on Topic Maps and Semantic Diversity

November 10, 2016

Here’s to the return of the journalist as malcontent

Filed under: Journalism,News,Reporting — Patrick Durusau @ 6:39 pm

Here’s to the return of the journalist as malcontent by Kyle Pope.

From the post:

JOURNALISM’S MOMENT of reckoning has arrived.

Its inability to understand Donald Trump’s rise over the last year, ending in his victory Tuesday night, clearly stand among journalism’s great failures, certainly in a generation and probably in modern times.

Reporters’ eagerness first to ridicule Trump and his supporters, then dismiss them, and finally to actively lobby and argue for their defeat have led us to a moment when the entire journalistic enterprise needs to be rethought and rebuilt. In terms of bellwether moments, this is our anti-Watergate.

Already the finger-pointing deconstructions have begun. Yes, social media played a role, enclosing reporters in echo chambers that made it hard, if not impossible, for them to hear contrarian voices; yes, the brutal economics of the news business hurt all our efforts, decimating newsrooms around the country and leaving fewer people to grapple with what was a gargantuan story; and yes, reporters can be forgiven, at least initially, for laughing off a candidate whose views and personality seemed so outside the norm of a serious contender for the White House.

While all those things are true, journalism’s fundamental failure in this election, its original sin, is much more basic to who we are and what we are supposed to be. Simply put, it is rooted in a failure of reporting.

(emphasis in original)

You should read this essay at the start of everyday. Even after opposition to and suspicion of every government, corporation or other statement is second nature.

It’s ironic that Pope points out:

[Trump] already has made clear that he is no friend of the press.

True enough but the press has made it clear it is the friend of government, for several administrations.

Regaining the trust of the public is going to be a long and hard slough.

Bursting Your News Bubble

Filed under: Journalism,News,Reporting — Patrick Durusau @ 11:55 am

It would not have helped the Clinton clones (a sense of entitlement makes people tone deaf and fact blind) but C.J. Adams and Izzie Zahorian explore a way to “see” news beyond your usual news bubble.

In If you are reading this, we might be in the same news bubble they write:

In Myanmar we met two journalists who, during a period of military rule, had smuggled newspapers in duffel bags to carry news between their country and the outside world. Their story stuck with us as a sort of personal challenge: these reporters had regularly risked their lives to read a just a few pages of news from outside their country; while we, with all our connectivity, rarely make the effort to do the same.

Even with the power of the internet, it can be surprisingly difficult to explore the diversity of global perspectives. Technology has made it easier for everyone share information, but it hasn’t made us better at finding viewpoints that are distant from our own. In some ways, a duffel bag full of newspapers would include a wider range of perspectives than many of us see on a daily basis.

Search engines, social media and news aggregators are great at surfacing information close to our interests, but they are limited by the set of topics and people we choose to follow. Even if we read multiple news sources every day, what we discover is defined by the languages we are able to read, and the topics that our sources decide to cover. Ultimately, these limitations create a “news bubble” that shapes our perspective and awareness of the world. We often miss out on the chance to connect and empathize with ideas beyond these boundaries.

How to “see” news without your usual filters?


We’ve just released a new experiment related to this idea: a data visualization called Unfiltered.News. The viz uses Google News data to show what the daily news topics are being published in every region. Headlines for these topics can be viewed from around the world, with translations provided in 40 languages. We hope the viz can become a useful tool to explore what shapes our different perspectives, and to help users discover topics and viewpoints they would have otherwise missed.

Push this one up to the top of your “sites/technology to explore” stack!

I’m having a mixed experience on Ubuntu 14.04. Chrome fails altogether, no support for WebGL. Mozilla displays the side bar of headlines but not the graph like presentation of stories.

I also tried to load the site on Windows 7 with IE and got no joy.

Understandable (but disappointing) that the site may be optimized for Windows but to exclude Chrome?

It’s a great idea, hopeful that during this beta shakedown that it becomes more widely accessible.

November 2, 2016

Does Verification Matter? Clinton/Podesta Emails Update

Filed under: Data Mining,Hillary Clinton,News,Reporting — Patrick Durusau @ 12:51 pm

As of today, 10,357 DKIM Verified Clinton/Podesta Emails (of 43,526 total). That’s releases 1-26.

I ask “Does Verification Matter?” in the title to this post because of the seeming lack of interest in verification of emails in the media. Not that it would ever be a lead, but some mention of the verified/not status of an email seems warranted.

Every Clinton/Podesta story mentions Antony Weiner’s interest in sharing his sexual insecurities and nary a peep about the false Clinton/Obama/Clapper claims that emails have been altered. Easy enough to check. But no specifics are given or requested by the press.

Thanks to the Clinton/Podesta drops by Michael Best, @NatSecGeek, I have now uploaded:

DKIM-verified-podesta-1-26.txt.gz is a sub-set of 10,357 emails that have been verified by their DKIM keys.

The statements in or data attached to those emails may still be false. DKIM verification only validates the email being the same as when it left the email server, nothing more.

DKIM-complete-podesta-1-26.txt.gz is the full set of Podesta emails to date, some 43,526, with their DKIM results of either True or False.

Both files have these fields:

ID – 1| Verified – 2| Date – 3| From – 4| To – 5| Subject -6| Message-Id – 7

Enjoy!

PS: Perhaps verification doesn’t matter when the media repeats false and/or delusional statements of DNI Clapper in hopes of…, I don’t know what they are hoping for but I am hoping they are dishonest, not merely stupid.

October 31, 2016

9,477 DKIM Verified Clinton/Podesta Emails (of 39,878 total (today))

Filed under: Data Mining,Email,Hillary Clinton,News,Reporting — Patrick Durusau @ 3:19 pm

Still working on the email graph and at the same time, managed to catch up on the Clinton/Podesta drops by Michael Best, @NatSecGeek, at least for a few hours.

DKIM-verified-podesta-1-24.txt.gz is a sub-set of 9,477 emails that have been verified by their DKIM keys.

The statements in or data attached to those emails may still be false. DKIM verification only validates the email being the same as when it left the email server, nothing more.

DKIM-complete-podesta-1-24.txt.gz is the full set of Podesta emails to date, some 39,878, with their DKIM results of either True or False.

Both files have these fields:

ID – 1| Verified – 2| Date – 3| From – 4| To – 5| Subject -6| Message-Id – 7

Question: Have you seen any news reports that mention emails being “verified” in their reporting?

Emails in the complete set may be as accurate as those in the verified set, but I would think verification is newsworthy in and of itself.

You?

Clinton/Podesta Emails 23 and 24, True or False? Cryptographically Speaking

Filed under: Data Mining,Hillary Clinton,News,Politics,Reporting — Patrick Durusau @ 10:21 am

Catching up on the Clinton/Podesta email releases from Wikileaks, via Michael Best, NatSecGeek. Michael bundles the releases up and posts them at: Podesta emails (zipped).

For anyone coming late to the game, DKIM “verified” means that the DKIM signature on an email is valid for that email.

In lay person’s terms, that email has been proven by cryptography to have originated from a particular mail server and when it left that mail server, it read exactly as it does now, i.e., no changes by Russians or others.

What I have created are files that lists the emails in the order they appear at Wikileaks, with the very next field being True or False on the verification issue.

Just because an email has “False” in the second column doesn’t mean it has been modified or falsified by the Russians.

DKIM signatures fail for all manner of reasons but when they pass, you have a guarantee the message is intact as sent.

For your research into these emails:

DKIM-complete-podesta-23.txt.gz

and

DKIM-complete-podesta-24.txt.gz.

For release 24, I did have to remove the DKIM signature on 39256 00010187.eml in order for the script to succeed. That is the only modification I made to either set of files.

October 22, 2016

Validating Wikileaks Emails [Just The Facts]

Filed under: Cybersecurity,Hillary Clinton,Journalism,News,Reporting,Wikileaks — Patrick Durusau @ 8:27 pm

A factual basis for reporting on alleged “doctored” or “falsified” emails from Wikileaks has emerged.

Now to see if the organizations and individuals responsible for repeating those allegations, some 260,000 times, will put their doubts to the test.

You know where my money is riding.

If you want to verify the Podesta emails or other email leaks from Wikileaks, consult the following resources.

Yes, we can validate the Wikileaks emails by Robert Graham.

From the post:

Recently, WikiLeaks has released emails from Democrats. Many have repeatedly claimed that some of these emails are fake or have been modified, that there’s no way to validate each and every one of them as being true. Actually, there is, using a mechanism called DKIM.

DKIM is a system designed to stop spam. It works by verifying the sender of the email. Moreover, as a side effect, it verifies that the email has not been altered.

Hillary’s team uses “hillaryclinton.com”, which as DKIM enabled. Thus, we can verify whether some of these emails are true.

Recently, in response to a leaked email suggesting Donna Brazile gave Hillary’s team early access to debate questions, she defended herself by suggesting the email had been “doctored” or “falsified”. That’s not true. We can use DKIM to verify it.

Bob walks you through validating a raw email from Wikileaks with the DKIM verifier plugin for Thunderbird. And demonstrating the same process can detect “doctored” or “falsified” emails.

Bob concludes:

I was just listening to ABC News about this story. It repeated Democrat talking points that the WikiLeaks emails weren’t validated. That’s a lie. This email in particular has been validated. I just did it, and shown you how you can validate it, too.

Btw, if you can forge an email that validates correctly as I’ve shown, I’ll give you 1-bitcoin. It’s the easiest way of solving arguments whether this really validates the email — if somebody tells you this blogpost is invalid, then tell them they can earn about $600 (current value of BTC) proving it. Otherwise, no.

BTW, Bob also points to:

Here’s Cryptographic Proof That Donna Brazile Is Wrong, WikiLeaks Emails Are Real by Luke Rosiak, which includes this Python code to verify the emails:

clinton-python-email-460

and,

Verifying Wikileaks DKIM-Signatures by teknotus, offers this manual approach for testing the signatures:

clinton-sig-check-460

But those are all one-off methods and there are thousands of emails.

But the post by teknotus goes on:

Preliminary results

I only got signature validation on some of the emails I tested initially but this doesn’t necessarily invalidate them as invisible changes to make them display correctly on different machines done automatically by browsers could be enough to break the signatures. Not all messages are signed. Etc. Many of the messages that failed were stuff like advertising where nobody would have incentive to break the signatures, so I think I can safely assume my test isn’t perfect. I decided at this point to try to validate as many messages as I could so that people researching these emails have any reference point to start from. Rather than download messages from wikileaks one at a time I found someone had already done that for the Podesta emails, and uploaded zip files to Archive.org.

Emails 1-4160
Emails 4161-5360
Emails 5361-7241
Emails 7242-9077
Emails 9078-11107

It only took me about 5 minutes to download all of them. Writing a script to test all of them was pretty straightforward. The program dkimverify just calls a python function to test a message. The tricky part is providing context, and making the results easy to search.

Automated testing of thousands of messages

It’s up on Github

It’s main output is a spreadsheet with test results, and some metadata from the message being tested. Results Spreadsheet 1.5 Megs

It has some significant bugs at the moment. For example Unicode isn’t properly converted, and spreadsheet programs think the Unicode bits are formulas. I also had to trap a bunch of exceptions to keep the program from crashing.

Warning: I have difficulty opening the verify.xlsx file. In Calc, Excel and in a CSV converter. Teknotus reports it opens in LibreOffice Calc, which just failed to install on an older Ubuntu distribution. Sing out if you can successfully open the file.

Journalists: Are you going to validate Podesta emails that you cite? Or that others claim are false/modified?

October 15, 2016

Why Journalists Should Not Rely On Wikileaks Indexing – Podesta Emails

Filed under: Journalism,News,Reporting,Wikileaks — Patrick Durusau @ 3:58 pm

Clinton on Fracking, or, Another Reason to Avoid Wikileaks Indexing

fracking-podesta-460

The quote in the tweet is false.

Politico supplies the correct quotation in its post:


“Bernie Sanders is getting lots of support from the most radical environmentalists because he’s out there every day bashing the Keystone pipeline. And, you know, I’m not into it for that,” Clinton told the unions, according to the transcript. “My view is, I want to defend natural gas. … I want to defend fracking under the right circumstances.”

I’m guessing that “…under the right circumstances.” must have pushed Wikileaks too close to the 140 character barrier.

Ditto for the Wikileaks mis-quote of: “Get a life.”

Which reported as in the tweet, appears to refer to unbridled fracking.

Not so in the Politico post:


“I’m already at odds with the most organized and wildest” of the environmental movement, Clinton told building trades unions in September 2015, according to a transcript of the remarks apparently circulated by her aides. “They come to my rallies and they yell at me and, you know, all the rest of it. They say, ‘Will you promise never to take any fossil fuels out of the earth ever again?’ No. I won’t promise that. Get a life, you know.”

Doesn’t read quite the same way does it?

I supposed once you start lying it’s really hard to stop. Clinton is a good example of that and Wikileaks should not follow her example.

It’s hard to spot these lies because Wikileaks isn’t indexing the attachments.

You can search all day for “defend fracking,” “get a life” (by Clinton) and you will come up empty (at least as of today).

So that you don’t have to search for: 20150909 Transcript | Building Trades Union (Keystone XL) at Wikileaks – Podesta Emails, I have produced a PDF version of that attachment, Building-Trades-Union-Clinton-Sept-09-2015.pdf (my naming), for your viewing pleasure.

October 13, 2016

George Carlin’s Seven Dirty Words in Podesta Emails – Discovered 981 Unindexed Documents

Filed under: Government,Hillary Clinton,Humor,Journalism,News,Reporting — Patrick Durusau @ 10:42 am

While taking a break from serious crunching of the Podesta emails I discovered 981 unindexed documents at Wikileaks!

https://www.youtube.com/watch?v=FMkNsMMvrqk

Try searching for Carlin’s seven dirty words at The Podesta Emails:

  • shit – 44
  • piss – 19
  • fuck – 13
  • cunt – 0
  • cocksucker – 0
  • motherfucker – 0 (?)
  • tits – 0

I have a ? after “motherfucker” because working with the raw files I show one (1) hit for “motherfucker” and one (1) hit for “motherfucking.” Separate emails.

For “motherfucker,” American Sniper–the movie, responded to by Chris Hedges – From:magazine@tikkun.org To: Podesta@Law.Georgetown.Edu

For “motherfucking,” H4A News Clips 5.31.15 – From/To: aphillips@hillaryclinton.com.

“Motherfucker” and “motherfucking” occur in text attachments to emails, which Wikileaks does not search.

If you do a blank search for file attachments, Wikileaks reports there are 2427 file attachments.

Searching the Podesta emails at Wikileaks excludes the contents of 2427 files from your search results.

How significant is that?

Hmmm, 302 pdf, 501 docx, 167 doc, 12 xls, 9 xlsx – 981 documents excluded from your searches at Wikileaks.

For 9,011 emails, as of AM today, my local.

How comfortable are you with not searching those 981 documents? (Or additional documents that may follow?)

October 11, 2016

Parsing Foreign Law From News Reports (Warning For Journalists)

Filed under: Journalism,News,Reporting — Patrick Durusau @ 6:48 pm

Cory Doctorow‘s headline: Scotland Yard charge: teaching people to use crypto is an act of terrorism red-lined my anti-government biases.

I tend towards “unsound” reactions when free speech is being infringed upon.

But my alarm and perhaps yours as well. was needlessly provoked in this case.

Cory writes:


In other words, according to Scotland Yard, serving a site over HTTPS (as this one is) and teaching people to use crypto (as this site has done) and possessing a secure OS (as I do) are acts of terrorism or potential acts of terrorism. In some of the charges, the police have explicitly connected these charges with planning an act of terrorism, but in at least one of the charges (operating a site served over HTTPS and teaching people about crypto) the charge lacks this addendum — the mere act is considered worthy of terrorism charges.

The concern over:


but in at least one of the charges (operating a site served over HTTPS and teaching people about crypto) the charge lacks this addendum — the mere act is considered worthy of terrorism charges.

is mis-placed.

Cory points to the original report here: Man arrested on Cardiff street to face six terror charges by Viram Dodd.

Cory’s alarm is not repeated by Dodd:


Ullah has been charged with directing terrorism, providing training in encryption programs knowing the purpose was for terrorism, and using his blog site to provide such training. His activities are alleged to have “the intention of assisting another or others to commit acts of terrorism”.

Beyond that (I haven’t seen the charging document), be aware that under English Criminal Procedure, the “charge” on which Cory places so much weight is defined as:

uk-charge-460

Pay particular attention to 7.3(1)(a)(i) (page 65):

…describes the offense in ordinary language, and…

A “charge” isn’t a technical specification of an offense under English criminal procedure. Which means you attach legal significance to charging language at your own peril. And to the detriment of your readers.

PS: I have contacted the Westminster Magistrates’ Court and requested a copy of the charging document. If and when that arrives, I will update this post with it.

October 6, 2016

Arabic/Russian Language Internet

Filed under: Journalism,News,Reporting — Patrick Durusau @ 1:01 pm

No matter the result of the 2016 US presidential election, mis-information on areas where Arabic and/or Russian are spoken will increase.

If you are creating topic maps and/or want to do useful reporting on such areas consider:

How to get started investigating the Arabic-language internet by Tom Trewinnard, or,

How to get started investigating the Russian-language internet by Aric Toler.

Any hack can quote releases from official sources and leave their readers uninformed.

A journalist takes monotone “facts” from an “official” release and weaves a story of compelling interest to their readers.

Any other guides to language/country specific advice for journalists?

September 27, 2016

How media coverage of terrorism endorses a legal [4-Ply or More] standard

Filed under: Journalism,News,Reporting — Patrick Durusau @ 8:45 pm

How media coverage of terrorism endorses a legal double standard by Rafia Zakaria.

From the post:

On June 17, 2016, Dylann Roof entered a predominantly black church In Charleston, South Carolina, and opened fire. When he was done, nine people lay dead around him. For a few days after Roof’s grisly act, a debate raged in the media over whether the committed white supremacist and mass murderer should be considered a terrorist. Many, including The Washington Post’s Philip Bump, vehemently opposed the label, insisting that even though the Justice Department had dubbed Roof’s killing spree “an act of domestic terrorism,” calling Roof a terrorist would confer upon him the very notoriety he sought.

Like other journalists and analysts, Bump analyzed the sociological and ethical dimensions of the terror label, concerns about whether all who terrify are terrorists, and whether the wider application of the label somehow lessens the potency of the evil it represents. However, like nearly all other journalists who write about terrorism, Bump missed the most crucial point concerning the media’s use of the term: that American law does not currently recognize “domestic terror” as a crime. For an act, however bloody and hateful, to be considered terrorism in the United States, it must be connected to a “foreign” terror organization.

Rafia makes an important point about the “pass” being given to white supremacists, while law abiding Muslims are viewed with suspicion if not being actively persecuted in the United States.

But Rafia misses the opportunity to point to the more than double standard in place for use of “terrorism” and “terrorist.”

What label other than “terrorist” would you apply to the unknown military personnel who attack a known hospital? It has been alleged those responsible have been punished, but then without transparency, how do we know?

Or even the garden variety cruise missile or drone attacks that end the lives of innocents with every strike. Aren’t those acts of terrorism?

Or does “terrorism” require a non-U.S. government actor?

Does that mean only the U.S. government?

How “terrorized” would you be by a phone call followed a “knock” by a missile on your roof, ordering you to leave immediately?

The claim that is “designed to minimize civilian casualties,” sounds like a quote from a modern day Marquis de Sade.

A little introspection by the media could explode the dishonest and manipulative use of the labels “terrorist” and “terrorism.”

Let’s hope that happens sooner rather than later.

Bulk Access to the Colin Powell Emails – Update

Filed under: Colin Powell Emails,Government,Journalism,News,Politics,Reporting — Patrick Durusau @ 7:31 pm

Still working on finding a host for the 2.5 GB tarred, gzipped archive of the Colin Powell emails.

As an alternative, working on splitting the attachments (the main source of bulk) from the emails themselves.

My thinking at this point is to produce a message-only version of the emails. Emails with attachments will have auto-generated links to the source emails at DCLeaks.com.

Other processing is planned for the message-only version of the emails.

Anyone interested in indexing the attachments? Generating lists of those with pointers shouldn’t be a problem.

Hope to have more progress to report tomorrow!

September 26, 2016

Value-Add Of Wikileaks Hillary Clinton Email Archive?

Filed under: Government,Hillary Clinton,Journalism,News,Politics,Reporting — Patrick Durusau @ 12:20 pm

I was checking Wikileaks today for any new document drops on Hillary Clinton, but only found:

WikiLeaks offers award for #LabourLeaks

Trade in Services Agreement

Assange Medical and Psychological Records

The lesson from the last item is to always seek asylum in a large embassy, preferably one with a pool. You can search at Embassies by embassy for what country, located in what other country. I did not see an easy way to search for size and accommodations.

Oh, not finding any new data on Hillary Clinton, I checked the Hillary Clinton Email Archive at Wikileaks:

wikileaks-hillary-460

Compare that to the State Department FOIA server for Clinton_Email:

state-dept-hillary-460

Do you see a value-add to Wikileaks re-posting the State Department’s posting of Hillary’s emails?

If yes, please report in comments below the value-add you see. (Thanks in advance.)

If not, what do you think would be a helpful value-add to the Hillary Clinton emails? (Suggestions deeply appreciated.)

September 25, 2016

Colin Powell Email Files

Filed under: Colin Powell Emails,Government,Journalism,News,Politics,Reporting — Patrick Durusau @ 8:43 pm

DCLeaks.com posted on September 14, 2016, a set of emails to and from Colin Luther Powell.

From the homepage for those leaked emails:

Colin Luther Powell is an American statesman and a retired four-star general in the United States Army. He was the 65th United States Secretary of State, serving under U.S. President George W. Bush from 2001 to 2005, the first African American to serve in that position. During his military career, Powell also served as National Security Advisor (1987–1989), as Commander of the U.S. Army Forces Command (1989) and as Chairman of the Joint Chiefs of Staff (1989–1993), holding the latter position during the Persian Gulf War. Born in Harlem as the son of Jamaican immigrants, Powell was the first, and so far the only, African American to serve on the Joint Chiefs of Staff, and the first of two consecutive black office-holders to serve as U.S. Secretary of State.

The leaked emails start in June of 2014 and end in August of 2016.

Access to the emails is by browsing and/or full text searching.

Try your luck at finding Powell’s comments on Hillary Clinton or former Vice-President Cheney. Searching one chunk of emails at a time.

I appreciate and admire DCLeaks for taking the lead in posting this and similar materials. And I hope they continue to do so in the future.

However, the access offered reduces a good leak to a random trickle.

This series will use the Colin Powell emails to demonstrate better leaking practices.

Coming Monday, September 26, 2016 – Bulk Access to the Colin Powell Emails.

September 23, 2016

14 free digital tools that any newsroom can use

Filed under: Journalism,News,Reporting — Patrick Durusau @ 8:54 pm

14 free digital tools that any newsroom can use by Sara Olstad.

From the post:

ICFJ’s Knight Fellows are global media innovators who foster news innovation and experimentation to deepen coverage, expand news delivery and better engage citizens. As part of their work, they’ve created tools that they are eager to share with journalists worldwide.

Their projects range from Push, a mobile app for news organizations that don’t have the time, money or resources to build their own, to Salama, a tool that assesses a reporter’s risk and recommends ways to stay safe. These tools and others developed by Knight Fellows can help news organizations everywhere find stories in complex datasets, better distribute their content and keep their journalists safe from online and physical security threats.

As part of the 2016 Online News Association conference, try out these 14 digital tools that any newsroom can use. If you adopt any of these tools or lead any new projects inspired by them, tweet about it to @ICFJKnight.

I was mis-led by the presentation of the “14 free digital tools.”

The box where African Network of Centers for Investigative Reporting (ANCIR) and Aleph appear has a scroll marker on the right hand side.

I’m not sure why I missed it or why the embedding of a scrolling box is considered good page design.

But the tools themselves merit your attention.

Enjoy!

5 lessons on the craft of journalism from Longform podcast

Filed under: Journalism,News,Reporting — Patrick Durusau @ 4:37 pm

5 lessons on the craft of journalism from Longform podcast by Joe Freeman.

From the post:

AT FIRST I WAS RELUCTANT to dive into the Longform podcast, a series of interviews with nonfiction writers and journalists that recently produced its 200th episode. The reasons for my wariness were petty. What sane freelancer wants to listen to highly successful writers and editors droning on about their awards and awesome careers? Not this guy! But about a year ago, I succumbed, and quickly became a thankful convert. The more I listened, the more I realized that the show, started in 2012 on the website Longform.org and produced in collaboration with The Atavist, was a veritable goldmine of information. It’s almost as if the top baseball players in the country sat down every week and casually explained how to hit home runs.

Whether they meant to or not, the podcast’s creators and interviewers—Aaron Lammer, Max Linsky, and Evan Ratliff—have produced a free master class on narrative reporting, with practitioners sharing tips and advice about the craft and, crucially, the business. As a journalist, I’ve learned a lot listening to the podcast, but a few consistent themes emerge that I have distilled into five takeaways from specific interviews.

(emphasis in original)

I’m impressed with Joe’s five takeaways but as I sit here repackaging leaked data, there is one common characteristic I would emphasize:

They all involve writing!

That is the actual production of content.

Not plans for content.

Not models for content.

Not abstractions for content.

Content.

Not to worry, I intend to keep my tools/theory edge but in addition to adding Longform podcast to my listening list, I’m going to try to produce more data content as well.

I started off with that intention using XQuery at the start of this year, a theme that is likely to re-appear in the near future.

Enjoy!

September 20, 2016

Betraying Snowden:… [Cynical, but not odd]

Filed under: Journalism,News,NSA,Reporting — Patrick Durusau @ 6:30 pm

Betraying Snowden: There’s a special place in journalism hell for The Washington Post editorial board by Daniel Denvir.

From the post:

There is a special place in journalism hell reserved for The Washington Post editorial board now that it has called on President Barack Obama to not pardon National Security Agency whistleblower Edward Snowden.

As Glenn Greenwald wrote, it’s an odd move for a news publication, “which owes its sources duties of protection, and which — by virtue of accepting the source’s materials and then publishing them — implicitly declares the source’s information to be in the public interest.” Notably, the Post decided to “inexcusably omit . . . that it was not Edward Snowden, but the top editors of the Washington Post who decided to make these programs public,” as Greenwald added.

The Post’s peculiar justification is as follows: While the board grudgingly conceded that reporters, thanks to Snowden, revealed that the NSA’s collection of domestic telephone metadata — which “was a stretch, if not an outright violation, of federal surveillance law” — it condemns him for revealing “a separate overseas NSA Internet-monitoring program, PRISM, that was both clearly legal and not clearly threatening to privacy.”

Washington Post opposition to a pardon for Edward Snowden isn’t odd at all.

Which story generates more PR for the Washington Post:

  1. The Washington Post, having won a Pulitzer prize due to Edward Snowden, joins a crowd calling for his pardon?
  2. The Washington Post, having won a Pulitzer prize due to Edward Snowden, opposes his being pardoned?

It’s not hard to guess which one generates more ad-views and therefore the potential for click-throughs.

I have no problems with the disclosure of PRISM, save for Snowden having to break his word as a contractor to keep his client’s secrets, well, secret.

No one could be unaware that the NSA engages in illegal and immoral activity on a daily basis before agreeing to be employed by them.

Although Snowden has done no worse than his former NSA employers, it illustrates why I have no trust in government agencies.

If they are willing to lie for what they consider to be “good” reasons to you, then they are most certainly willing to lie to me.

Once it is established that an agency, take the NSA for example, has lied on multiple occasions, on what basis would you trust them to be telling the truth today?

Their assurance, “we’re not lying this time?” That seems rather tenuous.

Same rule should apply to contractors who lie to or betray their clients.

September 12, 2016

Inside the fight to reveal the CIA’s torture secrets [Support The Guardian]

Filed under: Government,Government Data,Journalism,News,Politics,Reporting,Transparency — Patrick Durusau @ 3:19 pm

Inside the fight to reveal the CIA’s torture secrets by Spencer Ackerman.

Part one: Crossing the bridge

Part two: A constitutional crisis

Part three: The aftermath

Ackerman captures the drama of a failed attempt by the United States Senate to exercise oversight on the Central Intelligence Agency (CIA) in this series.

I say “failed attempt” because even if the full 6,200+ page report is ever released, the lead Senate investigator, Daniel Jones, obscured the identities of all the responsible CIA personnel and sources of information in the report.

Even if the full report is serialized in your local newspaper, the CIA contractors and staff guilty of multiple felonies, will be not one step closer to being brought to justice.

To that extent, the “full” report is itself a disservice to the American people, who elect their congressional leaders and expect them to oversee agencies such as the CIA.

From Ackerman’s account you will learn that the CIA can dictate to its overseers, the location and conditions under which it can view documents, decide which documents it is allowed to see and in cases of conflict, the CIA can spy on the Select Senate Committee on Intelligence.

Does that sound like effective oversight to you?

BTW, you will also learn that members of the “most transparent administration in history” aided and abetted the CIA in preventing an effective investigation into the CIA and its torture program. I use “aided and abetted” deliberately and in their legal sense.

I mention in my header that you should support The Guardian.

This story by Spencer Ackerman is one reason.

Another reason is that given the plethora of names and transfers recited in Ackerman’s story, we need The Guardian to cover future breaks in this story.

Despite the tales of superhuman security, nobody is that good.

I leave you with the thought that if more than one person knows a secret, then it it can be discovered.

Check Ackerman’s story for a starting list of those who know secrets about the CIA torture program.

Good hunting!

September 4, 2016

Plugins for Newsgathering and Verification

Filed under: Journalism,News,Reporting — Patrick Durusau @ 7:55 pm

7 vital browser plugins for newsgathering and verification by Alastair Reid.

From the post:

When breaking news can travel the world in seconds, it is important for journalists to have the tools at their disposal to get to work fast. When searching the web, what quicker way is there to have those tools available than directly in the browser window?

Most browsers have a catalogue of programs and software to make your browsing experience more powerful, like a smartphone app store. At First Draft we find Google’s Chrome browser is the most effective but there are obviously other options available.

Text says “five” but this has been updated to include “seven” plugins.

One of the updates is: Frame by Frame for YouTube, which like the name says, enables frame by frame viewing, is touted for verification.

I can think of a number of uses for frame-by-frame viewing. You?

See Alastair’s post for the rest and follow @firstdraftnews to stay current on digital tools for journalists.

September 2, 2016

Best and Worst Journalism of August 2016 [An Exercise]

Filed under: Journalism,News,Reporting — Patrick Durusau @ 9:18 pm

The best and worst journalism of August 2016 by David Uberti.

Before you read Uberti’s post:

Take a few minutes to find stories you recall from August and sort them into best and worst, along with your reasons.

It’s one thing to passively go along with the judgment of others, it takes real effort to form a judgment of your own.

Now, compare your stories to Uberti’s.

Same, different? Were your reasons different?

What stories did Uberti “miss?”

PS: The boosterism of the New York Times for Iraqi militias merits a “worst” place, at least to me.

Journalism Drone Operations Manual

Filed under: Journalism,News,Reporting — Patrick Durusau @ 3:12 pm

CoJMC’s Drone Journalism Lab launches drone operations manual

From the webpage:

To help newsrooms get started using drones for journalism, the Drone Journalism Lab at the University of Nebraska-Lincoln is releasing the “The Drone Journalism Lab Operations Manual,” a guide that covers everything from pre-flight checklists to ethical considerations.

A first of its kind, the manual is free, Creative Commons licensed and provided as an open source document online. The Drone Journalism Lab created it with support from the John S. and James L. Knight Foundation.

“As journalists look to become more relevant and responsive to community needs, this manual is an important step towards experimenting with new ways of gathering and presenting news and information. It is a resource for best practices and an exciting invitation to explore a fresh, emerging area of the field,” said Shazna Nessa, Knight Foundation director for journalism.

Dr. Maria Marron, dean of the College of Journalism and Mass Communications, praised Professor Matt Waite for producing the operations manual.

“Matt is a key innovator in journalism,” she said. “It was his prescience about the potential for drones in journalism that made UNL’s Drone Journalism Lab the leader in the field. The operations manual will be the go-to resource for anyone interested in using drones for journalistic purposes.”

Link for the manual: https://www.dropbox.com/sh/32pi2e2gv6huyzg/AAAwGq7b1mO5ekikCn-7JFiMa?dl=0.

What a great resource!

A great template for how to describe your use of drones for journalism.

August 27, 2016

Shield laws and journalist’s privilege: … [And Beyond]

Filed under: Journalism,News,Reporting — Patrick Durusau @ 1:26 pm

Jonathan Peters‘s Shield laws and journalist’s privilege: The basics every reporter should know is a must read … before a subpoena arrives.

From his post:

COMPELLED DISCLOSURE is in the air.

A federal judge has ordered Glenn Beck to disclose the names of confidential sources he used in his reporting that a Saudi Arabian man was involved in the Boston Marathon bombing. The man sued Beck for defamation after he was cleared of any involvement.

Journalist and filmmaker Mark Boal, who wrote and produced The Hurt Locker and Zero Dark Thirty, has asked a judge to block a subpoena threatened by military prosecutors who want to obtain his confidential or unpublished interviews with US Army Sgt. Bowe Bergdahl, accused of being a deserter.

A state judge has ruled that a New York Times reporter must testify at a murder trial about her jailhouse interview with the man accused of killing Anjelica Castillo, the toddler once known as Baby Hope. The judge said the interview included the only statements the man made about the crime other than those in his police confession.

If my inbox is any indication, those cases have prompted a surge of interest in shield laws and the practice of compelled disclosure. What is a shield law, exactly? When can a government official require a reporter to disclose sources or information? Who counts as a journalist under a shield law? What types of sources or information are protected? Is there a big difference between a subpoena and a search warrant?

Those are the questions I’ve been asked most often in this area, as a First Amendment lawyer and scholar, and this post will try to answer them. (Please keep in mind that I’m a lawyer, not your lawyer, and these comments shouldn’t be construed as legal advice.)

As useful as Jonathan’s advice, in conjunction with advice from your own lawyer, I would point out by the time a subpoena arrives, you have already lost.

Because of circumstances, a jail house interview where you are the only possible source, or bad OpSec, you have been identified as possessing information state authorities want.

As Jonathan points out, there are governments with shield laws and notions of journalist privilege, but even those have fallen on hard times.

Outside of single source situations, consider anonymous posting of information needed for your story.

You can cite the public posting, as can others, which leaves the authorities without a target for their “name of the source” subpoena. It’s public information.

No one will be able to duplicate months of research and writing with a week or two and public posting may keep the you out of the cross-hairs of local government.

Posting unpublished information is an anathema to some, who think hoarding is the only path to readers. They are the best judges of whether they are read because they hoard or because of their skills as story tellers and analysts.

As an additional precaution, I assume you have a documented story development trail that you can fight tooth and nail to keep, which when disclosed shows your reliance on the publicly posted data. Yes?

PS: Wikilinks is one example of a public posting venue. Dark web sites for states (or other administrative divisions) or cities might be more appropriate. My suggestion is to choose one that doesn’t censor data dumps. Ever.

August 19, 2016

Top Ten #ddj:… [18 August 2016]

Filed under: Journalism,News,Reporting — Patrick Durusau @ 8:34 pm

Top Ten #ddj: The Week’s Most Popular Data Journalism Links

A weekly feature of the Global Investigative Journalism Network and particularly good this week:

  • Animated Data Visualisation: Trends in Household Debts Reveals a Constant Increase in Student Loans
  • Discover Why and How The New York Times is Changing the Way They Present Interactive Content
  • Analysis of Trump’s Tweets: Trump Writes Angrier Tweets on Android While His Staff Tweets More Positively On His Behalf Using an iPhone
  • Analysis of Trump’s Tweets: Sharp Decline in Trump’s Own Tweets from 77 to 24 percent Suggests Tighter Campaign Control
  • Open Data-Driven Articles Using Olympic Data: Edit the Source Code and Create Your Own Visualisations
  • Interactive Map of Recreational Areas in Ravensburg, Germany
  • Onodo: Network Visualisation and Analysis Tool for Non-Tech Users
  • Opinion: Not Every Venn Diagram Has Something Worth Reporting
  • Data on Teenage Pregnancies and HIV rates in Kenya
  • Mapbox: How to Customise and Embed Maps on Websites

See the original post for links and very annoying “share” options. (Annoying to me, others may find them indispensable.)

Mark your calendars to check for new top ten lists and/or follow @gijn.

August 17, 2016

Double Standards At NPR

Filed under: Government,Journalism,News,Reporting,Wikileaks — Patrick Durusau @ 4:00 pm

NPR Host Demands That Assange Do Something Its Own Reporters Are Told Never to Do by Naomi LaChance.

From the post:

In a ten-minute interview aired Wednesday morning, NPR’s David Greene asked Wikileaks founder Julian Assange five times to reveal the sources of the leaked information he has published on the internet.

A major tenet of American journalism is that reporters protect their sources. Wikileaks is certainly not a traditional news organization, but Greene’s persistent attempts to get Assange to violate confidentiality was alarming, especially considering that there has been no challenge to the authenticity of the material in question.

NPR (National Public Radio) shows its true colors, not as a free and independent press but as a lackey of the Democratic Party in this interview with Assange.

David Greene (Morning Edition) was fixated on repeating the unconfirmed reports that the Russians (which Russians no one every says), were behind the leak of DNC emails.

You can read the transcript of Assange/Greene interview for yourself.

Greene never asks one substantive question about the 20,000 emails. Not one. The first leak of its kind and all Greene does is whine about rumors of Russian involvement.

Well, that’s not entirely fair, Greene does have this exchange with Assange:


GREENE: Well, let me – apart from the different investigations, could you see people in the U.S. government thinking that you might be a threat to national security?

ASSANGE: Well, I mean, there’s great people in the U.S. government – many of them are our sources – and there’s terrible people in the U.S. government. Unfortunately, the U.S. government is a – you know, a reflection, to some degree, of the rest of society. So it’s filled with its share of paranoid and sociopathic power climbers…

GREENE: But is it paranoid to look at these uncensored documents?

ASSANGE: …People who make errors of judgment, etc.

GREENE: Is it paranoid to look at these uncensored documents, these emails, that are released by you? And if they believe that that could change a U.S. presidential election, could be a threat to national security, why isn’t it logical…

ASSANGE: I just – I mean…

GREENE: …For them to see you as a possible threat?

Hmmm, telling the truth about DNC emails can be a threat to national security?

What a bizarre concept in a democracy! Disclosure of evidence of manipulation of the democratic process is a “…threat to national security?”

NPR can and should do better than David Greene shilling for the Democratic Party.

August 10, 2016

50 ways to measure your analytics

Filed under: Journalism,News,Reporting — Patrick Durusau @ 8:24 am

50 ways to measure your analytics (with apologies to Paul Simon) by Melody Kramer.

From the post:

“The problem is all inside your figures,” she said to me.
“The answer is easy if you think more than numerically.
I’d like to help you in your struggle to count your impact perfectly.
There must be (at least) 50 ways to measure success for a news article.”

She said, “It’s really not my habit to really think about the news.
Furthermore, I hope my meaning won’t be lost or misconstrued.
But I’ll repeat myself, at the risk of being crude:
There must be 50 ways to assess whether your piece is reaching the full potential audience it could.

Fifty ways to count your numbers.'”

You will have to find mechanisms to measure your analytics but Melody does give you fifty (50) things to measure!

Clever use of the Paul Simon lyrics.

Suggest a “trigger warning” that doesn’t give away the trigger in this case? 😉

August 9, 2016

How to avoid 10 common mistakes in data reporting [Plus #11]

Filed under: Journalism,News,Reporting — Patrick Durusau @ 2:51 pm

How to avoid 10 common mistakes in data reporting by Catherine Sheffo.

From the post:

After getting your hands on a data set, the hardest part of incorporating data analysis into your beat is getting started — and avoiding beginners’ pitfalls along the way.

From scrambled columns to unintelligible field names, every file you receive with comes with challenges for new and experienced data reporters alike.

We talked to Sean Mussenden, chief of the data and graphics bureau at the University of Maryland’s Capital News Service, about 10 mistakes to avoid while you establish a workflow and get comfortable with data sets in your day-to-day reporting.

Topic map fans will recognize #5:

Mistake No. 5 – Assuming you know what the field names mean

That can easily extend to what is in the fields as well.

I would add:

Mistake No. 11 – Assuming data is truthful and unbiased

Always bear in mind data given to you has been “cooked.”

Data has been omitted, changed or added, however “raw” the data may appear to you. The act of collecting data involves omission, changes and additions. All from a point of view.

Not to mention whoever gave you data had an agenda as well.

There’s no escape from bias but you can work at serving your own agenda and not those of others.

August 7, 2016

Hierarchy of Disagreement – Trump On Nuclear Weapons

Filed under: Argumentation,Journalism,News,Reporting — Patrick Durusau @ 3:50 pm

disagreement-levels-460

Politicians? Politicians?

Hell, I would be happy if news commentators and “experts” that appear on news shows would rise above contradiction.

Repetition, especially repeating what other commentators have said, isn’t evidence, it’s just noise.

If the medium you are using doesn’t support robust referencing of facts and analysis, you are using the wrong medium.

Or should that be … “you are following the wrong medium?”

You remember the Dilbert cartoon about the evening news, Sunday February 07, 1993 with the line:

A new poll shows that many voters have strong opinions on these issues despite the fact that we provide no useful contextual data.

That is a great summary of news reporting on top issues of the day. On occasion NPR will have an in-depth analysis but it repeats the stories of the day with little context, just like other media outlets.

Granting that is a limitation of the medium, why not use the Internet to deliver the context that video or radio media lack the time to deliver? Using video or radio as a highlights or awareness service, with further details collected and organized for viewer/listeners.

Despite timely, accurate and moving news reporting, I don’t have a regular source that provides in-depth contextual for everyday news stories.

For example, the internet was aflame with news of Trump asking “…why he could not use nuclear weapons?” Or at least that was the headline.

Some reports did pick up the contradiction in spending $billions on weapons you aren’t (don’t intend?) using, but few and far in between. And of those that did, how many examined the economic drivers that have created a useless product industry? The one that produces nuclear weapons.

In case you are curious, the United States has steadfastly refused to renounce first strike as a military strategy. (Report on Nuclear Employment Strategy of the United States, 2010, yes, during President Obama’s first term in office).

Do you recall seeing in depth reporting or analysis of either of those two aspects of the use of nuclear arms issue?

There was a lot of huffing, puffing and strutting around as I recall but little in the way of substantive or contextual analysis.

August 4, 2016

Joel Simon (@Joelcpj): Woodward and Bernstein Not “Ethical and Committed” Journalists

Filed under: Government,Journalism,News,Reporting,Wikileaks — Patrick Durusau @ 10:01 am

Joel Simon‘s opinion piece How journalists can cover leaks without helping spies, leaves you with the conclusion that Woodward and Bernstein (Watergate) were not “ethical and committed” journalists.

Skipping the nationalistic ranting and “compelling evidence,” which turns out to be the New York Times parroting surmises and guesses by known liars (U.S. intelligence community), Simon writes of the Wikileaks dump of DNC emails:


As for WikiLeaks, by publishing a data dump without verifying the source or providing its readers with the context to make informed decisions about the motivations of the leakers, it is allowing itself to be a vehicle for governments like Russia that are weaponizing information and using it to achieve policy objectives. Ethical and committed journalists should do all within their power to ensure they are never put in such a position. (emphasis added)

For more than thirty years, 1972 – 2005, the Watergate source known as “Deep Throat (W. Mark Felt),” and his motives, remained a mystery to the American public.

Yet, his revelations were instrumental in bringing down an American president (Richard Nixon).

Mark Felt was a friend of Bob Woodward and their meeting in a parking garage on October 9th, 1972, lead to the October 10, 1972 Washington Post story titled: FBI Finds Nixon Aides Sabotaged Democrats.

In case you don’t remember, 1972 was a presidential election year, with the election being held on November 7, 1972.

Consider those three dates, the discussion between Bernstein and Felt (October 9, 1972), the Washington Post story (October 10, 1972) and the presidential election (November 7, 1972). Or perhaps better:


October 9, 1972 – 29 days until voting begins in presidential election

October 10, 1972 – 28 days until voting begins in presidential election

November 7, 1972 (election day)

The timing of the leak and its publication by the Washington Post less than thirty (30) days prior to a presidential election certainly make the motives of the leaker a relevant question.

Yet, Deep Throat remained unknown and “…readers with[out] the context to make informed decisions about the motivations of the [Deep Throat/Mark Felt]…” for more than thirty years.

Contrary to Joel Simon’s criteria, Woodward and Bernstein verified and corroborated the information given to them by Deep Throat/Mark Felt to be truthful and did not explore for their readers, any possible motivations on his part.

The authenticity of the DNC emails has not been challenged and resignations of Wasserman Schultz (DNC Chair), Amy Dacey (DNC CEO), Brad Marshall (DNC CFO), Luis Miranda (DNC Communications Director) and an public apology to Bernie Sanders by the Democratic National Committee, are all supporting evidence that the DNC email leak is both accurate and authentic.

Unlike Joel Simon, I think Woodward and Bernstein were “ethical and committed” journalists during Watergate, providing their readers with accurate information in a timely manner.

Without exploring the motives of why someone would leak truthful information.

The CJR, Joel Simon and the media generally should abandon its attempt to twist journalistic ethics to exclude publication of truthful information of legitimate interest to a voting public.

Judging from the tone of Simon’s post, his concerns are driven more by rabid nationalism and jingoism than any legitimate concern for journalistic ethics.

August 3, 2016

First Draft – July Fake News Quiz

Filed under: Journalism,News,Reporting — Patrick Durusau @ 8:52 am

First Draft – July Fake News Quiz by Alastair Reid.

You think you keep up with current events?

Or can read a story, critically, to separate fact from fiction?

Hopefully you will do better than I did this month!

Enjoy!

PS: It just occurred to me that this is an excellent resource for secondary school teachers who are teaching students to keep up with the news. Pass this along on education/teaching channels.

July 31, 2016

Who Decides On Data Access?

Filed under: Ethics,Journalism,News,Reporting — Patrick Durusau @ 9:35 am

In a Twitter dust-up following The Privileged Cry: Boo, Hoo, Hoo Over Release of OnionScan Data the claim was made by [Λ•]ltSciFi@altscifi_that:

@SarahJamieLewis You take an ethical stance. @patrickDurusau does not. Note his regression to a childish tone. Also: schneier.com/blog/archives/…

To which I responded:

@altscifi_ @SarahJamieLewis Interesting. Questioning genuflection to privilege is a “childish tone?” Is name calling the best you can do?

Which earned this response from [Λ•]ltSciFi@altscifi_:

@patrickDurusau @SarahJamieLewis Not interested in wasting time arguing with you. Your version of “genuflection” doesn’t merit the effort.

Anything beyond name calling is too much effort for [Λ•]ltSciFi@altscifi_. Rather than admit they haven’t thought about the issue of the ethics of data access beyond “me too!,” it saves face to say discussion is a waste of time.

I have never denied that access to data can raise ethical issues or that such issues merit discussion.

What I do object to is that in such discussions, it has been my experience (important qualifier), that those urging ethics of data access have someone in mind to decide on data access. Almost invariably, themselves.

Take the recent “weaponized transparency” rhetoric of the Sunlight Foundation as an example. We can argue about the ethics of particular aspects of the DNC data leak, but the fact remains that the Sunlight Foundation considers itself, and not you, as the appropriate arbiter of access to an unfiltered version of that data.

I assume the Sunlight Foundation would include as appropriate arbiters many of the usual news organizations what accept leaked documents and reveal to the public only so much as they choose to reveal.

Not to pick on the Sunlight Foundation, there is an alphabet soup of U.S. government agencies that make similar claims of what should or should not be revealed to the public. I have no more sympathy for their claims of the right to limit data access than more public minded organizations.

Take the data dump of OnionScan data for example. Sarah Jamie Lewis may choose to help sites for victims of abuse (a good thing in my opinion) whereas others of us may choose to fingerprint and out government spy agencies (some may see that as a bad thing).

The point being that the OnionScan data dump enables more people to make those “ethical” choices and to not be preempted because data such as the OnionScan data should not be widely available.

BTW, in a later tweet Sarah Jamie Lewis says:

In which I am called privileged for creating an open source tool & expressing concerns about public deanonymization.

Missing the issue entirely as she was quoted as expressing concerns over the OnionScan data dump. Public deanonymization, is a legitimate concern so long as we all get to decide those concerns for ourselves. Lewis is trying to dodge the issue of her weak claim over the data dump for the stronger one over public deanonymization.

Unlike most of the discussants you will find, I don’t want to decide on what data you can or cannot see.

Why would I? I can’t foresee all uses and/or what data you might combine it with. Or with what intent?

If you consider the history of data censorship by governments, we haven’t done terribly well in our choices of censors or in the results of their censorship.

Let’s allow people to exercise their own sense of ethics. We could hardly do worse than we have so far.

« Newer PostsOlder Posts »

Powered by WordPress