Archive for the ‘FOIA’ Category

FOIA Success Prediction

Friday, June 16th, 2017

Will your FOIA request succeed? This new machine will tell you by Benjamin Mullin.

From the post:

Many journalists know the feeling: There could be a cache of documents that might confirm an important story. Your big scoop hinges on one question: Will the government official responsible for the records respond to your FOIA request?

Now, thanks to a new project from a data storage and analysis company, some of the guesswork has been taken out of that question.

Want to know the chances your public records request will get rejected? Plug it into FOIA Predictor, a probability analysis web application from Data.World, and it will provide an estimation of your success based on factors including word count, average sentence length and specificity.

Accuracy?

Best way to gauge that is experience with your FOIA requests.

Try starting at MuckRock.com.

Enjoy!

Open Data = Loss of Bureaucratic Power

Friday, June 9th, 2017

James Comey’s leaked memos about meetings with President Trump illustrates one reason for the lack of progress on open data reported in FOIA This! The Depressing State of Open Data by Toby McIntosh.

From Former DOJ Official on Comey Leak: ‘Standard Operating Procedure’ Among Bureaucrats:


On “Fox & Friends” today, J. Christian Adams said the leak of the memos by Comey was in line with “standard operating procedure” among Beltway bureaucrats.

“[They] were using the media, using confidential information to advance attacks on the President of the United States. That’s what they do,” said Adams, adding he saw it go on at DOJ.

Access to information is one locus of bureaucratic power, which makes the story in FOIA This! The Depressing State of Open Data a non-surprise:

In our latest look at FOIA around the world, we examine the state of open data sets. According to the new report by the World Wide Web Foundation, the news is not good.

“The number of global truly open datasets remains at a standstill,” according to the group’s researchers, who say that only seven percent of government data is fully open.

The findings come in the fourth edition of the Open Data Barometer, an annual assessment which was enlarged this year to include 1,725 datasets from 15 different sectors across 115 countries. The report summarizes:

Only seven governments include a statement on open data by default in their current policies. Furthermore, we found that only 7 percent of the data is fully open, only one of every two datasets is machine readable and only one in four datasets has an open license. While more data has become available in a machine-readable format and under an open license since the first edition of the Barometer, the number of global truly open datasets remains at a standstill.

Based on the detailed country-by-country rankings, the report says some countries continue to be leaders on open data, a few have stepped up their game, but some have slipped backwards.

With open data efforts at a standstill and/or sliding backwards, waiting for bureaucrats to voluntarily relinquish power is a non-starter.

There are other options.

Need I mention the Office of Personnel Management hack? The highly touted but apparently fundamentally vulnerable NSA?

If you need a list of cyber-vulnerable U.S. government agencies, see: A-Z Index of U.S. Government Departments and Agencies.

You can:

  • wait for bureaucrats to abase themselves,
  • post how government “…ought to be transparent and accountable…”
  • echo opinions of others on calling for open data,

or, help yourself to government collected, generated, or produced data.

Which one do you think is more effective?

FOIA Data Models for Everyone [If You Are Going To Ask]

Monday, May 8th, 2017

FOIA Data Models for Everyone by Jeremy B. Merrill.

From the post:

Listen to two FOIA practitioners describe their request strategies and you’ll probably get two very different answers. I know because I’ve done it. As someone with not much of a personal FOIA strategy—besides “wait and hope”—I was surprised that journalists skilled at prying obscure records from the government have wildly different approaches.

These differences in how to engage with the FOIA process can cover questions that are flashy—to us nerds—like whether to ask for “any and all” documents or to call the officer every week or so. But the idiosyncrasies in journalists’ mental models trickle down even into the little details, like how they keep track of agencies’ contact info.

When I began an internal FOIA tracker app for the New York Times, I knew I’d have to understand different mental models of the FOIA process in order to represent that process in a database. So, I put out a call to the friendly community of news nerds on Twitter and in the NewsNerdery Slack:

Tracking your FOIAs with a spreadsheet (or an app) is a best practice. But everyone’s chart is a little different and probably encodes different nuggets of hard-earned wisdom. Care to share the column headers from your spreadsheet?

Computers don’t know anything about FOIAs. Bless their hearts, but they’re dumb; data modeling is how we imbue computers with little morsels of our human wisdom hidden in row 1 of a spreadsheet. I collated the results—from eight individuals’ spreadsheets and two open-source FOIA tracker apps plus my own, so hopefully a lot of little morsels of wisdom—and analyzed them to see what I might have missed. I want to share the results back to the community.

A gold mine of curated advice and practices on FOIA data models.

Old pros and newbies at FOIA requests are going to benefit from Merrill’s post.

Be sure to ping him @jeremybmerrill to show your appreciation for this summary.

Writing Good FOIA Requests

Tuesday, January 31st, 2017

What makes a good FOIA request? We studied 33,000 to find out by By Nicholas Dias, Rashida Kamal and Laurent Bastien.

From the post:

EVERY JOURNALIST HAS IDEAS about what makes a good public records request. But surprisingly few people have actually tried to systematically analyze how requests can be written to improve their chances of success.

To fill this vacuum, we analyzed more than 33,000 Freedom of Information Act requests and identified a few characteristics that were typical of those that were fulfilled.

The requests were made to five federal agencies that publish to FOIAonline.gov: the Environmental Protection Agency, the Department of Commerce, Customs and Border Protection, the Department of the Navy, and the National Archives and Records Administration. All were filed between 2011 and 2016.

We defined success as the receipt of all records requested, as defined by the agency. There was no straightforward relationship between wait time and any of the characteristics we considered, so we factored it out as a measure of success.

For the requests we examined, the full-grant rate across all five agencies was around 23 percent. That’s the same success rate for requests across all federal agencies, according to Max Galka of FOIA Mapper, a project funded by the Knight Foundation that outlines the record systems of federal agencies.

Requesters in our sample typically waited around 142 days, or a little more than four months, to get responses. Less than 39 percent of requests received responses within 28 days, which is the longest amount of time an agency could spend fulfilling a request while still meeting FOIA’s 20-business-day time limit.

That’s a pretty bleak picture. So, how can you improve your chances?
… (emphasis in original)

What? Evidence-based FOIA practices? 😉

After reading this review of FOIA practices, get thee to MuckRock.

MuckRock has advice, tools, community, in short, it is a one-stop FOIA shop.

What FOIA request(s) are you going to file?

Do You Have Big Brass Ones*? FOIA The President

Wednesday, January 18th, 2017

Join our project to FOIA the Trump administration by Michael Morisy.

From the post:

Since June 2015, MuckRock users have been filing FOIA requests regarding a possible Trump presidency. In fact, so far there’s been over 160 public Trump-related requests filed through the site, all of which you can browse here.

We’ve also put together a number of guides and articles on the upcoming administration, ranging from what you can and can’t file regarding Trump to deep dives into what’s already out there:

We’ve launched a new project page for users to showcase their requests, find new documents regarding the Trump administration, or get inspiration for their own requests, and we’ve created a special Slack channel for you to join in and strategize on future requests, or help share big league FOIA stories that shed light on the President Elect’s team.

We’ve had a few users join us there already and they’ve helped file some really fun requests, so we’re excited about what else the transparency community can come up with.

An effort worthy of both your time and support!

One answered, remember that availability isn’t the same thing as meaningful access.

OCR, indexing, entity extraction, in short any skill you have is important in this effort.

* No longer a gender specific reference as you well know.

PS: I’ve signed up and need suggestions on what to ask for? Suggestions?

MuckRock Needs Volunteers (9 states in particular)

Monday, November 21st, 2016

MuckRock needs your help to keep filing in all 50 states by Beryl Lipton.

From the post:

Election time excitement got you feeling a little more patriotic than usual? Looking for a way to help but not sure you have the time? Well, MuckRock is looking for a few good people to do a big service requiring little effort: serve as our resident proxies.

A few states have put up barriers at their borders, limiting required disclosure and response to requests to only residents. One more thing added to the regular rigamarole of requesting public records, it’s huge block to comparative studies and useful, outside accountability.

This is where you come in.

proxymap-460

We’re looking for volunteers in the ten states that can whip out their residency requirements whenever they get the chance:

  • Alabama
  • Arkansas
  • Georgia
  • Missouri
  • Montana.
  • New Hampshire
  • New Jersey
  • Tennessee
  • Virginia

As a MuckRock proxy requester, you’ll serve as the in-state request representative, allowing requests to be submitted in your name and enabling others to continue to demand accountability. In exchange, you’ll get your own Professional MuckRock account – 20 requests a month and all that comes with them – and the gratitude of the transparency community.

Interested in helping the cause? Let us know at info@muckrock.com, or via the from below.

Despite my view that government disclosures are previously undisclosed government lies, I have volunteered for this project.

Depending on where you reside, you should too and/or contribute to support MuckRock.

CIA Raises Technical Incompetence Flag

Saturday, November 19th, 2016

The CIA‘s responded to Michael Morisy‘s request for:

“a copy of emails sent to or from the CIA’s FOIA office regarding FOIA Portal’s Technical Issues.”

gives these requirements for requesting emails:

We require requesters seeking any form of “electronic communications” such as emails, to provide the specific “to” and “from” recipients, time frame and subject.

(The full response.)

Recalling that the FBI requested special software to separate emails of Huma Abedin and Anthony Weiner on the same laptop, is the CIA really that technically incompetent in terms of searching?

Is the CIA is incapable of searching emails by subject alone?

With a dissatisfied-with-intelligence-community president-elect Donald Trump about to take office, I would not be flying the Technical Incompetence Here flag.

The CIA may respond it is not incompetent but rather was acting in bad faith.

In debate we used to call that the “horns of a dilemma,” yes?

I’m voting for bad faith.

How about you?

Attn: Secrecy Bed-Wetters! All Five Volumes of Bay of Pigs History Released!

Thursday, November 3rd, 2016

Hand-wringers and bed-wetters who use government secrecy to hide incompetence and errors will sleep less easy tonight.

All Five Volumes of Bay of Pigs History Released and Together at Last: FRINFORMSUM 11/3/2016 by Lauren Harper.

From the post:

After more than twenty years, it appears that fear of exposing the Agency’s dirty linen, rather than any significant security information, is what prompts continued denial of requests for release of these records. Although this volume may do nothing to modify that position, hopefully it does put one of the nastiest internal power struggles into proper perspective for the Agency’s own record.” This is according to Agency historian Jack Pfeiffer, author of the CIA’s long-contested Volume V of its official history of the Bay of Pigs invasion that was released after years of work by the National Security Archive to win the volume’s release. Chief CIA Historian David Robarge states in the cover letter announcing the document’s release that the agency is “releasing this draft volume today because recent 2016 changes in the Freedom of Information Act (FOIA) requires us to release some drafts that are responsive to FOIA requests if they are more than 25 years old.” This improvement – codified by the FOIA Improvement Act of 2016 – came directly from the National Security Archive’s years of litigation.

The CIA argued in court for years – backed by Department of Justice lawyers – that the release of this volume would “confuse the public.” National Security Archive Director Tom Blanton says, “Now the public gets to decide for itself how confusing the CIA can be. How many thousands of taxpayer dollars were wasted trying to hide a CIA historian’s opinion that the Bay of Pigs aftermath degenerated into a nasty internal power struggle?”

To read all five volumes of the CIA’s Official History of the Bay of Pigs Operation – together at last – visit the National Security Archive’s website.

Even the CIA’s own retelling of the story, The Bay of Pigs Invasion, ends with a chilling reminder for all “rebels” being presently supported by the United States.


Brigade 2506’s pleas for air and naval support were refused at the highest US Government levels, although several CIA contract pilots dropped munitions and supplies, resulting in the deaths of four of them: Pete Ray, Leo Baker, Riley Shamburger, and Wade Gray.

Kennedy refused to authorize any extension beyond the hour granted. To this day, there has been no resolution as to what caused this discrepancy in timing.

Without direct air support—no artillery and no weapons—and completely outnumbered by Castro’s forces, members of the Brigade either surrendered or returned to the turquoise water from which they had come.

Two American destroyers attempted to move into the Bay of Pigs to evacuate these members, but gunfire from Cuban forces made that impossible.

In the following days, US entities continued to monitor the waters surrounding the bay in search of survivors, with only a handful being rescued. A few members of the Brigade managed to escape and went into hiding, but soon surrendered due to a lack of food and water. When all was said and done, more than seventy-five percent of Brigade 2506 ended up in Cuban prisons.

100% captured or killed. There’s an example of US support.

In a less media savvy time, the US did pay $53 million (in 1962 dollars, about $424 million today) for the release of 1113 members of Brigade 2506.

Another important fact is that fifty-seven (57) years of delay enabled the participants to escape censure and/or a trip to the gallows for their misdeeds and crimes.

Let’s not let that happen with the full CIA Torture Report. Even the sanitized 6,700 page version would be useful. More so the documents upon which it was based.

All of that exists somewhere. We lack a person with access and moral courage to inform their fellow citizens of the full truth about the CIA torture program. So far.


Update: Michael Best, NatSecGeek advises CIA Histories has the most complete CIA history collection. Thanks Michael!

Six lessons from a five-year FOIA battle [Cheat Sheet]

Wednesday, September 28th, 2016

Six lessons from a five-year FOIA battle by Philip Eil

From the post:

I FILED MY FIRST Freedom of Information Act request on February 1, 2012. I was 26 years old, and chasing a story about my father’s med-school classmate, Dr. Paul Volkman, who had been convicted of a massive prescription drug dealing scheme the previous year. The aim of the request was simple: I wanted to see the evidence the jury saw during Volkman’s eight-week trial in Cincinnati for a book I’m writing about the case. (Volkman went to college and medical school with my father.) But everyone I asked—the US district court clerk, the appellate court clerk, the prosecutor, and the judge who presided over the case—declined to give me the documents. It was time to make an official request to the Department of Justice.

To make a very long story short, in March of 2015, that FOIA request turned into my first FOIA lawsuit. And, earlier this month, I received my first FOIA judgment, which, I’m happy to report, is also my first FOIA-lawsuit victory. In a 17-page decision, US District Court Judge Jack McConnell cited Serial and Making A Murderer, wrote “Public scrutiny of judicial proceedings produces a myriad of social benefits,” and ordered the Drug Enforcement Administration to fork over the requested documents within 60 days.

It took more than four and half years to receive that judgment. And, during that time I began joking that I had attended “FOIA University.” Today, the prospect of postgraduate study looms; if the government appeals, it could extend this ordeal by months, if not years. But, with Judge McConnell’s decision in hand, I’d like to share a few of the things I’ve learned. What follows is the cheat-sheet I wish I someone had handed me five years ago.
… (emphasis in the original)

Not only will you find Philip’s “cheat sheet” useful, it is also inspirational.

Enjoy!

Building A National FOIA Rejection Database (MuckRock)

Thursday, July 14th, 2016

MuckRock is launching a national database of FOIA exemptions by Joseph Licterman.

From the post:

In the 2015 fiscal year, the U.S. federal government processed 769,903 Freedom of Information requests. The government fully fulfilled only 22.6 percent of those requests; 44.9 percent of federal FOIA requests were either partially or fully denied. Even though the government denied at least part of more than 345,000 requests, it only received 14,639 administrative appeals.

In an attempt to make the FOIA appeals process easier and help reporters and others understand how and why their requests are being denied, MuckRock is on Thursday launching a project to catalog and explain the exceptions both the federal and state governments are using to deny requests.

MuckRock is a nonprofit site that helps its users file FOIA requests, and cofounder Michael Morisy said that the site is planning to create a “Google for FOIA rejections” which will help users understand why their requests were denied and learn what they can do to appeal the case.

If your FOIA request is rejected, who knows about it? You and maybe a few colleagues?

If you contribute your rejected FOIA requests to this MuckRock project, your rejected requests will join thousands of others to create a database on which the government can be held accountable for its FOIA behavior.

Don’t let your rejected FOIA request languish in filing cabinets and boxes, contribute them along with support to MuckRock!

The government isn’t the only party that can take names and keep records.

On-Again/Off-Again Democracy In New York

Friday, June 3rd, 2016

Andrew Denney reports in Panel Supports City’s Denial of Data on NYPD Surveillance that the NYPD can refuse to acknowledge the existence of records requested under the state equivalent of FOIA.

From the post:

Police properly applied a legal doctrine allowing it to refuse to acknowledge the existence of records, requested under state Freedom of Information Law, that related to surveillance programs, a Manhattan appeals court found.

The ruling by the Appellate Division, First Department, settles a dispute between two trial judges who disagreed in 2014 as to whether the New York City Police Department could use the “Glomar Doctrine.” The policy allows federal departments to cite security concerns to neither confirm nor deny the existence of records requested under the federal Freedom of Information Act.

The doctrine is named for an inquiry into a salvage operation of a Soviet nuclear submarine by a ship named the Hughes Glomar Explorer.

An NYPD spokesman commented:

“We are all safer because of this ruling, which confirms that the NYPD is not required to reveal the targets of counterterrorism surveillance,” department spokesman Nicholas Paolucci said.

I would agree with Paolucci had he said:

“Illegal, unauthorized and abusive ‘counterterrorism surveillance’ will be safer because of this ruling.”

National (think FBI) and local law enforcement authorities have long histories of illegal misconduct, a large amount of which is only discovered years or even decades later. There is no reason to believe that “counterterrorism surveillance” is any less prone to similar abuses.

Without public oversight and transparency, “counterterrorism surveillance” is a recipe for an ongoing abuse of the rights.

Having denied the access needed for meaningful public oversight, the courts and NYPD should not complain about uncontrolled releases of the same information.

When faced with an on-again/off-again democracy, what alternative does the public have?

I first saw this in a tweet by North Star Post.