Archive for the ‘Intelligence’ Category

3 Reasons to Read: Algorithms to Live By

Monday, April 24th, 2017

How Algorithms can untangle Human Questions. Interview with Brian Christian by Roberto V. Zican.

The entire interview is worth your study but the first question and answer establish why you should read Algorithms to Live By:

Q1. You have worked with cognitive scientist Tom Griffiths (professor of psy­chol­ogy and cognitive science at UC Berkeley) to show how algorithms used by computers can also untangle very human questions. What are the main lessons learned from such a joint work?

Brian Christian: I think ultimately there are three sets of insights that come out of the exploration of human decision-making from the perspective of computer science.

The first, quite simply, is that identifying the parallels between the problems we face in everyday life and some of the canonical problems of computer science can give us explicit strategies for real-life situations. So-called “explore/exploit” algorithms tell us when to go to our favorite restaurant and when to try something new; caching algorithms suggest — counterintuitively — that the messy pile of papers on your desk may in fact be the optimal structure for that information.

Second is that even in cases where there is no straightforward algorithm or easy answer, computer science offers us both a vocabulary for making sense of the problem, and strategies — using randomness, relaxing constraints — for making headway even when we can’t guarantee we’ll get the right answer every time.

Lastly and most broadly, computer science offers us a radically different picture of rationality than the one we’re used to seeing in, say, behavioral economics, where humans are portrayed as error-prone and irrational. Computer science shows us that being rational means taking the costs of computation — the costs of decision-making itself — into account. This leads to a much more human, and much more achievable picture of rationality: one that includes making mistakes and taking chances.
… (emphasis in original)

After the 2016 U.S. presidential election, I thought the verdict that humans are error-prone and irrational was unassailable.

Looking forward to the use of a human constructed lens (computer science) to view “human questions.” There are answers to “human questions” baked into computer science so watching the authors unpack those will be an interesting read. (Waiting for my copy to arrive.)

Just so you know, the Picador edition is a reprint. It was originally published by William Collins, 21/04/2016 in hardcover, see: Algorithms to Live By, a short review by Roberto Zicari, October 24, 2016.

Building a Keyword Monitoring Pipeline… (Think Download Before Removal)

Wednesday, April 19th, 2017

Building a Keyword Monitoring Pipeline with Python, Pastebin and Searx by Justin Seitz.

From the post:

Having an early warning system is an incredibly useful tool in the OSINT world. Being able to monitor search engines and other sites for keywords, IP addresses, document names, or email addresses is extremely useful. This can tell you if an adversary, competitor or a friendly ally is talking about you online. In this blog post we are going to setup a keyword monitoring pipeline so that we can monitor both popular search engines and Pastebin for keywords, leaked credentials, or anything else we are interested in.

The pipeline will be designed to alert you whenever one of those keywords is discovered or if you are seeing movement for a keyword on a particular search engine.

Learning of data that was posted but is no longer available, is a sad thing.

Increase your odds of grabbing data before removal by following Justin’s post.

A couple of caveats:

  • I would not use GMail, preferring a Tor mail solution, especially for tracking Pastebin postings.
  • Use and rotate at random VPN connections for your Searx setup.

Going completely dark takes more time and effort than most of us can spare, but you can avoid being like a new car dealership with search lights crossing the sky.

CIA To Silence Wikileaks? Donate/Leak to Wikileaks

Thursday, April 13th, 2017

CIA chief targets WikiLeaks and Julian Assange as ‘hostile,’ vows to take action by Tim Johnson.

From the post:

CIA Director Mike Pompeo on Thursday called the anti-secrecy group WikiLeaks a hostile intelligence service and said the group would soon face decisive U.S. action to stifle its disclosures of leaked material.

“It ends now,” Pompeo said in his first public remarks after 10 weeks on the job, indicating that President Donald Trump will take undefined but forceful action.

Pompeo lashed out aggressively against Julian Assange, the Australian founder of WikiLeaks – who has been holed up in the Ecuadorean embassy in London for nearly five years – calling him a narcissist and “a fraud, a coward hiding behind a screen.”

Really?

Given the perennial failure of the CIA to discover terror attacks before they happen, recognize when governments are about to fall, and maintain their own security, I can’t imagine Assange and Wikileaks are shaking in their boots.

I disagree with Wikileaks on their style of leaking, I prefer faster and unedited leaking but that’s a question of style and not whether to leak.

If, and it’s a big if, Wikileaks is silenced, the world will grow suddenly darker. Much of what Wikileaks has published would not be published by main stream media, much to the detriment of citizens around the world.

Two things you need to do:

The easy one, donate to support WikiLeaks. As often as you can.

The harder one, leak secrets to Wikileaks.

Repressive governments are pressing WikiLeaks, help WikiLeaks make a fire hose of leaks to push them back.

Raw SIGINT Locations Expanded

Tuesday, January 17th, 2017

President Obama has issued new rules for sharing information under Executive Order 12333, with the ungainly title: (U) Procedures for the Availability or Dissemination of Raw Signals Intelligence Information by the National Security Agency Under Section 2.3 of Executive Order 12333 (Raw SIGINT Availability Procedures).

Kate Tummarello, in Obama Expands Surveillance Powers On His Way Out by Kate Tummarello, sees a threat to “innocent persons:”

With mere days left before President-elect Donald Trump takes the White House, President Barack Obama’s administration just finalized rules to make it easier for the nation’s intelligence agencies to share unfiltered information about innocent people.

New rules issued by the Obama administration under Executive Order 12333 will let the NSA—which collects information under that authority with little oversight, transparency, or concern for privacy—share the raw streams of communications it intercepts directly with agencies including the FBI, the DEA, and the Department of Homeland Security, according to a report today by the New York Times.

That’s a huge and troubling shift in the way those intelligence agencies receive information collected by the NSA. Domestic agencies like the FBI are subject to more privacy protections, including warrant requirements. Previously, the NSA shared data with these agencies only after it had screened the data, filtering out unnecessary personal information, including about innocent people whose communications were swept up the NSA’s massive surveillance operations.

As the New York Times put it, with the new rules, the government claims to be “reducing the risk that the N.S.A. will fail to recognize that a piece of information would be valuable to another agency, but increasing the risk that officials will see private information about innocent people.”

All of which is true, but the new rules have other impacts as well.

Who is an “IC element?”

The new rules make numerous references to an “IC element,” but comes up short in defining them:

L. (U) IC element is as defined in section 3.5(h) of E.O. 12333.
(emphasis in original)

Great.

Searching for E.O. 12333 isn’t enough. You need Executive Order 12333 United States Intelligence Activities (As amended by Executive Orders 13284 (2003), 13355 (2004) and 13470 (2008)). The National Archives version of Executive Order 12333 is not amended and hence is misleading.

From the amended E.0. 12333:

3.5 (h) Intelligence Community and elements of the Intelligence Community 
        refers to:
(1) The Office of the Director of National Intelligence;
(2) The Central Intelligence Agency;
(3) The National Security Agency;
(4) The Defense Intelligence Agency;
(5) The National Geospatial-Intelligence Agency;
(6) The National Reconnaissance Office; 
(7) The other offices within the Department of Defense for the collection of 
    specialized national foreign intelligence through reconnaissance programs;
(8) The intelligence and counterintelligence elements of the Army, the Navy,
    the Air Force, and the Marine Corps;
(9) The intelligence elements of the Federal Bureau of Investigation;
(10) The Office of National Security Intelligence of the Drug Enforcement
     Administration;
(11) The Office of Intelligence and Counterintelligence of the Department
      of Energy;
(12) The Bureau of Intelligence and Research of the Department of State;
(13) The Office of Intelligence and Analysis of the Department of the Treasury;
(14) The Office of Intelligence and Analysis of the Department of Homeland 
     Security;
(15) The intelligence and counterintelligence elements of the Coast Guard; and
(16) Such other elements of any department or agency as may be designated by 
     the President, or designated jointly by the Director and the head of the 
     department or agency concerned, as an element of the Intelligence Community. 

The Office of the Director of National Intelligence has an incomplete list of IC elements:

Air Force Intelligence Defense Intelligence Agency Department of the Treasury National Geospatial-Intelligence Agency
Army Intelligence Department of Energy Drug Enforcement Administration National Reconnaissance Office
Central Intelligence Agency Department of Homeland Security Federal Bureau of Investigation National Security Agency
Coast Guard Intelligence Department of State Marine Corps Intelligence Navy Intelligence

I say “incomplete” because from E.O. 12333, it is missing (with original numbers for reference):

...
(7) The other offices within the Department of Defense for the collection of 
    specialized national foreign intelligence through reconnaissance programs;
(8) The intelligence and counterintelligence elements of ..., and the 
    Marine Corps;
...
(16) Such other elements of any department or agency as may be designated by 
     the President, or designated jointly by the Director and the head of the 
     department or agency concerned, as an element of the Intelligence Community.

Under #7 and #16, there are other IC elements that are unnamed and unlisted by the Office of the DOI. I suspect the Marines were omitted for stylistic reasons.

Where to Find Raw SIGINT?

Identified IC elements are important because the potential presence of “Raw SIGINT,” beyond the NSA, has increased their value as targets.

P. (U) Raw SIGINT is any SIGINT and associated data that has not been evaluated for foreign intelligence purposes and/or minimized.
… (emphasis in original, from the new rules.)

Tummarello is justly concerned about “innocent people” but there are less than innocent people, any number of appointed/elected official or barons of industry who may be captured on the flypaper of raw SIGINT.

Happy hunting!

PS:

Warning: It’s very bad OPSEC to keep a trophy chart on your wall. 😉

IC_Circle-460

You will, despite this warning, but I had to try.

The original image is here at Wikipedia.

CIA Cartography [Comparison to other maps?]

Monday, November 28th, 2016

CIA Cartography

From the webpage:

Tracing its roots to October 1941, CIA’s Cartography Center has a long, proud history of service to the Intelligence Community (IC) and continues to respond to a variety of finished intelligence map requirements. The mission of the Cartography Center is to provide a full range of maps, geographic analysis, and research in support of the Agency, the White House, senior policymakers, and the IC at large. Its chief objectives are to analyze geospatial information, extract intelligence-related geodata, and present the information visually in creative and effective ways for maximum understanding by intelligence consumers.

Since 1941, the Cartography Center maps have told the stories of post-WWII reconstruction, the Suez crisis, the Cuban Missile crisis, the Falklands War, and many other important events in history.

There you will find:

Cartography Tools 211 photos

Cartography Maps 1940s 22 photos

Cartography Maps 1950s 14 photos

Cartography Maps 1960s 16 photos

Cartography Maps 1970s 19 photos

Cartography Maps 1980s 12 photos

Cartography Maps 1990s 16 photos

Cartography Maps 2000s 16 photos

Cartography Maps 2010s 15 photos

The albums have this motto at the top:

CIA Cartography Center has been making vital contributions to our Nation’s security, providing policymakers with crucial insights that simply cannot be conveyed through words alone.

President-elect Trump is said to be gaining foreign intelligence from sources other than his national security briefings. Trump is ignoring daily intelligence briefings, relying on ‘a number of sources’ instead. That report is based on a Washington Post account, which puts its credibility somewhere between a conversation overhead in a laundry mat and a stump speech by a member of Congress.

Assuming Trump is gaining intelligence from other sources, just how good are other sources of intelligence?

This release of maps by the CIA, some 160 maps spread from the 1940’s to the 2010’s, provides one axis for evaluating CIA intelligence versus what was commonly known at the time.

As a starting point, may I suggest: Image matching for historical maps comparison by C. Balletti and F. Guerrae, Perimetron, Vol. 4, No. 3, 2009 [180-186] www.e-perimetron.org | ISSN 1790-3769?

Abstract:

In cartographic heritage we suddenly find maps of the same mapmaker and of the same area, published in different years, or new editions due to integration of cartographic, such us in national cartographic series. These maps have the same projective system and the same cut, but they present very small differences. The manual comparison can be very difficult and with uncertain results, because it’s easy to leave some particulars out. It is necessary to find an automatic procedure to compare these maps and a solution can be given by digital maps comparison.

In the last years our experience in cartographic data processing was opted for find new tools for digital comparison and today solution is given by a new software, ACM (Automatic Correlation Map), which finds areas that are candidate to contain differences between two maps. ACM is based on image matching, a key component in almost any image analysis process.

Interesting paper but it presupposes a closeness of the maps that is likely to be missing when comparing CIA maps to other maps of the same places and time period.

I am in the process of locating other tools for map comparison.

Any favorites you would like to suggest?

Why I Distrust US Intelligence Experts, Let Me Count the Ways

Sunday, October 16th, 2016

Some US Intelligence failures, oldest to most recent:

  1. Pearl Harbor
  2. The Bay of Pigs Invasion
  3. Cuban Missile Crisis
  4. Vietnam
  5. Tet Offensive
  6. Yom Kippur War
  7. Iranian Revolution
  8. Soviet Invasion of Afghanistan
  9. Collapse of the Soviet Union
  10. Indian Nuclear Test
  11. 9/11 Attacks
  12. Iraq War (WMDs)
  13. Invasion of Afghanistan (US)
  14. Israeli moles in US intelligence, various dates

Those are just a few of the failures of US intelligence, some of which cost hundreds of thousands if not millions of lives.

Yet, you can read today: Trump’s refusal to accept intelligence briefing on Russia stuns experts.

There are only three reasons I can think of to accept findings by the US intelligence community:

  1. You are on their payroll and for that to continue, well, you know.
  2. As a member of the media, future tips/leaks depends upon your acceptance of current leaks. Anyone who mocks intelligence service lies is cut off from future lies.
  3. As a politician, the intelligence findings discredit facts unfavorable to you.

For completeness sake, I should mention that intelligence “experts” could be telling the truth but given their track record, it is an edge case.

Before repeating the mindless cant of “the Russians are interfering with the US election,” stop to ask your sources, “…based on what?” Opinions of all the members of the US intelligence community = one opinion. Ask for facts. No facts offered, report that instead of the common “opinion.”

Threat Intelligence Starter Resources

Sunday, August 14th, 2016

Threat Intelligence Starter Resources by Amanda McKeon.

From the post:

Creating a threat intelligence capability can be a challenging undertaking, and not all companies are ready for it. Businesses that run successful threat intelligence teams generally:

  • Collect externally available data on threats and correlate it with internal events.
  • Be aware of threats driving proactive security controls.
  • Establish proactive internal hunting for unidentified threats.
  • Invest in employee and customer threat education.
  • Expand security industry peer relationships.
  • Apply methods for collecting and analyzing external threat data.

For more information, read our white paper on building an advanced threat intelligence team.

Now, if your company is just starting out with threat intelligence and doesn’t have the time or resources to dedicate an entire department to the task, there are some easy ways to begin integrating threat intelligence into your daily routine without breaking the bank.

The following resources can help build awareness of the threat landscape and prepare your company for defense.

Great starting points for collection of general threat intelligence.

Unfortunately, the elimination of repetition of the same information/reports from different sources, separation of surmises from facts, etc., remain the responsibility of the reader.

Failure of Thinking and Visualization

Wednesday, August 10th, 2016

Richard Bejtlich posted this image (thumbnail, select for full size) with the note:

When I see senior military schools create slides like this, I believe PPT is killing campaign planning. @EdwardTufte

enemy-is-ppt

I am loathe to defend PPT but the problem here lies with the author and not PPT.

Or quite possibly with concept of “center of gravity analysis.”

Whatever your opinion about the imperialistic use of U.S. military force, 😉 , the U.S. military is composed of professional warriors who study their craft in great detail.

On the topic “center of gravity analysis,” try Addressing the Fog of COG: Perspectives on the Center of Gravity in US Military Doctrine, Celestino Perez, Jr., General Editor. A no-holds barred debate by military professionals on COG.

With or without a background on COG, how do your diagrams compare to this one?

Cooked Intel, Again (Anyone Surprised?)

Wednesday, August 10th, 2016

ISIS Intel Was Cooked, House Panel Finds by Nancy A. Youssef and Shane Harris.

From the post:

A House Republican task force has found that officials from the U.S. military’s Central Command altered intelligence reports to portray the U.S. fight against ISIS and al Qaeda in a more positive light than lower-level analysts believed was warranted by the facts on the ground, three officials familiar with the task force’s findings told The Daily Beast.

A roughly 10-page report on the controversy is expected to be released by the end of next week, two officials said. While it contains no definitive evidence that senior Obama administration officials ordered the reports to be doctored, the five-month investigation did corroborate earlier reports that analysts felt the leaders of CENTCOM’s intelligence directorate pressured them to conclude that the threat from ISIS was not as ominous as the analysts believed, the officials said.

To paraphrase Nietzsche, “…there are no facts, only politically convenient interpretations.”

Publications that strive for accuracy should omit any claims or statements of U.S. intelligence sources unless confirmed independently by non-intelligence sources.

If you are competing for click-bait, use U.S. intelligence sources without verification.

Telephone Metadata Can Reveal Surprisingly Sensitive Personal Information

Wednesday, August 3rd, 2016

Stanford computer scientists show telephone metadata can reveal surprisingly sensitive personal information by Bjorn Carey.

The intelligence community assertion that telephone metadata only enables “connecting the dots,” has been confirmed to be a lie.

From the post:

Most people might not give telephone metadata – the numbers you dial, the length of your calls – a second thought. Some government officials probably view it as similarly trivial, which is why this information can be obtained without a warrant.

But a new analysis by Stanford computer scientists shows that it is possible to identify a person’s private information – such as health details – from metadata alone. Additionally, following metadata “hops” from one person’s communications can involve thousands of other people.

The researchers set out to fill knowledge gaps within the National Security Agency’s current phone metadata program, which has drawn conflicting assertions about its privacy impacts. The law currently treats call content and metadata separately and makes it easier for government agencies to obtain metadata, in part because it assumes that it shouldn’t be possible to infer specific sensitive details about people based on metadata alone.

The findings, reported today in the Proceedings of the National Academy of Sciences, provide the first empirical data on the privacy properties of telephone metadata. Preliminary versions of the work, previously made available online, have already played a role in federal surveillance policy and have been cited in litigation filings and letters to legislators in both the United States and abroad. The final work could be used to help make more informed policy decisions about government surveillance and consumer data privacy.

The computer scientists built a smartphone application that retrieved the previous call and text message metadata – the numbers, times and lengths of communications – from more than 800 volunteers’ smartphone logs. In total, participants provided records of more than 250,000 calls and 1.2 million texts. The researchers then used a combination of inexpensive automated and manual processes to illustrate both the extent of the reach – how many people would be involved in a scan of a single person – and the level of sensitive information that can be gleaned about each user.

From a small selection of the users, the Stanford researchers were able to infer, for instance, that a person who placed several calls to a cardiologist, a local drugstore and a cardiac arrhythmia monitoring device hotline likely suffers from cardiac arrhythmia. Another study participant likely owns an AR semiautomatic rifle, based on frequent calls to a local firearms dealer that prominently advertises AR semiautomatic rifles and to the customer support hotline of a major firearm manufacturer that produces these rifles.

One of the government’s justifications for allowing law enforcement and national security agencies to access metadata without warrants is the underlying belief that it’s not sensitive information. This work shows that assumption is not true.

See Carey’s post for the laypersons explanation of the Stanford findings or dive into Evaluating the privacy properties of telephone metadata by Jonathan Mayera, Patrick Mutchler, and John C. Mitchell, for more detailed analysis. (Thankfully open access.)

Would law enforcement and national security agencies think telephone metadata is not sensitive if hackers were obtaining it from telecommunication companies and/or from the electromagnetic field where communication signals are found?

If you were interested only in law enforcement, national security agencies and governments, a much smaller set of data for tracking and processing.

Sounds like a business opportunity, depending on what country, their degree of technology, market conditions for pro/anti government data.

U.S. government satellites collect such data but it is shared (or not) for odd and obscure reasons.

I’m thinking more along the lines of commercial transactions between willing sellers and buyers.

Think of it as a Rent-An-NSA type venture. Customers don’t want or need 24×7 rivals for power. Properly organized, they could buy as much or as little intelligence as they need. Exclusive access to some intelligence would be a premium product.

Intelligence Suicide By Data

Wednesday, June 8th, 2016

Facing Data Deluge, Secret U.K. Spying Report Warned of Intelligence Failure by Ryan Gallagher.

From the post:


The amount of data being collected, however, proved difficult for MI5 to handle. In March 2010, in another secret report, concerns were reiterated about the agency’s difficulties processing the material it was harvesting. “There is an imbalance between collection and exploitation capabilities, resulting in a failure to make effective use of some of the intelligence collected today,” the report noted. “With the exception of the highest priority investigations, a lack of staff and tools means that investigators are presented with raw and unfiltered DIGINT data. Frequently, this material is not fully assessed because of the significant time required to review it.”

Ironic this story appears less than two (2) weeks after reports of the FBI seeking NSL (national security letter) authority to obtain email records and browsing histories.

gun_suicide_silhouette_800x600-460

I should not complain about the FBI, NSA and other government agencies committing intelligence suicide by data.

Their rapidly growing ineffectiveness shields innocents from their paranoid fantasies.

At the same time, that ineffectiveness inhibits the performance of legitimate purposes. (The FBI, once upon a time, had a legitimate purpose, some of the others, well, that’s an issue for debate.)

So we are clear, I don’t consider contracts for “butts in seats” for either contractors or agencies to be for “legitimate purposes.” I reserve the phrase “legitimate purposes” for activities that further the stated goals of the agency, not padding staffing rolls, not occupying as much office space as possible, not having the most forms or whatever other criteria functions as the measure of success in a particular agency.

Hints for federal agencies already committing intelligence suicide by data or approaching that point:

  1. What data sources have proven valuable in the past? (Reminder: Phone metadata records have not. Not ever.)
  2. What data sources, in order of historical importance, are available in case X?
  3. Assemble the data from the top performing resources

For example, if an informant has direct contact with an alleged Islamic State supporter, isn’t that the best source of evidence for their plans and thinking? Do you really need their websearch history from an internet services provider? Considering that you will ask for everyone’s web search history to avoid disclosing the particular web history you are seeking.

To be sure, vendors will sell you as much data processing and storage capacity as you care to purchase, but you won’t be any closer to stopping terrorism. Just closer to the end of your budget for the current fiscal year.

Is intelligence suicide by data a goal of your agency?

Twitter Giveth and Taketh Away (NSA as Profit Center?)

Monday, May 16th, 2016

Twitter Giveth: GCHQ intelligence agency joins Twitter. Just about anyone can get a Twitter account these days.

Do see the GCHQ GitHub site for shared software.

Taketh Away Twitter Bars Intelligence Agencies From Using Analytics Service.

Twitter has barred Dataminr from providing services to government intelligence services.

Dataminr monitors the entire Twitter pipe and provides analytics based on that stream.

Will this result in the NSA sharing its signal detection in the Twitter stream with other intelligence agencies?

Or for that matter, the NSA could start offering commercial signal detection services across all its feeds. Make it a profit center for the government rather than a money pit.

BTW, don’t be deceived by the illusion of space between government and Twitter, or any other entity that cooperates with a national government. Take “compromised” as a given. The real questions are by who and for what purpose?

“Lite” Lists of Intelligence Agencies

Wednesday, May 4th, 2016

I referenced World Wide Intelligence (and defense) Agencies as a list of intelligence agencies, but looking at it later, it appears to be a bit “lite.”

There are one hundred and forty-five (145) agencies by my count.

I think what captured my attention is that there are no intelligence agencies for Latin or South America. Come to think of it, there are no intelligence agencies for Africa as well.

Whereas, the List of Intelligence Agencies (Wikipedia) gives a rough count of six hundred and sixty-four (664) intelligence/signal agencies.

The advantage of the World Wide Intelligence (and defense) Agencies list is that it has URLs for the agencies themselves.

The larger Wikipedia list has links to other Wikipedia pages. Useful I suppose for the social engineering required for hacking a security service but not useful as a quick list of URLs for intelligence agencies.

The Federation of American Scientists (FAS) maintains a set of webpages that start with World Intelligence and Security Agencies. Organized by country and below the country pages, the amount of details varies. Pages have been updated unevenly and should be checked before relying on the information you find.

The Crypto Museum also maintains a list of intelligence
organizations.

None of the lists appear to be “complete.”

I didn’t see any listing for the fifty (50) state police organizations in the United States. Nor any for major cities, such as Chicago which operates its own gulag.

I haven’t looked on the “Dark Web” but I assume useful lists there are fairly expensive.

Enjoy!

“No One Willingly Gives Away Power”

Friday, April 8th, 2016

Matthew Schofield in European anti-terror efforts hobbled by lack of trust, shared intelligence hits upon the primary reason for resistance to topic maps and other knowledge integration technologies.

Especially in intelligence, knowledge is power. No one willingly gives away power.” (Magnus Ranstorp, Swedish National Defense University)

From clerks who sort mail to accountants who cook the books to lawyers that defend patents and everyone else in between, everyone in an enterprise has knowledge, knowledge that gives them power others don’t have.

Topic maps have been pitched on a “greater good for the whole” basis but as Magnus points out, who the hell really wants that?

When confronted with a new technique, technology, methodology, the first and foremost question on everyone’s mind is:

Do I have more/less power/status with X?

A

approach loses power.

A

approach gains power.

Relevant lyrics:

Oh, there ain’t no rest for the wicked
Money don’t grow on trees
I got bills to pay
I got mouths to feed
And ain’t nothing in this world for free
No I can’t slow down
I can’t hold back
Though you know I wish I could
No there ain’t no rest for the wicked
Until we close our eyes for good

Sell topic maps to increase/gain power.

PS: Keep the line, “No one willingly gives away power” in discussions of why the ICIJ refuses to share the Panama Papers with the public.

Automatically Finding Weapons…

Wednesday, January 13th, 2016

Automatically Finding Weapons in Social Media Images Part 1 by Justin Seitz.

From the post:

As part of my previous post on gangs in Detroit, one thing had struck me: there are an awful lot of guns being waved around on social media. Shocker, I know. More importantly I began to wonder if there wasn’t a way to automatically identify when a social media post has guns or other weapons contained in them. This post will cover how to use a couple of techniques to send images to the Imagga API that will automatically tag pictures with keywords that it feels accurately describe some of the objects contained within the picture. As well, I will teach you how to use some slicing and dicing techniques in Python to help increase the accuracy of the tagging. Keep in mind that I am specifically looking for guns or firearm-related keywords, but you can easily just change the list of keywords you are interested in and try to find other things of interest like tanks, or rockets.

This blog post will cover how to handle the image tagging portion of this task. In a follow up post I will cover how to pull down all Tweets from an account and extract all the images that the user has posted (something my students do all the time!).

This rocks!

Whether you are trying to make contact with a weapon owner who isn’t in the “business” of selling guns or if you are looking for like-minded individuals, this is a great post.

Would make an interesting way to broadly tag images for inclusion in group subjects in a topic map, awaiting further refinement by algorithm or humans.

This is a great blog to follow: Automating OSINT.

50 Spies Say ISIS Intelligence Was Cooked

Thursday, September 10th, 2015

50 Spies Say ISIS Intelligence Was Cooked by Shane Harris and Nancy A. Youssef.

From the post:

More than 50 intelligence analysts working out of the U.S. military’s Central Command have formally complained that their reports on ISIS and al Qaeda’s branch in Syria were being inappropriately altered by senior officials, The Daily Beast has learned.

The complaints spurred the Pentagon’s inspector general to open an investigation into the alleged manipulation of intelligence. The fact that so many people complained suggests there are deep-rooted, systemic problems in how the U.S. military command charged with the war against the self-proclaimed Islamic State assesses intelligence.

“The cancer was within the senior level of the intelligence command,” one defense official said.

Two other examples of “cooked” intelligence come to mind:

S. Rept. 108-301 – REPORT OF THE SELECT COMMITTEE ON INTELLIGENCE on the U.S. INTELLIGENCE COMMUNITY’S PREWAR INTELLIGENCE ASSESSMENTS ON IRAQ together with ADDITIONAL VIEWS

Some of the results from that “cooked” intelligence include a costly war with Iraq and further destabilization of the Middle East.

The Pentagon Papers (Vietnam).

The “cooked” intelligence in Vietnam resulted in human and environmental costs that have never been adequately tallied.

Anyone, inside or outside the intelligence community who acts “shocked” that intelligence is “cooked” for political ends is either demented or extraterrestrial.

Cooked intelligence is used the intelligence community to justify its existence and in government departments to further their own budgets and agendas. Why would anyone be surprised that politicians cook intelligence for their own ends?

The cult of secrecy around intelligence is what enables the cooking of intelligence. If the information collected by the NSA, CIA and others was dumped onto GitHub on a regular basis, the ability of anyone to “cook” intelligence would be greatly diminished.

Or perhaps better, if intelligence data were available to everyone, then there would be a variety of dishes of “cooked” intelligence to chose from.

For all the frothing cries of “Danger!, Danger!,” that follow every leak of classified data, have you ever seen reports of anyone being called to account based on those leaks?

Of course not! The danger to others from TS/SCI classified data serves to enhance the status of those with clearance and avoids principled disagreement because “they know something you don’t.”

And that’s true, they do know something you don’t. What is often omitted is that what they know is often of no interest to anyone.

Decoding Satellite-Based Text Messages… [Mini-CIA]

Sunday, August 23rd, 2015

Decoding Satellite-Based Text Messages with RTL-SDR and Hacked GPS Antenna by Rick Osgood.

From the post:

[Carl] just found a yet another use for the RTL-SDR. He’s been decoding Inmarsat STD-C EGC messages with it. Inmarsat is a British satellite telecommunications company. They provide communications all over the world to places that do not have a reliable terrestrial communications network. STD-C is a text message communications channel used mostly by maritime operators. This channel contains Enhanced Group Call (EGC) messages which include information such as search and rescue, coast guard, weather, and more.

Not much equipment is required for this, just the RTL-SDR dongle, an antenna, a computer, and the cables to hook them all up together. Once all of the gear was collected, [Carl] used an Android app called Satellite AR to locate his nearest Inmarsat satellite. Since these satellites are geostationary, he won’t have to move his antenna once it’s pointed in the right direction.

You may have to ally with a neighbor who is good with a soldering iron but considering the amount of RF in the air, you should be able to become the mini-CIA for your area.

Not that the data itself may be all that interesting, but munging cellphone data with video surveillance of street traffic, news and other feeds, plus other RF sources, will hone your data handling skills.

For example, have you ever wondered how many of your neighbors obey watering restrictions during droughts? One way to find out is to create a baseline set of data for water usage (meters now report digitally) and check periodically when drought restrictions are in effect.

Nothing enlivens a town or county meeting like a color-coded chart of water cheats. (That will also exercise your mapping skills as well.)

Using topic maps will facilitate merging your water surveillance data other data, such as high traffic patterns for some locations of different cars. Or the periods of cars arriving and departing from some location.

Open Source Intelligence Techniques:… (review)

Wednesday, June 10th, 2015

Open Source Intelligence Techniques: Resources for Searching and Analyzing Online Information by CyberWarrior.

From the post:

Author Michael Bazzell has been well known and respected in government circles for his ability to locate personal information about any target through Open Source Intelligence (OSINT). In this book, he shares his methods in great detail. Each step of his process is explained throughout sixteen chapters of specialized websites, application programming interfaces, and software solutions. Based on his live and online video training at IntelTechniques.com, over 250 resources are identified with narrative tutorials and screen captures.

This book will serve as a reference guide for anyone that is responsible for the collection of online content. It is written in a hands-on style that encourages the reader to execute the tutorials as they go. The search techniques offered will inspire analysts to “think outside the box” when scouring the internet for personal information.

On the flip side, Open Source Intelligence Techniques is must reading for anyone who is charged with avoiding disclosure of information that can be matched with other open source intelligence.

How many people has your agency outed today?

Foreign Intelligence Gathering Laws

Wednesday, June 3rd, 2015

Foreign Intelligence Gathering Laws by Peter Roudik, Director of Legal Research, Law Library of Congress.

From the description:

This report contains information on laws regulating the collection of intelligence in the European Union, United Kingdom, France, Netherlands, Portugal, Romania, and Sweden. The report details how EU Members States control activities of their intelligence agencies and what restrictions are imposed on information collection. All EU Member States follow EU legislation on personal data protection, which is a part of the common European Union responsibility.

To the extent that you think intelligence services obey laws or if you need statute and case citations for rhetorical purposes, for the countries covered this report will be quite handy.

Whether you are in the United States or one of the countries listed in this report or elsewhere, your default assumption should be that you are under surveillance and the record light is on.

Bin Laden’s Bookshelf

Wednesday, May 20th, 2015

Bin Laden’s Bookshelf

From the webpage:

On May 20, 2015, the ODNI released a sizeable tranche of documents recovered during the raid on the compound used to hide Usama bin Ladin. The release, which followed a rigorous interagency review, aligns with the President’s call for increased transparency–consistent with national security prerogatives–and the 2014 Intelligence Authorization Act, which required the ODNI to conduct a review of the documents for release.

The release contains two sections. The first is a list of non-classified, English-language material found in and around the compound. The second is a selection of now-declassified documents.

The Intelligence Community will be reviewing hundreds more documents in the near future for possible declassification and release. An interagency taskforce under the auspices of the White House and with the agreement of the DNI is reviewing all documents which supported disseminated intelligence cables, as well as other relevant material found around the compound. All documents whose publication will not hurt ongoing operations against al-Qa‘ida or their affiliates will be released.

From the website:

bin-laden-bookcase

The one expected work missing from Bin Laden’s library?

The Anarchist Cookbook!

Possession of the same books as Bin Laden will be taken as a sign terrorist sympathies. Weed your collection responsibly.

Lies, Damned Lies, and Clapper (2015)

Sunday, March 8th, 2015

Worldwide Threat Assessment of the US Intelligence Community 2015 by James R Clapper (Director of National Intelligence).

The amazing thing about Director of National Intelligence (DNI) Clapper is that he remains out of prison and uncharged for his prior lies to Congress.

Clapper should get points for an amazing lack of self-awareness when he addresses the issue of unknown integrity of information due to cyber attacks:

Decision making by senior government officials (civilian and military), corporate executives, investors, or others will be impaired if they cannot trust the information they are receiving.

Decision making by members of congress (senior government officials) and member of the public are impaired when they can’t obtain trust information from government agencies and their leaders.

In that regard, the 2015 threat assessment is incomplete. It should have included threats that the US public faces, cyber and otherwise from its own government.

Basic Understanding of Big Data…. [The need for better filtering tools]

Saturday, February 21st, 2015

Basic Understanding of Big Data. What is this and How it is going to solve complex problems by Deepak Kumar.

From the post:

Before going into details about what is big data let’s take a moment to look at the below slides by Hewlett-Packard.

What_is_BigData

The post goes on to describe big data but never quite reaches saying how it will solve complex problems.

I mention it for the HP graphic that illustrates the problem of big data for the intelligence community.

Yes, they have big data as in the three V’s: volume, variety, velocity and so need processing infrastructure to manage that as input.

However, the results they seek are not the product of summing clicks, likes, retweets, ratings and/or web browsing behavior, at least not for the most part.

The vast majority of the “big data” at their disposal is noise that is masking a few signals that they wish to detect.

I mention that because of the seeming emphasis of late on real time or interactive processing of large quantities of data, which isn’t a bad thing, but also not a useful thing when what you really want are the emails, phone contacts and other digital debris of say < one thousand (1,000) people (that number was randomly chosen as an illustration, I have no idea of the actual number of people being monitored). It may help to think of big data in the intelligence community as consisting of a vast amount of "big data" about which it doesn't care and a relatively tiny bit of data that it cares about a lot. The problem being one of separating the data into those two categories. Take the telephone metadata records as an example. There is some known set of phone numbers that are monitored and contacts to and from those numbers. The rest of the numbers and their data are of interest if and only if at some future date they are added to the known set of phone numbers to be monitored. When the monitored numbers and their metadata are filtered out, I assume that previously investigated numbers for pizza delivery, dry cleaning and the like are filtered from the current data, leaving only current high value contacts or new unknowns for investigation. An emphasis on filtering before querying big data would reduce the number of spurious connections simply because a smaller data set has less random data that could be seen as patterns with other data. Not to mention that the smaller the data set, more prior data could be associated with current data without overwhelming the analyst. You may start off with big data but the goal is a very small amount of actionable data.

Intelligence Sharing, Crowd Sourcing and Good News for the NSA

Monday, February 16th, 2015

Lisa Vaas posted an entertaining piece today with the title: Are Miami cops really flooding Waze with fake police sightings?. Apparently an NBC affiliate (not FOX, amazing) tried its hand at FUD, alleging that Miami police officers were gaming Waze.

There is a problem with that theory, which Lisa points out quoting Julie Mossler, a spokes person for Waze:

Waze algorithms rely on crowdsourcing to confirm or negate what has been reported on the road. Thousands of users in Florida do this, both passively and actively, every day. In addition, we place greater trust in reports from heavy users and terminate accounts of those whose behavior demonstrate a pattern of contributing false information. As a result the Waze map will remain reliable and updated to the minute, reflecting real-time conditions.

Oops!

See Lisa’s post for the blow-by-blow account of this FUD attempt by the NBC affiliate.

However foolish an attempt to game Waze would be, it is a good example to promote the sharing of intelligence.

Think about it. Rather than the consensus poop that emerges as the collaboration of the senior management in intelligence agencies, why not share all intelligence between agencies between working analysts addressing the same areas or issues? Make the “crowd” people who have similar security clearances and common subject areas. And while contributions are trackable within a agency, to the “crowd,” everyone has a handle and their contributions on shared intelligence is voted up or down. Just like with Waze, people will develop reputations within the system.

I assume for turf reasons you could put handles on the intelligence so the participants would not know its origins as well, just until people started building up trust in the system.

Changing the cultures at the intelligence agencies, which hasn’t succeeded since 9/11, would require a more dramatic approach than has been tried to date. My suggestion is to give the Inspector Generals the ability to block promotions and/or fire people in the intelligence agencies who don’t actively promote the sharing of intelligence. Where “actively promotes” is measured by intelligence shared and not activities to plan to share intelligence, etc.

Unless and until there are consequences for the failure of members of the intelligence community to put the interests of their employers (in this case, citizens of the United States) above their own or that of their agency, the failure to share intelligence since 9/11 will continue.

PS: People will object that the staff in question have been productive, loyal, etc., etc. in the past. The relevant question is whether they have the skills and commitment that is required now? The answer to that last question is either yes or no. Employment is an opportunity to perform, not an entitlement.

Mercury [March 5, 2015, Washington, DC]

Saturday, February 14th, 2015

Mercury Registration Deadline: February 17, 2015.

From the post:

The Intelligence Advanced Research Projects Activity (IARPA) will host a Proposers’ Day Conference for the Mercury Program on March 5, in anticipation of the release of a new solicitation in support of the program. The Conference will be held from 8:30 AM to 5:00 PM EST in the Washington, DC metropolitan area. The purpose of the conference will be to provide introductory information on Mercury and the research problems that the program aims to address, to respond to questions from potential proposers, and to provide a forum for potential proposers to present their capabilities and identify potential team partners.

Program Description and Goals

Past research has found that publicly available data can be used to accurately forecast events such as political crises and disease outbreaks. However, in many cases, relevant data are not available, have significant lag times, or lack accuracy. Little research has examined whether data from foreign Signals Intelligence (SIGINT) can be used to improve forecasting accuracy in these cases.

The Mercury Program seeks to develop methods for continuous, automated analysis of SIGINT in order to anticipate and/or detect political crises, disease outbreaks, terrorist activity, and military actions. Anticipated innovations include: development of empirically driven sociological models for population-level behavior change in anticipation of, and response to, these events; processing and analysis of streaming data that represent those population behavior changes; development of data extraction techniques that focus on volume, rather than depth, by identifying shallow features of streaming SIGINT data that correlate with events; and development of models to generate probabilistic forecasts of future events. Successful proposers will combine cutting-edge research with the ability to develop robust forecasting capabilities from SIGINT data.

Mercury will not fund research on U.S. events, or on the identification or movement of specific individuals, and will only leverage existing foreign SIGINT data for research purposes.

The Mercury Program will consist of both unclassified and classified research activities and expects to draw upon the strengths of academia and industry through collaborative teaming. It is anticipated that teams will be multidisciplinary, and might include social scientists, mathematicians, statisticians, computer scientists, content extraction experts, information theorists, and SIGINT subject matter experts with applied experience in the U.S. SIGINT System.

Attendees must register no later than 6:00 pm EST, February 27, 2015 at http://events.SignUp4.com/MercuryPDRegistration_March2015. Directions to the conference facility and other materials will be provided upon registration. No walk-in registrations will be allowed.

I might be interested if you can hide me under a third or fourth level sub-contractor. 😉

Seriously, it isn’t that I despair of the legitimate missions of intelligence agencies but I do despise waste on ways known to not work. Government funding, even unlimited funding, isn’t going to magically confer the correct semantics on data or enable analysts to meaningfully share their work products across domains.

You would think going on fourteen (14) years post-9/11 and not being one step closer to preventing a similar event, that would be a “wake-up” call to someone. If not in the U.S. intelligence community, perhaps in intelligence communities who tire of aping the U.S. community with no better results.

Bulk Collection of Signals Intelligence: Technical Options (2015)

Saturday, January 17th, 2015

Bulk Collection of Signals Intelligence: Technical Options (2015)

Description:

The Bulk Collection of Signals Intelligence: Technical Options study is a result of an activity called for in Presidential Policy Directive 28, issued by President Obama in January 2014, to evaluate U.S. signals intelligence practices. The directive instructed the Office of the Director of National Intelligence (ODNI) to produce a report within one year “assessing the feasibility of creating software that would allow the intelligence community more easily to conduct targeted information acquisition rather than bulk collection.” ODNI asked the National Research Council (NRC) — the operating arm of the National Academy of Sciences and National Academy of Engineering — to conduct a study, which began in June 2014, to assist in preparing a response to the President. Over the ensuing months, a committee of experts appointed by the Research Council produced the report.

Believe it or not, you can’t copy-n-paste from the pre-publication PDF file. Truly irritating.

From the report:

Conclusion 1. There is no software technique that will fully substitute for bulk collection where it is relied on to answer queries about the past after new targets become known.

A key value of bulk collection is its record of past signals intelligence that may be relevant to subsequent investigations. If past events become interesting in the present, because intelligence-gathering priorities change to include detection of new kinds of threats or because of new events such as the discovery that an individual is a terrorist, historical events and the context they provide will be available for analysis only if they were previously collected. (Emphasis in the original)

The report dodges any questions about effectiveness or appropriateness of bulk collection of signals data. However, its number one conclusion provides all the ammunition one needs to establish that bulk signals intelligence gathering is a clear and present danger to the American people and any semblance of a democratic government.

Would deciding that all Muslims from the Middle East represented potential terrorist threats to the United States qualify as a change in intelligence-gathering priorities? So all the bulk signals data from Muslims and their contacts in the United States suddenly becomes fair game for the NSA to investigate?

I don’t think any practicing Muslim is a threat to any government but you saw how quickly the French backslide into bigotry after Charlie Hebdo. Maybe they didn’t have that far to go. Not any further than large segments of the U.S. population.

Our National Research Council is too timid voice an opinion other than to say if you don’t preserve signals records you can’t consult them in the future. But whether there is any danger or is this a good policy choice, they aren’t up for those questions.

The focus on signals intelligence makes you wonder how local and state police have operated all these years without bulk signals intelligence? How have they survived without it? Well, for one thing they are out in the communities they serve, not cooped up in cube farms with other people who don’t have any experience with the communities in question. Simply being a member of the community makes them aware of new comers, changes in local activity, etc.

Traditional law enforcement doesn’t stop crime as a general rule because that would require too much surveillance and resources to be feasible. When a crime has been committed, law enforcement gathers evidence and in a very large (90%+) number of cases, captures the people responsible.

Which is a interesting parallel to the NSA, which has also not stopped any terrorist plots as far as anyone knows. Well, there as that case in the State of Georgia where two aging alcoholics were boosting about producing Ricin and driving down I-285 throwing it out the window. The government got a convicted child molester to work as in informant to put those two very dangerous terrorists in jail. And I don’t think the NSA was in on that one anyway.

If the NSA has stopped a major terrorist plot, something that actually was going to be another 9/11, you know it would have been leaked long before now. The absence of such leaks is the best evidence for the lack of any viable terrorist threats in the United States that I can think of.

And what if we stop bulk signals data collection and there is another terrorist attack? So, what is your question? Bulk signals collection hasn’t stopped one so far so if we stop bulk signals collection and there is another terrorist attack, look at all the money we will have saved for the same result. Just as a policy matter, we shouldn’t spend money for no measurable result.

If you really think terrorism is a threat, take the money from bulk signal data collection and fund state and local police hiring, training and paying (long term, not just a grant) more local police officers out in their communities. That will do more to reduce the potential for all types of crimes, including those labeled as terrorism.

To put it another way, bulk signal data collection is a form of wealth sharing, wealth sharing from the public treasury to contractor’s. Wealth sharing that has been shown to be ineffectual against terrorism. Why continue it?

Defence: a quick guide to key internet links

Sunday, November 16th, 2014

Defence: a quick guide to key internet links by David Watt and Nicole Brangwin.

While browsing at Full Text Reports, I saw this title with the following listing of contents:

  • Australian Parliament
  • Australian Government
  • Military history
  • Strategic studies
  • Australian think tanks and non-government organisations
  • International think tanks and organisations
  • Foreign defence

The document is a five (5) page PDF file that has a significant number of links, particularly to Australian military resources. Under “Foreign defense” I did find the Chinese Peoples’ Liberation Army but no link for ISIL.

This may save you some time if you are spidering Australian military sites but appears to be incomplete for other areas.

Intelligence Community On/Off The Record

Wednesday, September 24th, 2014

While looking up a particular NSA leak today I discovered:

IC On The Record

Direct access to factual information related to the lawful foreign surveillance activities of the U.S. Intelligence Community.

Created at the direction of the President of the United States and maintained by the Office of the Director of National Intelligence.

and,

IC Off The Record

Direct access to leaked information related to the surveillance activities of the U.S. Intelligence Community and their partners.

IC Off The Record points to IC On The Record but the reverse isn’t true.

When you visit IC On The Record, tweet about IC Off The Record. Help everyone come closer to a full understanding of the intelligence community.

Fixing Pentagon Intelligence [‘data glut but an information deficit’]

Sunday, September 21st, 2014

Fixing Pentagon Intelligence by John R. Schindler.

From the post:

The U.S. Intelligence Community (IC), that vast agglomeration of seventeen different hush-hush agencies, is an espionage behemoth without peer anywhere on earth in terms of budget and capabilities. Fully eight of those spy agencies, plus the lion’s share of the IC’s budget, belong to the Department of Defense (DoD), making the Pentagon’s intelligence arm something special. It includes the intelligence agencies of all the armed services, but the jewel in the crown is the National Security Agency (NSA), America’s “big ears,” with the National Geospatial-Intelligence Agency (NGA), which produces amazing imagery, following close behind.

None can question the technical capabilities of DoD intelligence, but do the Pentagon’s spies actually know what they are talking about? This is an important, and too infrequently asked, question. Yet it was more or less asked this week, in a public forum, by a top military intelligence leader. The venue was an annual Washington, DC, intelligence conference that hosts IC higher-ups while defense contractors attempt a feeding frenzy, and the speaker was Rear Admiral Paul Becker, who serves as the Director of Intelligence (J2) on the Joint Chiefs of Staff (JCS). A career Navy intelligence officer, Becker’s job is keeping the Pentagon’s military bosses in the know on hot-button issues: it’s a firehose-drinking position, made bureaucratically complicated because JCS intelligence support comes from the Defense Intelligence Agency (DIA), which is an all-source shop that has never been a top-tier IC agency, and which happens to have some serious leadership churn at present.

Admiral Becker’s comments on the state of DoD intelligence, which were rather direct, merit attention. Not surprisingly for a Navy guy, he focused on China. He correctly noted that we have no trouble collecting the “dots” of (alleged) 9/11 infamy, but can the Pentagon’s big battalions of intel folks actually derive the necessary knowledge from all those tasty SIGINT, HUMINT, and IMINT morsels? Becker observed — accurately — that DoD intelligence possesses a “data glut but an information deficit” about China, adding that “We need to understand their strategy better.” In addition, he rued the absence of top-notch intelligence analysts of the sort the IC used to possess, asking pointedly: “Where are those people for China? We need them.”

Admiral Becker’s:

data glut but an information deficit” (emphasis added)

captures the essence of phone record subpoenas, mass collection of emails, etc., all designed to give the impression of frenzied activity, with no proof of effectiveness. That is an “information deficit.”

Be reassured you can host a data glut in a topic map so topic maps per se are not a threat to current data gluts. It is possible, however, to use topic maps over existing data gluts to create information and actionable intelligence. Without disturbing the underlying data gluts and their contractors.

I tried to find a video of Adm. Becker’s presentation but apparently the Intelligence and National Security Security Summit 2014 does not provide video recording of presentations. Whether that is to prevent any contemporaneous record being kept of remarks or just being low-tech kinda folks isn’t clear.

I can point out the meeting did have a known liar, “The Honorable James Clapper,” on the agenda. Hard to know if having perjured himself in front of Congress has made him gun shy of recorded speeches or not. (For Clapper’s latest “spin,” on “the least untruthful,” see: James Clapper says he misspoke, didn’t lie about NSA surveillance.) One hopes by next year’s conference Clapper will appear as: James Clapper, former DNI, convicted felon, Federal Prison Register #….

If you are interested in intelligence issues, you should be following John R. Schindler. A U.S. perspective but handling issues in intelligence with topic maps will vary in the details but not the underlying principles from one intelligence service to another.

Disclosure: I rag on the intelligence services of the United States due to greater access to public information on those services. Don’t take that as greater interest how their operations could be improved by topic maps over other intelligence services.

I am happy to discuss how your intelligence services can (or can’t) be improved by topic maps. There are problems, such as those discussed by Admiral Becker, that can’t be fixed by using topic maps. I will be as quick to point those out as I will problems where topic maps are relevant. My goal is your satisfaction that topic maps made a difference for you, not having a government entity in a billing database.

Storing and visualizing LinkedIn…

Saturday, June 21st, 2014

Storing and visualizing LinkedIn with Neo4j and sigma.js by Bob Briody.

From the post:

In this post I am going to present a way to:

  • load a linkedin networkvia the linkedIn developer API into neo4j using python
  • serve the network from neo4j using node.js, express.js, and cypher
  • display the network in the browser using sigma.js

Great post but it means one (1) down and two hundred and five (205) more to go, if you are a member of the social networks listed on List of social networking websites at Wikipedia, and that excludes dating sites and includes only “notable, well-known sites.”

I would be willing to bet that your social network of friends, members of your religious organization, people where you work, etc. would start to swell the number of other social networks that number you as a member.

Hmmm, so one off social network visualizations are just that, one off social network visualizations. You can been seen as part of one group and not say two or three intersecting groups.

Moreover, an update to one visualized network isn’t going to percolate into another visualized network.

There is the “normalize your graph” solution to integrate such resources but what if you aren’t the one to realize the need for “normalization?”

You have two separate actors in your graph visualization after doing the best you can. Another person encounters information indicating these “two” people are in fact one person. They update their data. But that updated knowledge has no impact on your visualization, unless you simply happen across it.

Seems like a poor way to run intelligence gathering doesn’t it?

Health Intelligence

Wednesday, June 4th, 2014

Health Intelligence: Analyzing health data, generating and communicating evidence to improve population health. by Ramon Martinez.

I was following a link to Ramon’s Data Sources page when I discovered his site. The list of data resources is long and impressive.

But there is so much more under Resources!

  • Data Tools
  • Database (DB) Blogs
  • Data Visualization Tools
  • Data Viz Blogs
  • Reading for Data Visualizations
  • Best of the Web…
  • Tableau Training
  • Going to School
  • Reading for Health Analysis

You will probably like the rest of the site as well!

Data tools/visualization are very ecumenical.