Another Word For It Patrick Durusau on Topic Maps and Semantic Diversity

April 2, 2015

Insane: Record Leakage or Security Incentives?

Filed under: Cybersecurity,Security — Patrick Durusau @ 6:32 pm

Did you know that record leakage in 2014 was “insane?” (2015 following the same security threat trend as 2014 by Neil Ford.)

IBM X-Force Threat Intelligence Quarterly reports that the number of records leaked in 2014 were 25% higher than 2013, or as shown on their graphic:

insane

Leaking 1 billion records in 2014 is a high water mark in terms of records but it isn’t “insane.”

Even cloud bursts, breaches of cloud infrastructure that increase the percentage of leaked records over the prior year by 1% or more, can’t fairly be called “insane.”

If you want to use “insane” with cybersecurity, here is the current status of incentives for security:

Financial incentives for security — None. See: The reason companies don’t fix cybersecurity [Same reason software is insecure].

Financial incentives for hacking — Numerous, ranging from credit card fraud, identity theft, corporate espionage, etc.

That is an “insane” situation.

What is being done about it?

Congress is considering: The Protecting Cyber Networks Act, which is summarized in part as:

2014 will be known as the year of the cyber breach. High profile attacks are a main topic of conversation in the boardroom and at the dinner table. Every day, nation-state actors and criminals target America’s businesses for cyber espionage and theft. These hackers steal our intellectual property, trade secrets and even sensitive government information. The same actors who conduct cyber espionage are also capable of significant offensive cyber attacks that could degrade or damage vital private-sector infrastructure.

The Protecting Cyber Networks Act enables private companies to share cyber threat indicators with each other and, on a purely voluntary basis, with the federal government but not through the NSA or the Department of Defense, all while providing strong protections for privacy and civil liberties. At the same time, the bill makes clear that defense contractors can continue to share cyber threat information with the Department of Defense when required to do so by another law, regulation, or contract.

Voluntary information sharing with the federal government helps improve the government’s ability to protect America against foreign cyber threats. It also gives our intelligence agencies tips and leads to help them find advanced foreign cyber hackers
overseas. That intelligence allows the government to provide, in turn, even better cyber threat indicators back to the private sector to help companies protect themselves.

Are you familiar with the term farce?

First, the summary claims information is not shared with the NSA or DoD (second paragraph), but then in the third paragraph, “…gives our intelligence agencies….” Either they are using “intelligence agencies” in a highly usual way or they have forgotten what they just wrote.

Second, “voluntary sharing” of cyber threat indicators does not address the imbalance of incentives for computer security.

The Protecting Cyber Networks Act is an “insane” response to current issues in computer security.

Want one step towards a “sane” solution for computer security? Take the Sony breach for example. The top 10% of their employees in terms of compensation forfeit 20% of their current retirement and stock benefits.

If Sony executives had “skin in the game,” security at Sony would be far more robust than it was last year.

How To Politic With $Money

Filed under: Government,Politics — Patrick Durusau @ 10:52 am

I mentioned Rep. John Carter (R, Texas) yesterday, Plumbing the Depths of Abject Stupidity, and I am sure much mirth was had over his misunderstanding of the nature of and need for encryption.

However, all the jests, jeers and humor aren’t going to change Carter’s ignorance nor his highly inappropriate position in the halls of power. All of those things make us “feel” superior to Carter but he is still the one with his hands on the levers of power.

Don’t get me wrong, I laugh just as hard as anyone at Comedy Central / MSNBC (I have trouble telling them apart) but insider humor isn’t going to dislodge anyone from public office. Or threaten to dislodge anyone from public office.

What do we know lies at the heart of success in American politics? Money. Everyone says it often enough and the history of elections bears that out. So if we want to change a position taken by a politician, shouldn’t we influence those that own him?

Finding ownership information for elected Federal officials:

One source of information about money in politics is OpenSecrets.org. There are others so this is just an illustration of hunting for someone to influence Rep. Carter.

Starting at the homepage of OpenSecrets.org, mouse over Politicians & Elections, which will display a drop down menu of choices. Select the first one, Congressional Races. The Congressional Races page loads and displays (in part):

congressional races

Enter “Carter” only:

congressional-carter

Don’t make the mistake I did and enter “John Carter.” You will get “no results.”

The results page displays in part:

2014-search-large

The results from OpenSecrets.orgOpenSecrets.org go back to 2004, the first time John Carter was elected to Congress.

Starting with 2014, select “John Carter (Texas – District 31)” and the Summary Data 2014 Race: Texas District 31 displays:

2014-summary-large

The results page also has “Outside Groups Spending Money in this Race” so you can trace outside money but I haven’t followed it in this example.

If you select John Carter, the Rep. John Carter page loads with a wealth of information.

There are tabs for: Summary (default), Elections, Industries, PACs, Donors, Geography, Expenditures, Legislation, In the News, and, Other Data. The information on the Summary page will suffice for our purposes today but if you are interested, I may return to OpenSecrets.org to explore the other information found there.

Under the default summary tab you will find:

2014-top-5-large

Now we are getting somewhere!

Which is more likely to have an impact on Rep. John Carter, a letter/email/phone call from a non-constituent or a visit from a representative from Bollinger Shipyards, Berkshire Hathaway, Dell Inc, American Bankers Assn, or American Crystal Sugar?

That seems pretty obvious but you can go even further and under the default display:

2014-rep-carter-large

You can select any individual election to see the top donors or you can select “career” to see the top donors overall:

2014-top-5-career-large

I think we have a winner!

You would have to check the sort of politics that Dell supports generally but a little “education” on encryption from Dell could go a long way.

If you need other options, check out the top twenty contributors from the 2014 election. Heavy on defense industry types, who have as much to loose from government back doors as anyone.

Financial disclosure records are a first step in making a case for your cause to a member of Congress. Without access, your argument isn’t going to be heard.

The next step is to create a commonality of interest with others. Say the defense industry, pointing out government intrusions could reach erroneous conclusions from materials not read in the proper context. No one wants that to happen with the attendant costs and expenses for lawyers, etc. It is much better for the government to request information so it can be disclosed in a useful and non-misleading way.

Be aware that creating a commonality of interest can include supporting causes you might not otherwise care to see as law. Increased military spending would be one for me. But I would gladly see the DoD budget increased if it mean crippling mass data collection at the NSA, which has already been found to be ineffectual. It’s a trade-off. Everyone wants something and it is a question of finding out what that something is.

(I didn’t cover individual donors but Carter only had four hundred and some odd in 2014. A comprehensive workup would include them and all the PAC donors as well.)

PS: Next year is a major election cycle in the United States. A majority of the seats in Congress are “safe” but that doesn’t mean you can’t leverage existing access relationships. Inquiries welcome.

PPS: I just happened to choose a Republican for this example. The lessons here apply to Democrats as well. Money is a non-partisan participant in American politics.

April 1, 2015

Full-Text Search in Javascript (Part 1: Relevance Scoring)

Filed under: Javascript,Lucene,Search Engines,Searching — Patrick Durusau @ 7:47 pm

Full-Text Search in Javascript (Part 1: Relevance Scoring) by Barak Kanber.

From the post:

Full-text search, unlike most of the topics in this machine learning series, is a problem that most web developers have encountered at some point in their daily work. A client asks you to put a search field somewhere, and you write some SQL along the lines of WHERE title LIKE %:query%. It’s convincing at first, but then a few days later the client calls you and claims that “search is broken!”

Of course, your search isn’t broken, it’s just not doing what the client wants. Regular web users don’t really understand the concept of exact matches, so your search quality ends up being poor. You decide you need to use full-text search. With some MySQL fidgeting you’re able to set up a FULLTEXT index and use a more evolved syntax, the “MATCH() … AGAINST()” query.

Great! Problem solved. For smallish databases.

As you hit the hundreds of thousands of records, you notice that your database is sluggish. MySQL just isn’t great at full-text search. So you grab ElasticSearch, refactor your code a bit, and deploy a Lucene-driven full-text search cluster that works wonders. It’s fast and the quality of results is great.

Which leads you to ask: what the heck is Lucene doing so right?

This article (on TF-IDF, Okapi BM-25, and relevance scoring in general) and the next one (on inverted indices) describe the basic concepts behind full-text search.

Illustration of search engine concepts in Javascript with code for download. You can tinker to your heart’s delight.

Enjoy!

PS: Part 2 is promised in the next “several” weeks. Will be watching for it.

« Newer Posts

Powered by WordPress