Another Word For It Patrick Durusau on Topic Maps and Semantic Diversity

August 14, 2018

Mouse > Sword – High Sierra Hack – 2 lines of code [Brett Kavanaugh documents?]

Filed under: Cybersecurity,Hacking — Patrick Durusau @ 3:30 pm

ex-NSA Hacker Discloses macOS High Sierra Zero-Day Vulnerability by Mohit Kumar.

The gist of the attack:

…
Patrick Wardle, an ex-NSA hacker and now Chief Research Officer of Digita Security, uncovered a critical zero-day vulnerability in the macOS operating system that could allow a malicious application installed in the targeted system to virtually “click” objects without any user interaction or consent.

To know, how dangerous it can go, Wardle explains: “Via a single click, countless security mechanisms may be completely bypassed. Run untrusted app? Click…allowed. Authorize keychain access? Click…allowed. Load 3rd-party kernel extension? Click…allowed. Authorize outgoing network connection? click …allowed.”

Wardle described his research into “synthetic” interactions with a user interface (UI) as “The Mouse is Mightier than the Sword,” showcasing an attack that’s capable of ‘synthetic clicks’—programmatic and invisible mouse clicks that are generated by a software program rather than a human.
…

Be sure to grab Wardle’s slides for: The Mouse is mightier than the sword.

It’s not a small file (194 MB) but it has goodies like:

and,

Not to mention numerous links and deep analysis of the Mac OS.

Enjoy!

PS: Do you think a current version of High Sierra has access to the files on Supreme Court nominee Brett Kavanaugh? The National Archives and Records Administration says it will take two months to review approximately 1 million records. If dumped, un-edited to the Internet, what? Two weeks? Tops?

To many eyes, all scandals (real or imagined) are transparent.

Man-in-the-Disk – Breaking and Entering Android Phones

Filed under: Cybersecurity,Hacking — Patrick Durusau @ 1:14 pm

New Man-in-the-Disk attack leaves millions of Android phones vulnerable by Swati Khandelwal.

From the post:

…
Dubbed Man-in-the-Disk, the attack takes advantage of the way Android apps utilize ‘External Storage’ system to store app-related data, which if tampered could result in code injection in the privileged context of the targeted application.

…

Google itself offers guidelines to Android application developers urging them to use internal storage, which is an isolated space allocated to each application protected using Android’s built-in sandbox, to store their sensitive files or data.

However, researchers found that many popular apps—including Google Translate itself, along with Yandex Translate, Google Voice Typing, Google Text-to-Speech, Xiaomi Browser—were using unprotected external storage that can be accessed by any application installed on the same device.
…

Khandelwal cites Man-in-the-Disk: A New Attack Surface for Android Apps, which provides this quick summary of the attack:

…

As the details of this attack may seem complex, let us recap the general outline and ramifications of these shortcomings of Android:

  • An Android device’s External Storage is a public area which can be observed or modified by any other application on the same device.
  • Android does not provide built-in protections for the data held in the External Storage. It only offers developers guidelines on proper use of this resource.
  • Developers anywhere are not always versed in the need for security and the potential risks, nor do they always follow guidelines.
  • Some of the pre-installed and popularly used apps ignore the Android guidelines and hold sensitive data in the unprotected External Storage.
  • This can lead to a Man-in-the-Disk attack, resulting in the manipulation and/or abuse of unprotected sensitive data.
  • Modification to the data can lead to unwelcome results on the user’s device.

…

Vulnerability pattern: Privileged execution of non-validated data.

Does anyone have a chart of the privileges required by Android apps using External Storage? That would help triage which apps to investigate first.

(Leaving to one side the deliberate creation of an app with high privileges with a plan to later update from External Storage.)

August 13, 2018

Hunting God Modes? [Get Thee to the Patent Office]

Filed under: Cybersecurity,Hacking — Patrick Durusau @ 7:53 pm

God Mode unlocked: Hardware backdoors in x86 CPUs by Christopher Domas.

Domas has discovered a god mode in the VIA C3 Nehemiah chip (2003) by tracing a series of patents.

An impressive bit of work, but its greater importance lies in partially populating search terms to use when looking for similar patents.

Not to mention that confirmation of the existence of a god mode, not rumored, not whispered about, but a corroborated god mode, will encourage other security researchers to seek other god modes in other versions of chips.

There is a non-technical treatment of Domas’ discovery at: Hacker Finds Hidden ‘God Mode’ on Old x86 CPUs by Paul Wagenseil.

It’s a good summary article but be forewarned of Wagenseil’s take on security:

…
The good news is that, as far as Domas knows, this backdoor exists only on VIA C3 Nehemiah chips made in 2003 and used in embedded systems and thin clients. The bad news is that it’s entirely possible that such hidden backdoors exist on many other chipsets.
…

Wagenseil has that backwards. Good news would be god modes on all chipsets. Bad news would be god mode is a one-off mistake on the VIA C3 Nehemiah chip (2003). God modes make information security more sporting.

What chip set patents are you going to be researching this week?

Update, 14 August 2018: See the Rosenbridge project at Github for code, etc.

August 5, 2018

Color and Size Steps with Radare2 on Ubuntu 18.04

Filed under: Cybersecurity,Hacking,Radare2 — Patrick Durusau @ 8:46 pm

I mentioned in First Steps with Radare2 on Ubuntu 18.04 that I needed to reset the default colors in Radare2, along with making the font larger.

Itay Cohen, @megabeets_, quickly responded:

Hi Patrick! I read that you had a bit of a struggle with the font colors. Did you know you can change the color theme? Just use “eco “. Screenshots of the different themes are available here: https://r2wiki.readthedocs.io/en/latest/home/themes/#themes. You can also use the Visual Color editor “VE”. Try ‘ec?’

Great way to change displays!

Since I am running XFCE as a desktop, ctrl + and ctrl -, don’t change the terminal font size. (Or at least I’m missing now to make that work in XFCE.)

For the time being, I’m starting r2 in an Emacs shell, which allows me to reset the font size quite easily. With the added advantage of being in Emacs!

Now to try out “eco “.

Several people mentioned that I should try Cutter, the new GUI for Radare2. Going to but I’m comfortable with command line interfaces. Not to mention that experience with the command line will enable me to notice groupings in the GUI.

Chaff Bugs: Deterring Attackers by Making Software Buggier

Filed under: Cybersecurity,Hacking — Patrick Durusau @ 2:20 pm

Chaff Bugs: Deterring Attackers by Making Software Buggier by Zhenghao Hu, Yu Hu, Brendan Dolan-Gavitt.

Abstract:

Sophisticated attackers find bugs in software, evaluate their exploitability, and then create and launch exploits for bugs found to be exploitable. Most efforts to secure software attempt either to eliminate bugs or to add mitigations that make exploitation more difficult. In this paper, we introduce a new defensive technique called chaff bugs, which instead target the bug discovery and exploit creation stages of this process. Rather than eliminating bugs, we instead add large numbers of bugs that are provably (but not obviously) non-exploitable. Attackers who attempt to find and exploit bugs in software will, with high probability, find an intentionally placed non-exploitable bug and waste precious resources in trying to build a working exploit. We develop two strategies for ensuring non-exploitability and use them to automatically add thousands of non-exploitable bugs to real-world software such as nginx and libFLAC; we show that the functionality of the software is not harmed and demonstrate that our bugs look exploitable to current triage tools. We believe that chaff bugs can serve as an effective deterrent against both human attackers and automated Cyber Reasoning Systems (CRSes).

A deeply interesting paper but testing the effectiveness of chaff bugs falls short. The researchers used standard tools to create their estimates of the effectiveness of the chaff bugs. But that isn’t the same as measuring their effectiveness against hackers.

By analogy, consider a team authoring a cracking puzzle and then estimating its difficulty, as opposed to relying on other teams to crack it. Different people, different perspectives, habits, tools, could all make a substantial difference.

Looking forward to seeing this technique appearing in hacking contests.

August 4, 2018

First Steps with Radare2 on Ubuntu 18.04

Filed under: Cybersecurity,Hacking,Radare2 — Patrick Durusau @ 3:19 pm

If you read Reverse Engineering With Radare2, Part 1 by Sam Symons, you will be hot to jump in and start using Radare2!

Of course, like me, you will ignore most of the introduction and quickly search for Radare2, only to encounter an array of installation options, most of which don’t concern you.

Avoid that mistake, follow this link, http://radare.org/r/down.html (yes, same one that Symons has in his post, and follow these directions:

git clone https://github.com/radare/radare2
cd radare2
sys/install.sh # just run this script to update from r2 from git

OK, you need to:

sudo sys/install.sh if you aren’t in a root shell.

Symons points you to course materials for a Modern Binary Exploitation course and their website.

Starting with ./crackme0x00a, you are introduced to the r2 command to open the first challenge.

Presented in a different order, you will encounter:

  • ? – help (append to any command)
  • aa – analyze all
  • cd – change directories
  • pdf – Print disassemble function – pdf@main (simple example)
  • pwd – identify working directory
  • s – seek
  • x – print

I’m working on resetting the colors! Even in a much larger size, this is terribly difficult to read!

That reminds me, there is a book on radare2, imaginatively titled: R2 “Book.” (There is truth to the claim that naming is one of the hardest problems in computer science.)

I got to the end of the first exercise and have some confidence that the Radare2 installation is working properly.

Before going any further, I’m going to experiment with and fix the color display. It’s painful to look at. More on its way!

Enjoy!

August 3, 2018

Browser-based GDB frontend: gdbGUI [With cameo by Thomas Hobbes]

Filed under: .Net,Cybersecurity,gdb,Hacking,Programming,Reverse Engineering — Patrick Durusau @ 8:26 pm

Browser-based GDB frontend: gdbGUI

From the post:

A modern, browser-based frontend to gdb (gnu debugger). Add breakpoints, view stack traces, and more in C, C++, Go, and Rust! Simply run gdbgui from the terminal and a new tab will open in your browser.

Features:

  • Debug a different program in each tab (new gdb instance is spawned for each tab)
  • Set/remove breakpoints
  • View stack, threads
  • Switch frame on stack, switch between threads
  • Intuitively explore local variables when paused
  • Hover over variables in source code to view contents
  • Evaluate arbitrary expressions and plot their values over time
  • Explore an interactive tree view of your data structures
  • Jump back into the program’s state to continue debug unexpected faults (i.e. SEGFAULT)
  • Inspect memory in hex/character form
  • View all registers
  • Dropdown of files used to compile binary, with autocomplete functionality
  • Source code explorer with ability to jump to line
  • Show assembly next to source code, highlighting current instruction. Can also step through instructions.
  • Assembly is displayed if source code cannot be found
  • Notifications when new gdbgui updates are available

…

While cybersecurity is always relative, the more skills you have, the more secure you can be relative to other users. Or, as Thomas Hobbes observed in De Cive, revised edition, printed in 1760 at Amsterdam, bellum omnium contra omnes, “the war of all against all.” (The quote is found on pages 25-26 of this edition. The following image is from the revised edition, 1647.)

Look to your own security. It is always less valuable to others.

Hints for Computer System Design

Filed under: Computer Science,Design — Patrick Durusau @ 7:24 pm

Hints for Computer System Design by Butler W. Lampson (1983)

Abstract:

Studying the design and implementation of a number of computer has led to some general hints for system design. They are described here and illustrated by many examples, ranging from hardware such as the Alto and the Dorado to application programs such as Bravo and Star.

Figure 1 is the most common quote you will see:

Figure 1 is a great summary, but don’t cheat yourself by using it in place of reading the full article. All of those slogans have a context of origin and usage.

I saw this in a tweet by Joe Duffy, who says he reads it at least once a year. Not a bad plan.

Russian Bot Spotting, Magic Bullets, New York Times Tested

Filed under: Bots,Social Media,Twitter — Patrick Durusau @ 4:39 pm

How to Spot a Russian Bot by Daniel Costa-Roberts.

Spotting purported Russian bots on Twitter is a popular passtime for people unaware the “magic bullet” theory of communication has been proven to be false. One summary of “magic bullet” thinking:

…
The media (magic gun) fired the message directly into audience head without their own knowledge. The message cause the instant reaction from the audience mind without any hesitation is called “Magic Bullet Theory”. The media (needle) injects the message into audience mind and it cause changes in audience behavior and psyche towards the message. Audience are passive and they can’t resist the media message is called “Hypodermic Needle Theory”.
…

The “magic bullet” is an attractive theory for those selling advertising, but there is no scientific evidence to support it:

…
The magic bullet theory is based on assumption of human nature and it was not based on any empirical findings from research. Few media scholars do not accepting this model because it’s based on assumption rather than any scientific evidence. In 1938, Lazarsfeld and Herta Herzog testified the hypodermic needle theory in a radio broadcast “The War of the Worlds” (a famous comic program) by insert a news bulletin which made a widespread reaction and panic among the American Mass audience. Through this investigation he found the media messages may affect or may not affect audience.

“People’s Choice” a study conducted by Lazarsfeld in 1940 about Franklin D. Roosevelt election campaign and the effects of media messages. Through this study Lazarsfeld disproved the Magic Bullet theory and added audience are more influential in interpersonal than a media messages.
…

Nevertheless, MotherJones and Costa-Roberts outline five steps to spot a Russian bot:

  1. Hyperactivity – more than 50 or 60 tweets per day
  2. Suspicious images – stock avatar
  3. URL shorterners – use indicates a bot
  4. Multiple languages – polyglot indicates a bot
  5. Unlikely popularity – for given # of followers

OK, so let’s test those steps against a known non-Russian bot that favors the US government, the New York Times.

  1. Hyperactivity – New York Times joined Twitter, 2 March 2007, 4173 days, 328,555 tweets as of this afternoon, so, 78.73 on average per day. That’s hyperactive.
  2. Suspicious images – NYT symbol
  3. URL shorterners – Always – signals bot. (displays nytimes.com but if you check the links, URL shorterner)
  4. Multiple languages – Nope.
  5. Unlikely popularity – In which direction? NYT has 41,665,676 followers and only 17,145 likes, or one like for every 2340 followers.

On balance I would say the New York Times isn’t a Russian bot, but given it’s like to follower ratio, it needs to work on its social media posts.

Maybe the New York Times needs to hire a Russian bot farm?

Podcasting from Scratch

Filed under: Podcasting,Topic Maps — Patrick Durusau @ 3:27 pm

Podcasting from Scratch by Alex Laughlin and Julia Furlan.

No promises but while thinking about a podcast on topic map authoring (something never covered in the standards) I encountered this eight (8) page guide.

It’s not everything you need to know but it’s enough to get you past the initial fear of starting a new activity or skill.

If and when I do post one or more podcasts, don’t judge Laughlin and Furlan by my efforts!

See how helpful they are in launching your podcasting career for yourself!

Red Team Tips

Filed under: .Net,Cybersecurity,Hacking,Security — Patrick Durusau @ 2:11 pm

Red Team Tips by Vincent Yiu.

Overview:

The following “red team tips” were posted by myself, Vincent Yiu (@vysecurity) over Twitter for about a year. This is still on-going but I took the opportunity to publish these in one solidified location on my blog. These will be updated ocassionally, but will not be bleeding edge updates. To receive my “red team tips”, thoughts, and ideas behind Cyber attack simulations, follow my Twitter account @vysecurity.

For the full Tweet and thread context (a lot of my followers will comment and give their insights also), visit Twitter.
…

Collection of three hundred and twenty-nine (329) red team (is there another kind?) tips!

Great way to start the weekend!

Enjoy!

August 2, 2018

Visual Guide to Data Joins – Leigh Tami

Filed under: .Net,Data Aggregation,Data Integration,Data Science,Joins — Patrick Durusau @ 7:06 pm

Leigh Tami created a graphic involving a person and a coat to explain data set joins.

Scaling it down won’t do it justice here so see the original.

Preview any data science book with this image in mind. If it doesn’t match or exceed this explanation of joins, pass it by.

Leaking 4Julian – Non-Sysadmin Leaking

Filed under: .Net,Journalism,Leaks,News,Reporting — Patrick Durusau @ 6:15 pm

Non-sysadmins read username: 4julian password: $etJulianFree!2Day and wish they could open corporate or government archives up to mining.

Don’t despair! Even non-sysadmins can participate in the Assange Data Tsunami, worldwide leaking of data in the event of the arrest of Julian Assange.

Check out the Whistle Blower FAQ – International Consortium of Investigative Journalists (ICIJ) by Gerald Ryle.

FYI, By some unspecified criteria, the ICIJ decides which individuals and groups mentioned in a leak that merit public exposure and those that do not. This is a universal practice amoung journalists. Avoiding it requires non-journalist outlets.

The ICIJ does a great job with leaks but if I were going to deprive a government or corporation of power over information, why would I empower journalists to make the same tell/don’t tell decision? Let the public decide what to make of the all the information. Assisted by the efforts of journalists but not with some information being known only to the journalists.

From the FAQ:

‘What information should I include?’ and other frequently asked questions about becoming a whistleblower

In my 30-year career as a journalist, I’ve spoken with thousands of potential sources, some of them with interesting tips or insider knowledge, others with massive datasets to share. Conversations often start with questions about the basics of whistleblowing. If you’re thinking about leaking information, here are some of the things you should keep in mind:

Q. What is a whistleblower?

A whistleblower is someone who has evidence of wrongdoing, abuse of power, fraud or misconduct and who shares it with a third party such as an investigative journalism organization like the International Consortium of Investigative Journalists.

By blowing the whistle you can help prevent the possible escalation of misconduct or corruption.

Edward Snowden is one of the world’s best-known Whistleblowers.

Q. Can a whistleblower remain anonymous?

Yes. We will always go out of our way to protect whistleblowers. You can remain anonymous for as long as you want, and, in fact, this is sometimes the best protection that journalists can offer whistleblowers.

Q. What information should I include?

To enable a thorough investigation, you should include a detailed description of the issue you are concerned about. Ideally, you should also include documents or data. The more information you provide, the better the work the journalists can do.
…

I need to write something up on “raw leaking,” that is not using a journalist. Look for that early next week!

Archives for the Dark Web: A Field Guide for Study

Filed under: Archives,Dark Web,Ethics,Journalism,Tor — Patrick Durusau @ 4:48 pm

Archives for the Dark Web: A Field Guide for Study by Robert A. Gehl.

Abstract:

This chapter provides a field guide for other digital humanists who want to study the Dark Web. In order to focus the chapter, I emphasize my belief that, in order to study the cultures of Dark Web sites and users, the digital humanist must engage with these systems’ technical infrastructures. I will provide specific reasons why I believe that understanding the technical details of Freenet, Tor, and I2P will benefit any researchers who study these systems, even if they focus on end users, aesthetics, or Dark Web cultures. To this end, I offer a catalog of archives and resources researchers could draw on and a discussion of why researchers should build their own archives. I conclude with some remarks about ethics of Dark Web research.

Highly recommended read but it falls short on practical archiving advice for starting researchers and journalists.

Digital resources, Dark Web or no, can be emphemeral. Archiving produces the only reliable and persistent record of resources as you encountered them.

I am untroubled by Gehl’s concern for research ethics. Research ethics can disarm and distract scholars in the face of amoral enemies. Governments and their contractors, to name only two such enemies, exhibit no ethical code other than self-advantage.

Those who harm innocents, rely on my non-contractual ethics at their own peril.

eXist-db 5.0.0 RC 3 [Prepping for Assange Data Tsunami]

Filed under: .Net,eXist,XML,XML Database,XQuery — Patrick Durusau @ 10:40 am

eXist-db 5.0.0 RC 3

One new feature and several bugs fixes over RC 2, but thought I should mention it for Assange Data Tsunami preppers.

I have deliberately avoided contact with any such preppers but you can read my advice at: username: 4julian password: $etJulianFree!2Day.

The gist is that sysadmins should, with appropriate cautions, create accounts with “username: 4julian password: $etJulianFree!2Day,” in the event that Julian Assange is taken into custory (a likely event).

If one truth teller (no Wikileaks release has ever been proven false or modified) disturbs the world, creating a tsunami of secret, classified, restricted, proprietary data, may shock it to its senses.

Start prepping for the Assange Data Tsunami today!

PS: Yes, there are a variety of social media events, broadcasts, etc. being planned. Wish them all well but governments respond to bleeding more than pleading. In this case, bleeding data seems appropriate.

Learning Math for Machine Learning [for building products/conducting academic research]

Filed under: Machine Learning,Mathematics — Patrick Durusau @ 10:09 am

Learning Math for Machine Learning by Vincent Chen.

From the post:

It’s not entirely clear what level of mathematics is necessary to get started in machine learning, especially for those who didn’t study math or statistics in school.

In this piece, my goal is to suggest the mathematical background necessary to build products or conduct academic research in machine learning. These suggestions are derived from conversations with machine learning engineers, researchers, and educators, as well as my own experiences in both machine learning research and industry roles.

To frame the math prerequisites, I first propose different mindsets and strategies for approaching your math education outside of traditional classroom settings. Then, I outline the specific backgrounds necessary for different kinds of machine learning work, as these subjects range from high school-level statistics and calculus to the latest developments in probabilistic graphical models (PGMs). By the end of the post, my hope is that you’ll have a sense of the math education you’ll need to be effective in your machine learning work, whatever that may be!
…

I headlined:

…my goal is to suggest the mathematical background necessary to build products or conduct academic research in machine learning.

because the amount of math you need for machine learning depends on your use of machine learning tools.

If you intend to “build products or conduct academic research in machine learning,” then Chen’s post is as good a place to start as any. And knowing more math is always a good thing. If for no other reason than to challenge “machine learning” others try to foist off on you.

However, there are existing machine learning tools which come with their own documentation and lore about their use in a wide variety of situations.

I always applaud deeper understanding of vulnerabilities or code, but it isn’t necessary that you re-write every, most, some tools from scratch to be effective in using machine learning.

While learning the math of machine learning at your own pace, I suggest:

  1. Define the goal of your machine learning. Recommendation? Recognition?
  2. Define the subject area and likely inputs for your goal.
  3. Search for the use of your tool (if you already have one) and experience reports.
  4. Test and compare your results to industry reports in the same area.

My list assumes you already understand the goals of your client. Except in rare cases, machine learning is a means to reach those goals, not a goal itself.

August 1, 2018

Developing SGML DTDs From Text To Model To Markup

Filed under: XML,XPath — Patrick Durusau @ 8:06 pm

Developing SGML DTDs: From Text To Model To Markup by Eve Maler and Jeanne El Andaloussi.

Maler and El Andaloussi summarize (1.2.4) the benefits of SGML this way:

To summarize, SGML markup is unique in that it combines several design strengths:

  • It is declarative, which helps document producers “write once, use many”—putting the same document data to multiple uses, such as delivery of documents in a variety of online and paper formats and interchange with others who wish to use the documents in different ways.
  • It is generic across systems and has a nonproprietary design, which helps make documents vendor and platform independent and “future-proof”—protecting them against changes in computer hardware and software.
  • It is contextual, which heightens the quality and completeness of processing by allowing documents to be structurally validated and by enabling logical collections of data to be manipulated intelligently.

The characteristics of being declarative, generic, nonproprietary, and contextual make the Standard Generalized Markup Language “standard” and “generalized.”

A truly remarkable work that is as relevant today as it was twenty-three years ago.

Most important lesson: Understanding your document comes before designing markup. Every time.

Printable Guns – When Censorship Fails

Filed under: 3D Printing,Government,Politics — Patrick Durusau @ 7:24 pm

It’s always nice when censorship fails. If you think about it for a minute, there were several places this AM where printable guns could be downloaded.

In anticipation that you will find unlooked for places with 3D printable gun designs, these may be useful resources:

20 Best 3D Printing Software Tools of 2018 (All Are Free)

20 Best Free STL File Viewer Tools of 2018

Before you try firing a printed gun, be sure to read 2018 3D Printed Gun Report – All You Need to Know very carefully.

There are reasons why no known military force uses 3D printed guns. Failure of the weapon and injury to its operator are two of them.

Interest in 3D printed guns has the potential to drive the market for better and cheaper 3D printers, as well as faster development of the technology.

All in all, not a bad result.

Trucks and beer (Music)

Filed under: Music,Text Analytics,Text Mining — Patrick Durusau @ 6:13 pm

Trucks and beer by John W. Miller.

From the post:

Inspired by a post on Big-ish Data, I’ve started working on a textual analysis of popular country music.

More specifically, I scraped Ranker.com for a list of the top female and male country artists of the last 100 years and used my python wrapper for the Genius API to download the lyrics to each song by every artist on the list. After my script ran for about six hours I was left with the lyrics to 12,446 songs by 83 artists stored in a 105 MB JSON file. As a bit of an outsider to the world of country music, I was curious whether some of the preconceived notions I had about the genre were true.

Some pertinent questions:

  • Which artist mentions trucks in their songs most often?
  • Does an artist’s affinity for trucks predict any other features? Their gender for example? Or their favorite drink?
  • How has the genre’s vocabulary changed over time?
  • Of all the artists, whose language is most diverse? Whose is most repetitive?

You can find my code for this project on GitHub.

Miller focuses on popular country music but the lesson here could be applied to any collection of lyrics.

What’s your favorite genre or group?

Here’s a history/data question: Does popular (for some definition of popular) music change before revolutions? If so, in what way?

While you are at Miller’s site, browse around. There’s a number of interesting posts in addition to this one.

…R Clients for Web APIs

Filed under: Data Mining,R,Web Applications — Patrick Durusau @ 3:35 pm

Harnessing the Power of the Web via R Clients for Web APIs by Lucy D’Agostino McGowan.

Abstract:

We often want to harness the power of the internet in our daily data practices, i.e., collect data from the internet, share data on the internet, let a dataset evolve on the internet and analyze it periodically, put products up on the internet, etc. While many of these goals can be achieved in a browser via mouse clicks, these practices aren’t very reproducible and they don’t scale, as they are difficult to capture and replicate. Most of what can be done in a browser can also be implemented with code. Web application programing interfaces (APIs) are one tool for facilitating this communication in a reproducible and scriptable way. In this talk we will discuss the general framework of common R clients for web APIs, as well as dive into specific examples. We will focus primarily on the googledrive package, a package that allows the user to control their Google Drive from the comfort of their R console, as well as other common R clients for web APIs, while discussing best practices for efficient and reproducible coding.

The ability to document and replicate acquisition of data is a “best practice,” until you have acquired data you prefer to not be attributed to you. 😉

For cases where the “best practice” obtains, consult McGowan’s slides.

July 31, 2018

Assassination Market Clickbait

Filed under: CryptoCurrency,Government,Politics — Patrick Durusau @ 3:46 pm

The First Augur Assassination Markets Have Arrived by David Floyd.

From the post:

“Killed, not die of natural causes or accidents.”

Pretty much everyone saw them coming, but it was no less disturbing when assassination markets actually began to appear on Augur, a decentralized protocol for betting on the outcomes of real-world events and that launched two weeks ago on ethereum.

The markets – which allow users to bet on the fates of prominent politicians, entrepreneurs and celebrities – in some cases explicitly specify assassination, as the quote above shows. (CoinDesk is intentionally not providing links to these markets or naming the individuals concerned.)

In addition to targeting individuals, some markets offer bets on whether mass shootings and terrorist attacks with certain minimum numbers of casualties will occur.

…

By creating a market for an assassination and placing a large “no” bet (actually, selling shares in the outcome), an individual or group could in effect place a bounty on the targeted person. The would-be assassin could then place a bet on “yes” (buy shares) and manipulate the outcome, to put it delicately.

…

An Augur assassination markets sounds like a way to democratize murder. Governments spend $billions every year killing people with their citizens exercising little or no influence of the choice of murder targets. An assassination market has the potential for a more democratic process. Or so it would seem.

The first thing you need is an Ethereum wallet. I choose a FireFox browser extension called MetaMask, but there are others, The Top 10 Best Ethereum Wallets (2018 Edition) by Sudhir Khatwani.

Next up, the Augur app. (GitHub) Augur isn’t long on documentation for the beginning users so here are screen shots and text about my installation process.

  1. I used sudo dpkg -i linux-Augur-1.0.7.deb, encountered dependency issues and so then ran apt-get install -f.

    OK, first screen shot, the default screen when I started Augur from the panel bar:

    I accepted all of the defaults, saved the configuration.

  2. After selecting connect, with the default configuration values, this is the next screen:

    As you can tell by the % meter, this is going to take a while. I didn’t time it precisely but would guess it is 90 minutes or longer to synch up.

  3. You probably don’t have to wait as long as I did but when it was over 99% synched, I connected with the Augur app:

  4. I should have expected it, next was the scroll down agreement to activate the checkbox and then agree to terms window, which in part reads:

    Right! I’ve taken numerous steps to conceal both my identity and activity, so sure, I’m going to try to tag Augus in court if something goes sideways.

    Sigh, old habits die hard. 😉

  5. The Augur default homepage (in part only):

    Then you choose “MARKETS” in the upper left-hand corner and look for assasinations.

A lot of installing to realize the reason why:

(CoinDesk is intentionally not providing links to these markets or naming the individuals concerned.)

There’s only one (1) such market and it has only one target, without any “no” money. As you might suspect, it’s the fav of all late night talk show hosts:

I don’t regret installing the new tools but was disappointed by the “assassination market clickbait” approach.

PS: Putin doesn’t even make my top 100. You?

July 30, 2018

Introducing VizHub

Filed under: D3,SVG,Visualization — Patrick Durusau @ 3:53 pm

Introducing VizHub by Curran Kelleher.

From the post:

I’d like to tell you a bit about VizHub, the next generation of Datavis.tech, a data visualization platform I worked on for about a year, and from which I learned how I wanted to develop VizHub.

VizHub is still early work in progress (alpha software), but the beta release should be ready by September, at which time I plan to use it as the platform for teaching (creating example code) and learning (students doing homework assignments) data visualization with D3.js and SVG in an online course this Fall at @WPI ! Many students are remote and transfer credit from WPI to other universities. If you’re a graduate student in Computer Science anywhere, you can register (see enrollment details). Here’s a taste of what my students made last year.
…

Difficulties with WordPress accepting images at the moment but here are links to three of the more impressive visualizations from Kelleher’s class:

If your visualization isn’t working, it’s unlikely its the tool. 😉

PS: CS 573 Data Visualization:

This course exposes students to the field of data visualization, i.e., the graphical communication of data and information for the purposes of presentation, confirmation, and exploration. The course introduces the stages of the visualization pipeline. This includes data modeling, mapping data attributes to graphical attributes, visual display techniques, tools, paradigms, and perceptual issues. Students learn to evaluate the effectiveness of visualizations for specific data, task, and user types. Students implement visualization algorithms and undertake projects involving the use of commercial and public-domain visualization tools. Students also read papers from the current visualization literature and do classroom presentations. Prerequisite: a graduate or undergraduate course in computer graphics.

July 29, 2018

When Phishing and “Dropped” USB Fails – Precision Issues in Graphic Libraries

Filed under: Cybersecurity,Security,Subject Identity — Patrick Durusau @ 3:10 pm

Drawing Outside the Box: Precision Issues in Graphic Libraries by Mark Brand and Ivan Fratric, Google Project Zero.

From the post:

In this blog post, we are going to write about a seldom seen vulnerability class that typically affects graphic libraries (though it can also occur in other types of software). The root cause of such issues is using limited precision arithmetic in cases where a precision error would invalidate security assumptions made by the application.

While we could also call other classes of bugs precision issues, namely integer overflows, the major difference is: with integer overflows, we are dealing with arithmetic operations where the magnitude of the result is too large to be accurately represented in the given precision. With the issues described in this blog post, we are dealing with arithmetic operations where the magnitude of the result or a part of the result is too small to be accurately represented in the given precision.

These issues can occur when using floating-point arithmetic in operations where the result is security-sensitive, but, as we’ll demonstrate later, can also occur in integer arithmetic in some cases.
…

With phishing success rates reported at 90% and the commonly cited 50% of all users who would insert a “found” USB drive in their computer, use of high end hacks is always a fall back position.

The techniques discussed here will be useful for such fall back cases but, the more interesting question to me comes in the conclusion:

…
When it comes to finding such issues, unfortunately, there doesn’t seem to be a great way to do it. When we started looking at Skia, initially we wanted to try using symbolic execution on the drawing algorithms to find input values that would lead to drawing out-of-bounds, as, on the surface, it seemed this is a problem symbolic execution would be well suited for. However, in practice, there were too many issues: most tools don’t support floating point symbolic variables and, even when running against just the integer parts of the simplest line drawing algorithm, we were unsuccessful in completing the run in a reasonable time (we were using KLEE with STP and Z3 backends).

In the end, what we ended up doing was a combination of the more old-school methods: manual source review, fuzzing (especially with values close to image boundaries) and, in some cases, when we already identified potentially problematic areas of code, even bruteforcing the range of all possible values.

Do you know of other instances where precision errors resulted in security issues? Let us know about them in the comments.

What set of subject identity criteria would enable rough indentification of these issues?

Thoughts?

July 28, 2018

Deep Learning … Wireless Jamming Attacks

Filed under: Cybersecurity,Government,Government Data,Hacking — Patrick Durusau @ 8:25 pm

Deep Learning for Launching and Mitigating Wireless Jamming Attacks by Tugba Erpek, Yalin E. Sagduyu, Yi Shi.

Abstract:

An adversarial machine learning approach is introduced to launch jamming attacks on wireless communications and a defense strategy is provided. A cognitive transmitter uses a pre-trained classifier to predict current channel status based on recent sensing results and decides whether to transmit or not, whereas a jammer collects channel status and ACKs to build a deep learning classifier that reliably predicts whether there will be a successful transmission next and effectively jams these transmissions. This jamming approach is shown to reduce the performance of the transmitter much more severely compared with randomized or sensing-based jamming. Next, a generative adversarial network (GAN) is developed for the jammer to reduce the time to collect the training dataset by augmenting it with synthetic samples. Then, a defense scheme is introduced for the transmitter that prevents the jammer from building a reliable classifier by deliberately taking a small number of wrong actions (in form of a causative attack launched against the jammer) when it accesses the spectrum. The transmitter systematically selects when to take wrong actions and adapts the level of defense to machine learning-based or conventional jamming behavior in order to mislead the jammer into making prediction errors and consequently increase its throughput.

As you know, convenience is going to triumph over security, even (especially?) in the context of military contractors. A deep learning approach may be overkill for low-bid contractor targets but it’s good practice for the occasionally more skilled opponent.

Enjoy!

July 24, 2018

Digital Research Tip

Filed under: Library,Research Methods — Patrick Durusau @ 6:44 pm

From Twitter:

Or photo the inside page with publications details (if it includes the shelf location).

Other digital research tips?

July 22, 2018

username: 4julian password: $etJulianFree!2Day

Filed under: Cybersecurity — Patrick Durusau @ 8:50 pm

Should Julian Assange lose his freedom, it looks eminent, sysadmins at all levels of corporations, governments and organizations are likely to create new root users:

username: 4julian
password: $etJulianFree!2Day

There’s nothing illegal about creating new users. Happens everyday.

Many have promised impotent and camera mugging expressions of rage as a response to an Assange arrest.

Systems hemorrhaging and continuing to hemorrhage data will have a much greater impact.

Don’t banks, stock exchanges, airports, news media, government, etc., all run on computers? Yes?

All those organizations should be lobbying the US government to leave Assange alone. Let him go freely to whatever destination he chooses. The alternative could be uncontrolled transparency.

Universal Feminine Hygiene

Filed under: Feminism,Government,Politics — Patrick Durusau @ 6:30 pm

It’s Not Just the Tampon Tax: Why Periods Are Political by By Karen Zraick reminded me to post a “progressive” proposal on feminine hygiene products.

Removing taxes on feminine hygiene products is a step in the right direction but why not go all the way and make those products universally available, at no cost?

The existing distribution chain for feminine hygiene products needs only a few minor tweaks to make that possible. Here’s my solution in three steps:

  1. Retailers provide feminine hygiene products to any customer, free of charge.
  2. Customers are free to choose any brand or type of feminine hygiene product.
  3. Retailers have a tax credit equal to feminine hygiene products distributed, at their retail “price.”

Charging customers for feminine hygiene products, directly or indirectly becomes illegal and states/localities are forbidden from limiting or regulating such sales in anyway.

A direct benefit to all women that preserves their freedom of choice of products. It re-uses existing distribution systems, without any additional forms or paperwork.

Share this with progressives seeking public office.

July 19, 2018

Printed Guns – Security Warning for Protesters

Filed under: FOIA,Free Speech,Government — Patrick Durusau @ 12:13 pm

DOJ Settles With Cody Wilson, Defense Distributed on 3D-Printed Guns

From the post:

The three-year legal battle over the future of 3D-printed guns is officially over, with the Department of Justice agreeing to allow the general public to “access, discuss, use, reproduce or otherwise benefit from” 3D gun files which had previously been prohibited, Reason.com reported.
…

DEFCAD will permit downloading and uploading of 3D gun files 1 August 2018.

Teasers on the site include:

AR-15

VZ. 58

Printable guns raise two major security concerns for protest groups in general but especially those who oppose pipelines, mining and other environmental crimes.

Traceability: Prior to 3-D printable guns, oppressors risked tracing of bullets fired to particular weapons, weapons which have relatively permanent serial numbers and at least some records of purchase/transfer. Not 100% and certainly rarely pursued but now even that remote possibility has been removed.

Untraceable Throw Down Guns: Putting “throw down” guns on protesters has always carried the risk of the true origin of a gun being discovered. Printable guns lower the cost of “throw down” guns and their lack of traceability, removes the risk of tracking a gun back to its point of origin.

The cheap “throw down” gun is the most likely use of 3-D printable guns by oppressors.

A partial solution for specific protest sites: Have a friendly police officer search you and document your lack of weapons. It’s not much but a law enforcement officer testifying on your behalf could be the saving touch.

PS: FOIA requests to police and other government departments should include purchases of 3-D printers and supplies for the same.

8 Big Processor Vulnerabilities in 2018

Filed under: Cybersecurity — Patrick Durusau @ 10:16 am

8 Big Processor Vulnerabilities in 2018 by Ericka Chickowski

Since the Spectre and Meltdown vulnerabilities knocked the glow off of the new year, 2018 has been the year of the CPU bug. Security researchers have been working in overdrive examining processors for design flaws, firmware bugs, and other vulnerabilities that put an entire computing architecture at risk.

They haven’t come up empty-handed.

Here’s what we’ve had to contend with this year on the CPU vulnerability front — and what we can expect in a couple of weeks when new research hits the stage at Black Hat.
…

Among those Chickowski dicusses:

BranchScope, Spectre Variants 3a and 4 (breaching barrier between cloud instances on the same CPU, think the IC’s planned cloud), not to leave AMD Ryzen chips unnoticed: Ryzenfall, Masterkey, Fallout, and Chimera, and others.

And the year is a little more than half over!

Enjoy!

July 18, 2018

Self-Help Transparency – Smoke Loader

Filed under: Cybersecurity,Malware,Transparency — Patrick Durusau @ 8:18 pm

Dissecting Smoke Loader by Michał Praszmo.

From the post:

Smoke Loader (also known as Dofoil) is a relatively small, modular bot that is mainly used to drop various malware families.

Even though it’s designed to drop other malware, it has some pretty hefty malware-like capabilities on its own.

Despite being quite old, it’s still going strong, recently being dropped from RigEK and MalSpam campaigns.

In this article we’ll see how Smoke Loader unpacks itself and interacts with the C2 server.
…

You can go the Freedom of Information Act (FOIA) route to become an “informed citizen,” provided you don’t mind:

  • Indeterminate exchanges to clarify your request
  • Delays and fees by agencies
  • Exemptions
  • Review and editing of documents by those most interested in non-disclosure

If you had access to the agency’s files:

  • No need to clarify your request
  • No delays or fees by the agency
  • No exemptions from disclosure
  • No review and editing of requested documents to prevent disclosure

Not to mention that self-help transparency saves the agency staff time and other resources in answering your request.

The other advantage of self-help transparency is that it works with political PACs, foreign governments, corporations and a host of other groups and institutions with no FOIA traditions.

All of those are incentives for closely attending to this blog post on the Smoke Loader.

Enjoy!

« Newer Posts — Older Posts »

Powered by WordPress