Archive for the ‘TensorFlow’ Category

TensorFlow 1.2 Hits The Streets!

Saturday, June 17th, 2017

TensorFlow 1.2

I’m not copying the features and improvement here, better that you download TensorFlow 1.2 and experience them for yourself!

The incomplete list of models at TensorFlow Models:

  • adversarial_crypto: protecting communications with adversarial neural cryptography.
  • adversarial_text: semi-supervised sequence learning with adversarial training.
  • attention_ocr: a model for real-world image text extraction.
  • autoencoder: various autoencoders.
  • cognitive_mapping_and_planning: implementation of a spatial memory based mapping and planning architecture for visual navigation.
  • compression: compressing and decompressing images using a pre-trained Residual GRU network.
  • differential_privacy: privacy-preserving student models from multiple teachers.
  • domain_adaptation: domain separation networks.
  • im2txt: image-to-text neural network for image captioning.
  • inception: deep convolutional networks for computer vision.
  • learning_to_remember_rare_events: a large-scale life-long memory module for use in deep learning.
  • lm_1b: language modeling on the one billion word benchmark.
  • namignizer: recognize and generate names.
  • neural_gpu: highly parallel neural computer.
  • neural_programmer: neural network augmented with logic and mathematic operations.
  • next_frame_prediction: probabilistic future frame synthesis via cross convolutional networks.
  • object_detection: localizing and identifying multiple objects in a single image.
  • real_nvp: density estimation using real-valued non-volume preserving (real NVP) transformations.
  • resnet: deep and wide residual networks.
  • skip_thoughts: recurrent neural network sentence-to-vector encoder.
  • slim: image classification models in TF-Slim.
  • street: identify the name of a street (in France) from an image using a Deep RNN.
  • swivel: the Swivel algorithm for generating word embeddings.
  • syntaxnet: neural models of natural language syntax.
  • textsum: sequence-to-sequence with attention model for text summarization.
  • transformer: spatial transformer network, which allows the spatial manipulation of data within the network.
  • tutorials: models described in the TensorFlow tutorials.
  • video_prediction: predicting future video frames with neural advection.

And your TensorFlow model is ….?

Enjoy!

Virtual Jihadists (Bots)

Saturday, March 4th, 2017

Chip Huyen, who teaches CS 20SI: “TensorFlow for Deep Learning Research” @Standford, has posted code examples for the class, along with a chatbot, developed for one of the assignments.

The readme for the chatbot reads in part:

A neural chatbot using sequence to sequence model with attentional decoder. This is a fully functional chatbot.

This is based on Google Translate Tensorflow model https://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/

Sequence to sequence model by Cho et al.(2014)

Created by Chip Huyen as the starter code for assignment 3, class CS 20SI: “TensorFlow for Deep Learning Research” cs20si.stanford.edu

The detailed assignment handout and information on training time can be found at http://web.stanford.edu/class/cs20si/assignments/a3.pdf

Dialogue is lacking but this chatbot could be trained to appear to government forces as a live “jihadist” following and conversing with other “jihadists.” Who may themselves be chatbots.

Unlike the expense of pilots for a fleet of drones, a single user could “pilot” a group of chatbots, creating an over-sized impression in cyberspace. The deeper the modeling of human jihadists, the harder it will be to distinguish virtual jihadists.

I say “jihadists” for headline effect. You could create interacting chatbots for right/left wing hate groups, gun owners, churches, etc., in short, anyone seeking to dilute surveillance.

(Unlike the ACLU or EFF, I don’t concede there are any legitimate reasons for government surveillance. The dangers of government surveillance far exceed any possible crime it could prevent. Government surveillance is the question. The answer is NO.)


CS 20SI: Tensorflow for Deep Learning Research

From the webpage:

Tensorflow is a powerful open-source software library for machine learning developed by researchers at Google Brain. It has many pre-built functions to ease the task of building different neural networks. Tensorflow allows distribution of computation across different computers, as well as multiple CPUs and GPUs within a single machine. TensorFlow provides a Python API, as well as a less documented C++ API. For this course, we will be using Python.

This course will cover the fundamentals and contemporary usage of the Tensorflow library for deep learning research. We aim to help students understand the graphical computational model of Tensorflow, explore the functions it has to offer, and learn how to build and structure models best suited for a deep learning project. Through the course, students will use Tensorflow to build models of different complexity, from simple linear/logistic regression to convolutional neural network and recurrent neural networks with LSTM to solve tasks such as word embeddings, translation, optical character recognition. Students will also learn best practices to structure a model and manage research experiments.

Enjoy!

Text To Image Synthesis Using Thought Vectors

Sunday, August 28th, 2016

Text To Image Synthesis Using Thought Vectors by Paarth Neekhara.

Abstract:

This is an experimental tensorflow implementation of synthesizing images from captions using Skip Thought Vectors. The images are synthesized using the GAN-CLS Algorithm from the paper Generative Adversarial Text-to-Image Synthesis. This implementation is built on top of the excellent DCGAN in Tensorflow. The following is the model architecture. The blue bars represent the Skip Thought Vectors for the captions.

OK, that didn’t grab my attention, but this did:

generated-images-tensorflow-460

Full size image.

Not quite “Tea, Earl Grey, Hot,” but a step in that direction!

Hello World – Machine Learning Recipes #1

Saturday, April 16th, 2016

Hello World – Machine Learning Recipes #1 by Josh Gordon.

From the description:

Six lines of Python is all it takes to write your first machine learning program! In this episode, we’ll briefly introduce what machine learning is and why it’s important. Then, we’ll follow a recipe for supervised learning (a technique to create a classifier from examples) and code it up.

The first in a promised series on machine learning using scikit learn and TensorFlow.

The quality of video that you wish was available to intermediate and advanced treatments.

Quite a treat! Pass onto anyone interested in machine learning.

Enjoy!

Spending Time Rolling Your Own or Using Google Tools in Anger?

Wednesday, March 30th, 2016

The question: Spending Time Rolling Your Own or Using Google Tools in Anger? is one faced by many people who have watched computer technology evolve.

You could write your own blogging software or you can use one of the standard distributions.

You could write your own compiler or you can use one of the standard distributions.

You can install and maintain your own machine learning, big data apps, or you can use the tools offered by Google Machine Learning.

Tinkering with your local system until it is “just so” is fun, but it eats into billable time and honestly is a distraction.

Not promising I immersing in the Google-verse but an honest assessment of where to spend my time is in order.

Google takes Cloud Machine Learning service mainstream by Fausto Ibarra, Director, Product Management.

From the post:

Hundreds of different big data and analytics products and services fight for your attention as it’s one of the most fertile areas of innovation in our industry. And it’s no wonder; the most amazing consumer experiences are driven by insights derived from information. This is an area where Google Cloud Platform has invested almost two decades of engineering, and today at GCP NEXT we’re announcing some of the latest results of that work. This next round of innovation builds on our portfolio of data management and analytics capabilities by adding new products and services in multiples key areas:

Machine Learning:

We’re on a journey to create applications that can see, hear and understand the world around them. Today we’ve taken a major stride forward with the announcement of a new product family: Cloud Machine Learning. Cloud Machine Learning will take machine learning mainstream, giving data scientists and developers a way to build a new class of intelligent applications. It provides access to the same technologies that power Google Now, Google Photos and voice recognition in Google Search as easy to use REST APIs. It enables you to build powerful Machine Learning models on your data using the open-source TensorFlow machine learning library:

Big Data and Analytics:

Doing big data the cloud way means being more productive when building applications, with faster and better insights, without having to worry about the underlying infrastructure. To further this mission, we recently announced the general availability of Cloud Dataproc, our managed Apache Hadoop and Apache Spark service, and we’re adding new services and capabilities today:

Open Source:

Our Cloud Machine Learning offering leverages Google’s cutting edge machine learning and data processing technologies, some of which we’ve recently open sourced:

What, if anything, do you see as a serious omission in this version of the Google-verse?

Suggestions?

TensorFlow – A Collection of Resources

Wednesday, November 11th, 2015

Your Twitter account is groaning from tweets and retweets about Google open sourcing TensorFlow, its machine learning system.

To help you cut through the clutter (and myself), I have gleaned the following resources from tweets about TensorFlow:

1. Ground zero for TensorFlow: TensorFlow: smarter machine learning, for everyone by Sundar Pichai, CEO, Google.

Just a couple of years ago, you couldn’t talk to the Google app through the noise of a city sidewalk, or read a sign in Russian using Google Translate, or instantly find pictures of your Labradoodle in Google Photos. Our apps just weren’t smart enough. But in a short amount of time they’ve gotten much, much smarter. Now, thanks to machine learning, you can do all those things pretty easily, and a lot more. But even with all the progress we’ve made with machine learning, it could still work much better.

So we’ve built an entirely new machine learning system, which we call “TensorFlow.” TensorFlow is faster, smarter, and more flexible than our old system, so it can be adapted much more easily to new products and research. It’s a highly scalable machine learning system—it can run on a single smartphone or across thousands of computers in datacenters. We use TensorFlow for everything from speech recognition in the Google app, to Smart Reply in Inbox, to search in Google Photos. It allows us to build and train neural nets up to five times faster than our first-generation system, so we can use it to improve our products much more quickly.

2FNLTensorFlow

We’ve seen firsthand what TensorFlow can do, and we think it could make an even bigger impact outside Google. So today we’re also open-sourcing TensorFlow. We hope this will let the machine learning community—everyone from academic researchers, to engineers, to hobbyists—exchange ideas much more quickly, through working code rather than just research papers. And that, in turn, will accelerate research on machine learning, in the end making technology work better for everyone. Bonus: TensorFlow is for more than just machine learning. It may be useful wherever researchers are trying to make sense of very complex data—everything from protein folding to crunching astronomy data.

Machine learning is still in its infancy—computers today still can’t do what a 4-year-old can do effortlessly, like knowing the name of a dinosaur after seeing only a couple examples, or understanding that “I saw the Grand Canyon flying to Chicago” doesn’t mean the canyon is hurtling over the city. We have a lot of work ahead of us. But with TensorFlow we’ve got a good start, and we can all be in it together.

2. Homepage: TensorFlow is an Open Source Software Library for Machine Intelligence, which reads in part:

TensorFlow™ is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. TensorFlow was originally developed by researchers and engineers working on the Google Brain Team within Google’s Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research, but the system is general enough to be applicable in a wide variety of other domains as well.

3. The TensorFlow whitepaper: TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems, by Martín Abadi, et al. (forty authors).

Abstract:

TensorFlow [1] is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms. A computation expressed using TensorFlow can be executed with little or no change on a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards. The system is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models, and it has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas of computer science and other fields, including speech recognition, computer vision, robotics, information retrieval, natural language processing, geographic information extraction, and computational drug discovery. This paper describes the TensorFlow interface and an implementation of that interface that we have built at Google. The TensorFlow API and a reference implementation were released as an open-source package under the Apache 2.0 license in November, 2015 and are available at www.tensorflow.org.

4. Tutorials: Tutorials and Machine Learning Examples. I have omitted the descriptions to provide a quick pick-list of the current tutorial materials.

5. Non-Google Tutorials/Examples of TensorFlow:

TensorFlow Tutorials by Nathan Lintz.

Introduction to deep learning based on Google’s TensorFlow framework. These tutorials are direct ports of Newmu’s Theano Tutorials.

TensorFlow-Examples by Aymeric Damien.

Basic code examples for some machine learning algorithms, using TensorFlow library.

BTW, as of Wednesday, November 11, 2015, 22:10 UTC, Github shows 70 repository results for TensorFlow The two I list above were in my Twitter stream.

Given the time lapse between being open-sourced and non-Google examples on GitHub, it looks like TensorFlow is going to be popular. What do you think?

6. Benchmark TensorFlow by Soumith Chintala.

This will show up in the GitHub search link but I wanted to call it out for the high quality use of benchmarks and the discussion that follows. (I am sure there are other high quality discussions but I haven’t seen and therefore have not captured them.)

7. TensorFlow: Second Generation Deep Learning System by Jeff Dean. (approximately 45 minutes)

8. Rob Story opines that TensorFlow is built on the same ideas as @mrocklin‘s Dask (dask.pydata.org/en/latest).

9. If you need a Popular Science article to pass onto management, Dave Gershgorn has you covered with: How Google Aims To Dominate AI: The Search Giant Is Making Its AI Open Source So Anyone Can Use It.

In November 2007, Google laid the groundwork to dominate the mobile market by releasing Android, an open ­source operating system for phones. Eight years later to the month, Android has an an 80 percent market share, and Google is using the same trick—this time with artificial intelligence.

Today Google is announcing TensorFlow, its open ­source platform for machine learning, giving anyone a computer and internet connection (and casual background in deep learning algorithms) access to one of the most powerful machine learning platforms ever created. More than 50 Google products have adopted TensorFlow to harness deep learning (machine learning using deep neural networks) as a tool, from identifying you and your friends in the Photos app to refining its core search engine. Google has become a machine learning company. Now they’re taking what makes their services special, and giving it to the world. (emphasis in original)

10. One short commentary: Google TensorFlow: Updates & Lessons by Delip Rao.

TensorFlow came out today, and like the rest of the ML world, I buried myself with it. I have never been more excited about a new open source code. There are actionable tutorials etc. on the home page that’s worth checking out, but I wanted to know if this was yet another computational graph framework — we already have Theano and CGT (CGT is fast; Theano is most popular).

Links for Delip’s post:

Computational Graph Toolkit (CGT)

Theano


In the time it took to collect these resources on TensorFlow, I am certain more resources have appeared but hopefully these will continue to be fundamental resources for everyone interested in TensorFlow.

Enjoy!