Skip to main content

Deep Learning - Tensorboard and TensorFlow Abstraction APIs: Estimator, Keras, and Layers

In this section of the course I'm taking, I'll be learning about three of the most used TensorFlow Abstraction APIs: Estimator, Keras, and Layers. Abstractions allow for simplification of code by rolling many of the lines of code that we commonly write into a built-in function. My code for this section can be found here.

While learning the Estimator API, I used a dataset of different types of wine. Each object had 13 attributes to describe it, most of them chemical. Each wine also had a label that could be either red, white, or rosé. I used the estimator.DNNClassifier API with three hidden layers of 13 neurons each. After the model was trained, it achieved a prediction accuracy (i.e. predicting if the wine is red/white/rosé given its attributes) of 97%.



While learning the Keras API, I used the same dataset and data organization code as before. It took a couple lines fewer to solve the same classification task, but the average was slightly lower -- 91% compared to the Estimator API's 97%. It's interesting that, while Keras is technically a separate framework from TensorFlow, all (or at least most) of its functionality has been ported over to be used on top of TensorFlow (as well as other frameworks). Using Keras to solve this classification task required approximately half of the number of lines of code as a classification example I did previously without using any abstraction APIs.


The Layers API is meant to abstract many pieces of code that's commonly used to build neural nets. The abstracted parameters are those that are rarely needed to be directly changed, so most of the time using the Layers API won't cause any issues. As an example of the code it can save, it only takes two lines of code to create 2 densely-connected layers using Layers. In a previous exercise (using MNIST), a convolutional network not using Layers required 15 lines of codes to create 2 layers. 7x savings!

Using 1000 training steps, the model had an accuracy of 100%, which is kind of fishy. So, I ran the model again using only 2 training steps. This modification should reduce the accuracy considerably, and indeed it did, to 57%. 

1000 training steps: 2 steps:


Tensorboard is built in to TensorFlow, and allows us to visualize complex graphs. My code for this can be found in the 'Intro to Tensorboard' file. After performing some operations and running the session, I followed these steps to see the visualization:
     1. Go to command line
     2. Use 'activate tfdeeplearning' -> 'tensorboard --logdir="./output" '
     3. Copy resulting address and paste it into a browser tab

My graph looked like this:

The names shown above are auto-generated by Tensorboard, but there is also the option to add custom names, as shown below. The implementation can be easily seen in the code via the 'name=' parameters. 


It's also possible to visually encapsulate sections of the graph by using 'with tf.name_scope('OPERATION_B'):  ' and then doing the operations within. These encapsulations can be expanded or minimized as necessary by double-clicking the node. 



...




An especially insightful tool in Tensorboard is the ability to see variable averages over time. By looking at the last block of my code, you can see that there is a Tensorflow variable called k that gets fed in numbers 1/400, 2/400, ... , 400/400 in a for loop using a feed dictionary. To see the average of k over time, run the provided code and then: 
    1. Go to Terminal and run 'tensorboard --logdir="./tmp/histogram_example"
    2. Copy address into a browser tab
    3. The graph shows the k values over time, for all values of N



Tensorboard makes it easy to find meaningful insights into machine learning models, which can be quite complex and difficult to trace back results to their source. TensorFlow's Abstraction APIs for common use cases can reduce the amount of repetitive code necessary to build a standard model, and also makes the code easier to follow. Overall, I think this section of the course will be quite useful when building more models in the future. 

















































Comments

Popular posts from this blog

Reading from and Modifying a DynamoDB Table using API Gateway

In many IoT (Internet of Things) projects, it is necessary to read from and write to a database. AWS has an application called DynamoDB that is a NoSQL database, exactly what we need. While it is very simple to add/modify/delete entries in this database via the user interface, it's not so obvious how to do so via HTTP API calls. So, in this post I'll continue the AWS portion of my journey into IoT to learn how to do just that. I'll be following  this  tutorial. The first thing I encountered is I wasn't sure if I was supposed to follow the steps under "Defining APIs", or if that was just knowledge for later. Turns out, that was knowledge for later. If I had any prior experience with APIs I probably would have realized this. Basically, the "API definition"s are just a high-level plan for what you want your API requests to look like in the end. In my opinion this information should go in the "Creating the APIs" section. Moving through "Cr...

Arduino Project: Sound Alert via Twitter

For my second IoT project using Arduino MKR1000, I'll employ a sound sensor to send out alerts via Twitter when a loud sound is detected. This could have many uses, ranging from informing the inhabitants of a house that there is someone at the door, informing a car owner that their alarm is going off, or residents that a gunshot has gone off in their neighborhood. See my previous posts to learn about the setup process for the Arduino MKR1000, which is wifi-enabled without any additional extensions. I will be using the Arduino IDE to compile the code. For hardware, I'll be using the sound sensor I purchased here , as well as my Arduino, three solder-free wires, a generic breadboard, and a Kindle power cable (this probably isn't the safest way to power Arduino, but it's what I had). I'll be following this  tutorial to set up the sound detector, as well as some code from here  to utilize Twitter. The sound setup tutorial was very simple and one of the clearest wiring g...

Deep Learning - Recurrent Neural Networks (RNNs)

The latest Deep Learning topic I've been exploring is the Recurrent Neural Network (RNN). It has many use cases, including: time series extrapolation (i.e. given historical data, predict future output); object trajectory; and natural language processing (NLP). The code I wrote during this section of the course I'm taking can be found here . For my summary of RNNs, click here . Here are the notes I took while watching the lectures: RNN Theory Examples of Sequences Time Series Data (e.g. stock prices, sales rates) Sentences (i.e. natural language) Audio/Music Car Trajectories (where a car will go in a situation) [1,2,3] shifted one time step into the future would be [2,3,4] will train a network to predict y-values for this A normal neuron has input, activation function, then output A Recurrent Neuron sends output back to itself this process is called “unrolling” the neuron note that the input of each neuron is a summation from both ...