In this section of the course I'm taking, I'll be learning about three of the most used TensorFlow Abstraction APIs: Estimator, Keras, and Layers. Abstractions allow for simplification of code by rolling many of the lines of code that we commonly write into a built-in function. My code for this section can be found here.
While learning the Estimator API, I used a dataset of different types of wine. Each object had 13 attributes to describe it, most of them chemical. Each wine also had a label that could be either red, white, or rosé. I used the estimator.DNNClassifier API with three hidden layers of 13 neurons each. After the model was trained, it achieved a prediction accuracy (i.e. predicting if the wine is red/white/rosé given its attributes) of 97%.
While learning the Keras API, I used the same dataset and data organization code as before. It took a couple lines fewer to solve the same classification task, but the average was slightly lower -- 91% compared to the Estimator API's 97%. It's interesting that, while Keras is technically a separate framework from TensorFlow, all (or at least most) of its functionality has been ported over to be used on top of TensorFlow (as well as other frameworks). Using Keras to solve this classification task required approximately half of the number of lines of code as a classification example I did previously without using any abstraction APIs.
Tensorboard is built in to TensorFlow, and allows us to visualize complex graphs. My code for this can be found in the 'Intro to Tensorboard' file. After performing some operations and running the session, I followed these steps to see the visualization:
1. Go to command line
2. Use 'activate tfdeeplearning' -> 'tensorboard --logdir="./output" '
3. Copy resulting address and paste it into a browser tab
My graph looked like this:
While learning the Estimator API, I used a dataset of different types of wine. Each object had 13 attributes to describe it, most of them chemical. Each wine also had a label that could be either red, white, or rosé. I used the estimator.DNNClassifier API with three hidden layers of 13 neurons each. After the model was trained, it achieved a prediction accuracy (i.e. predicting if the wine is red/white/rosé given its attributes) of 97%.
The Layers API is meant to abstract many pieces of code that's commonly used to build neural nets. The abstracted parameters are those that are rarely needed to be directly changed, so most of the time using the Layers API won't cause any issues. As an example of the code it can save, it only takes two lines of code to create 2 densely-connected layers using Layers. In a previous exercise (using MNIST), a convolutional network not using Layers required 15 lines of codes to create 2 layers. 7x savings!
Using 1000 training steps, the model had an accuracy of 100%, which is kind of fishy. So, I ran the model again using only 2 training steps. This modification should reduce the accuracy considerably, and indeed it did, to 57%.
1. Go to command line
2. Use 'activate tfdeeplearning' -> 'tensorboard --logdir="./output" '
3. Copy resulting address and paste it into a browser tab
My graph looked like this:
The names shown above are auto-generated by Tensorboard, but there is also the option to add custom names, as shown below. The implementation can be easily seen in the code via the 'name=' parameters.
It's also possible to visually encapsulate sections of the graph by using 'with tf.name_scope('OPERATION_B'): ' and then doing the operations within. These encapsulations can be expanded or minimized as necessary by double-clicking the node.
...
An especially insightful tool in Tensorboard is the ability to see variable averages over time. By looking at the last block of my code, you can see that there is a Tensorflow variable called k that gets fed in numbers 1/400, 2/400, ... , 400/400 in a for loop using a feed dictionary. To see the average of k over time, run the provided code and then:
1. Go to Terminal and run 'tensorboard --logdir="./tmp/histogram_example"
2. Copy address into a browser tab
3. The graph shows the k values over time, for all values of N
Tensorboard makes it easy to find meaningful insights into machine learning models, which can be quite complex and difficult to trace back results to their source. TensorFlow's Abstraction APIs for common use cases can reduce the amount of repetitive code necessary to build a standard model, and also makes the code easier to follow. Overall, I think this section of the course will be quite useful when building more models in the future.









Comments
Post a Comment