Skip to main content

Deep Learning - Crash Course in NumPy, Pandas, SciKit Learn, and Data Visualization


In this section of the course I'm taking, I'll get a hands-on introduction to selected portions of the Python libraries NumPy, Pandas, and SciKit-Learn. I'll also use these in combination with other libraries to create basic data visualizations. These lectures only cover a handful of functions/methods in each library, and don't go in depth -- they only cover common functionality for beginner's deep learning projects. Here I'll provide a brief set of notes I took in the beginning of this section, but the substance of this post will lay in the Jupyter Notebooks that I've created while following along with the lectures and while implementing the review challenge at the end of the section. I'll also mention any trouble I had.

Crash Course Intro Notes
  • Three types of data: training data, test data, and holdout data
  • To "fit data" means to divide each data point by the largest data point in the set, which means the all data will be between 0 and 1.
  • Will typically fit data to training data, not test data because we don’t want to assume anything about the test data (i.e. that it will be within the bounds of the training data)

Here are links to all of the code I've completed in this section. 








The Review Exercise was a "quiz" of sorts, reviewing everything that was covered in this section of the course. It was all pretty straightforward, although it did take me a few minutes to figure out how to make a scatter plot of column 0 vs column 1. I kept trying things like x=mat[:0], y=mat[:1], but Turns out, I just needed to specify x=0, y=1. Once again, a simple solution that evaded me. 

I think my favorite part of this section were the data visualization tools. In a world of GUI-free commands and data sets, it's nice to have a colorful overview of information. For example, I learned how to represent relatively large sets of (random) data points as scatter plots and color-coded blocks:






That was a fun section, but now it's time to finally get into some real deep learning, which I'll write about in the next post.










Comments

Popular posts from this blog

Reading from and Modifying a DynamoDB Table using API Gateway

In many IoT (Internet of Things) projects, it is necessary to read from and write to a database. AWS has an application called DynamoDB that is a NoSQL database, exactly what we need. While it is very simple to add/modify/delete entries in this database via the user interface, it's not so obvious how to do so via HTTP API calls. So, in this post I'll continue the AWS portion of my journey into IoT to learn how to do just that. I'll be following  this  tutorial. The first thing I encountered is I wasn't sure if I was supposed to follow the steps under "Defining APIs", or if that was just knowledge for later. Turns out, that was knowledge for later. If I had any prior experience with APIs I probably would have realized this. Basically, the "API definition"s are just a high-level plan for what you want your API requests to look like in the end. In my opinion this information should go in the "Creating the APIs" section. Moving through "Cr...

Arduino Project: Sound Alert via Twitter

For my second IoT project using Arduino MKR1000, I'll employ a sound sensor to send out alerts via Twitter when a loud sound is detected. This could have many uses, ranging from informing the inhabitants of a house that there is someone at the door, informing a car owner that their alarm is going off, or residents that a gunshot has gone off in their neighborhood. See my previous posts to learn about the setup process for the Arduino MKR1000, which is wifi-enabled without any additional extensions. I will be using the Arduino IDE to compile the code. For hardware, I'll be using the sound sensor I purchased here , as well as my Arduino, three solder-free wires, a generic breadboard, and a Kindle power cable (this probably isn't the safest way to power Arduino, but it's what I had). I'll be following this  tutorial to set up the sound detector, as well as some code from here  to utilize Twitter. The sound setup tutorial was very simple and one of the clearest wiring g...

Deep Learning - Recurrent Neural Networks (RNNs)

The latest Deep Learning topic I've been exploring is the Recurrent Neural Network (RNN). It has many use cases, including: time series extrapolation (i.e. given historical data, predict future output); object trajectory; and natural language processing (NLP). The code I wrote during this section of the course I'm taking can be found here . For my summary of RNNs, click here . Here are the notes I took while watching the lectures: RNN Theory Examples of Sequences Time Series Data (e.g. stock prices, sales rates) Sentences (i.e. natural language) Audio/Music Car Trajectories (where a car will go in a situation) [1,2,3] shifted one time step into the future would be [2,3,4] will train a network to predict y-values for this A normal neuron has input, activation function, then output A Recurrent Neuron sends output back to itself this process is called “unrolling” the neuron note that the input of each neuron is a summation from both ...