Skip to main content

Reading from and Modifying a DynamoDB Table using API Gateway

In many IoT (Internet of Things) projects, it is necessary to read from and write to a database. AWS has an application called DynamoDB that is a NoSQL database, exactly what we need. While it is very simple to add/modify/delete entries in this database via the user interface, it's not so obvious how to do so via HTTP API calls.

So, in this post I'll continue the AWS portion of my journey into IoT to learn how to do just that. I'll be following this tutorial.

The first thing I encountered is I wasn't sure if I was supposed to follow the steps under "Defining APIs", or if that was just knowledge for later. Turns out, that was knowledge for later. If I had any prior experience with APIs I probably would have realized this. Basically, the "API definition"s are just a high-level plan for what you want your API requests to look like in the end. In my opinion this information should go in the "Creating the APIs" section.

Moving through "Creating the DynamoDB Table", I experienced no trouble.

Under the subheading "Creating the Post Comments API", the instruction begins from what's called the "editor screen". There's a screenshot, but no indication of how to make that screen appear. I figured out that you must click "Actions" before you can select "Create Resource".




Similarly, the "Create Method" option is in this drop down, which is needed in the next step but the tutorial doesn't explain how to get there. 

The tutorial also asks us to create an IAM role that "has permission to call the DynamoDB API PutItem for the Comments table", but doesn't explain how to do so. Looking at some other AWS documentation I took the following steps: 
  1. From the AWS Console, go to the IAM service. 
  2. Select "Policies" on the left panel.
  3. Select "Create a Policy".
  4. Choose "Select" by "Copy an AWS Managed Policy". 
  5. Choose "Select" next to "AmazonDynamoDBFullAccess"
  6. Choose "Create Policy". 
Honestly, I'm not sure if it's appropriate to use the AmazonDynamoDBFullAccess policy, but I don't know what else to use. 


The next issue I ran into was that when I tried to test the API using the response body provided in the tutorial, I get a message with a response body of "internal server error". To verify that the request to post a new item didn't work, I navigated to the DynamoDB table and, sure enough, no item was added. Looking at the logs generated from the execution, I see "Execution failed due to configuration error: API Gateway does not have permission to assume the provided role". I suspect this an issue with the IAM role I assigned. 

SOLUTION: 
After scouring the internet for an answer, I finally resorted to post a question on Stackoverflow. After creating the policy as outlined above, you must take the following steps: 

  1. Go to "Roles" and click "Attach Policy". 
  2. Select the policy you just created and click "Attach Policy".
  3. Copy the value of "Role ARN", located near the top of the page. 
  4. Go to your API in API Gateway and click the method you are working on (in my case, "Post").
  5. Click "Integration Request".
  6. Click the edit icon next to "Execution Role" and paste the role ARN inside. 
  7. Click the small check mark next to the text box, and you should be good to test your API. 

Upon successful completion of a POST request, you should see Successfully completed execution near the bottom of the logs output. To verify that the request was successful, I went to my DynamoDB table and sure enough, a new entry had been inserted!






The GET request implementation went smoothly. Upon success, I received the following log output (partial): 
Mon Sep 18 21:29:34 UTC 2017 : Endpoint request body after transformations: {
"TableName": "Comments",
"IndexName": "pageId-index",
"KeyConditionExpression": "pageId = :v1",
"ExpressionAttributeValues": {
":v1": {
"S": "breaking-news-story-01-18-2016"
}
}
}
Mon Sep 18 21:29:34 UTC 2017 : Sending request to https://dynamodb.us-east-1.amazonaws.com/?Action=Query
Mon Sep 18 21:29:34 UTC 2017 : Received response. Integration latency: 32 ms
Mon Sep 18 21:29:34 UTC 2017 : Endpoint response body before transformations: {"Count":1,"Items":[{"userName":{"S":"Just Saying Thank You"},"message":{"S":"I really enjoyed this story!!"},"commentId":{"S":"test-invoke-request"},"pageId":{"S":"breaking-news-story-01-18-2016"}}],"ScannedCount":1}


That concluded the end of the tutorial. 

Here is a video demonstration of my results: 


Knowing how to perform GET and POST operations on a DynamoDB table will prove invaluable in the exploration of IoT. All sorts of data can be stored and retrieved by devices, available to query and analyze to unearth valuable insights about the world around us. 

Comments

Popular posts from this blog

Arduino Project: Sound Alert via Twitter

For my second IoT project using Arduino MKR1000, I'll employ a sound sensor to send out alerts via Twitter when a loud sound is detected. This could have many uses, ranging from informing the inhabitants of a house that there is someone at the door, informing a car owner that their alarm is going off, or residents that a gunshot has gone off in their neighborhood. See my previous posts to learn about the setup process for the Arduino MKR1000, which is wifi-enabled without any additional extensions. I will be using the Arduino IDE to compile the code. For hardware, I'll be using the sound sensor I purchased here , as well as my Arduino, three solder-free wires, a generic breadboard, and a Kindle power cable (this probably isn't the safest way to power Arduino, but it's what I had). I'll be following this  tutorial to set up the sound detector, as well as some code from here  to utilize Twitter. The sound setup tutorial was very simple and one of the clearest wiring g...

Deep Learning - Recurrent Neural Networks (RNNs)

The latest Deep Learning topic I've been exploring is the Recurrent Neural Network (RNN). It has many use cases, including: time series extrapolation (i.e. given historical data, predict future output); object trajectory; and natural language processing (NLP). The code I wrote during this section of the course I'm taking can be found here . For my summary of RNNs, click here . Here are the notes I took while watching the lectures: RNN Theory Examples of Sequences Time Series Data (e.g. stock prices, sales rates) Sentences (i.e. natural language) Audio/Music Car Trajectories (where a car will go in a situation) [1,2,3] shifted one time step into the future would be [2,3,4] will train a network to predict y-values for this A normal neuron has input, activation function, then output A Recurrent Neuron sends output back to itself this process is called “unrolling” the neuron note that the input of each neuron is a summation from both ...