It's always interesting to get an update on the different Cryptocurrency pricings. Sadly enough there are not that many services out there that do that out of the box for us, so let's create one!
Once the prerequisites are met, we can now set-up a flow that will scrape the cryptocurrency pricings from https://coinmarketcap.com/ every hour and send a Tweet once these got received. To do this, we need to create a flow that looks like this:
Scraper.AI -> Data Processor -> Twitter
The easiest way to do this is to have a Serverless function that gets executed on the Scraper.AI Webhook, which will manipulate the data and send this to Twitter. To send to Twitter, we utilize the IFTTT service, which has a connector to send a Tweet. In components we will thus get something like this:
Scraper.AI -> AWS Lambda -> IFTTT -> Twitter
Setting up AWS Lambda
The first thing we want to do is to set-up our AWS Lambda function. Here we want to configure a function that can be called through an HTTP endpoint, so we have to add a Trigger with the API Gateway to it.
Creating the function
Adding our API Gateway Trigger - Step 1
Adding our API Gateway Trigger - Step 2
Adding our API Gateway Trigger - Step 3
Adding our API Gateway Trigger - Step 4
Adding our API Gateway Trigger - Step 5
Once we then have our boilerplate and trigger configured, we can add a
Note: you will have to create this locally, install the NPM modules with
npm install, archive it into a
.zipfile and upload this to the Lambda portal.
Creating our Function - Package.json
Creating our Function - index.js
This was the hardest part! The rest is just click-click-click 😉 so let's continue!
IFTTT (If This, Then That) is an easy service to utilize. We can create an applet here that takes the URL called above in our AWS function (which we will get at the end here) and send it to Twitter.
Let's configure IFTTT
Connect Twitter - Authenticate Twitter Account
Connect Twitter - Select Action
Connect Twitter - Tweet Text Configuration
Now Twitter has been connected, the last thing we need to do is to get the Link for our Webhook.
Note: this URL you will have to enter in the AWS Lambda Function!!!
Configure Webhook - Fetch URL
Setting Up Scraper.AI
Navigate to https://coinmarketcap.com/ and select the properties
Then continue until the Website is scraped and displayed on Scraper.AI
Note: Configure scheduling to happen
hourly! This ensures that you post the newest tweets every time.
Finally, go to Notifications and enter the AWS Lambda endpoint you copied earlier into the "Webhook" field.
Congratulations! Tweets should now start appearing into your Twitter account.
Note: to test this easily, you can "Scrape Manually" on the dashboard!
An example of this can be seen below: