Tweet Cryptocurrency and Bitcoin prices every hour

Aug 16, 2020

It's always interesting to get an update on the different Cryptocurrency pricings. Sadly enough there are not that many services out there that do that out of the box for us, so let's create one!

Prerequisites

Flow

Once the prerequisites are met, we can now set-up a flow that will scrape the cryptocurrency pricings from https://coinmarketcap.com/ every hour and send a Tweet once these got received. To do this, we need to create a flow that looks like this:

Scraper.AI -> Data Processor -> Twitter

The easiest way to do this is to have a Serverless function that gets executed on the Scraper.AI Webhook, which will manipulate the data and send this to Twitter. To send to Twitter, we utilize the IFTTT service, which has a connector to send a Tweet. In components we will thus get something like this:

Scraper.AI -> AWS Lambda -> IFTTT -> Twitter

Setup

Setting up AWS Lambda

The first thing we want to do is to set-up our AWS Lambda function. Here we want to configure a function that can be called through an HTTP endpoint, so we have to add a Trigger with the API Gateway to it.

Creating the function
Adding our API Gateway Trigger - Step 1
Adding our API Gateway Trigger - Step 2
Adding our API Gateway Trigger - Step 3
Adding our API Gateway Trigger - Step 4
Adding our API Gateway Trigger - Step 5

Once we then have our boilerplate and trigger configured, we can add a package.json and index.js

Note: you will have to create this locally, install the NPM modules with npm install, archive it into a .zip file and upload this to the Lambda portal.
Creating our Function - Package.json
Creating our Function - index.js

This was the hardest part! The rest is just click-click-click 😉 so let's continue!

Setting up IFTTT

IFTTT (If This, Then That) is an easy service to utilize. We can create an applet here that takes the URL called above in our AWS function (which we will get at the end here) and send it to Twitter.

Let's configure IFTTT

Connect Twitter
Connect Twitter - Authenticate Twitter Account
Connect Twitter - Select Action
Connect Twitter - Tweet Text Configuration

Now Twitter has been connected, the last thing we need to do is to get the Link for our Webhook.

Note: this URL you will have to enter in the AWS Lambda Function!!!
Configure Webhook - Fetch URL

Setting Up Scraper.AI

Last but not least, we need to monitor https://coinmarketcap.com/ to changes. Luckily we can utilize Scraper.AI here!

Navigate to https://coinmarketcap.com/ and select the properties name, price and volume

Then continue until the Website is scraped and displayed on Scraper.AI

Note: Configure scheduling to happen hourly! This ensures that you post the newest tweets every time.

Finally, go to Notifications and enter the AWS Lambda endpoint you copied earlier into the "Webhook" field.

Congratulations! Tweets should now start appearing into your Twitter account.

Note: to test this easily, you can "Scrape Manually" on the dashboard!

An example of this can be seen below:

Great! You've successfully subscribed.
Great! Next, complete checkout for full access.
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.