August 16, 2020 - monitoring

Tweet Cryptocurrency and Bitcoin prices every hour

It's always interesting to get an update on the different Cryptocurrency pricings. Sadly enough there are not that many services out there that do that out of the box for us, so let's create one!



Once the prerequisites are met, we can now set-up a flow that will scrape the cryptocurrency pricings from every hour and send a Tweet once these got received. To do this, we need to create a flow that looks like this:

Scraper.AI -> Data Processor -> Twitter

The easiest way to do this is to have a Serverless function that gets executed on the Scraper.AI Webhook, which will manipulate the data and send this to Twitter. To send to Twitter, we utilize the IFTTT service, which has a connector to send a Tweet. In components we will thus get something like this:

Scraper.AI -> AWS Lambda -> IFTTT -> Twitter


Setting up AWS Lambda

The first thing we want to do is to set-up our AWS Lambda function. Here we want to configure a function that can be called through an HTTP endpoint, so we have to add a Trigger with the API Gateway to it.

Creating the function

Adding our API Gateway Trigger - Step 1

Adding our API Gateway Trigger - Step 2

Adding our API Gateway Trigger - Step 3

Adding our API Gateway Trigger - Step 4

Adding our API Gateway Trigger - Step 5 Once we then have our boilerplate and trigger configured, we can add a package.json and index.js

Note: you will have to create this locally, install the NPM modules with npm install, archive it into a .zip file and upload this to the Lambda portal.

Creating our Function - Package.json

Creating our Function - index.js

This was the hardest part! The rest is just click-click-click 😉 so let's continue!

Get Started
### Setting up IFTTT

IFTTT (If This, Then That) is an easy service to utilize. We can create an applet here that takes the URL called above in our AWS function (which we will get at the end here) and send it to Twitter.

Let's configure IFTTT

Connect Twitter

Connect Twitter - Authenticate Twitter Account

Connect Twitter - Select Action

Connect Twitter - Tweet Text Configuration

Now Twitter has been connected, the last thing we need to do is to get the Link for our Webhook.

Note: this URL you will have to enter in the AWS Lambda Function!!!

Configure Webhook - Fetch URL

Setting Up Scraper.AI

Last but not least, we need to monitor to changes. Luckily we can utilize Scraper.AI here!

Navigate to and select the properties name, price and volume

Then continue until the Website is scraped and displayed on Scraper.AI

Note: Configure scheduling to happen hourly! This ensures that you post the newest tweets every time.

Finally, go to Notifications and enter the AWS Lambda endpoint you copied earlier into the "Webhook" field.

Congratulations! Tweets should now start appearing into your Twitter account.

Note: to test this easily, you can "Scrape Manually" on the dashboard!

An example of this can be seen below:

Get Started

Scraper.AI © 2020

Twitter - LinkedIn