When we are working with Stocks, we want to be able to track the value of the different top performing ETFs each day. Luckily Yahoo provides us this data, but how can we actually get this data in a no-code kind of way?
Let's start by loading up our website: https://finance.yahoo.com/etfs this provides us an overview of the top performing ETFs. For our use-case we want to extract all the data provided (including the 50 and 200 day average since these are key metrics to track).
Now we can select these metrics through the extension:
But of course this is not everything, the page shows that there are 439 results. So how can we actually get all of those? Well when we click on the "Next" button in the extension, we eventually get presented with the monitoring options. In our case, we now want to utilize the Pagination option, that allows us to select the Next Pagination Button:
Important now is to correctly set the Page Limit (we don't know how many pages there will be!). So set this to 999. The engine will automatically detect recurrence and stop scraping when it has gathered all the data.
Our settings finally will look like the below. When you're comfortable with this, click the "Finish" button.
The scraper will now start and load all the pages. In our case it will stop after ~18 pages!
Once this is done we can see something like the below:
Very interesting to notice here is that it also detected the different links behind the stocks! Potentially allowing us to automate the gathering of top performing ETFs and utilizing these links in an application!