Screaming Frog SEO Spider is one of the best SEO tools available in the market. It is a comprehensive crawler that provides you with all of the information you need to know about your website.
It can crawl and analyse any website and extract keywords, meta tags, links, mentions on social media, errors and more.
This also provides additional data such as domain age and backlinks to help you understand how successful your site is online.
Screaming Frog SEO Spider has many benefits for both beginners and experienced users. It is not just an SEO tool but also a powerful crawler that can extract data from your site or someone else’s site.
Screaming Frog is a web crawler that can be used to crawl any website. It is an SEO tool that helps with data extraction, analysis and reporting.
In order to use this tool, you will need to download the software from the official website and install it on your computer. You will also need to start the software before you can start crawling a website. The first step of using Screaming Frog is configuring it. This includes setting up a connection with your target site, setting up filters for what you want to extract and what not and defining your crawler’s behaviour, like how many pages you want it to crawl at once or how frequent should it check for changes in a site’s content.
Once you have configured Screaming Frog, all that’s left is to start crawling. This is done by giving the crawler a URL, configuring your settings and letting it do its work. Once Screaming Frog has finished crawling, you will get an overview of all the content found on the site. You can then filter this content and view individual pages that have been crawled or export entire websites to CSV files for further analysis.
Enter a free PageSpeed Insights API key into the Config > API Access > PageSpeed Insights section of the Screaming Frog SEO Spider. Log in to your Google account and go to the Get started page for instructions on generating an API key.
Click the ‘Metrics’ tab after connecting to the PSI API. If you want desktop or mobile data, choose that option first (remember that CWVs will be applicable only to mobile data at first).
Select the data you want to be shown for each URL from PageSpeed Insights next. If you use the default choice, you’ll see data from the CRUX report, lab results from Google Lighthouse, and area for improvement data. Each metric can be selected or removed, however.
You can find Core Web Vitals under CrUX Metrics and lab data under Lighthouse Metrics if you choose manually.
In the ‘Enter URL to spider’ box of the SEO Spider, type or copy the address of the website you wish to crawl, and hit ‘Start.’
As another alternative, you can upload URLs for analysis in List mode (Mode > List).
After the crawl has been completed, the CVW data will be incorporated via the PSI API. Therefore, keep an eye on both progress bars until both have reached 100%.
By clicking on the PageSpeed tab, you will be able to view all discovered URLs with speed data that is reported from the API. If the API fails to fetch any data for a particular URL, the PSI Error and PSI Status columns will indicate this.
The CrUX metrics collected from the API are displayed to the right of the PageSpeed tab.
‘Core Web Vitals Assessment’ is the first column to appear (if enabled). Depending on whether all three Web Vitals are considered acceptable, it will be marked a Pass or a Fail.
As visible in the screenshot above, the top URL has an LCP of 1.99 seconds (under 2.5s), an FID of 37 milliseconds (under 100 milliseconds), and a CLS of 0.06 (under 0.1). This URL thus passes the CWV assessment in each metric.
It is important to examine which pages are failing and which metrics they are failing on. Once this is determined, improvements can be made to enhance the score.
When CWV URLs are not populated with data, it will be because the page doesn’t have enough visitors to provide sufficient real-world speed data. The Chrome UX report generally contains real-world field data only for the most popular pages.
If you enable the PageSpeed Insights API settings in the SEO Spider, you can see the opportunities to improve your web vital performance in a number of areas. Depending on whether PageSpeed is enabled, you will be able to see various filters for each opportunity.
The selected filter will show all pages that could be improved with the enhancement as well as the possible time savings. Selecting the lower PageSpeed Details tab underneath a URL will show you more details regarding that page – including all opportunities associated with that page, as well as individual resources associated with each opportunity.
The top menu item ‘Reports > PageSpeed’ allows you to export bulk opportunity data across the entire site.
The Largest Contentful Paint can be affected by many factors, and anything that reduces loading times will decrease it. Among these factors are:
Elaborate images tend to take up a lot of space, and they are also difficult to optimise. Make sure each image is properly sized and make use of lazy loading when it is appropriate. Using a more efficient image format, such as WebP, or improving the encoding of existing images might also be an option.
Performance Enhancement – A slow server slows down response times and increases loading times. When possible, try to minimise response times. In most cases, this can be accomplished by improving hosting or using Content Delivery Networks (CDNs).
LCP performance is slowed because rendering blockers are encountered while HTML is being parsed, namely stylesheets and CSS. If possible, you should defer non-critical CSS and JavaScript in order to speed up your site’s load time.
You should optimise your resources – JavaScript, CSS, fonts, etc., along with blocking the render. Each requires a significant amount of processing time. Try to optimise the files and remove unnecessary ones.
Whenever possible, try to minimise first input delays caused by large script execution times. Initial points to consider are:
Reducing the execution and rendering time of JavaScript files can be achieved by removing unnecessary portions (or entire scripts). Likewise, obsolete JavaScript is also included if it’s duplicated within a bundle and does not apply to modern browsers.
Delete unnecessary plugins and unused scripts – Eliminating unnecessary plugins will reduce the number of assets that need to be rendered and will mean fewer files to transfer. It may be possible to lazy-load third-party code in some cases to reduce the initial loading time.
On the Overview tab of the lower PageSpeed details, you can find information about how many third-party resources were used on a page.
Ensure that browsers minimise main thread work by switching to a background thread when the main thread is busy. User interactions will feel more responsive and immediate if you reduce the main thread execution time.
The number and size of resources on a page will be lower, so it will take less time to request, download, and execute. The execution of the web page will be sped up by reducing the number of requests as well as the size of the download.
CLS is most improved by making sure that pages are displayed in proper order and with defined spacing for each asset. There are a few areas that should be examined first:
Attributes for Image, Ad, Video and Iframes – All images, ads, videos, and iframes should have height or width attributes. Adding these resources to a page will ensure their spacing is appropriate, rather than shifting other content around.
Consider loading content from the bottom of the page to avoid adding content above and pushing other assets downwards – this is common with cookie banners.
FOUT/FOUT: Make sure custom fonts do not cause Flashing, Outlining, or Invisible Text – If custom fonts are applied to a page late, this can cause text to flash or become invisible until the custom font is loaded. It would be ideal if all custom fonts were preloaded so that they would be applied to text upon adding it to the page.
Don’t use non-composed animations – Non-composed animations can appear janky on low-end devices or when intense tasks are running simultaneously to page loading, causing layout shifts. Use compositor-only animations in order to avoid this.
Screaming Frog SEO Spider is a web crawler that can index and analyse the collected data in order to provide insights into the website’s SEO. It is a very powerful tool that can help in optimising websites for search engines and improving their rankings.
The benefits of using Screaming Frog SEO Spider are as follows:
-It can crawl through every page on the website
-It can find out which pages are being indexed by search engines -It provides details about how many links exist on each page
-It finds out how much duplicate content exists across different pages
-It helps in identifying what parts of the site are not being indexed by search engines
Page Speed Insights is a Google product that helps users to identify the performance of their website. It is an extension of the Google Chrome browser.
To connect Page Speed Insights to Screaming Frog, follow these steps:
1) Install Page Speed Insights in your Chrome browser
2) Add the extension to Screaming Frog
3) Click on the “Page Speed” tab in Screaming Frog and select “Google PageSpeed Insights” from the drop-down menu.
A backlink audit is a process that helps you to identify the links that are pointing to your website. It also tells you which links are broken, those that are followed, those that have a bad anchor text, and so on.
In order to do a backlink audit in Screaming Frog, you will need to first download the program. Then open it and click on “Site Explorer.” After this, you will need to input your domain name and click “Start.” This will start the process of crawling your website. Once this is done, all of the links pointing toward your website will be shown in Screaming Frog’s interface.
Exporting data from Screaming Frog is a very easy process. All you need to do is select the data that you want to export and then click on export.
Follow the steps below to export data from Screaming Frog:
1) Select the data that you want to export
2) Click on Export
3) Choose where and what format you would like to save the exported file in
