Screaming Frog In this Tutorial we will be learning about SEO spider Screaming Frog, a couple of tutorials for this software is already present in my site which i did in the past but these tutorials are little bit incomplete so in this post i will try to cover everything and also giving you a couple of things today for you to take home with you and basically a SEO template that many of you are familiar with, And also an explanation on how to sync up screaming frog with the template so that you can perform better SEO audits for your clients and your own websites. SEO Template Also from this post, you will be able fully understand how to use this tool and how to take all these different elements you’re going to be seeing in the tool and apply it to a SEO audit for a client or for your own websites so if you want to pick up any of my templates or you want to contact me about getting an audit any of that stuff means you can head over to the home page and you’ll be able to find all the resources and everything on the homepage. homepage   Also in this tutorial we are going to be doing a complete breakdown post for all the different things that you’re going to be finding in screaming frog so I actually wrote out all the different definitions for what’s going to be both in the template and in screaming frog so you could kind of tell what all these different things mean like crawl depth, unique out links, status codes, index ability so on. So this is just kind of one of those cheat sheets just in case you forget so first thing that you’re going to do when you are starting out is you’re going to : homepage template

  • Head over to the SEO audit template which again is on the home page of my website.
  • When you go to the homepage you can go ahead and click on “get SEO audit checklist” you can go ahead and download it.
  • After downloading it you’re just go ahead and open it up.
  • Now you’re gonna have a section called screaming frog in your template.

template So we’re gonna go to my website and as you guys probably know from one of my previous post (videos) that i have replaced screaming frog with Phase three because I was doing some keyword research. Now you would want to set up conditional formatting for these different tabs so that you can actually see whether or not certain things are above or below the average you’d like them to be at, when you’re going through your website but I’ll explain that a little bit more because that kind of starts to get complicated let’s just go over the basics of screaming frog so I’m just gonna exit out of this so you can just see it when it first pops up so I’m gonna go screaming frog. Template   Screaming frog is a free tool up for to 500 URLs and if you’re gonna buy it you can you could go up to 50,000 URLs crawled, And once you get about fifty past 50,000 URLs you have to go to configuration system and memory and you’re actually gonna need to increase the amount of ram that can be used to crawl on a website. Ideally if you’re starting out you’re going to want to probably get the free version until you get a few audits under your belt and then you’d probably want to start doing the paid. The really nice thing about the paid is that you can hook up Google Analytics and Google search console as well ahrefs so that you can get extra data which we’ll discuss later in this post. So let me just show you what it looks like when you crawl without a subscription without having a paid version and what it looks like without having any of that stuff hooked up so I’ll just put in my website and here you’re gonna see this is just gonna spit out a bunch of your URLs now what you’re gonna usually want to do first is you’re gonna want to filter by HTML you can filter by other stuff like PDFs, images. If you want to see if those are on your website and what things are on your website but for the most part I just only filter by HTML so I can just see all of the HTML pages on my website so here we go ; Screaming Frogs Seo audit template   Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs tree visualization Screaming Frogs As from the above pic we can see we got about 42 pages showing up, These are all crawlable by google obviously or by screaming frog. You can see here all of these different tabs are going to give us the idea of what’s going on so we can see the status code, So you know whether or not there are 301 or 200 status code and (if you don’t know about 301 it just basically means that the page is redirecting permanently to another page) If we keep scrolling over we can see all the titles for these different pages meta descriptions, meta robots and so on. Now we can take one of these columns and we can drag it over so if we want to see all the pages on my website with the titles of those pages we can actually just see them by dragging them over. However there are few problems i noticed with scrolling of screaming frog ;

  • It doesn’t let you drag over quickly.
  • You can’t just drag this and then it follows you and you have to drag and then move and then drag.
  • And also the other problem is that if you’re trying to save this layout to export this for later, unfortunately screaming frog still doesn’t support the ability to do that so after a certain amount of times of you closing your app this will actually reset and you can’t keep the layout that you previously had which makes it really annoying for exporting and saving this stuff to a template with the conditional formatting it really is irritating.

So anyways when we go into these pages one of the things that we can start doing is we can click here and we can see it’s called the tree table view and this will actually show you kind of how that urls are categorized. So you can see here the blog is then we have the category SEO so “” and then we have different posts under that category so : “” So this kind of gives you an idea of the architecture of your website, One other cool thing you can do is you can right click you can click on visualizations and you can look at the crawl tree graph and it will actually kind of show you how this stuff is being linked as well as, so you can see all of these pages are being linked from the home page, so the SEO coaching the SEO audit checklist the about page and then you can see how those pages are linking to other pages as well. Screaming frog crawl tree graph And that’s kind of cool because if for instance if you have all of these pages only being linked through the site map, you might want to consider linking them through the home page or through some of these other pages as well, because it kind of helps the crawler get to those pages easier, it passes more page influence when you’re linking them directly they’re not really far down in the crawl depth and the whole point of crawl depth by the way is you don’t want your big-money pages to be like five steps down, meaning that Google has to or crawlers have to go to five levels down the website to get to that page they ideally that you want your big pages to be linked from your home page so that it’s only two steps down so for instance SEO audit checklist is only two steps down you can see here which is a good thing because you want that to show up really soon when crawlers starts crawling it. Screaming Frog will actually show you the pages that are being affected or are being categorized as something so these are all the external links pointing from my website to all of these. So if I wanted to go click on external link and then see all the links that are 301 redirecting or if I wanted to see all the 404 links that I’m linking from my website to other websites I can just get an idea right here so we can see all of these are 301 and then you can actually click on here and see where it’s showing up. The page that’s linking is my best SEO tools page and it’s linking to a 301 redirect page now obviously you can go through here and manually change all these especially if you’re not on WordPress because sometimes you have to do that if you don’t have a plug-in, but if you do have WordPress you can use something like EPS redirects that’ll automatically pick up these pages and then just let you change the redirects for you yeah and then certain things like broken links that kind of stuff you want to check those. So if we go to protocol so this is everything that’s going to be HTTP or HTTPS so if I wanted to see find things that were HTTP or HTTPs I could click up here and filter by that we can see that only three pages and all of them are 301 so we don’t really have to worry about our SSL not being active on the website if we go to response codes we can see if we go to response codes we can see all of the different pages on the website internally so we can see which ones are 404 and which ones are 300 wanting and we can check out and make sure that these are different pages that we want to 301 and 404. This is just going to show you all the URLs on the website I believe and it’s going to let you filter by if there’s uppercase URLs, if there’s duplicate URLs, if there’s over 115 characters and again if you want to check all this stuff you can also do it on this tab as well so you can see URL over 115 characters duplicate so this is kind of a quicker way to look at all this stuff which is nice because here you’re going to be able to look at this stuff but you will have to dig through each of it to find what’s the problems is in this bar it’s kind of cool because when you just scroll down it’ll just show you if there’s anything you picked up that is an error or considered an error so for instance you know to HTTPS which we saw but they’re both 301 redirects response codes. Biggest issues with doing SEO audits is prioritization so what I mean by that is if you were to find let’s say on a website with 55 pages missing heading tags and you know hundred pages missing meta descriptions and then some pages that have low page speed and so on. Let’s say you found a thousand different things you could fix the question is what do you fix first. It could be missing hundred meta descriptions, it could be the low page speed, it could be the indexation, it could be the schema markup and so what ends up happening is when you get when you start to get to bigger websites this it gets even worse but even with smaller websites you know you look at the SEO issues and then you start to think about other stuff like keyword research and then adding content and then duplicate content and then internal linking and it can be extremely overwhelming thinking about all these different things that could be done to a website but you don’t know necessarily which one to do first. So how do you choose which task to do first, you do the content, you do the SEO, you do the social media. So the way you do this is really just aligned with the way I have my template laid out and if you look at my template you’ll see, I have four phases.. If you get the template that I have saved on my website it has four phases. You want to fix the things that are affecting every single page first so for instance if let’s say you have let’s say you have 22 missing heading tags you’re not going to want to fix those until you fix the page speed and why is that why do you want to fix the page speed or the SSL or the indexation or the duplicate content that kind of stuff why do you want to fix that stuff first, In phase one before phase two and the Answer is because in phase two when you start optimizing individual pages that’s when you’re going to get the smallest percent boost like let’s say you you add a missing heading tag to a page and you get let’s say a five percent boost on your SEO just for that page well just by fixing the page speed on the entire website you could get a 20 percent boost on every single page SEO flaw. SEO is same thing with SSL could be five percent boost except it would be on every page could be indexing a bunch of junk on your website that you get a ten percent boost on every page so site-wide issues are so much more important to fix first because you don’t want to start fixing individual pages and focusing on stuff like this like the heading tags missing or whatever until you fixed your site wide issues another reason why phase one is good to do first is because if you’re going to be getting rid of a bunch of pages in phase one then these 22 heading tags that are going to be missing might only be five missing heading tags because you just went indexed a bunch of pages that don’t need to be there on your website and so that’s sometimes what you’ll see people do is they’ll go and they’ll fix like 22 pages on their website individually but then they’ll be like oh I should have gotten rid of those because they have nothing to do with my website and that also has to do with the way your site. Architecture set up so if you go into your website and you see in here that a lot of your pages have let’s say nothing to do with what your overall theme is like let’s say you crawl your website and you really want to rank well for chiropractor in Denver Colorado so remember how I kind of showed you this tree graph before how things are sort of categorized so when you’re setting up your website you kind of want to think about it like a tree you want to think like okay here’s my main topic and here’s all of these other sub topics related to this. So there’s chiropractor in Denver there’s probably you know I don’t know I have no idea what goes with chiropractor but let’s just say massage pain relief sleep that kind of thing it now it’s your job to take the thing that’s most relevant to you first you don’t want to go for everything at once like you don’t wanna just go for everything around sleep everything around pain relief, everything around massage, everything around you know rank for everything around like just Denver. Let’s say we just want to make for like places to go in Denver and this is how a lot of people do SEO they go well you know this has a keyword difficulty three, this has a keyword difficulty five it has a six hundred search volume if I use my SEO tool to scan this I know it how it needs like a 1% keyword density and it needs six thousand words you don’t want to do that or you know ten links ten links going to the page ideally what you want to do is you want to skip all this distracting stuff and you. You want to stick with one thing that works well for you first so to put this in perspective what I’m doing on my own website and on my youtube channel and everything that I do is I want to go for everything around my most general keyword being SEO audit right so that doesn’t mean I can’t go and rank for other stuff it doesn’t mean that I can’t go for other general things like I could go try to rank for everything around SEO training or I could go try to rank for everything around SEO videos but what I found is that all of a lot of my money that I get and a lot of the conversions that I get a lot of my audience is built around people who want to know how to do an SEO audit not just how to do an SEO audit but also get an SEO audit done on their website so this is basically my first biggest conversion point I know that based on my experimenting that my biggest sort of free thing I can give away or thing that I can get people to sign up for is my SEO audits so that’s the first thing that I’m worried I want to focus on now when I think about all my related things around SEO audits now I know I have to kind of spend time you know looking into what are these topics that are gonna be related so you know we have SEO audit services we have SEO audit tutorials we have SEO audit templates SEO audit videos so where do I focus my effort first now generally you want to focus your effort where you have the most resources so like for me some my biggest resources right now are my templates and sort of my knowledge around you know screaming frog how to how to conduct audits things that are more around them like how to type stuff right versus I don’t have I don’t have as much resources around SEO services mainly because I don’t templatized services as much as I do templates and implementations which could be considered services but the point I’m trying to make is you want to really drill down as much as you can and start figuring out where is your sweet spot for the thing that you do the most of and the best and then you start creating a bunch of authority and content around that thing so for me I know like my again my specialty is like templates how to use certain softwares that kind of thing and and that’s why if you’ll see on the website why I made this post you know screen frog SEO audit tutorial here’s all of the breakdowns for the definitions I’m going to actually add this video in there get this broken down so that it’s a step-by-step sort of process as well that people can go out and use get off my website or get off my youtube channel and that’s going to be very valuable long term when I start building out the rest of this silo and so that’s basically what it is it’s called content siloing now in terms of prioritization you want to think okay well where does content silo and come in and content styling is really after your main things are fixed so you got your site wide issues and then you got your individual issues now your individual issues as well also have a prioritization and what I mean by this is let’s say you have a thousand pages on your website but let’s say you only have you know ten pages that are getting traffic are you going to go and fix those thousand other pages and what you’ll actually see from one of my recent videos is that you know in these pages that I have I’ll end up getting a rid of a lot of them not because they were just irrelevant but also because they just didn’t really have any traffic and I didn’t really feel the need to keep them around and my general rule of thumb is if you have pages that do get traffic but they’re not that relevant then you might want to keep them around so if you get these pages do get traffic but they’re kind of irrelevant to the website so for instance if you’re ranking for a denver chiropractor but you have like a page about you know best ways to eat right kind of irrelevant still sort of in the health niche but you know it’s bringing you traffic and you know maybe it’s bringing you some people signing up for your email list or whatever I would keep it however if the pages don’t get traffic and they’re also irrelevant then you need to get rid of them what’s the point in keeping a page on your website that doesn’t get traffic and is completely irrelevant to the website so if you go through your website and you start going through and you’re like oh I have a page about how to sell pairs in 2018 or how to how to fix your Apple computer but everything else is around chiropractics or something go get rid of those pages they have nothing to do with you as far as how to get rid of them this is another thing you have to think about when you get rid of a page do you know index it do you 301 it do you 302 it do you 410 it do you let it 404 do you canonical it I mean there’s all these different options right and and just to explain what these different things mean if you know index at page Google removes it from its index but users can still see it a canonical is Google does not keep this page in its index rather it prefers another page and it’s index and it now and you end users can still see this page a 301 means that this page now permanently redirects to another page at 302 means that it temporarily Redux to another page of 404 means that the pages temporarily not found in a 410 means that the page is permanently removed and again if you can’t remember all these things they’re all in this post so what do you do what do you do when you start removing let’s say 60 different pages on your website how do you assign that one page goes here three pages go here you know 10 pages go there right how do you assign that and the way you do it is really just based off what the purpose of the pages so really with canonical tags if you have duplicate pages you would prefer one page over the other obviously if you have duplicate pages that you both want to rank then you could make sure that the site has content that isn’t duplicated so that way you can now not have to use a canonical and the way you can do that which is another thing that you need to check is you can check with something called site liner for your internal duplicate content and you can check your website with something called copyscape to see your external duplicate content which again both of these are very important because if your website has plagiarized content on it it’s found through Scott copies gay you really want to change that or notify the website that has duplicated your content if you go into site liner and you see there’s internal duplicate content then you need to figure out whether or not you want to canonical that content and keep that page on your website no index it so that it still stays on your website but it’s not saying that it’s in about another page 301 and if you don’t want that page there anymore and it has traffic or links to it 302 if you temporarily need to redirect that page to another page because let’s say you just don’t want people going to that page right now but you want it to keep ranking for 10 if you want Google to completely remove it and 404 if you want Google to remove the page but go back and recrawl it later so I know this is all pretty confusing and really it doesn’t usually get this technical unless you start working with larger websites but for the most part when you look at your pages ideally again you want to fix all you want to fix their biggest site-wide issues first it’s affecting every page like PageSpeed bla bla bla then you want to go into your individual pages and start looking at the highest traffic pages first now if you don’t have screaming frog the pro version you’re not going to be able to do what I’m about to show you and what I’m about to show you is you’re going to be able to look at your pages one by one based off their traffic and that’s why it’s so important to get the pro version eventually if you can because the pro version is going to show you a lot of extra data once you hook it up to Google Analytics Google search console and H refs so I’m going to show you how to do this first thing you’re going to do if you have Pro and again if you don’t have Pro sure get it fitted do the other way the free version if you don’t but if you do you’re gonna click on Google Analytics you’re going to connect your account with the data that’s going to be on your website so you can see here it says chase Rhino Chase Turner com the view is all website data all users and I’m gonna just do the last month one other thing that I like to do is when I have goals set up in my Google Analytics is I like to track my different goal values so one thing that’s really important is if you don’t have goals set up on your website you need to go ahead and do that ASAP that’s like the first thing you need to do because if you’re not able to track how what you’re doing is leading to a better bottom line for your business meaning more conversions more leads that kind of thing then there’s really no point in doing marketing in the first place because traffic alone it does isn’t valuable you could be getting a thousand visits for the most irrelevant keyword you have a 90% bounce rate on that page with a zero conversion rate and guess what you’re making no money so the way you track that is you go and do your admin on your cogwheel you’re gonna go to goals and you’re gonna go ahead and set up your goals now you can see here my two main goals are goal ID 8 and goal ID 15 so this is people requesting an audit from me this is people out bout and clicking to my mini chat a also they’re signing up for my mini chat and basically that’s my two biggest ways to make money if I end up having more things in the future that I want to put on here like let’s say I want to sell a course or whatever then I can add this in here as well but for now my two biggest conversions are outbound clicks too many chats so people signing up for my list and then people requesting an SEO audit so what I’m going to do is I’m going to go look at these goal eight and goal 15 and I’m gonna plug that in here so here we go go eight and go fifteen now that’s just showing me like the completions on those I also want to see what the conversion rates are because you can see that there’s there might be people converting on those URLs but you actually want to see the conversion rates as well so that’s really just the main ones I use everything else is pretty much just checked by default I’m going to press ok and then I’m also going to go back to API access and go to Google search console now if I click into here and I go to chase runner again I can go ahead and look at the date range last month it’s matching Google Analytics and I’ll press ok and the last thing I’ll do is if you also have href switch is a good idea but you don’t have to have it you’re gonna just go press ok and you’re gonna restart the crawl so what’s the crawl is working with the API you’ll see they’re all at a hundred percent meaning that they’re working and we don’t have to really worry about them messing up okay because I don’t have a lot of pages of my website that should finish pretty quickly we’re gonna filter by whoops we’re gonna actually stroll over here let’s go if we can go over here can we go uh looks like we’re gonna have to just go up here and we’re gonna go to internal we’re gonna go back to the regular view and we’re gonna go HTML now what we just did is if we scroll over we’re gonna be able to see the pages with the most clicks so here we go these are all the pages with the most clicks on here now if I scroll back over I can see these are all of my highest traffic pages and a good thing that note is that within the top 20 pages here all of them have pretty much title tags right except for this page which is being redirected so that’s a good sign you don’t really want to see you know your top 20 pages not have title tags or maybe have bad title tags that are all matching each other you want them all to be unique you want them to look like there’s something that people want to click on blah blah blah so what we can do is we can just press export save and we can go over to Google sheets now I use Google sheets because I like to save this stuff on the internet I don’t like to have to you know edit this stuff and then find out a day later that I didn’t save it on my computer so what I’ll do is I’ll go to quick quick access upload the sheet and now what I’m gonna do is I’m going to grab all of this I’m gonna bring it in to my SEO audit template copy and paste I’m gonna view freeze one row and I’m gonna bold it just so I can see a little bit better now if I stretch this out boom I could start organizing my tabs and the reason why I do this guys again is because when you’re thinking about optimizing pages on a website you want to look at as much data as possible you don’t want to be doing something called gessie oh and what guess EO is is you’re basically taking some sort of idea that you think will work and you’re just applying it to a website so you’re like oh well I think this website would do better if they put a picture of blah because you know that’s just what I like I like to see pictures of this while I’m when I’m surfing the internet or I think they should put a call to action that’s rad B instead of yellow because you know I just don’t like orange that doesn’t look good or you know based on some SEO checker I found you know their keyword density isn’t high enough so I’m going to change this from rank and cell to rank in whatever you don’t want to do that that’s all guess EO anytime you start using data outside of your own website so that means your own data sorry your own mind dad I guess anything that isn’t direct analytics right analytics is your is your best friend when it comes to SEO anything else meaning like your own mind or your own can your own opinions which people think are facts a lot of people look at their own opinions that go no that’s a fact your opinion is not a fact it never will be I’m sorry or other people’s opinions or other tools outside your website if you’re using a tool and there’s thousands of tools right now and they say you know we think that you should add 10 more keywords or you need to add the heading tag even screaming frog if you go and see it says you know heading tags missing or whatever it’s still not a direct signal that’s not a direct signal and the reason why it’s not a direct signal is because if you add 20 heading tags to your website there’s a chance that your what your ranking might improve but your ranking as well is not an indication that your page is performing better so ranking and traffic which a lot of people think so like the the archaic Stone Age SEO s are things that their opinions and other tools are going to help them the people that are a little bit more advanced think that their rankings and their traffic are helping them because it’s like well how can you say that you know the things that I’m doing are bad if I have rankings in traffic but the real pros rely only on analytics or basically mainly on analytics and they use these other things as maybe like a 1 to 5 percent boost and the reason why rankings and traffic don’t really help you as a signal is because again if you have thousands of visits to your website and you’re not checking your analytics data and let’s say you’ve a 90 percent bounce rate or like a 1 percent conversion rate or let’s say you don’t even have conversions set up then these really at the end of the day are just sort of vanity metrics they don’t really help you they just kind of distract you so when you look at your data you look at your analytics you want to see certain things like frizzes if I look at my ranking right guide and I scroll over here one thing that I know is that the crawled is let’s say it’s let’s say it’s five so that means it takes five clicks from the homepage to get to my rank and rent guide now from a user experience standpoint I know that that’s not great if I think that this is a very valuable post that is relevant to what the users looking for when they think of my brand that’s something that I would want to present to them as probably the second or third click in crawl depth now why is it that I would assign of ranking factors to that or why would I say that that would help my rankings and it’s because anytime you’re helping the user you’re basically helping yourself you’re helping your website no matter what in rankings in in conversions and whatever you want to call it if I look at the in links to a page I know that if the in links are low meaning that this page only has five internal links going to it that means there’s only five pages that this page can be found from and if I have let’s say a thousand pages of my website and only five pages are pointing to it I’m not really giving a great idea to the users that this is an important page on my website same thing with Google if I go over to my bounce rates and my pageviews and my conversion rates which is the real data I can see okay well the bounce rate on this page is pretty high it’s at 72% sorry yeah 72% I think I actually mix those it’s at forty nine percent it’s eighty seven eighty two percent so if it’s at eighty two percent and this is more common with blog posts you’ll find but if this is a website or a page that’s very valuable on my website and it’s something that I really want to bring a lot of conversions to then one thing that I would want to do is try to figure how to lower the bounce rate might be the page speed it might be the layout it might be that there’s not something for somebody to kind of land on and stick to like a video so there’s different things you can test around that but if we start looking at let’s say the conversion rates like let’s say we look and see goal conversion goal conversion eight which is the many check completions so anybody who signs up for my mini chat and then over here is anybody who gets an audit so if I look at these two bring them over boom I can see that this page is getting zero audits off of it nobody’s asking for an audit once they hit this page however 17 people in the last 30 days have gone to my mini chat and then I can see the conversion rate so if I go over to the conversion rates and I see that the amount of people where did I put the conversion rates here we go so many chat conversion rate whoops is somewhere over here here it is mini chat conversion rate I just usually do percent and audit percent so scrolling over boom so many check conversion rate on the home page that’s 70 percent that’s nuts that means 70% of people who hit my website sign up for my mini chat like that’s really good and then about 1.5 percent of people who hit my home page they go and get my SEO on it which could be better so certain things like maybe adding a video adding some sort of you know call to action that might be more clear even just by changing the call to action from the mini chat to the audit might get more people rank and rent you can see the conversion rate too many chats 2.8 so again these are different things that you need to look at these are all things that affect the bottom line of your business and it also gives you really good indicators on how to improve the website because you have your analytics data and then you have external data as we said you have the tools you have other people’s opinions you have your own opinions but instead of doing that you should really just use Google’s data or Google’s indicators well Google’s indicators are and this is really the most important thing in my opinion is you have two things that Google looks Google looks at and this is straight out of their documentation this is how they assign people who manually go and rate search results because just give you guys an FYI google doesn’t have all of its search results algorithmic some of them are actually based on their Google page raider team so like if you look up like payday loans that kind of stuff there’s actually a manual algorithm for that meaning that people go and manually rate those websites there’s different queries that have to do with something called your money your life why my L and Google takes us a lot more seriously because if anything affects your money or your health they don’t want it to be like some sort of algorithmic mistake where it actually ends up hurting somebody so they’ll have manual raiders going rate that so what they look at and this is the way they they explain it in this one hundred fifty five page document which I read and fell asleep countless times while I was reading it because it’s so boring you but the biggest thing they talk about is they have you have to there’s different sections in your website there’s your your menu like your main menu there’s your sidebars like over here your footer you guys know that website looks right and then you have your your main content area this is where all of your sort of your unique content supposed to be so the way they they describe this and they abbreviate it is in the middle you have something called MC which stands for main content and in your sidebars you have something called SC which stands for supplementary content so the difference between main content supplementary content is that main content is supposed to be as unique as possible why because when somebody hits a page they’re not expecting your menu to look any different than it would on any other page right your home pit your menu right up here let me get a little line draw your menu right up here is going to basically be the same thing everywhere you’re gonna have your homepage you can have your logo blah blah your sidebar is probably going to look pretty pretty familiar anywhere else as well like for instance all your product pages will probably have the same sidebars all your blog post will probably the same sidebars same thing and then your footer is probably now the same thing on every page as well you know you’ve got your copyright you got your Terms of Service whatever you’d be expecting so all of this stuff here it’s not supposed to really be unique that Google doesn’t look at this and think oh this is supposed to be unique content it’s supposed to change on every page so the point of this is that if you have links in these sidebars if you have text images etc none of these really get any SEO benefit because it’s basically duplicate content and it’s kind of weighed out of the algorithm now your main content is supposed to be all unique why because this page is supposed to look as unique as possible to Google and its users mm-hmm so if you have content that’s for instance duplicated let me put it in red so we know that that’s bad if you have content that’s duplicated you have content that’s keyword stuffed meaning it’s just over optimized let’s say your layout doesn’t really match what people are looking for or let’s say this page is just made to traffic people to other pages which are called doorway pages maybe it’s just made for links if it’s really just deceptive the user has nothing to do with your brand it’s your job to get rid of that stuff because a lot of websites will do this they’ll take pages that they think are going to be helpful to the user but if you look at it especially older websites you’ll actually usually see hundreds if not thousands of pages that are really just there to make money they don’t really care about the user and why is that a problem why is it a problem to not care about the user because you have let’s say on Google millions of pages per results up sorry millions of results per search query if not billions competing for the same keyword now let’s say one of these pages of your thousand pages has this stuff it’s duplicated it’s keyword stuff it’s irrelevant it’s just there to point it to another page for the purpose of links has nothing to do with your brand now what Google’s going to do is go okay well this page isn’t really relevant compared to the one other page that the competitor has that has you know the opposite of all of this stuff it’s really unique content it’s not keyword stuffed it’s really matching what the user wants blah blah blah now if you start having this happen for thousands of your pages Google is going to start picking up that your website is a low quality website because all of your pages on it are matching these themes meaning you have let’s say hundreds of pages that are duplicated you have hundreds of pages that are keyword stuffed you have hundreds of pages that have nothing to do with your brand or what the user is looking for and that’s going to bring down the entire sort of value of your website so instead of doing that what you do is you use Google’s indicators and the way you use Google’s indicators is you start looking up okay so let’s say for instance my theme as we talked about before was things around SEO audits now after getting rid of all my things that are irrelevant kind of match the stuff we were talking about before with the page quality what I’m gonna do is I’m gonna start going through these pages I’m gonna go and do a Content audit and if you guys remember how to do a Content audit from one of my previous videos you’ll see what I’ll do is I’ll go and I’ll look and find the related things that pages are ranking for and try to include those in my page so if I want to rank for something called rank and rent and the way I would assign keywords to my pages is I would move this column over after getting the screaming frog crawl and I would start mapping keywords through search console and this is also how you’re going to do the basically the data syncing that we’re talking about in screaming frog with the API so if you don’t have it so what you would do is you go into something like search console you’d look at your top pages based on traffic and then you would just start putting those into your template so if I go to my website and I go to chase runner on search console I can click on open report and I can sort by pages now this is going to show me the pages with the most traffic now again the way you’ve the way you map your keywords is you click on your page and you click on queries and you find the query that you think is most applicable to your page that’s usually getting the most traffic and has the highest amount of impressions so you can see here rank and rent is the keyword that I’m going to choose out of position five which is where it’s ranking currently on Google and all I have to do is just take that keyword and plug it right there I can also benchmark the data of where it’s at by taking those other columns and putting them where they’re at for the last 30 days or 90 days whatever it’s at and I can plug those into here as well for when I rebec in a month or three months I can I can take that data and I can retest it and see if it changed based off the optimizations I made so boom I could just add it like that but I’m just gonna keep just the keyword because I don’t really care too much about the data right now because I’m just giving an example and this is more for the content audit so now we know rank and Rand is our main keyword for that page now so what I’m going to do is I’m going to go to something like HR F’s or summarize if you can’t afford a dress whatever you whatever you can use to extract competitor data and you’re going to go over to keyword Explorer and put that keyword in so if we’re gonna press search we’re gonna see keyword difficulties at 6 and if I scroll down here I can see this is the top post that’s ranking right now so what I can do is I can go ahead and go click on the little down arrow and look at the organic keywords that this is ranking for so what this is going to do is it’s going to give me all of the different things that this is ranking for so you can see rank and rent ranking about SCO regular websites ranking Rhett Nisha is ranking right so these are all topics that I would want to expand on within this post because they’re all hyper relevant to what people are searching for if Google is ranking this number one chances are Google’s gonna be ranking this for at least 100 other keywords because Google finds it relevant to those other keywords but most people don’t expand on the things that they’re already ranking well for so you just take this you go okay well I’m going to a drink and rent niches in my article expand on that and boom it’s probably already included improve the article relevancy now you again you don’t want to just think about it from Google’s indicators which is just the inside data meaning the stuff that we know is it’s ranking for but also the outside data and outside data is this if this website is ranking well for rank and rent we know there’s probably a specific layout for it let’s say maybe there’s comparison tables maybe there’s reviews maybe there’s these social shares at the top maybe there’s a table of contents so when you go through these posts you usually want to look and see manually now I’m not talking about with just like a page check earning that stuff manually go through the top you know ten results or whatever and see what they have in common you can see oh another page with a table of contents and let’s say most of these are blog posts let’s say there’s videos on most of these pages so you gather data and you start kind of putting together a game plan you go okay well first of all I want to include these different topics and this is really just based off the content audit so if we go into the content on it checklist we can see okay well the keyword is this here’s all the general search phrases that we grabbed from 8 reps and then the related search phrases whether there’s a video or whatever but then you can also come up with a game plan so you can put in here in your notes you can say alright well in this page insert note in this page I found that you know we need to have a table of contents we should probably add a video about whatever towards the middle we should have a better call to action that indicates what the other company is also probably doing we need to have but you know more internal links whatever it is we need to switch this from a service page to a blog post that people are going to read instead of just going to like a service we need to add reviews we need to get more comments because the top pages that are ranking have hundreds of comments and we have zero follow along so again these are all theories but you’re doing it based off what you can see already working for other people so you combine all that stuff together with what we’re talking about you know fixing the site when issues and optimizing the individual pages based on the top 10 most relevant pages to your website you start figuring out your content so I strategy and architecture and then phase four you start building out your authority and the way you build out Authority is what with what I do cut which is called Authority hacking and if you guys haven’t checked out my     authority action course it’s pretty cool by the way if you haven’t checked out at any of my courses you can go to chase senator duck teachable and you can go check out all the courses they have I’m thinking about doing a Black Friday special if you’re interested in that let me know I’ll probably make the code just Black Friday special when you check out but the authority hacking course is where I cover how to actually go and take the principle of what we’re doing here and start optimizing a strategy to build as much relevancy to your pages as possible using social media tactics call to action tactics like for instance the one I use with these videos a lot of the reason why you’ll see is what these videos get as much traction as they do is because I will offer people some sort of compensation which is usually some sort of free thing that I give out for them to leave a like or a comment on this video or the post I’m gonna be making which ends up getting my content distributed to more and more people and there’s a pretty cool strategy around that but that’s it for today really in terms of all this other stuff that goes on with screaming frog, you can go through these and you can look at you know I’m setting up your robots.txt.     Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Siteliner Copyscape Siteliner Siteliner Siteliner Screaming Frogs Screaming Frogs Screaming Frogs   Screaming Frogs Screaming Frogs Screaming Frogs Google analytics Google Analytics Google Analytics Screaming Frogs Screaming Frogs Google Search Console Site structure template   Chasereiner template Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs Screaming Frogs     Screaming Frogs Robots.txt or respects it you can go through and you can just you can grab all different response codes and export them you can generate XML sitemaps but for the most part a lot of this stuff you don’t really need to focus on a lot of it is really more prioritization you can only get so far with data you can have the best data in the world from analytics you have the best data from screaming frog or whatever crawl or outside SEO tools but what’s really going to come down what it’s really gonna come down to is prioritization and figuring out what is going to be a long term not just SEO plan for your website but marketing plan to improve the conversions on your website as in your business online and offline. happy seo’ing and happy marketing guys! later bye 🙂   Some Commonly Used Terms ; Query : A query is a type of search process such as a request for information from a database. This data or information may be from chosen parameters of a menu, query by examples, and query language. Clicks :  Clicks are a metric available to determine the count of clicks with search results. Clicks can refer to the way a user interacts with a web page. Calculating mouse clicks can be helpful for watching metrics with website monitoring. Impressions :  Impression metrics are counted when the user visits that page of results. If a user views only page 1 but the result is on page 2, the impression metric will not include it. CTR : This is the click-through rate which is the click count divided by the impression count. Position : The value of a position represents the average position for the search with the results from the item in that specific row. For example, a query grouping will give the average position for the query in the search results. Title 1 : The first page discovered on the page. Meta Description 1 : This is the first meta description on the webpage. Address : The address URL. Ahrefs URL Rating – Exact : This metric shows the strength of a backlink profile with a target URL. This could be on a scale from 1 to 100. Indexability : The Indexability is whether the URL is Indexable or Non-Indexable. The Indexability Status is the reason why a URL is non-indexable. For example, if it is canonicalised to another URL. Copyscape Pass : Copyscape is available to check articles and look all over the web to see if it is copied. Following the search, it says no results found or might have links to sites and the % copied. Clicks : Clicks are a metric available to determine the count of clicks with search results. GA Sessions : Sessions, sometimes referred to as visits, are units of measurements for a user’s actions within a specific time period. Session can be based on continuity in user activity and based on a continuity in a chain of requested pages. Indexability : The Indexability is whether the URL is Indexable or Non-Indexable. GA Bounce Rate : The Bounce rate is the percent of sessions that leave the site without any inteaction. GA Goal Conversion Rate All : A goal can be a way to measure how often someone completes a specific action on the website. When a user completes that specific action, there is an occurrence of a conversion. GA Avg Session Duration : The total duration of all Google Analytic sessions in a particular time period divided by number of sessions. Word Count : The word count is all the words inside the body tag, not including the HTML markup. Rendering settings may affect what HTML is considered. The definition of a word is the text split with spaces with no consideration given to the visibility of content, such as text inside a div set to hidden. Ahrefs Backlinks : Ahrefs is an SEO tool that can be used to perform backlink audits. Auditing an existing link profile is a way to get helpful data for making good decisions for the link profile. Unique Inlinks : The number of internal hyperlinks to the URL. The internal inlinks are the links pointing to a URL from the same subdomain that is being crawled. Crawl Depth : This is part of the page depth calculations with the depth of the page from the start page as the number of clicks away from the start page. Redirects are counted as a level for the page depth calculations. Unique Outlinks : The number of unique internal outlinks from the URL. Title 1 Length : The length of Title 1 is the character length of the page title. Meta Description 1 Length : The Meta Description 1 Is the first meta description on the page. The Meta Description 1 length is the character length of the meta description. H1-1 length : The h1 – 1 is the first heading on the page. The h1 – 1 length is the character length of the heading. H2-1 length : The h2 – 1 is the first h2 heading on the page. The h2 – 1 length is the character length of the h2. Status Code : The status code is the HTTP response code. Content : Content type of the URL Status : HTTP header response. The status code is the HTTP response code. Indexability Status : This is the reason why a URL is Non-Indexable. For example, if a URL is canonicalised to another URL, the Indexability Status will give the reason why the URL is non-indexable. Title 1 Pixel Width : The pixel width of the page title. Meta Description 1 Pixel Width : This is the pixel width of the meta description. Meta Keyword 1 : The meta keywords. Meta Keywords 1 Length : The meta keywords with the character length provided for the meta keywords. H1-1 : The first heading h1 on the page. H1-2 : An h1 heading on the page. H1-2 length : The character length of a heading h1 on the page. H2-1 : The first heading found on the page is the h2. H2-2 : The second h2 heading on the page. H2-2 length : The character length of the second h2 heading on the page. Meta Robots 1 : These are the meta robots directives identified on the URL. All instances will be found with The Spider if there are multiples. X-Robots-Tag 1 : These are the X-robots-tag HTTP header directores identified for the URL. Meta Refresh 1 : The meta refresh data Canonical Link Element 1 : This is the element data for the canonical link. The Spider will find all instances if there is canonical link element data on the URL in any multiples. rel=”next” 1 : Indicates the relationship between URLs with a paginated series. The SEO Spider collects these HTML link elements that are designed to provide the relationship between URLs. rel=”prev” 1 : Indicates the relationship between URLs with a paginated series. HTTP rel=”next” 1 : The link elements from the HTTP that indicates the relationship between URLs with a paginated series. HTTP rel=”prev” 1 : The link elements from the HTTP that indicates the relationship between URLs with a paginated series. Size (bytes) : The size in kilobytes. If the content-length header is provided, the value is set from the this amount. If it is not provided, the value is set to zero. For HTML pages this is updated to the uncompressed size in KB. After exporting, the size is in bytes. Text Ratio : The number of non-HTML characters found in the HTML body tag on a page as the text, divided by the notal number of characters the HTML page is made up of, and displayed as a percentage. Link Score : A metric between 0 and 100 that calculates the relative value of a page based on its internal links similar to Google’s PageRank. Inlinks : The number of internal hyperlinks to the URL. The Internal inlinks are links pointing to a give URL from the same subdomain that is being crawled. % of Total : The percentage of unique internal inlinks to the URL. The internal inlinks are the links pointing to a given URI from the same subdomain that is being crawled. Outlinks : The number of internal outlinks from the URL. Internal outlinks are considered the links from a given URL to other URLs on the same subdomain that is being crawled. External Outlinks : External outlinks are links from a given URL to another subdomain. Unique External Outlinks : This is the number of unique external outlinks from the URL. These external outlinks are the links from a given URL to another subdomain. Hash : This is helpful to check the duplicate content. Pages are exactly the same in content if two hash values match. As a duplicate content check, the hash values can show if there is any unique content. Response Time : The time in seconds to download the URL. Last Modified : This can be helpful for knowing content version control and web page maintenance. The read from the last-modified header in the servers HTTP response. The value will be empty if their server does not provide this. Redirect URL : If the address URL redirects, this column will include the redirect URL target. The status code will display the type of redirect. Redirect Type : The HTTP Redirect is triggered by an HTTP header. The JavaScript Redirect is triggered by execution of JavaScript, which can only happen when using JavaScript rendering. A MetaRefresh Redirect is triggered by a meta refresh tax in the HTML. GA % New Sessions : Sessions, sometimes referred to as visits, are units of measurements for a user’s actions within a specific time period. GA Users : Google Analytics users and active users metrics show how many users engaged with a website. GA New Users : A unique id is counted as a new unique user with GA. Whenever a new id is detected, GA counts a new user. GA Page Views Per Session : This is the average number of pages viewed during a session on a website. The more pages per session means that website users are more engaged and possibly exploring more of the website. GA Goal Completions All : This is the total number of visitors who have completed all of the defined elements for a particular goal. GA Goal Value All : With a GA account, you have access to create goals and input the goal values. The displays of goal values are available to review and customize. GA Page Value : A page value can be considered a measurment of influence that can be used to help you better understand which pages on a website drive conversions and revenue. Pages with a higher page value are more influential than pages with a low page value. CTR : The click through rate is the number of clicks that an ad receives divided by the number of times the ad is shown. The clicks divided by impressions is the CTR. Each of the keywords and ads can have its own CTRs that can be seen listed in an account. Ahrefs RefDomains – Exact : Refdomains contain the referring domains that contain backlinks to the target. URL Encoded Address : The URL requested by the SEO Spider. Position : The value of a position represents the average position for the search with the results from the item in that specific row.