A Marketing Guide to How Keyword Stemming Works for SEO *

A Marketing Guide to How Keyword Stemming Works for SEO *

Keyword Stuffing Used to be the Norm

 

Content writing years ago was much different than it is today.  Stuffing as many keywords as possible into your content was what everybody did.  The following type of sentence was the standard.  “Orlando Eye is one of many things to do near Orlando.”  Sometimes we’d spell “Orando” or another word incorrectly to account for common typos.  

 

Google search query list of things to do near Orlando

When stuffing, writers might include five different variations of “things to do near Orlando” in a 300 word article.  Keyword stuffing was encouraged by how search engines handled search queries.  Content writers tried to duplicate the exact phrases or wording that people might search for.  Then they used Google to help people find them.

However, Google progressed, as did marketers and content writers.  Today, you barely have to think about SEO when crafting an article.  If you’re a solid writer and you’re staying on-topic, it gets you there naturally when it comes to Search Engine Optimization. 

 

Kayaking near Orlando Google Search with keyword Orlando misspelled

Google has become awesome at reading and analyzing text.  In the last decade it’s as if Google has gone from kindergarten to graduating from college.  If you want to optimize your content for “kayaking near Orlando” it’s much easier today.  Google understands that “kayaking springs near Orlando” and “kayaking tours near Orlando” are basically asking for the same thing.  You no longer have to match keywords word-for-word.  Don’t stuff your keywords, because that will result in low-quality articles, and Google hates bad content.

 

What is Keyword Stemming?

 

Google’s algorithms developed the ability to understand different forms of a keyword or phrase.  This is called “keyword stemming,” stemming  takes the root of a word (the stem) and determines what variations of the word deliver the information appropriate to the seach query.

If the stem word is “kayak” Google would recognize kayaking with manatees, kayaking orlando, kayak rentals near me and many other variations on that theme.  Google recognizes the variations in addition to the stem word.

 

Kayak Google search query screenshot

Stemming has been a developing concept at Google for many years.  What has changed is how Google has become better at understanding word variations and how they are used.  Because Google seach has evolved it is no longer necessary to stuff your content with word-for-word keyword matches.

Here’s a great article on how to succeed doing local SEO.

 

Google is Great for Keyword Suggestions

 

For most people the simplest solution is the free option available using Google.  Entering your keyword into Google, it will automatically populate with suggestions.  You may not want to use any of Google’s suggestions, however they may provide you with some pretty good ideas.  For “hiking in Florida,” this is what Google comes up with:

 

Hiking in Florida Google Search query screenshot

Their suggestions are nothing I don’t already know.  However, when I click on a search result and scroll to the bottom, I get helpful suggestions like “hiking in florida everglades” and “hiking in Florida with dogs.”

 

How You Can Find Keyword Variations

 

It’s second nature for many of us to write using stem word variations, and it’s easy to see a sentence that needs to be tweaked for optimization.

As an example, let’s optimize the last sentence for “keyword stemming.”  Keyword stemming is second nature for many of us, and I can easily spot a sentence to tweak for optimization.

To find these keyword variations there are many tools that are available.  Doing this is different than doing SEO research.  If you already have an idea what keyword or phrase you want to optimize for, you just need to find it’s variations.  

The following tools will help you create a list of variations.  However, the best thing to do is to train yourself to think in keyword variations so it comes to you naturally.

 

Karooya Keyword Variation Tool

 

The simplest keyword variation tool is from Karooya.   It’s a stripped down search tool that gives you straightforward variations, nothing more.  Use it if your brain is stuck and you can’t think of common forms of single words.

 

Karooya Free Keyword Variant Tool seach query for kayak screenshot

SEMrush Keyword Magic Tool

 

The Keyword Magic Tool from SEMrush lets you enter a seed keyword (the stem) and generate more keyword ideas.  I entered “kayaking in Florida” and selected “Broad Match,” which gives any variation of the phrase in any order.  Here are the results:

 

Semrush Keyword Magic Tool kayaking in Florida search query screenshot

Several of the variations still have “kayaking in Florida” written exactly like that, and I already know I can use that phrase as-is.  What’s helpful are the suggestions for “best kayaking in Florida,” “best places to kayak in Florida” and “kayaking in Clearwater Florida.”

 

Ubersuggest Keyword Ideas

 

Neil Patel’s browser-based app Ubersuggest can give you keyword ideas from a stem word or phrase, and also tell you your chances of ranking for the keyword, plus the top search engine results.

 

Ubersuggest kayaking in Florida search query screenshot

Updating Your Content for Optimization

 

If your content isn’t optimized there’s a quick way to optimize.  Seek out partial keyword phrases and update the sentence to maximize the SEO possibilities.  For example, if I want to rank for “keyword stemming” I would search for “stemming” to find sentences where I didn’t use the full phrase.  If I found one like: “Stemming isn’t new to Google.”   I’d simply add “keyword” to the beginning of the sentence and boom, a bit more optimization.

Here’s another example.  Let’s say I want to optimize for “biking in New York City.”  If my original sentence is, “The best biking routes run along the river.”  I can easily change it to any of these options:

  • The best biking in New York City is along the river.
  • New York City biking is prettiest along the river at sunrise.
  • Check out these riverside biking routes in New York City.

I’ve optimized for my keyword and can also experiment with stemming if other variations fit better.

 

When Not to Use Keyword Stemming

 

Are there times that you would not want to use keyword stemming?  It’s important to keep your keywords in sync with the context of your article.  Also avoid overusing or misusing keyword stemming because you can create low-quality content that tries to trick Google and disappoints readers.  Here are two examples:

 

  • If I want to rank for “hiking in New York,” Google may recognize “hiker in New York,” but that’s not the context of my article.  I’m not talking about hikers, I’m covering hiking trails.  This variation doesn’t fit the context of my article, and it may seem like a bait-and-switch to Google or my readers.
  • “Hiking in NYC” may be a Google-friendly variation to “hiking in NY,” but it’s changed the stem word too much and it’s out of context with the article.

Don’t use keyword stemming if it doesn’t fit the context or if the variation is unrelated to the stem word. 

 

Final Thoughts About Keyword Stemming

 

It’s important to know what Google hates because it’s very similar to what your readers will hate.  Google serves the user, and its algorithm changes are all about creating a better user experience.  It’s as if Google has done some audience research for you.

Google doesn’t like content that seems engineered or uses clumsy wording.  Keyword stemming gives you the ability vary your wording and focus on your writing quality while including plenty of SEO.

Want to learn more about how Google ranks your web pages?  Check out Elegant Themes Beginner’s Guide to Google’s 4 Most-Important Ranking Algorithms.

 

5 Low Hanging SEO Fruits You can Harvest for Your Business *

5 Low Hanging SEO Fruits You can Harvest for Your Business *

SEO Elements Improve Your Organic Search Ranking

 

Since I building my first website, my SEO hat has always been on.  When I assess any website, I focus on SEO as much as your design, your User Experience, and ADA compliance and general accessibility.  I’m always amazed by how often I find the basics missed. 

The SEO failures I regularly see, blow me away.  These are simple and easy things for the most part, yet are too often missed.  Google assesses over 200 different factors to determine their search ranking.  As Google has grown more sophisticated, now where it’s Artificial Intelligence understands natural language to assess a page’s theme.

These major algorithm changes shifted the focus of SEO experts from keywords to the quality and depth of the content.  Keywords remain relevant, but SEO providers have since needed to apply more finesse to ensure they’re not forced at the cost of “natural language”.  But we’re getting ahead of ourselves.

 

Your HTML Title Tags Should be Unique

 

This is the first thing I check.   You won’t see this in the viewport of the browser.  You see it up in the tab for the page, and more importantly in the search engine results – where it is most prominent.  

 

Title Tags should be unique to each page, and accurately and briefly describe the page

If I see “Home”, or just the company name, or a marketing tagline, I know right away that an SEO professional has not worked on the site.  The title tag should be no longer than approximately 70 characters.  

 

Google Search Query Snippets describe individual web pages

Title tags should be unique to each page, and accurately and briefly describe the page.  Google displays these unique titles in the search query snippets.  Keywords should naturally be included.  Your title should begin or include your top keyword.

<H1 Heading Tags>

 

Headings on your page are structured H1, H2, H3, etc.  Your H1 is the most influential and for it to have the most impact, only use one per page.  Your keywords should be finessed and reinforce those in the title.

 

Your H1 is the most influential and for it to have the most impact

The main purpose of categorizing your content with different headings so that your design become more user-friendly and people will get a notion about your website more easily by just reading the Sub-headings.

Now we know H1 tag is most important tag so whenever you going to use it for your webpage heading just try to make it like that people get an idea about your website just reading your heading. So that h1 tag should be in that way which describes everything about your web page just in one shot.

 

Headings Tag – Hierarchy

Hierarchy on your web page should be like <h1> comes first which is followed by <h2>, <h2> followed by <h3>, <h3> followed by <h4>, <h4> followed by <h5> and <h5> followed by <h6>.  For better SEO of your web page, one should have to follow the hierarchy in this way.

Headings Tags with Keywords

Keywords are first and foremost priority of any SEO Expert. In the matter, we first do the research of best keywords according to the web page and then categorize these keywords into focus keyword, primary keyword, secondary keyword and tertiary keyword. Focus Keyword is always one for the web page. It’s better to include your focus keyword into the title, Meta tags, and h1 of the webpage.

Heading Tags Frequency of Use

It is recommended that one should have to use h1 tag per web page because h1 tag should be like a newspaper heading and other content should be structured with subheadings by following the hierarchy of heading tags. Using of subheadings depend on the way your content is structured.

Ex: Single page websites.

Well, Heading tags are crucial for the proper On page SEO of your blog posts. One mistake which is common among newbie bloggers is excessive use of H2 or H3 tags & at times skipping H2 tags & only using H3 tags.

 

Good Content – Pages & Blogs

 

In the history of SEO, nobody has ever said that content quality isn’t important.  Of course it is.  So pages with good, deep content that reflect the themes established in the title and H1 provide search engines with something to really sink teeth into.  

Make sure to add alt tags to all imags – both to comply with ADA/WCAG but also for a little SEO boost as well.   Use blogs to feed fresh and timely content. And content length also matters.  Go deep. 

 

Keyword Density and Keyword Intent

 

Since Google’s Hummingbird update, where AI began reading, density has played less of a role.  And there is too much debate about how much is “just right”.  Just be careful not to over do it. 

 

Web Page Speed

 

This is especially relevant for mobile search results.  While the influence of speed is still just for searches made on mobile, we know what’s coming next.  There are many steps you can make to improve speed.   Amongst the top of my list is to 

  1. Write clean efficient code.
  2. Optimize images (sized correctly, blurring, quality setting no higher than 70%)
  3. Responsive Server Side (RESS) to appropriately resize images at server for the device
  4. Content Delivery Networks (CDN) to cache your content (images) closer to your visitors

 

My fav speed tester : Pingdom

 

Improve Your Organic Search Rankings

 

Those are my top 5 SEO items that I’d consider the low hanging fruit to improve your organic search rankings.  If you need any help, give us a shout!

 

11 SEO Ranking Tips to Make Your Long Form Content Engaging *

11 SEO Ranking Tips to Make Your Long Form Content Engaging *

Optimize Your Content for SEO Ranking

 

Have you ever written an eBook or any blog post content of 4000 words or more?

If you haven’t, you probably don’t understand the benefits of writing longer posts and may think that it’s just a waste of your time.  However if you have written longer content, your efforts will be rewarded and can mean a huge difference in your search engine rankings.

 

Let’s imagine that you want to set up your own business.  You probably would lie awake at night, worrying that your business might fail.  While researching business ideas, you find a comprehensive guide for starting any business.

This guide is loaded with everything you need to know and clears up all your doubts and fears.  After reading it you begin to trust the people that created it.  You have now found a go to information source to go to.

That is the benefit of creating informative, in-depth, long-form content.  Long-form content will get you more of what you want from your blog.  It will give you more visibility, improved engagement and social shares, and more backlinks which are an integral part of your SEO.  Most important of all, it will make your audience start seeing you as an expert in your industry.

 

Begin With Keyword Research

 

After deciding on your blog topic, but before writing your post, you need to do some keyword research.   You need to take some time to search for the “focus keyword” that best describes or relates to your chosen topic.  You should also consider related keywords that might have less competition.

This is an important step that needs to be done correctly because it will ultimately determine your content’s success in search engines.  There are many amazing tools on the internet that can make your keyword research task much easier.  

One of my favorites is called Keyword Tool which is a free online keyword research tool that uses Google Autocomplete to generate hundreds of relevant long-tail keywords for any topic.

 

Free Keyword Tool Finds Great Keywords Using Google Autocomplete

Another important consideration in your keyword research is keyword intent. Keyword intent is simply a representation of the your user’s purpose for the search.  It’s what we think the user is likely to do during a search since we can’t be sure.  Keyword intent is a very important concept when it comes to keyword research.  

 

Target Long Tail Keywords

 

Now that you understand how important your focus keyword is as the first step in writing your long form blog post.  The next step builds on your focus keyword.  Using your focus keyword you also need to be targeting long tail keywords.

These are keyword phrases that get very specific about any topic.  You should be interetsed in long tail keywords because thay are usually much easier for you to rank with and drive highly targeted traffic toward your content.

Below is a great example of long tail keywords which I got using the popular keywordtool.io tool:

 

Below is an example of long tail keywords which I got from the popular keywordtool.io tool

You can also use the same tool to gather as many long tail keywords as possible.  If you’re having trouble coming up with more keywords like these, another good place to look at is the “related searches” section in Google:

 

Optimize and Clean Your URL

 

Cleaner and shorter URLs help search engines understand what your post is about.  They also provide a better user experience.  

If possible, it’s great for your URL to also include your target keyword.  Doing this helps it to rank better.  Remember, it’s always better for your URL to be short and concise.

 

Use A Proper H1 Tag

 

Because your H1 Tag is usually your blog post title, it will always be the first thing people see when they arrive at your blog posts.  Selecting a great H1 tag can significantly impact your search engine ranking.

The chart below shows the search engine ranking result gotten from a local car parts store in Houston, Texas after they changed some of their page titles and H1 tags:

 

The chart below shows the search engine ranking result gotten from a local car parts store in Houston, Texas after they changed some of their page titles and H1 tags

When you have optimized your H1 tag it enables the search engines ability to index your content efficiently.

Your H1 tags should:

  • Be short and to the point.
  • Provide a clear idea of what the blog post is about.
  • Have a long tail keyword (your target keyword).

Once your headings have these qualities, you’re good to go.

 

Add Good Subheadings 

 

When you write your content, it’s important to have catchy headlines.  Your writing must be amazing, using well-crafted words, punchy sentences, and incredible insights into your readers needs.  If your post is longer than 150 words, you need to add sub-headings.

This is true because we all know that large blocks of text are a chore to read.  The subheadings you need to use are the h2, h3, and h4 tags usually contained in a blog post.

You need to use subheadings to characterize and break up your content into easy to follow chunks.  Doing this makes your content snackable by those who want to skimthrough your content and hit the high points.  

 

Add Inbound Links

 

Internal links are any links from one page on a website that points to another page on the same site.  Smart bloggers and content marketers use these internal links to tell Google and website visitors that a particular page of content is relevant.

There are many benefits associated with internal linking, the benefits include:

  • It helps to boost your ranking for certain keywords.
  • It helps search engines easily crawl your site.
  • It can help you to promote some paid services.
  • It gives your audience further reading options.

 

 

Add Outbound Links

 

Like inbound links, when you link out to related pages it tells Google that your page has high quality content.  Your outbound links also help search engines figure out your exact topic.

 

Brian from Backlinko's take on the need for outbound links in your content

Optimize Your Title Tag

 

Title tags are incredibly essential to your content.  Using a properly optimized title tag enhances your page’s visibility in search engine results pages (SERPs).  They’re usually the very first thing people see when your content shows up in the search results.

For the sake of clarity, a title tag is the title of each result that comes up.

Here’s an example:

While writing your page’s title tag, ensure it contains your focus keyword and also describes the benefits the page offers.

 

Optimize Your Meta Description For SEO

 

Your content meta description is a 160 character snippet that summarizes a page’s content.   Search engines like Bing and Yahoo support the description attribute while Google falls back on this tag when information about the page itself is requested in a search query.

A well written meta description attribute provides a concise explanation of your web page’s content.  This is  a Web page authors opportunity to give a meaningful description of the pages content.

Your meta description is often, but not always, displayed on search engine results pages.  The effectiveness of your meta description is easily measured by how it can affect your click-through rates.  Clicks may be a positive sign of effective title and description writing. 

 

Like the title tag, your page’s meta description enables your post to stand out in the search engine result pages.

A properly optimized meta description should:

  • Include a focus keyword.
  • Be brief (about 160 characters).
  • Be crystal clear and descriptive.
  • Contain a call-to-action (CTA).
  • Be enchanting and entice the visitor to read the entire page.

 

Multimedia Enhances Your SEO Ranking

 

You already understand that your content marketing is not about just text.  Text alone can only do so much.  Including entertaining videos, images, screenshots, diagram, and charts can take your content more engaging and useful for your audience.

Including multimedia will boost the time visitors spend on your site.  Multimedia will also reduce your bounce rate by keeping visitors on your site.

These are the two important user-interaction ranking factors that Google has been using to judge web pages recently.  I often use a lot of images, charts, and screenshots on my blog posts.  

I understand how multimedia makes my content authentic and genuine to my audience.  I know how using multimedia enhances the perceived value of the page, which will equally increase the likelihood of people linking to it.

While writing your long form content, try and add as many relevant multimedia as possible.

 

Add Your Target and Related Keywords 

 

It is important to include your focus keyword in the body of your blog post.  Don’t overdo it, just ensure it’s in there a few times.

This makes Google understand that your focus keyword is important while crawling your content.  Also, include some related, low competition keywords throughout your content to enable you to rank for even more focused long tail keywords.

It’s advisable to mention your keyword toward the end of your article, and in some of your subheadings.  Remember to tread with caution and don’t over do it.

 

Perform Proper SEO Of Long Form Content

 

Your long form content is effective in positioning you as an authority on your topic.  It also enhances your search engine ranking.  You can achieve your goals using solid SEO practices, like these 11 points.

This is not a complete list of everything needed for you to have great SEO.  These are the basics that will help you craft the best long form content on any topic.

Search engine optimization is a long term strategy and not something you should expect seeing results from immediately.  For your stategy to be effective, you need to be patient.  Embrace and implement these techniques, and then watch, with patience, as your rankings skyrocket.

 

How These 4 SEO Tactics Will Damage Your Brand’s Reputation *

How These 4 SEO Tactics Will Damage Your Brand’s Reputation *

4 SEO Tactics That Damage Your Brand’s Reputation  

 

From a marketing perspective, you’ve got to feel at least a little sorry for the utter devastation of Seaworlds’ sterling brand.  For decades, its parks were a fun, wholesome place to take the family.  Then, the movie Blackfish ruined our love of watching Shamu shows, and SeaWorld has been in PR damage control mode ever since.

Managing corporate reputations is tricky in the modern era, as SeaWorld proves.  You must take your company’s brand integrity into consideration in everything you do, including your search engine optimization efforts.

SEO is all about boosting your company in search engine rankings, but you can’t step all over your brand during the climb.  Many companies use tired and dated SEO tactics to reach for the coveted first page of rankings.

Google is way too smart at this point to reward black hat SEO practices, but bad SEO can also taint a company’s brand.  To avoid shredding your brand’s values in the process, employ “white hat” SEO practices instead.

 

Duplicate Content is SEO Brand Destroyer #1

 

It’s natural to assume that the more places your content can be found, the better.  So why not try to throw your latest blog post against every conceivable online wall to see if it will stick?

Google’s algorithms are extremely adept at identifying duplicate content, even if you tweak a few words here and there.  It doesn’t want to give equal SEO juice to the same content over and over, so it may not index the page at all or even punish sites that host duplicate content.

White Hat SEO Tip:   Generate new, unique content instead of re-posting content over and over again.  Every piece of new content may appear in search rankings and give your audience new ways to engage with you.

There’s a  wireless SD card manufacturer called Eyefi, that wanted to boost interest in its brand.  Because SD cards are used in cameras, Eyefi began profiling talented photographers on its website.  It’s a fun, visually compelling series that makes it easy for Eyefi to keep producing fresh content.

 

Fake Profiles are SEO Brand Destroyer #2

 

Seventy percent of consumers check out company or product reviews before making an online purchase. It’s no wonder, then, that some practitioners of black hat SEO regularly create fake profiles to post falsely positive reviews or to drum up support for their companies and products through social media.

 

 

Review websites like Yelp and major retailers like Amazon are getting better at eliminating fake reviews

Review websites like Yelp and major retailers like Amazon are getting better at eliminating fake reviews. Your brand’s reputation will take a major hit if you get caught.

White Hat SEO Tip:   Develop relationships with brand advocates and influencers who already love your product and are happy to mention it to their audiences.

You can get enthusiastic reviews or mentions simply by offering free samples to bloggers, YouTube stars, and well-known personalities in your sphere of influence.  The best way to boost your reputation is just to step back and let someone else sing your brand’s praises.

Keyword Stuffing is SEO Brand Destroyer #3

 

The era of SEO keyword stuffing is dead.  In the past, it was common to see poorly written content stuffed with keywords in an attempt to fool search engines.  User experiences suffered and reputations were damaged by articles inflated with duplicate keywords.  Using this method don’t expect to see your search rankings climb.

White Hat SEO Tip:   Search engines still care what your content is about, but they’ve become much better at understanding themes rather than counting keyword repeats. Quality matters, and it always will.

Focusing on great content creation that’s relevant to your business is still a terrific way to move up in the search rankings.

 

Link-Baiting is SEO Brand Destroyer #4

 

Links have been an important part of a solid SEO strategy.  Many believe that it’s better to have more backlinks.  This belief led to the development of link farms and pay-for-link schemes.  Google will penalize your site if you have too many backlinks from poor-quality sites.

White Hat SEO Tip:   Don’t be afraid to seek backlinks.  Be sure you do it in the correct way.  Search out quality websites looking for opportunities to partner in authentic ways.  The legitimate sharing of links between websites is still a valid SEO technique.

You need to create excellent content that serves your readers, who will share your content across diverse platforms.  This will naturally and organically generate backlinks.

 

How Does Your Business Appear in Search Queries

If consumers search for you today what will they find?  Today as many as 80 percent of consumers search for a product or service before buying.  Customers aresearching for your business now.

If you create quality content that’s useful, informative, and entertaining, your site will be an SEO superstar.

In SEO, as in life, strong reputations build strong bridges.  Taking shortcuts, cheating, or lying is the fastest way to burn those bridges and lose your reputation and your customers forever.

 

10 Steps to Get Your Site Properly Indexed by Google *

10 Steps to Get Your Site Properly Indexed by Google *

If Google Doesn’t Index Your Site, You’re Invisible

 

If Google doesn’t index your website, then you’re pretty much invisible.  To the rest of the world your site doesn’t even exist.  You won’t show up in any search queries, and you have almost no chance of getting any organic web traffic.

Given that you’re here reading this article, I’m guessing that you understand that you need to do something to make your website visible to the world.  So let’s get straight down to business.

This article will teach you how to fix these three problems:

  1. Your entire website isn’t indexed.
  2. Some of your pages are indexed, but others aren’t.
  3. Your newly‐published web pages aren’t getting indexed fast enough.

 

But first, let’s make sure we’re on the same page and fully‐understand what indexing really does.

 

 

Google discovers new web pages by crawling the web, and then they add those pages to their index.  They do this using a web spider called Googlebot.

Confused?  Let’s define a few of the key terms you need to understand.

  • Crawling:  The process of following hyperlinks on the web to discover new content.
  • Indexing:  The process of storing every web page in a vast database.
  • Web Spider:  A piece of software designed to carry out the crawling process at scale.
  • Googlebot:  Google’s web spider.

 

Here’s a video from Google that explains the process in more detail:

 

Google- How Search Works

When you search or Google something, you’re asking Google to return all the relevant pages from their index.  Because there are often millions of pages that fit the bill, Google’s ranking algorithm does its best to sort the pages so that you see the best and most relevant results first.

You may have noticed that Google places their sponsored pages at the top of their list – so they might not actually be the most relevant for YOU.

The critical point I’m making here is that indexing and ranking are two different things.  Indexing is showing up for the race, ranking is winning is winning the race.

You can’t possibly win without showing up for the race in the first place.

 

 

Go to Google, enter your website then search for site:yourwebsite.com

 

Google Indexing Check

This number shows roughly how many of your pages Google has indexed.

If you want to check the index status of a specific URL, use the Same site:yourwebsite.com/web-page-slug operator.

 

Google Find Out How Much Traffic a Website Gets

No results will show up if the page isn’t indexed.

Now, it’s worth noting that if you’re a Google Search Console user, you can use the Coverage report to get a more accurate insight into the index status of your website.  Just go to: Google Search Console > Index > Coverage for a full report.

 

Google Search Console Go to Index Go to Coverage

Look at the number of valid pages both with and without warnings.

If these two numbers total anything but zero, then Google has at least some of the pages on your website indexed.  If not, then you have a severe problem because none of your web pages are indexed.

SIDENOTE:  Not a Google Search Console user?  Sign up.  It’s free.  Everyone who runs a website and cares about getting traffic from Google should use Google Search Console.  It’s that important.

You can also use the search Console to check whether a specific page is indexed.  To do that, paste the URL into the URL Inspection tool.

If that page is indexed, it’ll say “URL is on Google.”

 

Google Search Console if the page is indexed it says URL is on Google

If the page isn’t indexed, you’ll see the words “URL is not on Google.”

 

Google Search Console If the page is not indexed it says URL is not on Google

 

Have you found that your website or web page isn’t indexed in Google?  If so, you need to try this:

 

  1. Go to Google Search Console
  2. Navigate to the URL inspection tool
  3. Paste the URL you’d like Google to index into the search bar.
  4. Wait for Google to check the URL
  5. Click the “Request indexing” button

 

This process is good practice when you publish a new post or page.  You’re effectively telling Google that you’ve added something new to your site and that they should take a look at it.

However, requesting indexing is unlikely to solve underlying problems preventing Google from indexing your old pages.  If that’s the case, follow the checklist below to diagnose and fix the problem.

 

1) Remove Crawl Blocks in Your robots.txt File

 

Is Google not indexing your entire website?  It could be due to a crawl block in something called a robots.txt file.  To check for this issue, go to yourdomain.com/robots.txt.

Look for either of these two snippets of code:

 

Google Robots Look for either of these two snippets of code

Both of these tell Googlebot that they’re not allowed to crawl any pages on your site.  To fix the issue, simply remove them.

A crawl block in robots.txt could also be the culprit if Google isn’t indexing a single web page.  To check if this is the case, paste the URL into the inspection tool in Google Search Console.  Click on the Coverage block to reveal more details, then look for the “Crawl allowed?  No: blocked by robots.txt” error.

This indicates that the page is blocked in robots.txt.

If that’s the case, recheck your robots.txt file for any “disallow” rules relating to the page or related subsection.

 

Google robots.txt remove where necessary

2) Remove Rogue Noindex Tags

 

Google won’t index pages if you tell them not to.  This is useful for keeping some web pages private.  There are two ways to accomplish this:

Method 1: meta tag

Pages with either of these meta tags in their <head> section won’t be indexed by Google:

 

Google Remove Rogue Noindex Tags

This is a meta robots tag, and it tells search engines whether they can or can’t index the page.  The key part of this is the “noindex” value.  If you see that, then the page is set to noindex.

To find all pages with a noindex meta tag on your site, run a crawl with Ahrefs’ Site Audit.  Go to the Internal pages report and look for the “Noindex page” warnings.

 

Google Meta Noindex Pages

Click through to see all affected pages. Remove the noindex meta tag from any pages where it doesn’t belong.

Method 2:   X‐Robots‐Tag

Crawlers also respect the X‐Robots‐Tag HTTP response header.  You can implement this using a server‐side scripting language like PHP, or in your .htaccess file, or by changing your server configuration.

The URL inspection tool in Search Console tells you whether Google is blocked from crawling a page because of this header.  Just enter your URL, then look for the “Indexing allowed?  No: ‘noindex’ detected in ‘X‐Robots‐Tag’ http header”

 

Google URL inspection tool in Search Console tells you whether Google is blocked from crawling a page because of this header

If you want to check for this issue across your site, run a crawl in Ahrefs’ Site Audit tool, then use the “Robots information in HTTP header” filter in the Data Explorer:

 

Google If you want to check for this issue

Tell your developer to exclude pages that you want indexing from returning this header.

 

3) Include the Page in Your Sitemap

 

A sitemap tells Google which pages on your site are important, and which aren’t.  It may also give some guidance on how often they should be re‐crawled.

Google should be able to find pages on your website regardless of whether they’re in your sitemap, but it’s still good practice to include them.  After all, there’s no point making Google’s life difficult.

To check if a page is in your sitemap, use the URL inspection tool in Search Console.  If you see the “URL is not on Google” error and “Sitemap: N/A,” then it isn’t in your sitemap or indexed.

 

Google To check if a page is in your sitemap, use the URL inspection tool in Search Console

Are you not using Search Console?  Head to your sitemap URL which is usually, yourdomain.com/sitemap.xml and search for the page.

 

Google Head to your sitemap URL usually, yourdomain.com:sitemap.xml and search for the page

These pages should be in your sitemap, so add them.  Once done, let Google know that you’ve updated your sitemap by pinging this URL:

http://www.google.com/ping?sitemap=http://yourwebsite.com/sitemap_url.xml

Replace the last part of the above URL with your sitemap URL.  You should then see something like this:

 

Sitemap Notification Received

That should speed up Google’s indexing of the page.

 

4) Remove Rogue Canonical Tags

 

A canonical tag tells Google which is the preferred version of a page.  It looks something like this:

<link rel="canonical” href="/page.html/">

Most pages either have no canonical tag, or what’s called a self‐referencing canonical tag.  That tells Google the page itself is the preferred and probably the only version.  In other words, you want this page to be indexed.

But if your page has a rogue canonical tag, then it could be telling Google about a preferred version of this page that doesn’t exist.  In which case, your page won’t get indexed.

To check for a canonical, use Google’s URL inspection tool.  You’ll see an “Alternate page with canonical tag” warning if the canonical points to another page.

 

Google To check for a canonical, use Google’s URL inspection tool

If this shouldn’t be there, and you want to index the page, remove the canonical tag.

IMPORTANT:   Canonical tags aren’t always bad.  Most pages with these tags will have them for a reason.  If you see that your page has a canonical set, then check the canonical page.  If this is indeed the preferred version of the page, and there’s no need to index the page in question as well, then the canonical tag should stay.

If you want a quick way to find rogue canonical tags across your entire site, run a crawl in Ahrefs’ Site Audit tool.  Go to the Data Explorer.  Use these settings:

 

If you want a quick way to find rogue canonical tags across your entire site, run a crawl in Ahrefs’ Site Audit tool

This looks for pages in your sitemap with non‐self‐referencing canonical tags.  Because you almost certainly want to index the pages in your sitemap, you should investigate further if this filter returns any results.

It’s highly likely that these pages either have a rogue canonical or shouldn’t be in your sitemap in the first place.

 

5) Check That the Page Isn’t Orphaned

 

Orphan pages are those without internal links pointing to them.  Because Google discovers new content by crawling the web, they’re unable to discover orphan pages through that process.  Website visitors won’t be able to find them either.

To check for orphan pages, crawl your site with Ahrefs’ Site Audit.  Next, check the Incoming links report for “Orphan page (has no incoming internal links)” errors:

 

To check for orphan pages, crawl your site with Ahrefs’ Site Audit

This shows all pages that are both indexable and present in your sitemap, yet have no internal links pointing to them.

 

IMPORTANT:   This process only works when two things are true:
  1. All the pages you want indexing are in your sitemaps
  2. You checked the box to use the pages in your sitemaps as starting points for the crawl when setting up the project in Ahrefs’ Site Audit.

 

Not confident that all the pages you want to be indexed are in your sitemap?  Try this:

  1. Download a full list of pages on your site (via your CMS)
  2. Crawl your website (using a tool like Ahrefs’ Site Audit)
  3. Cross‐reference the two lists of URLs

 

Any URLs not found during the crawl are orphan pages.

You can fix orphan pages in one of two ways:

  1. If the page is unimportant, delete it and remove from your sitemap.
  2. If the page is important, incorporate it into the internal link structure of your website.

 

6) Fix Nofollow Internal Links

 

Nofollow links are links with a rel=“nofollow” tag.  They prevent the transfer of PageRank to the destination URL.  Google also doesn’t crawl nofollow links.

Here’s what Google says about the matter:

Essentially, using nofollow causes us to drop the target links from our overall graph of the web.  However, the target pages may still appear in our index if other sites link to them without using nofollow, or if the URLs are submitted to Google in a Sitemap.

In short, you should make sure that all internal links to indexable pages are followed.

To do this, use Ahrefs’ Site Audit tool to crawl your site.  Check the Incoming links report for indexable pages with “Page has nofollow incoming internal links only” errors:

 

To do this, use Ahrefs’ Site Audit tool to crawl your site. Check the Incoming links report for indexable pages with “Page has nofollow incoming internal links only”

Remove the nofollow tag from these internal links, assuming that you want Google to index the page.  If not, either delete the page or noindex it.

Recommended reading:   What Is a Nofollow Link?  Everything You Need to Know

 

7) Add “Powerful” Internal Links

 

Google discovers new content by crawling your website.  If you neglect to internally link to the page in question then they may not be able to find it.

One easy solution to this problem is to add some internal links to the page.  You can do that from any other web page that Google can crawl and index.  However, if you want Google to index the page as fast as possible, it makes sense to do so from one of your more “powerful” pages.

Why should you do this?  Because Google is likely to recrawl such pages faster than less important pages.

To do this, head over to Ahrefs’ Site Explorer, enter your domain, then visit the Best by links report.

 

The Best Pages by Incoming Links

This shows all the pages on your website sorted by URL Rating (UR).  In other words, it shows the most authoritative pages first.  Skim this list and look for relevant pages from which to add internal links to the page in question.

For example, if we were looking to add an internal link to our guest posting guide, our link building guide would probably offer a relevant place from which to do so.  And that page just so happens to be the 11th most authoritative page on our blog:

 

Google will then see and follow that link next time they recrawl the page.

 

PRO TIP:   Paste the page from which you added the internal link into Google’s URL inspection tool.  Hit the “Request indexing” button to let Google know that something on the page has changed and that they should recrawl it as soon as possible.  This may speed up the process of them discovering the internal link and consequently, the page you want indexing.

8) Make Sure the Page is Valuable and Unique

 

Google is unlikely to index low‐quality pages because they hold no value for its users.  Here’s what Google’s John Mueller said about indexing in 2018:

 

He implies that if you want Google to index your website or web page, it needs to be “awesome and inspiring.”

If you’ve ruled out technical issues for the lack of indexing, then a lack of value could be the culprit.  For that reason, it’s worth reviewing the page with fresh eyes and asking yourself: Is this page genuinely valuable?  Would a user find value in this page if they clicked on it from the search results?

If the answer is no to either of those questions, then you need to improve your content.

You can find more potentially low‐quality pages that aren’t indexed using Ahrefs’ Site Audit tool and

URL Profiler.  To do that, go to Data Explorer in Ahrefs’ Site Audit and use these settings:

 

9) Remove Low‐quality Pages to Optimize Your “Crawl Budget”

 

Having too many low‐quality pages on your website serves only to waste crawl budget.

Here’s what Google says on the matter:

Wasting server resources on low‐value‐add pages will drain crawl activity from pages that do actually have value, which may cause a significant delay in discovering great content on a site.

Think of it like a teacher grading essays, one of which is yours.  If they have ten essays to grade, they’re going to get to yours quite quickly.  If they have a hundred, it’ll take them a bit longer.  If they have thousands, their workload is too high, and they may never get around to grading your essay.

Google does state that “crawl budget […] is not something most publishers have to worry about,” and that “if a site has fewer than a few thousand URLs, most of the time it will be crawled efficiently.”

Still, removing low‐quality pages from your website is never a bad thing.  It can only have a positive effect on your crawl budget.

You can use ahrefs content audit template to find potentially low‐quality and irrelevant pages that can be deleted.

 

Go to Data Explorer in Ahrefs’ Site Audit and use these settings

This will return “thin” pages that are indexable and currently get no organic traffic.  In other words, there’s a decent chance they aren’t indexed.

Export the report, then paste all the URLs into URL Profiler and run a Google Indexation check.

 

Export the report, then paste all the URLs into URL Profiler and run a Google Indexation check
IMPORTANT:   It’s recommended to use proxies if you’re doing this for lots of pages (i.e., over 100).  Otherwise, you run the risk of your IP getting banned by Google.  If you can’t do that, then another alternative is to search Google for a “free bulk Google indexation checker.”  There are a few of these tools around, but most of them are limited to <25 pages at a time.

Check any non‐indexed pages for quality issues.  Improve where necessary, then request reindexing in Google Search Console.

You should also aim to fix issues with duplicate content.  Google is unlikely to index duplicate or near‐duplicate pages.  Use the Content quality report in Site Audit to check for these issues.

 

alt=""

 

 

10) Build High‐Quality Backlinks

 

Backlinks tell Google that a web page is important.  After all, if someone is linking to it, then it must hold some value.  These are pages that Google wants to index.

For full transparency, Google doesn’t only index web pages with backlinks.  There are billions of indexed pages with no backlinks.  However, because Google sees pages with high‐quality links as more important, they’re likely to crawl—and re-crawl—such pages faster than those without.  That leads to faster indexing.

We have resources on building high‐quality backlinks on our blog.

 

Accessibility Toolbar