drupal seo pitfalls how to avoid them

Drupal SEO Pitfallsand

How To Avoid Them- by Brent Gees and Wouter De Bruycker

1ABOUT USPUBLIC ENTITIES 2

Drupal SEO Pitfalls and How To Avoid Them - DrupalCon Amsterdam 2019 3

ABOUT US

Brent Gees

● Drupal Architect and Drupal Trainer at Dropsolid

● Speaker at Drupal Dev Days Lisbon 2018

● Speaker at Drupalcamp Ghent 2018


ABOUT US

Wouter De Bruycker

● SEO Strategist & Evangelist at Dropsolid

● Speaker at Digital Content Marketing Congress: "Technical SEO 101 (& 102)"

● Closing Keynote Speaker at DMB Day “Today is Yesterday's Future: SEO in Evolution"

2PUBLIC ENTITIESALL PAGES SHOULD BE AN ENTITY

ABOUT US

3

1


PUBLIC ENTITIES

By default, entities might be publicly available on their own unique URL.

Example: a “team member” module could generate a public node for each team member, while they are really only used for a “team overview

page” and the site doesn’t need detail pages for each team member.

https://mysite.com/node/42, https://mysite.com/taxonomy/term/42, ...

The case


PUBLIC ENTITIES

This results in low value and “thin content” pages, indexable by Google.

You don’t want these pages on your site because they are a waste of resources (bandwidth, crawl budget, database storage, …).

The problem


PUBLIC ENTITIESPrevent those entities from being accessed by visitors

This can be done by modules such as rabbit_hole

The solution

https://www.drupal.org/project/rabbit_hole

3ALL PAGES SHOULD BE AN ENTITYINDEXED INTERNAL SEARCH

PUBLIC ENTITIES

4

2


ALL PAGES SHOULD BE AN ENTITY

Oftentimes, some pages on websites are generated based on other content. These pages are not an “editable node” in the backend.

Examples: homepage, overview pages, ...

The case



When a page isn’t editable as a node, there’s no easy way to edit the meta tags, or to configure the XML sitemap inclusion for this page, or ...

The problem


ALL PAGES SHOULD BE AN ENTITY• Use the core drupal layout builder to construct the homepage

The solution


ALL PAGES SHOULD BE AN ENTITY• Use paragraphs together with modules such as block_field or

overview_field to add blocks to the page.

The solution

https://www.drupal.org/project/block_field

https://www.drupal.org/project/overview_field

4INDEXABLE INTERNAL SEARCHINDEXED TEST ENVIRONMENTS AND PAGES


5

3


INDEXABLE INTERNAL SEARCH

By default, internal search result pages are often indexable by search engines.

The case



This results in low value and “thin content” pages, indexable by Google.

You don’t want low-value pages in the index.

Google also implicitly mentions in its quality guidelines to not let Googlebot index internal search results.

The problem

https://support.google.com/webmasters/topic/6001971?hl=en&ref_topic=6001981


INDEXABLE INTERNAL SEARCHWhen using views for your search pages

install metatag and metatag_views module

The solution

https://www.drupal.org/project/metatag


INDEXABLE INTERNAL SEARCHWhen using a node for your search pages

Install metatag and add the field to your content type

The solution


5INDEXED TEST ENVIRONMENTSAND PAGES

ASSETS BLOCKED BY ROBOTS.TXT


6

4


INDEXED TEST ENVIRONMENTS AND PAGES

Development and staging environments are often crawlable and indexable by search engines because of configuration issues.

Or because of laziness ¯\_(ツ)_/¯

Temp content is also something that often shows up in search results.

The case



Everybody probably agrees we don’t want our staging environments or paragraph testing pages to be in the Google index, or even be publicly

available, for a wide range of reasons.

The problem


INDEXED TEST ENVIRONMENTS AND PAGES The problem


INDEXED TEST ENVIRONMENTS AND PAGESTest pages on live environments

Unpublish if possible or prevent from indexing with metatag module

The solution



INDEXED TEST ENVIRONMENTS AND PAGESTest environments

Use htpassword protection for your environments

The solution

6ASSETS BLOCKED BY ROBOTS.TXTMODULE OVERLOAD


7

5



Sometimes website assets (like favicons or images) are located inside a folder that’s blocked for crawlers via the robots.txt.

The case



We want Google to understand our entire page and “see” it as a regular

website visitor would see it.

If some assets are blocked via robots.txt, Google can’t view the

page as if it was a regular website visitor.

The problem

Image: notification in Google Search Console due to blocked site assets.



Make sure your website assets (images, icons, favicon, …) are in a publicly available folder, not blocked by robots.txt.

Keep an eye on your Google Search Console property for notifications regarding blocked resources.

The solution

https://search.google.com/search-console

7MODULE OVERLOADREDIRECT-REDIRECT-REDIRECT


8

6


MODULE OVERLOAD

There are a lot of modules available for a wide range of use cases, which is mostly a good thing.

The case


MODULE OVERLOAD

Be careful not to overload your website with bulky or unnecessary modules.

Each one could require multiple resources to be downloaded and executed by the browser, which will have an impact on your website

speed.

The problem


MODULE OVERLOAD

“We encourage you to start looking at your site's speed “...” — not only to improve your ranking in search engines, but also to improve everyone's

experience on the Internet.”

Source: Google Webmaster Central Blog - “Using site speed in web search ranking” https://webmasters.googleblog.com/2010/04/using-site-speed-in-web-search-ranking.html

The problem

https://webmasters.googleblog.com/2010/04/using-site-speed-in-web-search-ranking.html


MODULE OVERLOAD

Think twice before installing a module. Do you really need it or is there an easier solution?

Check if there are unused modules on your site with something like unused_modules.

The solution

https://www.drupal.org/project/unused_modules

8REDIRECT-REDIRECT-REDIRECTSECURITY LEAKS IMPACTING SEO

MODULE OVERLOAD

9

7


REDIRECT-REDIRECT-REDIRECT

Not paying close attention to redirect setups could result in multiple redirects following each-other.

The case



Redirect chains are not search-engine-friendly.

When Googlebot visits a page which returns a redirect statuscode, it adds that page to the bottom of its “to visit” list for that website.

For small websites this is not a big problem, but for big websites this could result in slow indexation times.

The problem



You want as little redirects as possible.

The problem


REDIRECT-REDIRECT-REDIRECTDon’t just uncomment the lines in htpasswd

The solution


REDIRECT-REDIRECT-REDIRECTBut think along with your site

The solution

9SECURITY LEAKS IMPACTING SEOROBOTS.TXT DISALLOW != NOINDEX


10

8


SECURITY LEAKS IMPACTING SEO

Allowing public file-uploads could result in lower organic traffic when not setup correctly.

The case



If files can be uploaded without some form of authentication or CAPTCHA, this could result in thousands of files being uploaded by

spammers, and indexed by Google.

The problem


SECURITY LEAKS IMPACTING SEO The problem



When Google notices spam on your website, they could punish you with

a “manual action”, potentially lowering your organic search

visibility.

The problem



“Google issues a manual action against a site when a human reviewer at Google has determined that pages on the site are not compliant with

Google's webmaster quality guidelines”

“If a site has a manual action, some or all of that site will not be shown in Google search results.”

Source: https://support.google.com/webmasters/answer/9044175?hl=en

The problem

https://support.google.com/webmasters/answer/9044175?hl=en


SECURITY LEAKS IMPACTING SEOUse a module to implement reCAPTCHA, recaptcha or simple_recaptcha

The solution

https://www.drupal.org/project/recaptcha

https://www.drupal.org/project/simple_recaptcha


SECURITY LEAKS IMPACTING SEOKnow when to use private files and place them outside of Drupal’s web

root.

The solution

10ROBOTS.TXT DISALLOW != NOINDEXGOOGLE ANALYTICS HORROR


11

9


ROBOTS.TXT DISALLOW != NOINDEX

Contrary to popular belief, blocking a page (or folder) via robots.txt or adding a noindex directive to the meta robots tag are not the same thing.

Robots.txt instructions impact crawling, not indexing.Noindex directives using a meta tag impact indexing, not crawling.



It sounds weird, but Google is able to stumble upon a link on an external website linking to a page blocked by robots.txt, and still index it.

The result will most likely be a snippet in the search results without a title or description, since Google can’t read the title or meta-description.



“Now wait a second Wouter. I understand it could happen in theory a link is indexed even though it’s blocked by robots.txt, but do things like this

really happen in practice?”


ROBOTS.TXT DISALLOW != NOINDEXWhy yes, yes they do:


ROBOTS.TXT DISALLOW != NOINDEXA weird “/home//home” page is indexed on drupal.org.

Let’s take a look at the robots.txt: https://www.drupal.org/robots.txt …

Bingo:

They noticed “strange homepage URLs” and tried removingthis page from the Google index by blocking Google fromcrawling the page.

https://www.drupal.org/robots.txt

11GOOGLE ANALYTICS HORRORRAPID-FIRE BEST PRACTICES


12

10


GOOGLE ANALYTICS HORROR

Correct data is very important when analysing SEO-efforts

Pay close attention to sudden drops and spikes in Google Analytics data.There might be a configuration issue.


GOOGLE ANALYTICS HORROR The case


GOOGLE ANALYTICS HORROR

https://www.drupal.org/project/eu_cookie_compliance updated

Each pageview started a new session as long as website visitors didn’t accept the cookies

The problem

https://www.drupal.org/project/eu_cookie_compliance


GOOGLE ANALYTICS HORRORTwo-step fix:

The solution

1. Anonymize the visitor IP-addresses (GA or GTM setup)

2. Whitelist the Google Analytics cookies (Drupal module setup)


GOOGLE ANALYTICS HORROR The solution

Update/patch the google analytics module to comply with the eu_cookie_compliance so that the snippet isn't loaded until approval was

given.

More information: https://www.drupal.org/project/google_analytics/issues/3060312

https://www.drupal.org/project/google_analytics/issues/3060312

12RAPID-FIRE BEST PRACTICES

GOOGLE ANALYTICS HORROR 11


RAPID-FIRE BEST PRACTICES

Use the Google Analytics module (google_analytics) or the Google Tag Manager module (google_tag).

Not both.

https://www.drupal.org/project/google_analytics

https://www.drupal.org/project/google_tag



Aggregate and minify css and js files where possible.

https://www.drupal.org/project/advagg

https://www.drupal.org/project/advagg



Make sure each page has a correct/well-configured canonical tag.

Not enough time to go into detail because of the complexity (what about paging parameters? facet filtering parameters? combinations with hreflang? ...)

come find us after the talk or search the interwebs for resources :)



Use Pathauto module to generate nice urls for your pages

https://www.drupal.org/project/pathauto



Follow-up on the amount of pages indexed by Google via Google Search Console.

Seems too high?

Maybe some rabbithole setup is needed to remove separate pages

from the index.

Seems too low?

Maybe important pages are no-indexed?



Make sure your base url is not ‘http://default/’ in your sitemap XML, we’ve seen this issue occur a lot in live sites.

Setup your cron job correctly, more info:https://www.drupal.org/project/xmlsitemap/issues/1944518

https://www.drupal.org/project/xmlsitemap/issues/1944518



Create checklists!

• Things that have to be done before going live• Things to check when site is live

• Things to check when site is live for x days

Image: WARNER BROS. ENTERTAINMENT INC.


QUESTIONSTime left?

Fire away!

No more time?

Come find us at the Dropsolid booth (#13)

Get in touch:https://www.linkedin.com/in/brentgees/

https://www.linkedin.com/in/wouterdebruycker/

+ Keep an eye on the Dropsolid Twitter feed for these slides!

https://www.linkedin.com/in/brentgees/

https://www.linkedin.com/in/wouterdebruycker/

https://twitter.com/dropsolid

Drupal SEO Pitfalls and How To Avoid Them - DrupalCon Amsterdam 2019

TODO

ABOUT US

70

Join us for contribution opportunitiesThursday, October 31, 2019

Mentored Contribution

9:00-18:00Room: Europe Foyer 2

First TimeContributor Workshop

9:00-14:00Room: Diamond

Lounge

GeneralContribution

9:00-18:00Room: Europe Foyer 2

#DrupalContributions

Drupal SEO Pitfalls and How To Avoid Them - DrupalCon Amsterdam 2019

TODO

ABOUT US

71

What did you think?

Locate this session at the DrupalCon Seattle website:https://drupal.kuoni-congress.info/2019/program/

Take the Survey!https://www.surveymonkey.com/r/DrupalConAmsterdam

Thank You!

https://drupal.kuoni-congress.info/2019/program/

https://www.surveymonkey.com/r/DrupalConAmsterdam

drupal seo pitfalls how to avoid them

Documents