[logo] a small computer

Adding A Sitemap To Your React App

adding a sitemap to your react app

Bradley Kofi
Published a year ago.
12 minute read
logo of undefined

Adding A Sitemap To Your React App

Having a sitemap is one of the most repeated pieces of advice in the SEO community. A sitemap is a structured file that has information about pages and other files on your site and the relationships between them. It's presence on makes it much easier for search engines to crawl your site.

Think of a sitemap as a blueprint to a house or a treasure map to treasure that would otherwise take a lot of random searching to find. Aside from telling Google (and other search engines) exactly where to find what, it also provides additional information such as when the page was last updated and how often it’s changed. If your site is available in multiple languages, it can also help point Google to the right direction.

The immediate problem you’re going to face is that manually updating a sitemap doesn’t scale very well. Once your site gets large, adding new routes is going to be a chore. Ironically enough, the larger your site is, the more urgent the need for a sitemap.

Installing Dependencies

If you come from the WordPress world, adding a sitemap to your blog might have been as simple as installing a plugin. With more modern technologies like React, you have to get your hands dirty.

In our case, we’ll make use of a few dependencies, but this tutorial assumes you used create-react-app to start your project. However, it should work fine in either case.

  • react-router-sitemap
  • react-router-dom
  • babel-preset-es2015
  • babel-preset-react
  • babel-register

react-router-dom provides routing functionality to our app and react-router-sitemap has several convenience methods to help us save some time.

To install them run,

npm install react-router-dom react-router-sitemap babel-register babel-preset-react babel-preset-es2015 --save

//or

yarn add react-router-dom react-router-sitemap babel-register babel-preset-react babel-preset-es2015

Creating The Project

If you already have a project up and running, feel free to skip to the final section of this tutorial. For this project, we’ll use six simple pages that look like this:

import React from 'react';

function PageOne() {
  return (
    <div>
      <p>Page 1</p>
    </div>
  );
}

export default PageOne;
//And so on..

And our routes.js file should be structured like so:

import React from 'react';
import {BrowserRouter, Route, Switch} from 'react-router-dom';
import PageOne from "./pages/page-1";
import PageTwo from "./pages/page-2";
import PageThree from "./pages/page-3";
import PageFour from "./pages/page-4";
import PageFive from "./pages/page-5";
import PageSix from "./pages/page-6";
import App from "./App";
function Routes() {
  return (
    <BrowserRouter>
      <Switch>
        <Route exact path='/' component={App}/>
        <Route exact path='/page-1' component={PageOne}/>
        <Route exact path='/page-2' component={PageTwo}/>
        <Route exact path='/page-3' component={PageThree}/>
        <Route exact path='/page-4' component={PageFour}/>
        <Route exact path='/page-5' component={PageFive}/>
        <Route exact path='/page-6' component={PageSix}/>
      </Switch>
    </BrowserRouter>
  );
}
export default Routes;

Using react-router-dom itself is outside the scope of this project, but most of the code should be easy to follow.

We have created a Routes component that returns BrowserRouter - a react component that allows us to use the Route, Switch and Link components. A quick rundown is as follows:

  • The Route component allows us to render a UI when the path provided matches the current URL.
  • The Switch component renders the first Route or Redirect that matches the provided location. This is distinctly different from using a bunch of Routes without it because it renders a route exclusively, rather than inclusively. That is, it allows you to render exact paths.
  • The Link component provides accessible navigation to your app.

Generating the sitemap

All that’s left now is to plug in react-router-sitemap to our application and we should be good to go.

//Babel allows us to convert modern js code into backwards compatible versions
//This includes converting jsx into browser-readable code

const es2015 = require('babel-preset-es2015');
const presetReact = require('babel-preset-react');
require("babel-register")({
  presets: [es2015, presetReact]
});
//Import our routes
const router = require("./routes").default;
const Sitemap = require("react-router-sitemap").default;

function generateSitemap() {
  return (
  new Sitemap(router())
  .build("https://www.example.com")
 //Save it wherever you want
  .save("../public/sitemap.xml")
  );
}

generateSitemap();

Now, generating a sitemap should be as simple as running

node sitemap-generator.js

Which produces the following code:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>https://www.example.com/</loc>
    </url>
    <url>
        <loc>https://www.example.com/</loc>
    </url>
    <url>
        <loc>https://www.example.com/</loc>
    </url>
    <url>
        <loc>https://www.example.com/page-1</loc>
    </url>
    <url>
        <loc>https://www.example.com/page-2</loc>
    </url>
    <url>
        <loc>https://www.example.com/page-3</loc>
    </url>
    <url>
        <loc>https://www.example.com/page-4</loc>
    </url>
    <url>
        <loc>https://www.example.com/page-5</loc>
    </url>
    <url>
        <loc>https://www.example.com/page-6</loc>
    </url>
</urlset>

If your site is fairly small, that should be enough. We can make it so that you don’t have to bother with manually running that script every time by integrating it in our build environment.

Automating creation of the sitemap

All we need to do is edit our package.json so the sitemap is generated before every build and saved to the public directory. Of course, you won’t be adding new routes with every build, but it’s such little overhead, you’ll barely notice.

{
    //...
    "scripts": {
        //...
      "generate-sitemap": "node src/sitemap-generator.js",
       "prebuild": "npm run generate-sitemap",
       "build": "react-scripts build",
        //...
    },
    // ...
}

Adding dynamic routes

One of the pieces of functionality that react-router allows is for the addition of dynamic routes. This makes it considerably easier to reuse components.

Consider the following jsx code

//imports excluded for brevity

function Routes() {
  return (
      <BrowserRouter>
          <Switch> 
              <Route exact path='/' component={App}/>
              <Route exact path='/page-1' component={PageOne}/>
              <Route exact path='/page-2' component={PageTwo}/>
              <Route exact path='/page-3' component={PageThree}/>
              <Route exact path='/page-4' component={PageFour}/>
              <Route exact path='/page-5' component={PageFive}/>    
              <Route exact path='/page-6' component={PageSix}/>
              <Route path={/product/:slug’} component={Product}/>  
          </Switch>
      </BrowserRouter>
  );
}
export default Routes;

It’s exactly the same as last time with the exception of one additional route:

 <Route path={'/product/:slug'} component={Slug}/>

This allows us to dynamically load different 'products' using the same logic.

To generate a sitemap that contains all the products in our database, we could either:

  • Crawl your site once all the necessary routes have been generated and automate uploading of the xml file to your server. There are several services online that can do this for you, but hardly any are free.
  • Provide react-router-sitemap with a list of urls to append to the “:slug” parameter so that it completes the routes on its own.

We’re going to cover the second method.

It’s worth noting that properly structuring files is one of the good development practices you should pick up along the way. This might be useful for a blog that converts all markdown files stored in a certain directory into HTML the same way Gatsby does.

For example, this example, we'll just have a list of names stored in a products.json file, which looks like this:

{
 "products": [
 	"sad-china",
 	"fine-china",
 	"blue-plate",
 	"red-plate"
 ]
}

Of course, if you have a list of products stored in the database and want Google to index all of them, retrieving them via an API will achieve the same effect.

The products.js file is just as simplistic:

function Products(props) {
  return (
    <div>
      {
        productObj.products.find(element => element.toLowerCase() === props.match.params.slug.toLowerCase()) ?
          <div>
            {props.match.params.id}!
          </div> :
          <div>
            Could not find a product with slug: {props.match.params.slug}
          </div>
      }
    </div>
  );
}

export default Products;

All it does is look for a match from our array of products using the Array.find function. If it finds a match, it renders Welcome #id. If not, the error message is rendered instead.

To plug this into react-router-sitemap, we use the applyParams method as follows:

//...
function generateSitemap() {
    const pathsConfig = {
        '/product/:id': [
            {
                id: productObj.products
            }
        ]
    };
    return (
        new Sitemap(router())
            .applyParams(pathsConfig)
            .build("https://www.example.com")
            .save("./public/sitemap.xml")
    );
}
//...

Adding a sitemap to your site without react-router-sitemap

The react-router-sitemap library is pretty handy if you depend on react-router on your site, have lots of routes and want to make minimal changes to your code. However, if you you're willing to put in a bit more work, you'll end up reducing the complexity of your code and eliminate another extra library.

As it turns out, the sitemap protocol is short and simple. Your sitemap file can either be a simple text file with a single url on each line, or an xml file like you're probably used to. In the latter case, the xml file must:

  • Begin with an opening <urlset> tag and end with a closing </urlset> tag.
  • Specify the namespace (protocol standard) within the <urlset> tag.
  • Include a <url> entry for each URL, as a parent XML tag.
  • Include a <loc> child entry for each <url> parent tag.

Additionally, every url should start with a valid protocol and end with a trailing slash, if your server requires it.

In other words, it should look something like this:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
    <url>
        <loc>http://www.example.com/</loc>
        <lastmod>2021-02-04</lastmod>
        <changefreq>monthly</changefreq>
        <priority>0.8</priority>
    </url>
</urlset> 

Or be a simple txt file like:

http://example.com

lastmod, changefreq and priority are optional. The last two attributes are ignored entirely by Google.

First, we need a simple way to import all our routes. Instead of specifying them directly inside our routes.js file, you can create a separate routes-list.js for the purpose, or use react-router-to-array to convert your routes to an array if you'd rather not rewrite them.

module.exports.regularRoutes = [
    {
      path: '/',
      exact: true,
      component: App
    },
    {
      path: "/page-1",
      exact: true,
      component: PageOne
    },
    {
      path: "/page-2",
      exact: true,
      component: PageTwo
    },
    {
      path: "/page-3",
      exact: true,
      component: PageThree
    },
    {
      path: "/page-4",
      exact: true,
      component: PageFour
    },
    {
      path: "/page-5",
      exact: true,
      component: PageFive
    },
    {
      path: "/page-6",
      exact: true,
      component: Page6
    },
]

And then change our routes.js file to:

import {regularRoutes} from './routes-list'
function Routes() {
  return (
    <BrowserRouter>
      <Switch>
        {
          regularRoutes.map(route=> (
            <Route exact={route.exact} component={route.component} path={route.path}/>
          ))
        }
      </Switch>
    </BrowserRouter>
  );
}

Now that we have a central point for importing all our routes, generating a text file sitemap should be easy:

const fs = require('fs');
const {regularRoutes} = require('./routes-list')

function generateSitemap(baseUrl){

  fs.writeFileSync('./sitemap.txt', regularRoutes.map(item => baseUrl + item.path).join('\n'));
}

generateSitemap('https://example.com');
  

To generate an xml sitemap, we will need to install a new dependency:

npm install xml2json -S

Now, say we had a sitemap that looks like this

<?xml version="1.0" encoding="UTF-8"?>   
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>   
    <loc>http://www.example.com/</loc>
    <lastmod>2020-01-01</lastmod>
    <changefreq>monthly</changefreq>
    <priority>0.8</priority>
  </url>
  <url>  
    <loc>http://www.example.com/page-1</loc>
    <lastmod>2020-01-01</lastmod>    
    <changefreq>monthly</changefreq>
    <priority>0.8</priority>
  </url>    
</urlset>

When parsed by xml2json, the resulting file would be shaped like:

{
  "urlset": {
    "xmlns": "http://www.sitemaps.org/schemas/sitemap/0.9",
    "url": [
      {
        "loc": {
          "$t": "http://www.example.com/"
        },
        "lastmod": {
          "$t": "2020-01-01"
        },
        "changefreq": {
          "$t": "monthly"
        },
        "priority": {
          "$t": "0.8"
        }
      },
      {
        "loc": {
          "$t": "http://www.example.com/page-1"
        },
        "lastmod": {
          "$t": "2020-01-01"
        },
        "changefreq": {
          "$t": "monthly"
        },
        "priority": {
          "$t": "0.8"
        }
      }
    ]
  }
}

Which means that we can edit the parsed JSON, convert it back to an xml file and write it out as a sitemap.xml file.

const fs = require('fs');
const {regularRoutes} = require('./routes-list')
const parser = require('xml2json');

function generateSitemap(allRoutes, baseUrl, type,){

  function mapRoutes(existingRoutes){

    // concatenate all the routes together
    const concatedRoutes = allRoutes.concat(existingRoutes);

    return concatedRoutes
      // remove any duplicate paths
      .filter((route, index, self) =>
        index === self.findIndex(t => t.path === route.path),
      ).map(route => {
        // and return the structure that we want
          if (route.path) {
            return {
              loc: {
                $t: route.path,
              },
              changefreq: {
                $t: 'monthly',
              },
            };
          }
        },
        // and, finally, remove any items that are undefined
      ).filter(item => item !== undefined);
  }

  if (type === 'txt')
    fs.writeFileSync('../public/sitemap.txt', allRoutes.map(item => baseUrl + item.path).join('\n'));
  else if (type === 'xml') {
    //read the xml file
    fs.readFile('../public/sitemap.xml', {}, (err, data) => {
      let sitemapData = data;
      // if the file doesn't exist
      if (err && err.code === 'ENOENT') {
        // create it
        fs.writeFileSync(
          '../public/sitemap.xml',
          '<?xml version="1.0" encoding="UTF-8"?>',
          { encoding: 'utf-8' },
        );
        // and re-read it
        sitemapData = fs.readFileSync('../public/sitemap.xml');
      }

      // we can be sure the file exists at this point
      const sitemapJSON = parser.toJson(sitemapData, { reversible: true });
      const sitemapObject = JSON.parse(sitemapJSON);

      sitemapObject['urlset'] = {
        xmlns: 'http://www.sitemaps.org/schemas/sitemap/0.9',
        url: mapRoutes(sitemapObject.urlset.url),
      };

      const sitemapXML = parser.toXml(JSON.stringify(sitemapObject));
      fs.writeFileSync('../public/sitemap.xml', `<?xml version="1.0" encoding="UTF-8"?>${sitemapXML}`);
    });
  } else {
    console.log("Doing nothing. type: ", type)
  }
}

generateSitemap(regularRoutes, 'https://example.com', 'xml');

Since there's no obvious way of knowing when a file was last modified, I've ignored that field. You can use statSync for this but it would be too convoluted for my use-case.

This article doesn't cover how to cover params (yet) but I'll probably update it some time in the future,

Conclusion

This article has covered two simple ways of generating a sitemap for your website. The first allows you to plug your routes from react-router into react-router-sitemap, but you then have to deal with extra libraries like Babel. The second goes into how to do it from scratch.

Regardless of what you prefer, a sitemap remains one of the simplest and most important ways of boosting your website SEO.

Further reading

Copyright © 2021 The Kenyan Dev