<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Derek Hawkins on Medium]]></title>
        <description><![CDATA[Stories by Derek Hawkins on Medium]]></description>
        <link>https://medium.com/@derek-m-hawk?source=rss-3e84d2639e65------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*DiN4w_FwVlQIkIMI7bOC4A@2x.jpeg</url>
            <title>Stories by Derek Hawkins on Medium</title>
            <link>https://medium.com/@derek-m-hawk?source=rss-3e84d2639e65------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 04:11:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@derek-m-hawk/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Automating Article Schema Implementation in Hubspot]]></title>
            <link>https://derek-m-hawk.medium.com/automating-article-schema-implementation-in-hubspot-f600933d1e71?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/f600933d1e71</guid>
            <category><![CDATA[marketing-automation]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[schema-markup]]></category>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[hubspot]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Wed, 23 Feb 2022 14:12:59 GMT</pubDate>
            <atom:updated>2022-04-11T20:02:03.345Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Hubspot logo on a piece of card stock" src="https://cdn-images-1.medium.com/max/1024/1*I6bEriE6crdX-6PggM52dw.jpeg" /></figure><p>I have to admit, having made the switch from agency to in-house SEO has come with a slew of changes both in working style and mindset. Working in a tech startup with limited design/development resources and an emphasis on making sure my time is spent on high-impact activities, typical SEO recommendations and activities find themselves further down the backlog. In the continual hunt for not just implementing quick win optimizations, but scalable systems for doing so, one such activity was bulk Schema building/implementation, a not-so-simple feat in Hubspot.</p><p><strong>Update (4/11/22):</strong> I presented this business case with the Hubspot Developer Advocacy team. You can see the broad overview of this article in the video below:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F9PDFNfVuN1I%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D9PDFNfVuN1I&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F9PDFNfVuN1I%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/77a0d4f7edcbfd2c82b244a1df04589f/href">https://medium.com/media/77a0d4f7edcbfd2c82b244a1df04589f/href</a></iframe><h3>Why Not Just Use a Plug-in?</h3><p>Funny enough, that is exactly what I wanted to do from the start! What I had mainly found in looking for the pathway of least resistance is one of the following.</p><ul><li>Plug-ins that have unnecessary subscription costs that simultaneously have little to no (human) support or documentation</li><li>3rd party applications that I would have to loop in multiple players in building the integration. ROI of taking marketing ops and dev time on schema is an all-uphill battle.</li><li>Universally no support for non-standard Schema types (A nice to have but not a need to have).</li></ul><p>Ultimately, what I find going with the plug-in route is creating long-term dependencies in tools that you don’t have complete control over. For small organizations (sub 50 person companies who only have 1 or 2 people working on a website), I would still recommend using a plug-in (provided you want to pay) for ease of access. But, for SEOs and website managers who need a simple way of dynamic generation and injection of Article schema (along with a framework for additional automation of schema building/injection) in a Hubspot environment, this is the solution for you.</p><h3>Getting Started</h3><p>To start, you will need the following libraries in Python:</p><ul><li><a href="https://github.com/HubSpot/hubspot-api-python">Hubspot</a>: The official Hubspot Python API SDK</li><li><a href="https://pypi.org/project/genson/">Genson</a>: a JSON schema building library</li><li>A few core essentials that usually come packaged with any IDE in Python (Pandas, JSON, Requests, time)</li></ul><h3>Pulling the Right Data</h3><p>There are a few different options for pulling out data, either directly from Hubspot or via API call. In our case, we will be pulling all of the information we need from the Hubspot API, including all public blog posts, their correlating IDs, and the information we need to build out the Article schema.</p><p>If you have never used the Hubspot API, you can get your <a href="https://knowledge.hubspot.com/integrations/how-do-i-get-my-hubspot-api-key">API key by following these directions</a> and brush up on the basics <a href="https://developers.hubspot.com/docs/api/developer-guides-resources">in their documentation</a>.</p><p>Rather than using the Hubspot API SDK for this call, we will call the Blog Post API endpoint using requests for ease of access. Because of the 100 post limit, the Hubspot Blog Post API has, we will loop over iterations of 100 until we collect every blog post on our site. In this instance, we are working with a site with approx.1000 blog posts.</p><p>As we go through each iteration, we will be parsing the JSON response for relevant information for our schema, including the article title, author, description, publish date, modification date, and featured image. We will store our elements in a dictionary, pass that to a list and build a DataFrame. At the end of the loop, we will concat all our articles into a single DataFrame.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/03320bb3aab75e434c97b563d1eda059/href">https://medium.com/media/03320bb3aab75e434c97b563d1eda059/href</a></iframe><h3><strong>Building the Article Schema.</strong></h3><p>I’ve written about a <a href="https://searchwilderness.com/automating-video-object-schema-with-python">Pythonic framework to build schema</a> in the past, which we will be leveraging here as well. The function below uses similar elements to a function I’ve built in the past. While we will pull most of what we need via the Hubspot API, you will still want to manually incorporate the additional elements you’ll need for the Article schema to be valid.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/68b4be1d05bce0eb652f587b8deb3a95/href">https://medium.com/media/68b4be1d05bce0eb652f587b8deb3a95/href</a></iframe><p>With the data all in one place and the tools to build the schema outlined, we can iterate over our DataFrame and build the schema in a new column. We will use an f-string to make sure we have the JSON-LD &lt;script&gt; tag in place prior to adding to the post.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f9973797afd572fd0e7b43c2e5e1f1b8/href">https://medium.com/media/f9973797afd572fd0e7b43c2e5e1f1b8/href</a></iframe><h3><strong>Implementation</strong></h3><p>Now that we have the schema built out, we can inject the schema directly into the html head of our blog post. Using the Hubspot API wrapper, we can simplify the authentication necessary to do the POST request. We’ll take the id we pulled from our initial Hubspot API call and our newly built schema and iterate over the entire DataFrame. I have added a sleep timer to ensure that the loop doesn’t timeout due to Hubspot API limitations.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a169e76fd22f61484ae8b9a83dcdf0cd/href">https://medium.com/media/a169e76fd22f61484ae8b9a83dcdf0cd/href</a></iframe><p>With that, you’ll be all set! The Article schema will be live on-site and ready to validate using your preferred toolset. The implementation example above also has additional uses for site owners looking to bulk upload recommendations given by internal or external SEO resources.</p><p>Overall, in a time-sensitive position in which you want to ensure quick wins are quickly implemented but you find yourself resource-strapped and looking for an immediate way to act, this process is suited for you!</p><p>Feel free to reach out to me directly on <a href="https://www.linkedin.com/in/derek-hawkins-seo/">LinkedIn</a> or <a href="https://twitter.com/Derek_M_Hawk">Twitter</a> with any questions on this script or process!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f600933d1e71" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[(Almost) End to End Log File Analysis with Python]]></title>
            <link>https://derek-m-hawk.medium.com/almost-end-to-end-log-file-analysis-with-python-59d4cb30f930?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/59d4cb30f930</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[technical-seo]]></category>
            <category><![CDATA[automation]]></category>
            <category><![CDATA[seo-tips]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Wed, 28 Jul 2021 19:37:14 GMT</pubDate>
            <atom:updated>2021-07-28T19:37:14.634Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*yjmHvhdTRCLxYt478QzpIQ.jpeg" /></figure><p>Often seen as a nuanced exercise and a buzzword to sound extra technical, log file analysis serves to bring data into a conversation of assumptions. Trying to understand bot crawl behavior through third party tooling such as Google Search Console and Bing Webmaster Tools alone only gives you part of the story. In instances where you want to conduct SEO experimentation with internal linking or you want to see crawl behavior changes with a site migration, you will need the level of granularity log files provide. If you are looking to scale up your understanding of crawl behavior, whatever the reason, you will want to make sure you have the tools at your disposal to do so.</p><p>* Note that this is not an article on the value of log file analysis (of where there are many floating around), but if you are looking for a place to start I would recommend the following:</p><ul><li><a href="https://moz.com/blog/log-file-analysis">Moz’s White Board Friday on Log File Analysis</a></li><li><a href="https://jetoctopus.com/the-ultimate-log-file-analysis-checklist/">Jet Octopus’ Log File Analysis Checklist</a></li><li><a href="https://www.botify.com/learn/basics/log-file-analysis">Botify’s Log File Analysis 101</a></li></ul><p><strong>Why Shouldn’t I Just Use SEMRush/Screaming Frog/JetOctopus/Botify and Excel?</strong></p><p>Jean said it best on his comment from the White Board Friday</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/782/0*BwE0MmJJ9V-rGvrf" /></figure><p>Often, especially from an enterprise standpoint, trying to leverage third party tools for log file parsing and data analysis falls into the trap of scale. Working with any brand who wants to do log file analysis at scale will be immediately stopped the second they want to look at more than a day’s worth of data. For one time, ad-hoc analysis using a Screaming Frog or SEMRush for parsing and Excel for analysis works fine. If you are a brand leveraging Botify for log file analysis, you are most likely in the camp of having a well off budget to bring in both the tool as well as their internal team to assist. Going through the exercise of navigating UI’s, drag and dropping files and waiting for the tool to load results to then have to manually export also adds to the overall time suck of the process, and that’s without having Excel ultimately crash when you try and open the file. If you are looking to do work on an enterprise tool in a custom solution without having to spend incrementally to do so, utilizing Python is probably the best way to go.</p><p><strong>Extracting the Log File Itself Within Python</strong></p><p>Not the most exciting part, but I found when requesting log files across a larger timeframe, I would end up with files types beyond .log. For work in the past, I have seen files come back as both .log and .gz. Accessing .log files can be done with a simple <em>with open()</em> statement. For a .gz file, you can use the library gzip and use a <em>with gzip.open(‘file.gz’) </em>statement, similar to a .log file.</p><p><strong>Parsing Separate Logs from a Log File</strong></p><p>Looking at a log file, you would expect that a simple <em>.split( )</em> would allow for a clean, parseable file, but in practicality, log files do not always maintain the same consistency every time (even though the format stays the same, sometimes the server logs actions which omit a specific element of the file itself). Instances, where certain aspects of the log are omitted, can through a <em>.split( )</em> approach completely off. Instead, I have relied on using a series of regexes to isolate elements necessary to analyze log files from an SEO standpoint. These have all been consolidated into the function<em> log_parse()</em>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/eec756b4a965f88bb8a4fb5faef1bb43/href">https://medium.com/media/eec756b4a965f88bb8a4fb5faef1bb43/href</a></iframe><p><strong>Running The Script and Data Storage</strong></p><p>Once you have your log files stored in an accessible area, you can run the function against your log file in a loop (in the instance below I’m using a nested loop to parse multiple log files in one run).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c78eccabd767813f509daae8556c0c1b/href">https://medium.com/media/c78eccabd767813f509daae8556c0c1b/href</a></iframe><p>After parsing your file, there are multiple approaches you can take to store your data based on the scale and needs of your site. If you are planning on leveraging this approach in the short term or for a smaller site, you may want to store locally in a pickle. For building processes on a larger scale, you should rely on migrating data into a SQL server, a BigQuery instance or your preferred data warehousing solution.</p><p><strong>Googlebot Verification</strong></p><p>When going through your log files, you will want to make sure that you are only pulling out valid crawl data from Googlebot/Bingbot/your bot of choice. One approach is to validate via reverse DNS lookup directly in Python within your DataFrame. To do this, you can run a lambda function against the IP address in your DataFrame.</p><p>Alternatively, you can use <a href="https://github.com/moskrc/CrawlerDetect">CrawlerDetect</a>, a third-party library for validating crawlers.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8f52e9380b26a41598e0bcb33fce76f0/href">https://medium.com/media/8f52e9380b26a41598e0bcb33fce76f0/href</a></iframe><p><strong>Pivoting &amp; Plotting</strong></p><p>Once you have your data in a manageable form, then it’s time to analyze. You can take your newly filtered DataFrame and go through an series of pivots.</p><p><strong>Distribution by Status Code</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/88189201c2a7ad9856b5950a9881c78a/href">https://medium.com/media/88189201c2a7ad9856b5950a9881c78a/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*T4Fzo92wRb12DhRRBi2-oA.png" /></figure><p><strong>Top Files Crawled</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/93c6528adc8ac57614d9fe5a6e5e6894/href">https://medium.com/media/93c6528adc8ac57614d9fe5a6e5e6894/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/824/1*UAWWIki02M8gdZ_gM8Ag6Q.png" /></figure><p><strong>Crawls by Date</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b671c1ec664996ba0e13073d4f7ea6a5/href">https://medium.com/media/b671c1ec664996ba0e13073d4f7ea6a5/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/936/1*UtQ2JRUblVpfDJ018heZnw.png" /></figure><p><strong>Conclusion</strong></p><p>Logfile analysis often can be a separator between theory and reality how Google and other search engines understand your sites. Taking the time to scale these types of analyses to understand the correlative effects of SEO changes/experiments on search engine behavior can quickly give your team the upper hand with all technical SEO programs moving forward. Leveraging Python for log file analysis allows for the most seamless approach to gain quick, continuous insight into your SEO initiatives without having to rely on manual tool configuration and big data issues which most tools cannot handle for.</p><p>Want to talk more about log files? Trying to figure out specific problems in your own implementation of Python-based log file analysis? Feel free to reach out to me directly via my <a href="https://www.linkedin.com/in/derek-hawkins-seo/">LinkedIn</a> or <a href="https://twitter.com/Derek_M_Hawk">Twitter</a>!</p><p><strong>Credit is Where Credit is Due</strong></p><p>There are plenty of other Python practitioners who had tackled this same problem with their unique approaches. Here are some of the resources I consulted as I approached building out my log file analysis systems that I would recommend you read if you are still interested in learning more!</p><ul><li><a href="https://www.suganthan.com/blog/logfile-analysis-seo/">Log File Analysis with Cloudflare and Google Data Studio</a> by Suganthan Mohanadasan</li><li><a href="https://searchengineland.com/leverage-python-and-google-cloud-to-extract-meaningful-seo-insights-from-server-log-data-329199">Leveraging Python and Google Cloud to Extract Meaningful SEO insights from server log data</a> by Charly Warginer</li><li><a href="https://www.seo-garden.com/post/how-i-analyzed-millions-of-log-files-with-python">How I Analyzed Millions of Log Files with Python</a> by SEO Garden</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59d4cb30f930" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging Semantic Keyword Research to Expedite your SEO]]></title>
            <link>https://derek-m-hawk.medium.com/leveraging-semantic-keyword-research-to-expedite-your-seo-2b0a2f4861e3?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b0a2f4861e3</guid>
            <category><![CDATA[search-engine-optimizati]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[keyword-research]]></category>
            <category><![CDATA[seo]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Wed, 03 Feb 2021 14:34:20 GMT</pubDate>
            <atom:updated>2021-03-04T06:45:29.087Z</atom:updated>
            <content:encoded><![CDATA[<h3>Leveraging Semantic Keyword Research to Expedite Your SEO</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XPnZIUjfg72y_HMx_WTI7Q.jpeg" /></figure><p>It’s not often that you find a page that holds a top position for a highly coveted keyword that doesn’t simultaneously hold various top positions for similar ranking keywords. As SEOs, we look to create content targeting a specific query that also attracts similar terms to that core keyword. To find these correlating keywords, we are often left to scrape the “People also search for” section of the SERP or leverage third-party platform’s keyword research tools. But, these approaches are limited in scope, lack scalability, and can leave us wanting much more from this data. Instead, we can leverage Python to give us a more streamlined solution.</p><p>The idea for this script came up in a conversation between two former colleagues of mine that I happened to eavesdrop my way into. In discussing searches around “ADHD quizzes,” we were aiming to interpret the user’s true intent of the search and find out other ways users were looking to understand ADHD symptoms. A user can search for “ADHD quiz” with the intent to see if they have ADHD. Then again, searching for “ADHD quiz” can be seen as a search around your knowledge of ADHD, in the event you are looking to diagnose someone else or if you are trying to become more informed about ADHD without having it yourself. How do we create distinctions in queries to understand their relevance, and how can we quantify these learnings to expedite both keyword research and keyword insights gathering? I took that conversation more as a challenge and took my shot at it.</p><p>For that, we will aim to emulate Google’s interpretations of a keyword and how they connect multiple queries to a single result. By inputting a core term, we will gather top results, we will collect highly ranked keywords associated with the SERP listings, look for top listing results associated with THOSE results and then quantify relevancy against the original first input. In other words, let’s create a spider web of the primary term to other associated terms to see what other relevant queries Google associates with our search.</p><h4>Getting Started</h4><p>To do this, we will be using the SEMRush API (of which I am <a href="https://medium.com/@Derek_M_Hawk/want-a-ton-of-seo-data-consider-the-semrush-api-in-your-workflow-5fe1fc7fc49e">a huge, unpaid advocate</a>). Note, that with this script, the API credit usage can vary based on the original input keyword used. Safeguards have been placed within functions to limit credit usage, but I have seen this script consume many (MANY) API credits for more intensive projects.</p><p>We will begin with importing the necessary libraries and functions.</p><ul><li>build_seo_urls(): allows us to ping the SEMRush API and return the top 10 results for a keyword.</li><li>parse_response(): parser for extracting data from SEMRush API call.</li><li>url_org(): pings the SEMRush API to give us every keyword in positions 1–10 for a given URL.</li><li>secondary_later(): runs through our the the process of collecting keywords across our URL set found via url_org().</li><li>third_layer_setup(): takes the second stage and compares it to the core results, calculating relevancy of semantically similar keywords through an occurrence score.</li></ul><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/441772113f9855ba1997166d4d5365e0/href">https://medium.com/media/441772113f9855ba1997166d4d5365e0/href</a></iframe><h4>Running the Script</h4><p>You will want to add your SEMRush API key to the<em> api_key </em>variable and set the “database” key in <em>build_seo_urls</em> to whatever market you are looking to search within.</p><p>The core of the script is around 15 lines where each function in a spider-like fashion will execute. When you run the script, enter in your starting keyword via the input to begin. Once the script runs, it will tell you how many semantically similar keywords it could find and export the file into a CSV with correlating search volume and an occurrence score of 1 to 10. Your starting query will always score 10 (10 out of 10 URLs rank for your original input because…you know…that’s where we started). From there, it will give you a score using that same notion.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/22c72d03e712f7f3899917624678407d/href">https://medium.com/media/22c72d03e712f7f3899917624678407d/href</a></iframe><p>The rule of thumb I have leveraged with this script is any keyword with an occurrence score above 7 is highly relevant to your source. 6 through 3 provide great added content opportunities or new areas of exploration. 2 through 1 give you a good sense for what other relevant searches are in the overall space of your original query.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZiCJgshsTw03PaEEvmNKGw.png" /></figure><h4>Conclusion</h4><p>There are many use cases for this script that go beyond traditional SEO research. From expedited keyword exploration (as often needed for new business initiatives, enterprise-scale keyword research and researching queries that are brand new to you), content ideation, gauging consumer sentiment around a particular subset of searches and data gathering for training clustering models, the world is your oyster! For the full script, check it out via <a href="https://github.com/DerekHawkins/semantic_keyword_research_tool/tree/main">my GitHub</a>!</p><p>Thinking about more automated ways of keyword research? Check out my article on automating keyword research via APIs, or reach me via my <a href="https://www.linkedin.com/in/derek-hawkins-seo/">LinkedIn</a> or <a href="https://twitter.com/Derek_M_Hawk">Twitter</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b0a2f4861e3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My Story]]></title>
            <link>https://derek-m-hawk.medium.com/my-story-4939a3669334?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/4939a3669334</guid>
            <category><![CDATA[social-justice]]></category>
            <category><![CDATA[blacklivesmatter]]></category>
            <category><![CDATA[racial-equality]]></category>
            <category><![CDATA[racial-bias]]></category>
            <category><![CDATA[racism]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Mon, 01 Jun 2020 16:54:28 GMT</pubDate>
            <atom:updated>2020-06-01T16:54:28.030Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/744/0*64TCER1eAH3HjCRj" /></figure><p>My story is not an unfamiliar one.</p><p>My story consists of the trials and tribulations that many who look like me experienced from their first exposures to broader society. Women whispering to each other about me and my mom (who is white) walking in the store together (I was 9 at the time). Having multiple high schoolers use racial expletives at my siblings and my father (I was 12 at the time). Being told I couldn’t date someone because “my father wouldn’t like you,” (I was 13 at the time). Having competing schools during swim meets says that they could beat me in a race because “black people aren’t good swimmers (I was 15 at the time, also regularly beat them).</p><p>My story in the professional space rings the same, just in a different tone. Self-described liberal and open-minded coworkers in the past who have touched my afroed hair without my consent during work outings. Having to explain my blackness to others because someone let a joke slip out they wouldn’t dare utter unless cognitive dissonance didn’t allow them to think of me as colored. Being heavily suggested to explain your blackness because you are one of the only POCs during a project. Imposter syndrome because you look around and realize that no one in your department looks like you.</p><p>My story now looks a lot like my story from the start, just with more questions. What are other people whispering to each other while they are staring at me when I shop? Will I be safe if I go to this (neighborhood, bar, a section of town)? If I go to this party or this restaurant or this networking event, will anyone there look like me? What will my future kid’s stories look like down the line?</p><p>My story is not an unfamiliar one. Like many, it ebbs and flows. Highlights and low-lights. Not defined but any one particular trait or moment or decision. It rings the same, just in a different tone.</p><p>My story exists within those who you work next to who look like me. In those who are marching on the streets who look like me, whose stories can only be heard through a megaphone because no one wants to listen otherwise.</p><p>For those who are wondering what they can do during a time of extreme civil unrest to better educate themselves and become more aware — listen to these stories.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4939a3669334" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Core Web Vitals]]></title>
            <link>https://derek-m-hawk.medium.com/understanding-core-web-vitals-d4884da17664?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/d4884da17664</guid>
            <category><![CDATA[google-algorithm-changes]]></category>
            <category><![CDATA[technical-seo]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[website-development]]></category>
            <category><![CDATA[seo]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Mon, 01 Jun 2020 14:10:38 GMT</pubDate>
            <atom:updated>2020-06-01T14:10:38.648Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/744/0*UhSrTDkC03ZJu03a" /></figure><p>What seemed to be almost inevitable, Google has made the <a href="https://webmasters.googleblog.com/2020/05/evaluating-page-experience.html">recent announcement of a new ranking signal</a> based around page experience in the next six months. With Google’s continual emphasis on usability over the past couple of years and the subsequent updates that have pushed that message forward (mobile-first indexing, page speed as a ranking factor, etc) sites can no longer rely solely on domain strength and quality content alone. With only a short period between announcement and implementation, SEOs should be preparing their sites (along with their own skill sets) to embrace the change. The first step in preparing, jumping right into Google Search Console in the newly-out-of-beta Core Web Vitals.</p><h3>What are Core Web Vitals?</h3><figure><img alt="Diagram showing Google’s Core Web Vitals" src="https://cdn-images-1.medium.com/max/960/0*Qvi8mKpx5qfmYiIR" /></figure><p>Core Web Vitals, as described by Google, is an offshoot of their large Web Vitals initiative that highlights the page metrics that matter most. Where Google has continued to provide toolsets and documentation on web performance optimization, many in the development, tech, and SEO community have had trouble keeping up (and really, can you blame them?).</p><p>Core Web Vitals make up into three distinct metrics:</p><h4>LCP (Largest Contentful Paint)</h4><p>Typical speed metrics to determine page loads such as Speed Index (a more formulaic way of determining overall site speed) or First Contentful Paint (first appearance of any part of a page’s content being loaded in) have since been proven as not the most viable means of highlighting speed. Google, in response, has taken a more in-depth view of the amount of time used to render the largest element in a viewport. The load in of large resources such as images or video signals that once the waterfall hit these elements that a URL is loading.</p><h4>FID (First Input Delay)</h4><p>Similar to First Contentful Paint, First Input Delay is a measurement of activity to respond. So for example what is the time between clicking an icon on a page to when the site responds to that click.</p><h4>CLS (Cumulative Layout Shift)</h4><p>Often experienced but rarely measured, Cumulative Layout Shift is the measurement of site layout movement in response to unexpected shifts that occur on a page. So, if you are interacting with an element on a page (let’s say, a survey) and you misclick an answer due to a banner floating in and moving elements around. The measurement of movement in unstable elements in an unexpected fashion is CLS.</p><h3>How do I Understand My Own Web Vitals?</h3><p>Jumping into your own Google Search Console (or pulling Web Vitals via API), you’ll see a series of red, yellow and green lines for both mobile and desktop. You’re probably thinking to yourself “what do I do with all this information? What’s even the difference from good, bad or in between?” Which is a totally understandable response.</p><p>Let’s jump into a couple of different questions you can ask yourself for each of these measurements.</p><p>For LCP issues Detected:</p><ul><li><em>“What does my current resource load look like on my site? Are there a lot of redundant elements that are being loaded in?”</em></li><li><em>“Are there render blocking elements on my site which can be optimized?”</em></li><li><em>“Where can I provide streamline load-in, whether that be through code minification or compression?”</em></li></ul><p>For FID issues detected:</p><ul><li><em>“How does JavaScript execute within my site? Are there serious issues between input action, execution and response for important user interactions?”</em></li><li><em>“Similar to LCP, what areas within the JavaScript of my site can be improved using minification, compression or deferment?”</em></li></ul><p>For CLS issues detected:</p><ul><li><em>“What is the nature of movement on my site? What triggers unnatural movement of elements on my site”</em></li><li>(If you host ads on your site) <em>“Are there intrusive ad units which dynamically alter content on my site?”</em></li><li><em>“What are the dimensions of my images on site? Do I have control of image assets sizing on site / have they been determined via the CSS?”</em></li><li><em>“How responsive are fonts on my site? Do they cause issues in load-in or displaying to users?”</em></li></ul><p>Core Web Vitals often points to issues which are broader, more sitewide issues, though you can find nuances in how different pages across your site. Make sure you are asking questions constantly across your organization. Often for SEOs solving these issues, you will be working alongside developers, UX and creatives to find the most optimal solution.</p><h4><strong>Okay, but How do I Analyze my Competition?</strong></h4><p>With around six months worth of time to prepare, I would hope SEOs are looking to their SERP competitors to see where they stand in comparison. With updates PageSpeed Insights and Chrome UX Report to highlight specific vitals imperative to page experience, we can create automated means of competitive analysis. Using a framework for site speed analysis <a href="https://www.linkedin.com/pulse/automating-seo-site-speed-analysis-using-pagespeed-api-derek-hawkins/">I’ve used in the past </a>(leveraging the PageSpeed Insights API), the <a href="https://www.linkedin.com/pulse/want-ton-seo-data-consider-semrush-api-your-workflow-derek-hawkins/">SEMRush API</a> for competitive landscape data from the SERP, and Plotly for data visualization, we can build quick vitals readouts on competitors based on your relevant keywords.</p><figure><img alt="Graph showing Largest Contentful Paint of competing terms on the SERP" src="https://cdn-images-1.medium.com/max/1024/0*FuDmU_gPkjYKTH_5" /></figure><p>In this example view, we can see that for the majority “best summer shoes” ranking URLs, Largest Contentful Paint have under 2.5s loads on mobile over 55% of the time. We can use this methodology across multiple web vital elements and essentially create full measurements of competing URLs against your own site, understanding the baseline landscape prior to the updates. To run the script yourself and experiment with these views, see the <a href="https://gist.github.com/DerekHawkins/b9641b8205e21de070477ce6d2b4bb65">source code here</a>.</p><p>If you wanted to take this process further and create competitive forecasts leading up to the update in 2021, you could run this script on a regular basis against your competitive URLs and store results in a database to track progression of how other SEOs/dev teams are preparing for the update. That way, you always have a birds eye view on the competitive set and the shifts they are making, creating a roadmap to prioritization for technical updates.</p><h3>Final Thoughts</h3><p>Google’s inclusion of page experience as a ranking factor is a turn in the right direction for a better internet. SEOs who want to stay vigilant in their abilities to optimize their sites will have to take more reigns on the technical side than they ever have before. For many, this means taking more time building up their programming skill sets. For others, it means building their relationships further with development, tech and UX teammates. No matter which way you go, having a grasp on Core Web Vitals early gives you a proactive advantage to your SEO competitors.</p><p>Thoughts on page experience and preparing your site for the newest update? Tell me about them! Feel free to <a href="https://www.linkedin.com/in/derek-hawkins-seo/">reach out to me directly</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d4884da17664" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leveraging Keyword Data for COVID-19 Trends Analysis]]></title>
            <link>https://medium.com/swlh/leveraging-keyword-data-for-covid-19-trends-analysis-16044619a636?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/16044619a636</guid>
            <category><![CDATA[search-engine-marketing]]></category>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Tue, 21 Apr 2020 14:13:15 GMT</pubDate>
            <atom:updated>2020-04-22T16:13:52.122Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pA47rEu3o-6JoqrG.jpeg" /></figure><p>Amid the recent COVID-19 pandemic, search has never been more at the forefront of highlighting changes in users’ behavior. We have seen dramatic growth in demand for particular topics along with dramatic drops in others. Keeping marketers and search engines alike on their toes, strategic shifts are being made daily to adjust to the ever-changing landscape that we are collectively experiencing. Many SEOs (and SEMs, we are all in this together), the need for new data has exploded. With many search engine providers not supplying COVID-related search data through conventional tools, we need to look to new sources of data that can truly inform strategies and tactics.</p><p><strong>Google Trends for Aggregation and Day Parting Analysis</strong></p><p>You’ll see many articles regarding COVID search strategies incorporating <a href="https://searchengineland.com/adapting-your-seo-strategy-to-soften-the-impact-of-covid-19-on-organic-traffic-333020">Google Trends</a> in the wake of necessary keyword data not being present at this time. But, where many analyses fall flat leveraging Google Trends is in the limitations of data utilized. Similar to standard keyword research, you wouldn’t base your strategy on only a small handful of keywords. The power of Google Trends lies in taking a larger data set and hyper focusing on the granular data points in between the lines. And, with rapidly changing schedules due to work from home initiatives, schools being closed and <a href="https://www.kiro7.com/news/trending/coronavirus-sleep-issues-crop-up-because-covid-19-stress-experts-say/6FMKVANIAFF4BGAOW4D7KZUPDA/">sleep patterns shifting</a>, having an understanding of volatility in search patterns will help you further evolve your strategies.</p><p>To take your Trends analysis to the next level, you can use the unofficial Google Trends API, <a href="https://github.com/GeneralMills/pytrends">PyTrends</a>. PyTrends allows for the same raw data exports found on Google Trends without having to manually navigate the user interface and submit keywords one at a time. Using a combination of PyTrends and a handful of other Python libraries, we can aggregate a larger keyword base’s demand together, create a pivot from the raw data and visualize the results in a fraction of the time.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c1388eb293ef4cb269cf9688aa7de463/href">https://medium.com/media/c1388eb293ef4cb269cf9688aa7de463/href</a></iframe><p>Add your keywords into the list (or bring in a column from a CSV using pandas) and set your time range within the PyTrends function. What you’ll end up with is an easy to navigate data visualization, along with a raw data file which you can use for your further analysis.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/0*4B4GEtjl8Wc0rkWy" /></figure><p>For those who are looking for continuous real time data over a particular keyword base, you can set this script to run regularly using datetime qualifiers within the PyTrends function, create an automated command to run this script automatically and path the raw data to a SQL server or separate workflow for data ingestion.</p><p><strong>Google Keyword Planner for Forecasting Search Demand</strong></p><p>Looking to understand and forecast shifts in demand within your verticals? Consider both historical search behavior data on top of existing shifts. While Google Keyword Planner is used far and wide for both SEO and SEM tasks, taking a step further into data outputs can allow us to create views of the month over month shifts to determine future demand. Simply log into Google Ads, input your keyword base and export historical metrics as a CSV for whatever time range you want to determine forecasts. From there, path your data export into the python script below and from there you’ll be given a series of line graphs showing historical month over month percentage changes in search demand for your keyword set.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3925957f112fa578a14d8dd1ab2ffd5d/href">https://medium.com/media/3925957f112fa578a14d8dd1ab2ffd5d/href</a></iframe><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*fPHJZnHWciqz9ocI" /></figure><p>Similar to Google Trends, you can take this script a step further by incorporating the Keyword Planner API to pull data for your keywords across separate niches or topics of exploration. From there, sync up this script into a Dash dashboard and have it regularly updated on a monthly cadence with newly updated data. Leave this monitor up and continue running even after the pandemic to create more compelling demand forecasts based on pre/post-pandemic activity.</p><p><strong>Conclusion</strong></p><p>Whether your business is seeing upticks or downticks in activity during the COVID-19 pandemic, there is a need to meet your audience directly on search. The data you begin to collect and analyze today serves as the basis for pointing your organization in the right direction moving forward. So, if you are trying to either optimize a paid search campaign, or looking to develop/optimize content for organic search during this time, hopefully some of these tools and techniques can help you along the way.</p><p>Want to talk further analyzing search or social data? You can always reach me via my <a href="https://www.linkedin.com/in/derek-hawkins-seo/">LinkedIn </a>or <a href="https://twitter.com/Derek_M_Hawk">Twitter</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=16044619a636" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/leveraging-keyword-data-for-covid-19-trends-analysis-16044619a636">Leveraging Keyword Data for COVID-19 Trends Analysis</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Learn Python as an SEO]]></title>
            <link>https://medium.com/swlh/how-to-learn-python-as-an-seo-d56528b1dedf?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/d56528b1dedf</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[search-engine-optimizati]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[professional-development]]></category>
            <category><![CDATA[seo]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Mon, 11 Nov 2019 14:24:12 GMT</pubDate>
            <atom:updated>2019-11-13T16:48:21.316Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*58twqItpOjtMRuQquE2l5w.jpeg" /></figure><p>It’s the cutting-edge, tech-driven SEOs that have begun to shine in both modern agency and in-house organizations. For those who have adopted programming, automation, and innovation into their everyday lives, the rewards are significant results in their organic search performance and best-in-class SEO strategies/processes. Looking to up my marketing game with something more advanced drove me to take on the intensive process of picking up programming from scratch. Two years later, many of my day to day processes being rooted almost exclusively in Python. Learning the language has been one of the most important steps in my career so far. But, with no formal training under my belt, no degree that states “Python Master,” and no pre-set pathway to picking up the language, the question has been asked many times over has arisen. “How did you learn Python?” Or just as much, “How should I learn Python?” While no pathway is perfect and mine had many bumpy roads along the way, I wanted to share the tips and advice I wish I had gotten before diving headfirst into a brand new programming world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kzXg7bDib72IfMEibB1fMA.jpeg" /></figure><h3>Learning the Fundamentals</h3><p>I spent (and still spend) a large amount of time reading/watching anything I can get my hands on. I never used a specific book or online course (though there are many out there that cover fundamental Python). Rather, I gave myself a lot of time to immerse myself in code and piece together an understanding of what I was looking at. Answering the basics for myself (“what is an IDE?” “what is a string” “what is a dictionary?”) while also tackling slightly more complicated code bases over time. From technical documentation, Stack Overflow threads, Github repo documentation, learning Python is just as much an exercise in reading the language as it is writing the language. Before you start with jumping into technical docs, I would highly recommend just turning to videos online. YouTube is a great resource for learning on the fly fundamentals through visual examples (and probably remains one of the biggest sources of education for me nowadays). While the realm of Python videos is vast online, some of the channels I heavily relied on to start were <a href="https://www.youtube.com/user/sentdex">sentdex</a>, <a href="https://www.youtube.com/user/dataschool">Data School</a> and <a href="https://www.youtube.com/channel/UCxX9wt5FWQUAAz4UrysqK9A">CS Dojo Reading</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*L4GgDWNMc3oUOaGEjZ5f_g.jpeg" /></figure><h3>Coding with Realistic Objectives</h3><p>A lot of what I did personally in terms of self-learning came out of necessity. When I first started I made two goals for myself to First, always attempt to implement a pythonic approach to any work task presented to me. Even when the task would take more time compared to conventional methods, understanding how to deconstruct a process into its core steps allows for developing the means to build code which can further optimize and automate the process. Does your day to day SEO work include reporting for example? When developing a weekly or monthly report, take the time to write down every step of the process in its most granular form, and don’t skip a detail. Once you have each step written out, approach each chunk of the process purely with Python. This allows for a step-by-step process rather than trying to drink straight from the fire hydrant.</p><p>Secondly, always have a side project that you are building for your own personal development/interest. For myself that has been building GUI applications for trend spotting in historical keyword data, along with web scrapers for sneaker websites to track price fluctuation. Having separate, non-work based projects allows for your mind to create and solve problems at your own pace and with your interests, making it less of a chore to learn but more so something you can enjoy throughout.</p><p>Overall, for both steps, you’re coding goals should have reachable goals. Start simple and work your way up.</p><h3>Follow by Example</h3><p>There are many SEOs in the community who are breaking new ground in applying Python into the SEO space. From <a href="https://www.searchenginejournal.com/bert-generate-meta-descriptions/332585/">automated meta description writing</a> to <a href="https://searchwilderness.com/mozcon-2019/">finding expired domains online</a>, the SEOs who are working in the language day to day serve as beacons for those who want context into what can be done by blending Python and SEO. Whether it is setting up a time to meet with them in person, or just sending a friendly LinkedIn message or email, look to further learn from those who are actively in the same world. While learning and collaborating with other SEOs in the niche we are all in, I have had the great pleasure of getting to know other SEO-programmers such as</p><ul><li><a href="https://www.linkedin.com/in/paulnshapiro/">Paul Shapiro</a> of Catalyst</li><li><a href="https://www.linkedin.com/in/michaelkingphilly">Michael King</a> of iPullRank</li><li><a href="https://www.linkedin.com/in/hamletbatista">Hamlet Batista</a> of Ranksense</li></ul><p>The community of Python focused SEOs is small (but ever-growing). If you are interested in what they are working on or want to know more about some of their processes, take the steps to network around the space!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/664/1*a0YRT7cGGX9nDFDDATU_ZA.jpeg" /></figure><h3>Collaboration</h3><p>No marketer works in a silo. They interact with all aspects of the organization from sales to strategy to tech and everything in between. It should work in the same way for learning. Taking up the challenges of learning not only with your peers in the SEO community but with internal teams across all facets of the organization. I am thankful that my office has a ton of other people who work in Python so I spent a lot of time working alongside teammates on more complex problems, picking brains along the way. Asking questions about what they are working on, showing genuine interest, engaging and trying to understand their code for myself. This can be achieved digitally via Github. Find over SEOs who are looking to learn and build something together, exchanging and editing GitHub repos. It goes a long way when you are working together with others during the learning process, much like being in your everyday office setting.</p><p>Want to get started learning Python? Check out my article on all things <a href="https://www.linkedin.com/pulse/python-approach-seo-automating-boring-stuff-derek-hawkins/">Python for streamlining SEO</a>! Also, reach out to me directly through <a href="https://www.linkedin.com/in/derek-hawkins-2487b185/">LinkedIn</a> or <a href="https://github.com/DerekHawkins">GitHub</a> with any questions or interest in collaborating!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d56528b1dedf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/how-to-learn-python-as-an-seo-d56528b1dedf">How to Learn Python as an SEO</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automating SEO Site Speed Analysis Using the PageSpeed API]]></title>
            <link>https://derek-m-hawk.medium.com/automating-seo-site-speed-analysis-using-the-pagespeed-api-3c4622850b27?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/3c4622850b27</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[page-speed]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[search-engine-optimizati]]></category>
            <category><![CDATA[seo]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Tue, 17 Sep 2019 15:09:21 GMT</pubDate>
            <atom:updated>2019-09-17T15:09:21.265Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uY0ORchE4b5qP19r2-iUkQ.jpeg" /></figure><p>Whether your site is 5 or 50,000 pages, the way your site loads in content and backend code can be one of the most significant factors in your site performing strongly in organic search. Google over the past few years has been continuing to emphasize the importance of page speed in SEO ranking, going so far as to have <a href="https://webmasters.googleblog.com/search/label/speed">entire sections of the Webmaster Blog </a>associated with site speed. Between mobile’s continual influence in organic search (and the many, MANY variables mobile faces with loading content) and site complexity in their builds continue to grow, having a streamlined method for understanding your site’s loading capabilities.</p><p>Thankfully, Google has a plethora of methods for understanding how your site stands in load time and speed. On the quick and easy end is the frontend facing, web-hosted <a href="https://developers.google.com/speed/pagespeed/insights/">PageSpeed Insights</a>. Many SEOs already live and breath by this site (myself included). Simply plug in the URL you want to check and get back all the metrics you are looking for across both mobile and desktop. But, going through the tedious process of copying and pasting URLs into the site, waiting for the test to be conducted and then recording the results you want onto a separate file is very time-consuming. On the other hand, Google Analytics provides site speed monitoring capabilities built into the platform to showcase the entirety of a site. But, for the many sites online who are utilizing other web analytics monitoring platforms or for SEOs who want to conduct competitive analysis on other site’s speed performance, this option is out of the question. So, how can we scale what Google has already provided without having to radically change technology?</p><p><a href="https://support.google.com/analytics/answer/1205784?hl=en">Google’s PageSpeed Insights API</a> is an amazing, easy to use, and scalable method for automating the site speed testing process for any site. Utilizing a REST framework, the API is designed to be flexible for all use case scenarios whether it is for your site initiatives or for conducting competitive analysis. Using Python (though this API works across many other programming languages), we’ll highlight how to leverage the API over multiple URLs in an automated fashion.</p><h3>Authenticating the PageSpeed Insights API</h3><p>The authentication process is straightforward. All you need is an account on Google Developer Console (which you can get via this link). From your account, start a new project from the homepage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/739/1*J8MZqAWS4pjRhT3iVFQCjg.jpeg" /></figure><p>Once you create the project, go to the <strong>API Library </strong>and search for “PageSpeed Insights API.” Click “Enable API” which should generate the API key you’ll need to start.</p><h3>Using the API</h3><p>Once you have your credentials, you will be able to construct your first API call and begin experimenting with automated site speed testing. Grab a list of URLs that you want to test (either from Screaming Frog, an XML sitemap or wherever works best for you) and we can begin. For ease of access, I have coded the API into a simple UI that allows for ease of access for those who are not familiar with coding. I will be providing instructions for using the API either with or without the UI.</p><h3>Working with the User Interface</h3><p>The <a href="https://github.com/DerekHawkins/Pagespeed-Insights-API">UI wrapper</a> for the PageSpeed Insights API uses <a href="https://pypi.org/project/easygui/">EasyGUI</a>, an easy to use interface which relies on a linear series of interactions rather than a more experiential software application. While the backing is simple, the ability to use the API without having to hardcode your inputs/output paths has its benefits in terms of streamlining the process.</p><p>When using the interface, the only coding you will have to do is inputting your key into the code itself. You can do so on line 13.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/625852ea4bf6edd9da6da993ef3fefcb/href">https://medium.com/media/625852ea4bf6edd9da6da993ef3fefcb/href</a></iframe><p>Once you save that into the code, you can run the whole script and run through the interface to do your testing. You’ll have the option to either submit an Excel file (with the first cell labeled “<strong>URL</strong>”) or via copying and pasting URLs into the program.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/581/1*FPrthCLWAWi23XYFKFRCqA.png" /></figure><p>The site speed testing will happen in the background while you are free to do other tasks or just stare at your screen being proud of bringing some new automation to your job! Once the script is finished running, you will be prompted to save a .xlsx file containing the data. Your output should look something like this. The file will have three different speed metrics across desktop and mobile. For more information about each of these metrics, refer to the <a href="https://developers.google.com/speed/docs/insights/v5/about">PageSpeed Insights documentation</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kEMwiiq-3JECQVK2Gwzlng.png" /></figure><h3>Without the User Interface</h3><p>For those who want additional metrics from PageSpeed, or want to just tinker around with the source code I’ve written out, there are two parts to working with the API which I would emphasize.</p><p><strong>Constructing the URLs/Making the API Call</strong></p><p>The URL building is done in a function using Urllib, which encodes out the URL into a REST-friendly format while also allowing for flexibility. Once the URL is developed, you can call using requests to receive the nested JSON output file.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7c4293bc0df544baf60b308955bc743f/href">https://medium.com/media/7c4293bc0df544baf60b308955bc743f/href</a></iframe><p><strong>Retrieving the Data from the JSON File</strong></p><p>Every successful API call outputs a JSON file constructed something like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*Q7jmvxOMm_XMqpDkwuj61w.png" /></figure><p>You can find a wealth of data within this output, from how the test loaded into the results. While my script takes only three site speed metrics, there are over 40 metrics which come with each call including DOM sizing, every web resource on the tested page, and speed optimization recommendations. Navigating the JSON file can be intensive, but very rewarding for the inquisitive SEO who wants to maximize their data. An example of how I have navigated the data output can be seen below. Starting with my main output (“Response”), I had dug into the specific data points needed to pull out the numeric values given by the API.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7b89e1418dc6c8dabb16cd2be2eb1897/href">https://medium.com/media/7b89e1418dc6c8dabb16cd2be2eb1897/href</a></iframe><h3>Conclusion</h3><p>The PageSpeed API is one of my absolute favorite offerings by Google. With the ability to make multiple data queries and retrieve so much information about either your own or a competitors site in an automated fashion, it has become a major asset in my technical SEO toolbelt. For those who want to make it a part of their own, the link to the file is on my <a href="https://github.com/DerekHawkins/Pagespeed-Insights-API">GitHub</a>. As you begin running your automated speed tests, feel free to reach out to me personally on my <a href="https://www.linkedin.com/in/derek-hawkins-2487b185/">LinkedIn</a> or <a href="https://twitter.com/Derek_M_Hawk">Twitter</a> for any questions about the script. But overall, happy speed testing!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3c4622850b27" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automating SEO Reporting: A Case Study]]></title>
            <link>https://derek-m-hawk.medium.com/automating-seo-reporting-a-case-study-cb6d55abcd5a?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb6d55abcd5a</guid>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[search-engine-optimizati]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Mon, 29 Jul 2019 13:21:02 GMT</pubDate>
            <atom:updated>2019-07-29T13:21:02.892Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="“Automate All the Things” written on a board" src="https://cdn-images-1.medium.com/max/477/1*aLHc9teOlIC3PolQ4H0Bzg.jpeg" /></figure><p>Usually, in an agency or in-house scenario, an SEO will be brought in to do more than just day to day optimizations. From consulting with developers and stakeholders, doing keyword/market research, and building out new content, the last thing you have time for is the mundane gut-check that is reporting. While the actual optimization process can involve critical thinking, navigating JavaScript, crafting copy, you know, things that you might have to rub your chin like “The Thinker,” most reports I have ever dealt with primarily involve clicking around through Excel and Powerpoint. So, you are probably thinking, why wouldn’t I just use the built-in reporting tools that come with pretty much every SEO software? Well, for starters:</p><ul><li>Most SEO software’s reporting tools lack the customization needed for client-specific metrics. With out-of-the-box SEO reporting tools, you are more often than not left with either too broad of a view or far too granular for what your client or internal stakeholders want.</li><li>The actual analysis/insights that make most reports meaningful are lost in translation. The time you spend with your data in gathering and formatting could be used for actual insights gathering.</li><li>Aggregation of other data sources. While many platforms integrate other data sources through API connections internally, you would be hard-pressed to find any tool that does it so in a clean way.</li></ul><figure><img alt="The Thinker sclupture" src="https://cdn-images-1.medium.com/max/222/1*S_CIGmLT9gx6mcsnGLw9ZQ@2x.jpeg" /></figure><h3><strong>The Problem</strong></h3><p>Our client wanted to track SEO metrics that were associated with the brand’s perception as thought leaders in a niche vertical. Utilizing internal website data, social media engagement and search engagement, the client wanted to see if their content efforts established them as the go-to source for information for their industry. To understand how the client was perceived from the lens of search, the following KPIs would be tracked as indicators.</p><ul><li>Number of keywords in top 10 positions</li><li>Number of quality backlinks acquired (the threshold determined by domain authority)</li><li>Click-through Rate (CTR) on the SERP</li></ul><p>This exercise, from going to the different sources and pulling data and compiling through data to get consistent results and place them into their report every quarter would take around 2 hours per month for a total of 6 hours a quarter. While that does not seem like a lot of time, that’s 6 hours that could be used to better my clients. So, let’s come up with a solution that ultimately makes the reporting process come out to around 3 minutes per month.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E4hNh5mReGzL24roRI7HGQ.jpeg" /></figure><h3><strong>The Report Automation Framework</strong></h3><p>Let’s start with the libraries that we used for this process.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aace25ea2db16a201232f1c798a4f22c/href">https://medium.com/media/aace25ea2db16a201232f1c798a4f22c/href</a></iframe><p>With all of our tools in places, let’s look through each of the client’s ask and see what we can do about automating them.</p><h4><strong>Tracking top keyword positions</strong></h4><p>We will be using SEMrush’s flexible REST API (which I use for <a href="https://medium.com/@Derek_M_Hawk/want-a-ton-of-seo-data-consider-the-semrush-api-in-your-workflow-5fe1fc7fc49e">a majority of my automation projects</a>) as our data source for keyword ranking. In this process, pull out the previous month’s date feed it into the function for the API, call for the keyword ranking data for our client’s site, compile the ranking data into a DataFrame and filter out keywords to count the number of top keywords we have.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5d7391cea1f51a34f2d1a41a1ae60adc/href">https://medium.com/media/5d7391cea1f51a34f2d1a41a1ae60adc/href</a></iframe><p>The result, when you export out the file will ultimately look something like this.</p><figure><img alt="DataFrame with SEO data" src="https://cdn-images-1.medium.com/max/562/1*8Ge9bybNJPtpCWatru4VCQ.jpeg" /></figure><h4>Compiling SERP CTR</h4><p>For understanding our engagement on search, we will be utilizing Google Search Console and a <a href="https://github.com/joshcarty/google-searchconsole">third-party wrapper that allows for easy API access</a>.</p><p>To utilize the Search Console API, you will need to enable access and obtain the necessary credentials. Google provides a<a href="https://developers.google.com/webmaster-tools/search-console-api-original/v3/prereqs"> very comprehensive guide to doing so here</a>. Once you have accomplished that, you can load your credentials into the wrapper and write up a function similar to what I have written (or just copy and paste mine, I won’t be offended).</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b3fc4075ae8a697bcdb0b8000d12b32b/href">https://medium.com/media/b3fc4075ae8a697bcdb0b8000d12b32b/href</a></iframe><h4><strong>Collecting Quality Backlink Data</strong></h4><p>We will once again be compiling data via SEMrush’s API. This method builds out the URL needed for the API call within a function and then utilizes previous functions to retrieve the data. Once the API call has been made, the timestamps on “First Seen” are converted into a legible time. From there, the DataFrame filters based on the month of the report and runs a secondary filter to only leave backlinks which are greater than a specific page score. For this example, the page score threshold has been artificially determined, but in your usage of these scripts, you can replace it with what you see fit.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/75138795ee56da50af2336360f853827/href">https://medium.com/media/75138795ee56da50af2336360f853827/href</a></iframe><h4><strong>Putting it All Together</strong></h4><p>With all of the necessary data for this exercise sitting within their unique DataFrames, we can compile all the data together into one easy to read, a navigatable spreadsheet that can be handed off to the team. ExcelWriter drafts up a new spreadsheet labeled based on the specific month the report is run and stitches together all of our examples into one.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c48648b374681d9c905d3f6770cd8dfd/href">https://medium.com/media/c48648b374681d9c905d3f6770cd8dfd/href</a></iframe><p>Utilizing this method of reporting, we were able to not only pull the necessary data in a streamlined fashion, but we can also routinely pull this data without interacting with separate platforms or compiling methods. In this instance, the report was set on a timer to run at a specific time and emailed out (using a separate script) to all necessary channels.</p><p>Automating the SEO reporting process means returning hours to the SEOs on your team (even if you are a team of one) that can be leveraged for other more critical tasks. While this example just highlights one instance of report automation from the perspective of search, essentially all metrics associated with search (organic traffic compilations, log file analysis, paid/organic synergy, etc) can be streamlined using a Pythonic workflow. So take the challenge for yourself and make your life just a little bit easier</p><p>Want to learn a little more about automating SEO using Python? Check out my article on all things <a href="https://medium.com/@Derek_M_Hawk/a-python-approach-to-seo-automating-the-boring-stuff-4a9eb2e6d9d5">SEO automation here</a>! Or, reach out to me directly through <a href="https://www.linkedin.com/in/derek-hawkins-2487b185/">LinkedIn</a> or <a href="https://github.com/DerekHawkins">GitHub</a>!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb6d55abcd5a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[3 APIs for Automating Keyword Research]]></title>
            <link>https://medium.com/swlh/3-apis-for-automating-keyword-research-1857eb295200?source=rss-3e84d2639e65------2</link>
            <guid isPermaLink="false">https://medium.com/p/1857eb295200</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[keywords-tool]]></category>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[keyword-research]]></category>
            <category><![CDATA[api]]></category>
            <dc:creator><![CDATA[Derek Hawkins]]></dc:creator>
            <pubDate>Wed, 12 Jun 2019 14:31:00 GMT</pubDate>
            <atom:updated>2019-06-13T17:18:45.937Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vT6NabAjzHyl_ziqT9xv-A.png" /></figure><p>Take tedious data entry, whiteboard brainstorming sessions, and a couple of hundred mouse clicks and you have the formula for many keyword research exercises. While this works for one-time efforts (and trust me, I will be the first to pull out a dry erase marker if the time calls for it), enterprise-level research requires a more sophisticated approach to data sourcing and analysis. From automating the day-to-day keywords sourcing to creating programs that semantically connect keywords together by SERP listing, tapping into new avenues of collecting keyword data will save you time and unlock new opportunities for your next SEO campaign.</p><p>So, whether you represent a larger organization, or want to bring your company/clients a larger pool of resources, an API driven keyword research process is right up your alley.</p><p>All of the examples present will be in Python (<a href="https://www.linkedin.com/pulse/python-approach-seo-automating-boring-stuff-derek-hawkins/">my preferred programming language</a>), but all the APIs presented are accessible via other languages such as Java, PHP, and JavaScript.</p><h3>SEMrush’s Analytics API</h3><p>I’ve talked about the <a href="https://medium.com/@Derek_M_Hawk/want-a-ton-of-seo-data-consider-the-semrush-api-in-your-workflow-5fe1fc7fc49e">SEMrush API in the past</a>, and with new updates being added regularly to reflect SEMrush’s current offerings, it has never been a better time to begin exploring. SEMrush’s API uses a REST framework for lean access to SEMrush’s massive and continuously updated keyword database. So it’s quick and easy to begin development and implementation with little hassle in the initial setup. And, with new additions such as their Phrase Questions (pulling questions-focused terms based on root inputs), and Broad Match Keywords (similar to their Keyword Magic Tool or Google Ads’ Broad Match Modifiers), keyword collection is a breeze. Here is a quick example script of how quick you can get started in the API.</p><pre>import pandas as pd<br>import requests<br>import urllib</pre><pre>api_key = &#39;insert your api key&#39;<br>service_url = &#39;<a href="https://api.semrush.com&/#39;">https://api.semrush.com&#39;</a></pre><pre>### For streamline URL development for call<br>def build_urls(phrase):<br>    params = {<br>        &quot;?type&quot;: &quot;phrase_related&quot;,<br>        &#39;key&#39;: api_key,<br>        &#39;phrase&#39;: phrase,<br>        &#39;export_columns&#39;: &quot;Ph,Nq&quot;,<br>        &#39;database&#39;: &#39;us&#39;<br>        }<br>    data = urllib.parse.urlencode(params, doseq=True)<br>    main_call = urllib.parse.urljoin(service_url, data)<br>    main_call = main_call.replace(r&#39;%3F&#39;, r&#39;?&#39;)<br>    return main_call</pre><pre>url_call = build_urls(phrase=&#39;keyword research&#39;)<br>r = requests.get(url_call)<br>data = r.json</pre><pre>df = pd.DataFrame(data)</pre><p>Speaking from my own personal experiences, SEMrush’s keyword capacities are best utilized for mass sourcing when you have a bulk of root terms/topics that you know will have greater depth and areas of opportunity. Being able to iterate over hundreds of root terms and storing the data in one place as led this workflow to be a core part of my own process that has saved me countless hours and given me keywords I never would have thought of on my own.</p><h3>Google Ads API — Keyword Forecasting.</h3><p>Taking the data right from the source, Google Ad’s API featured a robust backend data stream for grabbing the best source for search volume. So why wouldn’t I put this one at the top?</p><p>Google’s APIs, especially those connected to their core advertising and analytics offerings, can have a steep learning curve in terms of jumping right in. Their <a href="https://developers.google.com/google-ads/api/docs/start">documentation is impeccable</a> and their full SDK can unlock endless opportunities. But, it can feel like drinking from a fire hydrant when first starting. SEO’s with not as strong developing skills may have larger issues with accessing the data and building lean keyword research tools compared to some of the other APIs in this article.</p><pre>from __future__ import absolute_import</pre><pre>import argparse<br>import six<br>import sys</pre><pre>from google.ads.google_ads.client import GoogleAdsClient<br>from google.ads.google_ads.errors import GoogleAdsException</pre><pre>def main(client, customer_id, keyword_plan_id):<br>    keyword_plan_service = client.get_service(&#39;KeywordPlanService&#39;)<br>    resource_name = keyword_plan_service.keyword_plan_path(customer_id,<br>                                                           keyword_plan_id)</pre><pre>try:<br>        response = keyword_plan_service.generate_forecast_metrics(resource_name)<br>    except GoogleAdsException as ex:<br>        print(&#39;Request with ID &quot;{}&quot; failed with status &quot;%s&quot; and includes the &#39;<br>              &#39;following errors:&#39;.format(ex.request_id, ex.error.code().name))<br>        for error in ex.failure.errors:<br>            print(&#39;\tError with message &quot;{}&quot;.&#39;.format(error.message))<br>            if error.location:<br>                for field_path_element in error.location.field_path_elements:<br>                    print(&#39;\t\tOn field: {}&#39;.format(field_path_element.field_name))<br>        sys.exit(1)</pre><pre>for i, forecast in enumerate(response.keyword_forecasts):<br>        print(&#39;#{} Keyword ID: {}&#39;.format(i + 1,<br>          forecast.keyword_plan_ad_group_keyword.value))</pre><pre>metrics = forecast.keyword_forecast</pre><pre>click_val = metrics.clicks.value<br>        clicks = &#39;{:.2f}&#39;.format(click_val) if click_val else &#39;unspecified&#39;<br>        print(&#39;Estimated daily clicks: {}&#39;.format(clicks))</pre><pre>imp_val = metrics.impressions.value<br>        impressions = &#39;{:.2f}&#39;.format(imp_val) if imp_val else &#39;unspecified&#39;<br>        print(&#39;Estimated daily impressions: {}&#39;.format(impressions))</pre><pre>cpc_val = metrics.average_cpc.value<br>        cpc = &#39;{:.2f}&#39;.format(cpc_val) if cpc_val else &#39;unspecified&#39;<br>        print(&#39;Estimated average cpc: {}\n&#39;.format(cpc))</pre><pre>if __name__ == &#39;__main__&#39;:<br>    google_ads_client = GoogleAdsClient.load_from_storage()</pre><pre>parser = argparse.ArgumentParser(<br>        description=&#39;Generates forecast metrics for a keyword plan.&#39;)<br>    # The following argument(s) should be provided to run the example.<br>    parser.add_argument(&#39;-c&#39;, &#39;--customer_id&#39;, type=six.text_type,<br>                        required=True, help=&#39;The Google Ads customer ID.&#39;)<br>    parser.add_argument(&#39;-k&#39;, &#39;--keyword_plan_id&#39;, type=six.text_type,<br>                        required=True, help=&#39;A Keyword Plan ID.&#39;)<br>    args = parser.parse_args()</pre><pre>main(google_ads_client, args.customer_id, args.keyword_plan_id)</pre><p>That being said, if you can leap over the learning curve, Google Ad’s keyword tool offers a ton of opportunities to decrease time and pull the most accurate data. Most importantly, it’s free! Well, the test accounts are free and limit the number of requests you are able to make in a day. But, regardless, for those who run with smaller budgets but still wish to bring API frameworks into their keyword research process, I would highly recommend giving it a look.</p><h3>Searchmetrics</h3><p>Typically used for enterprise-level SEO, Searchmetrics features a comprehensive keyword and domain database with a scalable API to match. With access to over 900 search engines and countries, Searchmetric’s API is a comprehensive way of gathering keyword data in niche markets where data is limited via conventional channels.</p><p>While the SDK for Searchmetrics is written in PHP, its <a href="http://api.searchmetrics.com/v3/documentation">thoroughly-detailed API documentation</a> allows for smooth integration and ease of access for any programming language.</p><pre>import requests<br>import urllib</pre><pre>API_KEY = &quot;xxxxx&quot;<br>API_SECRET = &quot;xxxxx&quot;<br>service_url = &#39;<a href="http://api.searchmetrics.com/v3/&#39;">http://api.searchmetrics.com/v3/&#39;</a><br>call_type = &#39;ResearchKeywordsGetListKeywordinfo.json/&#39;</pre><pre>data = {<br>    &#39;grant_type&#39;: &#39;client_credentials&#39;<br>}</pre><pre>response = requests.post(&#39;<a href="http://api.searchmetrics.com/v3/token&#39;">http://api.searchmetrics.com/v3/token&#39;</a>, data=data, auth=(<br>    API_KEY, API_SECRET)).json()</pre><pre>access_token = response[&#39;access_token&#39;]</pre><pre>def build_urls(keyword):<br>    params = {<br>        &quot;?countrycode&quot;: &quot;us&quot;,<br>        &#39;keyword&#39;: keyword,<br>        &#39;access_token&#39;: access_token<br>        }<br>    data = urllib.parse.urlencode(params, doseq=True)<br>    first_url = urllib.parse.urljoin(service_url, call_type)<br>    second_url = urllib.parse.urljoin(first_url, data)<br>    main_call = second_url.replace(r&#39;%3F&#39;, r&#39;?&#39;)<br>    return main_call</pre><pre>keyword_call = build_urls(keyword=&#39;seo&#39;)<br>r = requests.get(keyword_call)<br>keyword_data = r.json()</pre><p>Searchmetric’s API is only available with their “Suite Enterprise” package, so utilizing their data will require a larger investment by either your clients or organization. But, with access to a larger pool of data, such as additional access to social media data and YouTube data, Searchmetric’s API is for the keyword researcher who truly wants it all.</p><p><strong>The Bottom Line</strong></p><p>Going off of my own personal experiences working with and writing code for each API, I would say each has its clear pros and cons. Scalability, request limits, pricing, all of these aspects can make or break a keyword research API into your workstream. And while there are many other excellent tools that I haven’t personally worked with (<a href="https://twitter.com/seosmarty?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor">Ann Smarty </a>wrote a<a href="https://searchenginewatch.com/2017/03/02/comparing-keyword-research-apis-spyfu-vs-semrush-vs-serpstat-and-more/"> great article highlighting some other APIs</a> that I haven’t had experience with), no matter which you choose it’ll be hard for you to ever imagine going back to manual keyword research.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1857eb295200" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/3-apis-for-automating-keyword-research-1857eb295200">3 APIs for Automating Keyword Research</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>