catchpoint Essential CDN Monitoring for Digital-First Organizations User Guide

: June 3, 2024
: catchpoint

Table of Contents

Introduction
Why CDN Monitoring Is Important
How eBay And Walmart Evaluate And Monitor Their CDNs
CDN Monitoring Best Practices
CDN Monitoring Checklist
RUM vs. Synthetic
Digital-First Organizations Must Monitor Their CDNs
Building your Observability Strategy with Catchpoint
FURTHER RESOURCES
Documents / Resources

Essential CDN Monitoring for
Digital-First Organizations:
The Authoritative Guide

Introduction

Content delivery networks (CDNs) have been around for over two decades. In addition to their foundational role in serving content closer to the end user, CDNs have evolved from simply measuring an HTTP object from different locations to include features like the ability to upload content, optimize performance, and provide load balancing, as well as enabling faster speed, heightened security, higher reliability, and greater scale and reach.
CDNs are crucial to many industries; for instance, they are the key enablers of the rise of video streaming, both on demand and live. They are also fundamental to Commerce as many enterprises rely on them to execute online commerce within acceptable quality of service (QoS) parameters including latency, performance, and high availability, according to IDC’s MarketScape: Worldwide Commercial Content Delivery Network Services 2022 Vendor Assessment.
As such, CDNs have become an inherent part of digital-first organizations since they play a crucial role in ensuring the ability to deliver content to every single end user, while delivering a great and secure experience. In fact, the commercial CDN market now carries a substantial portion of the world’s internet traffic. IDC projects the worldwide CDN market will reach $18.8 billion by 2025 at a fiveyear CAGR of 17.1%.
“CDNs have become an essential tool to handle the demands created by the massive amount of web content and large downloads on the internet today,’’ IDC writes.

Why CDN Monitoring Is Important

Since CDNs greatly impact the global end-user experience and all data moves through the CDN infrastructure, it is critical to gain visibility into that network. CDN Monitoring lets businesses understand whether a high web page response time is due to objects that are either not cached at all or not cached for long periods. Evaluating the cache hit/miss ratio provides a clearer understanding of performance.
With CDN Monitoring, you’ll be able to detect and mitigate significant outages that impact end user experience, evaluate the DNS performance of your CDN and improve global performance levels. This gives you the ability to make better decisions around key CDN-related strategies, such as implementing multi-CDN, evaluating cost vs offload, or picking geo-specific CDN vendors.
When evaluating a CDN, organizations should consider bandwidth cost, the ability to deliver in different geographies, computation at the edge, the application team requirements, and baseline compatibility. You also need to be able to monitor a CDN’s performance against its origin; a typical DNS override method where you can bind it to the origin host name or origin IP, according to Ankit Kumar, technical manager of partnerships at observability solution provider Catchpoint.
You want to be able to monitor how your CDNs are performing against your origin, as well as monitor them from different regions. “Measuring geographical distribution always helps to tell you if performance is consistent across certain regions or if it’s varying,’’ Kumar says.

How eBay And Walmart Evaluate And Monitor Their CDNs

At eCommerce giant eBay, improving the seller experience is always top of mind and availability to the business is a key metric that officials track. The company has invested significantly in its CDN efforts and has a multi-CDN provider strategy.
The focus has become on how to simplify and automate their usage, says Junaid Akhtar, principal architect for edge service alliances at eBay. Akhtar is part of the CDN team at eBay and says nimbleness is an important metric when choosing a CDN provider. That involves determining whether they can work in a multi-CDN architecture. This is one of the key reasons eBay turned to Catchpoint for CDN Monitoring.
Certain CDNs are very product-rich,
Akhtar explains, but if eBay needs to onboard a new platform, the team needs to know if all their other CDNs can support it. Akhtar’s team gets real-time logs from four different CDN vendors and is using synthetic tools for its monitoring and observability strategy. eBay also undertakes deep data analytics to look at the big picture–the future of CDN monitoring. Real-time logs that come out of each CDN are put into data analytics provider Sumo Logic, and the team creates a snapshot dashboard where they can look at error analysis for latency, performance, and cache optimizations.
“It starts from DNS. We invest so much into CDNs, we really need to understand and track how our connectivity in terms of how our origin is performing across different providers. A lot of times we monitor each layer.’’
The goal is use the rich data to automate decisions, Akhtar says. He recalled an incident where the team saw multiple errors in a regional location, thanks to having set up several alerts and doing proactive monitoring of each CDN.
“It starts from DNS. We invest so much into CDNs, we really need to understand and track how our connectivity in terms of how our origin is performing across different providers. A lot of times we monitor each layer, ’’ he says. Additionally, it is critically important to monitor certificate management, which synthetic monitoring enables.
“If they expire, you really need to monitor and probe, which is what we are doing with synthetic monitors,’’ Akhtar says. “And then we can quickly ask … what are our popular workflows? You want to observe what your users are experiencing. The core of this is finding real- time errors.” That way, the team can make more accurate decisions.

Shubham Shrivastava, a software engineer at Walmart Labs, evaluates different CDNs by looking at the company’s applications and the application team’s requirements. Then his team explores each CDN’s respective features thoroughly. Walmart Labs similarly rely on Catchpoint for the CDN Monitoring they need.
When evaluating a CDN, Walmart Labs looks at availability, performance, and feature capabilities as well as baseline compatibility. Every CDN has a certain baseline that it follows, such as the header length they can support, Shrivastava explains.
“Real user monitoring is great because you’re able to sample all your users from every single location,’’ Kumar says. “Where synthetic monitoring comes into its own is in being able to look at specific, granular areas to diagnose what the problem is.”
But everything boils down to developer requirements, he adds. For example, if an application team needs delivery in China and in Southeast Asia, that is a specific
requirement. “They don’t need global delivery, they only need the best local CDN in that case.”
Availability has become a major concern with some CDNs, Shrivastava says.
So when planning for a multi-CDN strategy, “we want to ensure that our compounded SLA for availability has increased,’’ he says. “We don’t account for only a single CDN availability, we account for all the solutions that we create with multi-CDNs.”
Providing a platform of multiple CDNs means they can commit to their users with a better SLA of availability and performance.
“Real user monitoring is great because you’re able to sample all your users from every single location,’’ Kumar says. “Where synthetic monitoring comes into its own is in being able to look at specific, granular areas to diagnose what the problem is.”
For example, you may have a problem with an ISP in a certain location. Real user monitoring (RUM) will be able to tell you that you have a problem, but not why it is happening. You won’t be able to determine if it is because the user is going to a distant PoP, if there is some kind of packet loss happening on the network, or if the CDN is taking too long to go back to the origin server. That’s where synthetic comes in.
So the challenge becomes, how do you manage the deployment and monitoring of the different CDNs?
For eBay, it starts with the big picture. The company works closely with its managed DNS provider to do CDN monitoring in real time for availability, performance, and cost, says Akhtar.
“Since availability is critical, we monitor the payload globally using synthetic,’’ he says. “We take the same payload and do it every time users on eBay go and use our pages to see how our CDNs are performing.”
This gives Akhtar’s team the ability to see which CDNs outshine the others.

CDN Monitoring Best Practices

There is a myth that if you are in the cloud, your availability is assured and you will never have a problem. That’s not the case, which is why robust CDN Monitoring is critical. The way Shrivastava sees it, “Cloud is just someone else’s computer that you’re using. And just like any system can go down, any network can go down. A CDN network is also subject to downtime, which can lead to a disastrous effect on your application delivery.”
It is important to monitor your CDN quite simply because it is a part of your infrastructure, he points out. “Just as you monitor for your application health, your VM health, your network health, you need to monitor CDN network health and system health or delivery health, as we call it,” says Shrivastava, who has previously worked at two different CDN providers.
As part of the process of monitoring your CDNs, you should be checking for availability SLAs or performance SLAs, especially as they may differ from region to region. These are numbers that can be crunched through a synthetic monitoring tool.
At Walmart Labs, performance availability is the main criteria, but Shrivastava’s team also looks at cost. And with a multi-CDN infrastructure, you have the ability to negotiate with your CDN partners because you can steer your traffic anytime you want—something the vendors look to avoid, he notes.
“Just as you monitor for your application health, your VM health, your network health, you need to monitor CDN network health and system health or delivery health, as we call it.”
“So having this data to hand gives you an upper hand when you’re negotiating your contracts as well as being able to optimize your cost of delivery on a regular basis,’’ he says.
For example, in the U.S., most CDNs perform well most of the time. If availability is questionable, then you can steer your traffic accordingly. The catch is being able to use the best CDN possible with availability and performance metrics, as well as the one with the lowest cost. “The cost can vary from a 1:10 ratio, and hence, you can save a lot of cost with your content delivery budget,’’ Shrivastava says.

CDN Monitoring Checklist

This monitoring checklist provides ten key practices for optimal results.

What should you monitor?	How should you monitor?
Check DNS resolution	Monitor whether CDN nameservers are slow to respond,

resulting in performance degradation.
Monitor DNS performance of CDN vs. origin| Correlate dips in availability with spikes in CDN DNS response time.
Check CDN mapping| Monitor domain names mapped to the CDN, domain names overriding IP addresses to that of the origin services, number of hops, and performance metrics of the CDN against origin.
Check cache hit ratio| Monitor CDN cache vs. CDN origin to compare the origin vs. cache KPIs per city, average ping round trip times, and so on.
Measure end-user-to-edge location latency| Track performance degradation between the end user and a specific edge server or across multiple edge servers.
Use performance metrics to uncover bottlenecks| Check page response or availability to find bottlenecks on a page once hosts have been segregated based on first party, CDN, etc.
Balance loads| Ensure optimal load balancing and alerts for unusual traffic surges.
Monitor the last mile network| Verify optimal CDN performance and ensure it is mapping end-users to the relevant PoP.
Track performance across multiple devices, networks, and locations| Ensure consistent performance.
Perform A/B tests| Evaluate how content changes impact end-user experience
Track CDN performance| Keep an eye out for SLA breaches.

Another monitoring consideration is that CDNs don’t have unlimited capacity and they can’t build unlimited capacity because they must stay within the parameters of their PoP within their ISP.
“The moment your traffic goes above their capacity, they will start routing your traffic from alternate regions, which is not really the best thing for your users and for your performance,’’ he notes. If you use two major CDNs you can failover from one to the other whenever something goes wrong.
Additionally, if the performance is not up to par, then you have a redundant platform to switch over your traffic for better performance. Having a distributed CDN or a multi-CDN environment ensures that users are not impacted by a CDN outage.
“I have seen enough outages in my career and that’s why I say that unless you monitor your infrastructure—and CDN is a part of your infrastructure–you cannot be 100% sure when it might go down or affect your application,’ ’ Shrivastava says. “You must ensure that you are getting what you’re buying. Look deeper into metrics and build a better platform offering.”
Another scenario might be an application that onboards your multiCDN platform offering that starts failing for some user requests. There is no way you can track these requests or identify right away that it is the result of a certain limitation on a CDN, he notes. So you need to identify the key components of your application’s requirements and build a regression strategy with your CDNs. That way, if the CDN upgrades or changes its software, you can catch those changes much faster and identify if it’s affecting your application. This can happen for caching issues, he says.
“There are endless things that you can ignore very easily when you are not doing a regression test for a CDN and the complete stack features that you’re offering,” Shrivastava says.

RUM vs. Synthetic

With RUM, you can use the data you capture to understand if a certain CDN is performing correctly or not in terms of its cache ability, its regional PoP allocation, and its correct header delivery.
If you’re completely dependent upon RUM however, you’re not getting insights into all the requests from the various geographies or distributions. That means you cannot look at what those users might face when they come online.
By contrast, synthetic agent-based monitoring gives you the opportunity to constantly test from different networks, PoPs, regions, and countries to identify performance and availability levels. You can also constantly track them so that when a peak time occurs, you will be aware that a PoP is out, a CDN is down or an ISP is having issues, and make decisions accordingly.
You also want to observe cache efficiency, and where there are issues with outages. This lets you drill down to understand if there is latency between the end user to the edge or the edge to the origin, or if it is at the origin, whether any specific infrastructure is causing that latency.
Also on the checklist, you should be monitoring your cache hit vs. cache miss ratio. Even when many optimizations are performed, there is still a lot of content based on cached content stored at the CDN. So it is important to monitor edge latency vs. middle mile latency up to origin. And of course, make sure you’re monitoring 24/7.

Digital-First Organizations Must Monitor Their CDNs

If global traffic is important to your enterprise, you cannot afford downtime, which leads to lost revenue. Monitoring large, complex websites with users spread across the globe, and websites or mobile apps with lots of dynamic content is critical to reducing the time to detect problems and drastically improving mean time to resolution.
CDN Monitoring enables businesses to understand whether a high web page response time is due to objects that are either not cached or not cached for longer duration.
Without it, you will not have visibility into:

Latency
Anomalous usage patterns
Site availability
Content delivery
Network and web performance

The content delivery path has also grown more complex as organizations are increasingly utilizing geo-specific CDNs and real-time performance-based CDN routing. This makes CDN Monitoring indispensable for regularly assessing CDN performance and evaluating any new CDN vendors to fill in potential gaps.
Using analytics and multiple data sources, Catchpoint prioritizes monitoring and observability so that the companies they support can deliver an excellent experience to their end users.

Building your Observability Strategy with Catchpoint

Active observability provides all the insights you need to optimize and maintain consistent CDN performance, delivering the most consistently positive user experience. That’s why it’s important to partner with a visibility platform like Catchpoint with a
Network Experience Observability Solution that can support you on your full CDN journey.
With Catchpoint’s CDN Monitoring, you can:

Monitor the last mile network to verify optimal CDN performance and ensure it is mapping end-users to the relevant PoP.
Track performance across multiple devices, networks, and locations to ensure consistent performance.
Use performance data to optimize applications, including content, code, user journey, and so forth.
Identify any bottlenecks, latency, or availability issues within the application infrastructure.
Perform A/B test changes to content in order to evaluate how such changes impact end user experience.
Benchmark performance of CDNs in a multiCDN environment.
Track CDN performance for SLA breaches.

Catchpoint’s CDN Monitoring strategy will allow you to monitor the performance of your chosen CDN, from vendor selection, to implementation, to continual optimization.
Whether you are an SRE on call or a CDN monitoring strategist, Catchpoint CDN Monitoring helps you proactively track CDN performance and ensure your content is being delivered efficiently, with minimum latency and without compromising digital experience, enabling you to gain a competitive advantage.
It’s time to enhance CDN performance and start making the most of your CDN investment.

FURTHER RESOURCES

How to Maximize Your CDN Investment – eBook
How Fastly Speeds Troubleshooting with Catchpoint – Customer Story
CDN Monitoring – Why You Must Monitor Your Extended Infrastructure – Blog
CDN Monitoring Extends to 5G Mobile Edge Nodes – Blog

Ready to take the next step?
Visitwww.catchpoint.com to check out
Catchpoint’s CDN Monitoring offering today!

About Our Sponsor
Catchpoint is the Internet Resilience CompanyTM . The top online retailers, Global2000, CDNs, cloud service providers, and xSPs in the world rely on Catchpoint to increase their resiliency by catching any issues in the Internet stack before they impact their business. The Catchpoint platform offers synthetics, RUM, performance optimization, high fidelity data and flexible visualizations with advanced analytics. It leverages thousands of global vantage points (including inside wireless networks, BGP, backbone, last mile, endpoint, enterprise, ISPs and more) to provide unparalleled observability into anything that impacts your customers, workforce, networks, website performance, applications, and APIs.
Learn more at www.catchpoint.com

Documents / Resources

| catchpoint Essential CDN Monitoring for Digital-First Organizations [pdf] User Guide
Essential CDN Monitoring for Digital-First Organizations, Essential CDN Monitoring, Digital-First Organizations
---|---