Free data sources for researching hosting markets

Web hosting companies often work on an international scale. When doing research on hosting markets that you don’t already know inside and out, you need data sources, and we have collected a bunch of them in this post. We hope you’ll find them useful.

Internet user statistics

• Internet penetration over time (countries and regions), courtesy of the World Bank.
• Internet users by country at Internet Live Stats.
• Internet users (countries and regions), usually fast to update with the latest numbers, courtesy of Internet World Stats. (Pro tip: You can use the Internet Archive’s Wayback Machine to explore numbers ranging all the way back to 2002.)
• Internet users by language. The internet is international, so maybe you’d rather target languages than regions or countries. Here’s the size of those markets, once again courtesy of Internet World Stats.
• And for completeness, here’s a list of the number of overall speakers per language, and another that also lists where they are the official language.
• The world in 2015 (PDF), a report from ITU that has plenty of internet usage numbers.
• The OECD Digital Economy Outlook 2015, a massive report about the digital economy and full of relevant data and trends related to internet usage.

Internet infrastructure data

• Broadband internet subscriptions for both cellular and fixed broadband, by country. Courtesy of Wikipedia (using ITU data).
• The OECD Broadband Portal, with tons of downloadable data that you can examine.
• Ookla used to have a great portal of freely available data based on their famous speed test, but it’s now a paid service of some kind. However, you can find a small subset of free data on their “Speedtest Awards” page, divided by country.
• Akamai operates one the world’s largest content delivery networks and their State of the Internet report has plenty of interesting data. This includes connection speeds by country, broadband adoption, mobile connectivity, IPv6 adoption, as well as security-related information. There are interactive charts, but the real meat is in the downloadable PDFs.
• Global real-time web traffic monitor by Akamai, a visualization based on data from its CDN.
• 4G LTE penetration by country, a very nice report page from OpenSignal that also lets you dig into providers and more. The linked page is for Q3 2015.
• Keynote’s Internet Health Report. Real-time status of major Tier 1 US internet backbones as measured by Keynote, a monitoring service that’s been around for a very long time.
• Ping times between cities, courtesy of WonderNetwork.
• Internet exchange points (IXPs) by size. Keep in mind that the list is based on publicly available data, so it’s not complete.
• DNS root server locations and numbers.
• We should mention this report from ITU about internet usage (PDF) that we included earlier, since it belongs in this category as well.

Web technology trends and statistics

• BuiltWith has a ton of statistics about web technology usage, covering everything from CDN providers, CMSs, Javascript libraries, and much more. They even have stats for hosting providers by country and some other interesting data samples.

Domain name and hosting statistics

• Top registrars, registries, DNS hosts and overall gTLD numbers from Webhosting.info.
• Top web hosts based on gTLDs (worldwide and by country), courtesy of Webhosting.info.
• Statistics for the new gTLDs (numbers, registries, registrars, etc).
• Another view of the new gTLDs.
• Want to more closely research a specific ccTLD or gTLD? You’re probably best off getting the info directly from the registry responsible for that top-level domain. IANA has a list of all TLDs and their registries that’s a good starting point.

Website and web server statistics

• The people over at Netcraft have been doing their monthly web server survey for ages. Want to see how many websites there are in the world, an estimate of how many web servers, or how popular Apache is compared to other web server software, this is probably your first stop.
• HTTP Archive keeps track of trends in web technology, mainly the composition of websites. It’s a good resource if you want to find out things such as average website sizes, number of requests per page, etc.
• Alexa has been ranking websites for a very long time. Here you can find the global top 500 sites, as well as the top 500 per per country. (There’s also a web service, and a downloadable global top 1M list (CSV).)
• Top sites as measured by Quantcast, by country. Note that you can download the site lists as well.
• Majestic Million, worldwide top million sites based on inbound links. There’s also a downloadable version.
• Moz Top 500, the top 500 websites (or at least domains) in the world based in inbound links.

Web browser and OS statistics

• Web browser statistics (global, regional, and by country), courtesy of StatCounter. Also provides OS numbers and distribution between desktop/mobile/tablet usage. (Pro tip: download the spreadsheet for a report if you want more data than what’s available in the graph.)
• Wikipedia has a page with more web browser usage stats from various sources.

Economy and buying power

While this is hardly technical, evaluating the buying power of a population can be highly relevant when considering new markets.

• GDP per capita, by country, over time. Once again courtesy of the World Bank.
• GDP per capita, by country, from several sources. Courtesy of Wikipedia.
• The Big Mac Index, another purchasing power indicator from The Economist.

Honorable mentions

• The CIA World Factbook is a classic and remains a great resource for getting an overview of a country. You’ll find everything from economic factors, communications, population statistics, and much more. Quite comprehensive.
• Wolfram|Alpha has a ton of data you can query and combine in interesting ways, and this includes internet-related data as well as population statistics, and so on. Some tech-related examples here.
• Wikipedia. Because it’s Wikipedia. Looking up a country will usually give you a ton of information in a relatively compact format. Not to mention that if you’re looking for a list, any kind of list, someone, somewhere, has made a Wikipedia page. 😉

There is more, of course

Since we’re in the hosting industry ourselves (we make hosting software), we tend to think a lot about the global hosting market and the opportunities that exist. This is the beauty of the internet, isn’t it? It has given web-based companies a reach that would have seemed impossible just a couple of decades ago. We come across interesting resources all the time, so this post was our way of sharing some of that with you.

However, the internet is a vast place and one heck of a resource, so of course what we have listed here is just a drop in the ocean. As the saying goes, Google is your friend.

Do you know of a great hosting-related data source that we didn’t list here? Drop me an email: peter -at- atomia -dot- com.