Technology & software

Two thirds of retail and e-commerce companies now scraping external data

From ‘The Growing Importance of External Data in the Retail and E-Commerce Industry’ white paper

Retail and e-commerce companies in the UK are increasingly relying on external real-time data to inform business decisions, with 57% now scraping external public web data to gain market insights and help them make more profitable business decisions. This is one of the key findings from the Oxylabs white paper: ‘The Growing Importance of External Data in the Retail and E-Commerce Industry,’ which reveals a huge rise in demand for external data scraping over the last twelve months.

251 senior data decision-makers including Directors of Engineering, CTOs, VPs of Engineering, Chief Data Officers, Heads of Business Intelligence, and Heads of Data & Analytics were questioned to gain insight into the use of external data and the challenges they face. Respondents were selected from a broad spectrum of leading UK-based retail and e-commerce businesses.

The findings reveal that businesses in the retail and e-commerce sector have a growing need for public web data, and the interest in old school sources of information is waning. Companies are becoming more interested in novel pricing strategies, shipping information, stock levels, customer reviews, and customer sentiment.

However, traditional sources, such as internal business systems data, remain relevant and useful for the retail and e-commerce sector. Based on our data, 51% of companies still employ data from traditional sources such as CRM, POS, and supply chain systems. In addition, some 38% use third-party data, market research, and statistical database aggregators, such as Nielsen and Euromonitor.

Andrius Palionis VP of Enterprise Solutions at, comments: “Traditional data sources are slowly losing relevance, although they will continue to be widely used. However, with external data it is now possible to plug in the gaps of knowledge left by traditional sources, allowing for greater predictive accuracy. In turn, this enables companies to significantly improve the ROI of nearly all operations.”

Online retail and e-commerce firms are now able to generate, process, and rapidly analyse data in real-time resulting in actionable business insights. Such data enables highly data-driven organisations to gain a significant proportion of growth and revenue from analytics, according to a McKinsey study.

Gediminas Rickevičius, Director of Strategic Partnerships at, comments:” E-commerce has experienced explosive growth in the past few years. Any data left on publicly accessible websites becomes fair game for companies looking to gain insight into market movements, consumer sentiment, etc. Therefore, it is completely natural that data-savvy companies will jump at the opportunity to improve their profitability.

To download your copy of the Oxylabs’ ‘From ‘The Growing Importance of External Data in the Retail and E-Commerce Industry’ white paper,’ white paper, please visit HERE.