With a significant presence in professional markets worldwide, a well-known website gathers a massive amount of data related to its many products and services. In order to provide meaningful analytics to its global customer base, the company relies on data storage and management technology – an analytics database – that supports many thousands of concurrent customer interactions. Because the performance of the technology directly affects the customer experience on the website, it is vital that the database provide rapid responses without errors.
The company’s new analytics experience is built with consistency and usability in mind. The response time for queries needs to be in the sub-second range in most cases, but the scale of the data driving the user experience is enormous and requires special big data technology in order to satisfy requirements and expectations.
Proof of Concept (POC) Details
With a need to improve the existing data analytics capability, which was based on Google Big Query, the company’s engineering team decided to test two software-as-a-service (SaaS) solutions – Vertica Accelerator and another leading solution – to determine which would best satisfy its criteria around performance and concurrency.
The team devised several tests to simulate effective performance for a multi-product analytics and insights tool – considering both current and future needs. The team was primarily interested in response time, but another decision factor was the cost of the new system. The team identified seventeen queries to be used in the benchmarking exercise – ranging from requests submitted through the user interface to subqueries.
The engineering team tested both SaaS solutions using data volumes typical for daily traffic, as follows:
|Customers (determined by the number of transactions under that customer)||Traffic Percentage|
|Date Range||Traffic Percentage|
|Under 30 days of data||95%|
|Between 30-60 days of data||2%|
|Between 60-90 days of data||1%|
|More than 90 days of data||2%|