Vertica 9 Frequently Asked Questions

Machine Learning

• What are the new Machine Learning algorithms supported with Vertica and when can customers use these algorithms?

  Here are the following new Machine Learning algorithms and their associated use cases since Vertica 8.0:

  • Naïve Bayes – a popular classification algorithm that can be used on huge data sets to perform multiclass predictions.
    • Spam filtering – Model predicts the probability that an email is a spam or not by using words normally associated with spam. Same can apply to news classification.
    • Document classification – Model determines whether a given document corresponds to one or more categories of document. In this case, it leverages the presence or absence of key words

  • Support Vector Machine (SVM) – a popular classification algorithm that can be used on huge data sets to perform multiclass predictions.
    • Loan processing – Model predicts whether a loan applicant has high or low chance of default based on factors such as income, education, mortgage, credit score, and more.
    • Medicine – Determine the likelihood of contracting common diseases like diabetes based on factors like family history, age, race and ethnicity, weight, height, waist circumference, and body mass index (BMI).

  • Random Forest – a popular classification algorithm that can be used on huge data sets to perform multiclass predictions.
    • Loan Categorization – Predicting whether a loan applicant has high, medium, or low chance of default based on factors like income, education, mortgage, and credit score.
• Can I take a Machine Learning model from one Vertica cluster to another?

  Vertica now offers the ability to copy models between Vertica clusters. A model can be exported to a binary file on disk, and then imported. This way data scientists can train machine learning models on a given Vertica cluster and then deploy it on to another cluster. This new capability is particularly important for embedded analytics customers who want to train their Machine Learning models on their data in Vertica and ship them with their solutions to run on their customers’ clusters.

• Is there a tool to select a suitable Machine Learning algorithm for a customer’s data?

  Choosing, comparing, and applying the right Machine Learning model can be very daunting for a data scientist for greater insights for a given use case. Vertica 9 empowers data scientists by adding a cross-validation function, enabling them to save time by comparing Machine Learning models, avoiding overfitting and getting reliable performance reports on each model for selection purposes

• How do I convert categorical data to numerical data with Vertica?

  Customers frequently work with categorical data such as US state data that is represented by 2 letters. A number of algorithms require users to manually convert categorical data to numerical data. Vertica 9 provides new data-preparation functions to convert categorical data to numerical, enabling organizations to derive greater insight from the data, while improving the quality of analysis.

Vertica in the Cloud

• What are the major cost differences between installing Vertica on-premise and Vertica in the cloud?

  On-premises costs could include software, hardware, data center, networking, data storage, electricity, and labor costs. For cloud, it could include virtualization, network hardware, maintenance, labor (cloud admins), and any shared costs. It’s important to understand that for “on-premises,” it is not just how much new hardware/software is needed to put a solution into place. And for “cloud” it is not just recurring monthly service cost. The real answer is for the businesses to determine what which deployment method is better suited for their data-driven business – cloud or on-premises.

• How do my customers deploy Vertica on Google Cloud?

  As more and more customers deploy big data in the cloud, Vertica is now proven and optimized to run on Google Cloud Platform, in addition to current support for AWS and Azure. This release also makes it easier to deploy Vertica via the Google Cloud Launcher, making it the fastest way to get started with Vertica on Google Cloud Platform.

• How do I deploy Vertica in AWS Cloud?

  Vertica can be downloaded from the AWS marketplace. With Vertica 9, AWS users can leverage the Vertica Management console to provision and add additional nodes to the cluster via an easy-to-use wizard interface.

• Does Vertica support backup to S3?

  Vertica provides backup and restore operations directly with S3 via the Vbr utility, reducing cost and the time to backup a Vertica cluster.

• Is Vertica v9 available for VMware?

  Yes, Vertica is available for deployments on VMware infrastructure.

Eon Mode Beta for AWS

• • 
    What is Eon Mode Beta and what are its benefits?

    With Vertica 9, organizations deployed in AWS can capitalize on cloud economics through rapid compute scaling, combined with affordable S3 storage while enjoying the same fast query processing that they have come to expect from Vertica. Vertica 9 enables this new mode through the separation of compute and storage for AWS deployments by operating the Vertica database in Eon Mode Beta. This new architecture provides rapid elastic scaling of the Vertica cluster, which is ideal for just-in-time workload based provisioning and for scaling concurrency linearly.

• 
  Who is an ideal candidate for Eon Mode Beta?

  An ideal beta customer for Eon Mode Beta will be deployed in the AWS cloud. These ideal customers will also have a need for variable workloads and require running queries that are accessed via dashboards against a window of data. These enterprises are also looking for elasticity – for e.g., retail outlets and bank branch offices that provision for the morning rush hour and need to scale back infrastructure for normal/off-peak periods. Eon Mode Beta is also suited for organizations that have an S3 data lake with data stored in Parquet format as Vertica 9 can now analyze both the hot data in Vertica and cold data in S3 via its support for direct querying of Parquet on S3.

Open Source and BI Tools

• Does Vertica support Hadoop Sentry?

  Vertica 9 introduces support for Hadoop Sentry. This support enables security policies and privileges associated with Hadoop users to govern access control in Vertica, reducing operational burden and centralizing secure access for organizations deploying Vertica in a Hadoop environment.

• What is Hadoop security realms and why does it matter?

  As data grows and more business functions within an enterprise access Hadoop, security is becoming an important aspect within these large enterprises. Security realms help to separate different groups of users as the data lake holds everything from sensitive finance data to clickstream data. For example, some Hadoop administrators need to –prevent marketing to access the finance organization’s data. This access control will now be feasible via Vertica’s support for Kerberos realms that enables granular control over different business units, accessing data residing in a Hadoop data lake.

• What is the latest version of the Apache Spark and Apache Kafka connectors that Vertica v9 supports?

  Vertica 9 supports Apache Kafka versions 0.8 – 0.10 and Apache Spark versions 1.6 – 2.1.

• How do I check Kerberos configuration for Vertica?

  Most data pipeline for analytics do use Kerberos for data authentication and it isn’t easy to configure and troubleshoot issues. Vertica provides Kerberos utility functions to validate configuration and to make recommendations for fixes, simplifying Vertica deployments and for streaming of IoT data into Vertica.

• Can I use Microsoft PowerBI with Vertica for insights into my data?

  Both Microsoft and Vertica have collaborated to ensure that PowerBI and Vertica interact via a direct-connect approach rather than downloading batches of data, resulting in a faster, scalable, and secure solution.

Management and Performance Improvements

• Can I use Microsoft PowerBI with Vertica for insights into my data?

  Since Vertica 8.0, there have been significant management enhancements in the area of provisioning/deployment, security/mobility, and monitoring and management. These enhancements include:

  Provisioning and ease of deployment

  • Quick Vertica deployment and provisioning in AWS Cloud – AWS users can leverage the Vertica Management Console to provision and add additional nodes to the cluster via an easy-to-use wizard interface.
  • New Web browser-based SQL query editor for Vertica

  Ease of managing security/mobility

  • Self-service mechanism for administrator to change IP addresses – With increasing movement of Vertica databases from on-premises Docker containers to clouds, Vertica now enables customers to change IP addresses easily.
  • License compliance – Capacity alert notifications include a built-in proactive management and tracking function within Vertica on license-use for compliance purposes.
  • Kerberos configuration utility – Vertica provides Kerberos utility functions to validate configuration and recommendations for configuration fixes, simplifying Vertica’s deployment and for streaming IoT data into Vertica.
  • Option to disable username auto-complete feature – Gives users control over the username being auto-filled for the Management Console during the re-login process.

  Ease of monitoring and management

  • Monitor and alert on catalog size – Vertica administrators receive email and dashboard notifications about catalog memory growth, ensuring optimal memory utilization and query performance.
  • Add-On query statistics information for Database Designer – Shows users the actual query timestamp, frequency, and run duration on the screen, resulting in users selecting the right query for the Database Designer assessment.
  • Management Console highlights new features in latest Vertica release, empowering customers to be more productive than before.
  • Retain and append existing extended monitoring data – Management Console now offers the added option to retain and append to a storage database that already has existing monitoring data, resulting in data retention options to balance business needs vs. storage resource allocation for monitoring the data.
• What are some of the performance improvements in Vertica 9?

  Vertica runs mission-critical big data analytical initiatives at extreme scale. Thousands of concurrent users access Vertica to extract meaningful insight in the moment. Vertica with every release improves query performance, scales number of concurrent queries, optimizes on resource utilization, speeds up node recovery time, reduces time to ingest data, and more.

  Here are some of the key improvements:

  • Improved node recovery – During node recovery, Vertica will report the corrupted partitions and give users the opportunity to handle these separately via drop/move/swap corrupted partitions, so that the system can quickly focus on the recovery of the good tables.
  • Analytic queries and complex subquery joins with Directed Queries – Organizations often have optimized a query for their environment and data but the Vertica optimizer chooses a different SQL plan that they may consider sub-optimal. Vertica now allows organizations to represent a plan as a SQL statement fully annotated with hints, enabling them to customize it and save it for use with future queries.
  • Hierarchical partition management – This latest version provides greater management of massive amounts of historical data with hierarchical partition management that improves query performance at Exabyte scale.
  • Faster refresh of Live Aggregate Projections – Vertica 9 also includes consistently faster refresh of live aggregate projections, improving Vertica cluster performance and speeding up node recovery.
• What are Flattened Tables and what are its benefits?

  Flattened Tables facilitates the task of performing complex JOINs across multiple tables that are much less cumbersome and much more performant. Analysts can quickly write straight-forward, fast-running queries as if the data resided in one big flat table without the need to alter their existing schemas, simplifying and speeding the process and management of big data analytics in databases with complex schemas.

• Does Vertica store Universally Unique Identifier (UUID)?

  Vertica 9 now supports UUID as a new data type, allowing users to store UUID columns in a space-efficient manner than having to store them as text strings.