High Velocity Analytics in Real-Time

Mike Allen, Zoomdata’s Vice President of Products, recently joined Eric Kavanagh, the host of the DM Radio, in discussing Streaming Analytics. The webinar, High Velocity – The What, Why, and How of Streaming Analytics, examined the need for Big Data Streaming Analytics and Real-Time Data Visualization. During the webinar, Allen stated:

“Streaming Analytics is a mainstream requirement of any modern Business Information platform. To be able to do modern Business Intelligence you need to be able to deal with data streams.  So, this leads to the concept of a kind of a Real-Time BI and high velocity Analytics.”

According to Allen, there all kinds of applications that can take advantage of Streaming Analytics and such platforms are the future. He said every BI platform will ultimately have to be written on Streaming Architecture, “to handle what’s coming, in terms of Big Data sets, like Hadoop.”

Streaming Analytics allows organizations to create Real-Time Analytics computations while processing data that is streaming from social media, sensors, applications, devices, and websites. Streaming Analytics supports quick and efficient time-sensitive processing, and language integration. It does this by making use of a SQL variant, and simplifies stream processing systems.

Zoomdata and Analytics

Zoomdata provides software that allows researchers to see and interact with all the data available to a modern organization, in real-time. It offers an intuitive, simple, and collaborative way to work with the data visually, and to solve business problems quickly. This is accomplished through the use of patented micro-query, microservices, containers, and data sharpening technologies. Additionally, Zoomdata makes it easy to embed Visual Analytics into any application. Organizations can use its interface, while developers are able to use it in embedding Visual Analytics into custom applications.

In a recent DATAVERSITY® interview, Nick Halsey, President and CEO of Zoomdata, said:

“With Zoomdata, a researcher can analyze data, and do calculations on it, projections, and whatever kind of Analytics and Data Visualization they want to. It provides composite view of all of the data, from all of these different sources in real-time.  You can set the update, refresh rate on our visualizations to whatever speed you want. Every second, every five seconds, every 10 seconds, whatever is appropriate for your situation.”

Zoomdata’s Data DVR helps to visualize and interact with the data, similar to the way streaming video technology is used. Data DVR combines the velocity of Big Data with the ability to fast forward, rewind, pause, and replay. This helps teams to identify and track recent anomalies. The Data DVR computes rolling or static windows, continuously allowing researchers to monitor data coming from both historical and real-time data streams.

According to Halsey, Zoomdata focuses on two basic markets:

• Software and technology businesses wanting to embed Analytics and visualizations into their applications.
• Businesses of any industry wanting stand-alone Analytics and visualizations for modern data platforms.

Big Data Research and Data Visualization

Big Data Visualization tools are necessary for fast, self-service, Visual Analytics. Such tools are needed to support a broad range of traditional and modern data sources, making it simple to compare historical trends with real-time streaming data, and making it easy to merge data from multiple sources.

Analysts using Zoomdata, said Halsey, “can easily develop dashboards that combine multiple data sources, and share their results with other members of the research team.” Its patented data sharpening technology converts Big Data into informative visualizations in seconds. As the Zoomdata stream uses more and more data, by way of various micro-queries, the visualization becomes sharper.

Traditionally, Business Intelligence Data Visualization tools have emphasized gathering and aggregating data in preparation for research. According to Halsey, Zoomdata streamlines this step by querying the data directly at its source, filtering it, and providing intelligent data, which in turn, provides insights more quickly and efficiently, than unfiltered Big Data. “With Zoomdata,” he said. “Big Data research can cover data ecosystems that include modern resources,” such as:

• Traditional relational databases
• Spark
• Hadoop
• NoSQL
• Cloud data stores
• Search engines
• Streaming sources

Open Source Apache Spark

Zoomdata has been certified for integration with the Apache Spark system. Zoomdata applies Spark data processing as a complementary process within the server. The program queries original data sources, filtering data close to where it is stored. Filtered results are retrieved from the source, and Spark stores the data as frames. Whenever a new request for data is submitted, it recovers the data from Spark, whenever possible.

Microservices Integration

Most “modern” distributed applications are designed to work as independently deployable Microservices, which are distributed over clusters of hardware. Recent advances in such things as Cloud Computing, Data Containers, continuous integration, and Data Virtualization, have made the software solutions now available quite different from those used a few years ago. The software of today is focused on delivering modern distributed applications. “The incorporation of Microservices allows for easy scaling by adding more servers, as needed, and improved flexibility by allowing for swap outs and upgrades,” said Halsey.

Containers carry discrete amounts of application logic and use only the minimum resources necessary for doing their job. Microservices, like Containers, are not a new concept, but before containers were developed, building applications while using microservices architecture was remarkably difficult. Containers can enable Microservices, which is a significant change in how applications are built.

Consequently, many organizations chose to stay with the traditional monolithic applications, even while knowing that the monolithic practices of building applications are inflexible. Containers changed the situation by making it easier to build and maintain applications, based on Microservices. Microservices provide an efficient method for working within the Cloud. Hasley commented:

“We know that people are deploying in the Cloud. Half our customers are deploying in the Cloud.  So, a real Microservices-based architecture where you can, you know, elastically scale and manage services, has become a necessity.”

During the webinar, Mike Allen noted that:

“Zoomdata is a modern BI platform that has a lot of powerful capabilities for dealing with both Big Data, fast-moving data, and a lot of different varieties of data. So, its front end is web-based, and the back end is all about Microservices and Smart Technology. And we offer lots of different dashboards.”

Group Discussion Podcast

There is also a related DMRadio podcast titled Finders Keepers: The New Age of Data Discovery sponsored by Zoomdata in association with DATAVERSITY®, which had additional guests who discussed the topic of Analytics, Data Discovery, Big Data, and new approaches to exploring data sources. The guest list included:  Jen Underwood of Impact Analytix, Ashley Kramer of Alteryx, Ian Fyfe, and Deborah Baron of Tom Sawyer Software. The host was Eric Kavanagh, who is also CEO of the Bloor Group.

Photo Credit: Graf Vishenka/Shutterstock.com