Partnering with Kinetica for Advanced Analytics Excellence

Kinetica, one of HEXstream’s partners, is an advanced GPU-based database which enables advanced filtering, aggregation, and visualization capabilities. It is a columnar, memory-first database that is optimized for both CPU and GPU processing.  It is useful for streaming analytics, artificial intelligence, machine learning, geospatial analytics, and many other functions.  For the last six months, my team and I have worked with a large retail client to help them decrease the response times of their customer-facing applications.  Building these applications using cutting-edge technologies like Spark, Kafka, and Java Spring Boot has increased customer satisfaction by making them much faster and more responsive.

We are supporting critical customer-facing applications that handle more than 100 million read requests and ingest more than 100 million records every day.  During these first six months, our team has built a couple of applications for the client which save time spent processing and aggregating using the Kinetica database’s powerful GPU-accelerated capabilities.  One of the applications supports the real-time ingestion of records from Kafka into Kinetica and performs complex queries, including group by and aggregations, and provides the response to the client in microseconds to milliseconds. We have also worked on an extension project to update half a million records every hour and provide the response to the downstream applications in a second.

Some highlights from our current engagement:

  • One of our newly built applications programmatically loads data into the Kinetica database, distributed across all the ranks and capability to read records directly from respective ranks (shards).
  • There had been an ingestion application that ran once a day, taking more than three hours to complete the job. We helped the client to reduce the time taken by the job by 90%, which in turn saved 160 minutes of resource utilization and time. Now, it takes only 20 minutes to complete data ingestion instead of three hours, and our client is very happy with the work we have done.

As an application designer and developer, I love the fact that Kinetica’s product development team is always available to me and my team for collaboration and innovation. We have direct access to their entire development team, including their CTO who, despite his very busy schedule, always makes himself available to listen to our ideas and clients’ feature requests.

When we face a challenge, we all group together and find a solution as a team. Case in point, we recently had a use case where the client was having issues understanding database response times. We quickly decided to create an application to ingest real-time database log data after parsing them and storing them in the Kinetica database.  We then created charts and dashboards to monitor database logs in a real time. We completed this application, deployed it across the entire cluster, and started monitoring them, all in less than a week. This helped us to capture database response times which were always in the range of microseconds to milliseconds and built trust between client and customer applications.

Kinetica is a great partner to HEXstream! We love using Kinetica’s robust and nimble platform to serve our clients’ real-time analytics needs. Together we are helping our clients handle voluminous and dynamic data and build innovative solutions that were not possible before. We at HEXstream look forward to continue working with Kinetica to help our clients for many years to come.

About the Author

Manikanth Koora

Manikanth Koora

Manikanth Koora is a Senior Software Developer for Advanced Analytics at HEXstream. He has many years of experience in various programming languages, big data, machine learning, blockchain, and real-time data analytics.  He completed his Master of Science in Information Technology at Southern New Hampshire University and has completed many certifications for big data and cloud technologies.  Manikanth enjoys learning new technologies and watching TV shows.

Back to Blog

Related Articles

Three Questions to Ask When Evaluating Software ROI

In the weeks, months, and years following “successful” implementations, the residual costs of...

CPUs vs. GPUs: Why You Should Care

As the unwavering demand for analytics increasingly strains processing systems, it is important to...

Using Oracle Work & Asset Cloud Service to Assess Utility Supplier Performance

As complex, multifunctional organizations, utilities usually enlist the services of various...