SigmaStream and Hazelcast Helps the Energy Industries Save Millions
Headquartered in Houston, TX, SigmaStream develops systems for energy companies, specifically in the oil and gas industries, to collect, persist, transport and analyze a broad range of operational data. SigmaStream specializes in the capture and analysis of high-frequency data streams generated in the drilling process.
Drilling for Data to Deliver Highly Complex, Streaming KPIs
A drilling rig is one of the most recognizable symbols of the oil and gas industries. The operating costs of a drilling rig are very high and any downtime throughout the drilling process can have a significant effect on the rig operator’s bottom line. The rigs are equipped with a large number of sensors to detect small vibrations during the drilling process. SigmaStream can gather, monitor and analyze real-time high-frequency data, and uses this to feed analytic systems that enable operators to immediately act on the streaming data to prevent costly, catastrophic failures in the drilling process.
The data generated from these sensors is for machine consumption; at any time during rig operations, up to 60 to 70 channels of high-frequency data enters the system at various frequencies. With no system in place to process, store and analyze the data, many critical events went unnoticed or even recorded.
Facing a large amount of “data exhaust,” which is discarded data, the founders of SigmaStream recognized an opportunity to develop and provide an event-based technology solution that addressed a pervasive and expensive problem for drill operators.
After contracting with its first customer, SigmaStream quickly realized that acquiring the data and making it transportable was insufficient. To address this challenge and offer a complete solution unavailable from any other company or source, SigmaStream integrated its operational platform with Hazelcast Jet. Analytics are now part of the process, which morphed into a data-gathering and a complete closed-loop control automation platform. SigmaStream integrates Jet with a visualization tool called Hummingbird, which is capable of analyzing and displaying high-frequency data.
The Hazelcast Solution
One of its clients was using a proprietary stream processing product, which was not a financially feasible solution for SigmaStream, so they searched for an alternative streaming engine and discovered Hazelcast. They evaluated Hazelcast Jet and were impressed by its relative simplicity and ease of use, as well as with the light footprint of the product.
“My team and I were attracted to Hazelcast because it is lightweight and easy to integrate,” said Koduru. “It was also more affordable compared to other products. We installed it, tried it and very quickly decided Hazelcast Jet was the solution SigmaStream and our clients needed.”
SigmaStream has been a Hazelcast Jet customer since 2017. The highly time-sensitive data stream entering SigmaStream’s analytics engine and the associated insights are stored in multiple NoSQL databases, and the analytic engines are homebuilt R and Python.
With Hazelcast Jet, SigmaStream can record or sample high-frequency data or events that occur, and then apply its proprietary algorithms to make very fine-tuned adjustments to the drilling process. An example of this is the realtime adjustment of the RPM of the drilling string and bit; this type of drilling must be managed at a very exacting level to prevent equipment failure and costly delays to the drilling process.
Typically, operational stakeholders, backed by their technical teams, contact SigmaStream to help extract more information from their existing data. Previously, there wasn’t a pre-existing solution from an operational KPI perspective. The combination of SigmaStream and Hazelcast Jet technologies allows for the streaming of highly complex KPIs.Additionally, SigmaStream provides its clients with dashboarding for real-time data visualization and interpretation.
"Because of the integration of Hazelcast Jet with our technology, our clients are now able to separate the data into windows and perform streaming analytics on it that was previously technically impossible." "Hazelcast Jet is exceeding our expectations; the training is outstanding and we have just moved onto the pipeline API version of Jet."— Hari Koduru
The operational and business benefits are in being able to gather, monitor, analyze and apply delicate tweaks to drilling parameters in near real-time, allowing clients to manage physical resources better through high-frequency feedback on a per-well basis. They are often able to reduce the drilling time by as much as 20 percent, from a typical 15 days to 12 days.
The most significant benefit for SigmaStream clients is the creation of streaming KPIs using Hazelcast Jet with its operation stack. By doing so, SigmaStream is reducing the amount of time to drill a well from spud to the true vertical depth or TVD, potentially saving its clients millions of dollars.
According to Koduru, the company is planning to expand its customer base to other industries that would benefit from the capabilities of SigmaStream and Hazelcast Jet to gather, monitor and analyze high-frequency streaming data.
“We are focused on using artificial intelligence and machine learning with Hazelcast Jet. This has the potential to be a big opportunity for SigmaStream because of the many rapidly spinning parts in these industries generating considerable amounts of high-frequency data that we can bring insight to” said Koduru.
While this particular use case revolves around streaming high-frequency data for oil and gas drilling, the capability delivered by a high-speed in-memory stream processing engine is applicable across a broad array of industries and applications.