Looking for clear diagnosis
The Quest Diagnostics team wanted to capitalize on its existing skillsets and capabilities, while building out a scalable solution that would enable it to move to the next level of clinical analysis. Potentially this could involve heavy investment in data warehousing solutions, which Quest Diagnostics was keen to avoid.
Nilesh Saratkar explains, “We have a 15-year background in traditional data warehousing experience, and deep knowledge of the tools required for a big data environment. We soon realized that IBM’s big data analytics offerings could solve our technical and performance challenges, and enable new capabilities for our diagnostics services.”
Quest Diagnostics decided to extend its diagnostics analysis services by adopting some of the capabilities of IBM BigInsights. BigInsights supplements a standardized, Open Data Platform-compliant Hadoop cluster with a user-friendly data management tool that helps to extract, transform and load [ETL] the diverse clinical lab data-sets. It also provides an analysis tool that helps Quest Diagnostics gain insight from its data easily and efficiently.
“IBM is particularly good at providing services we need: grid computing, robust tools, and analysis of very large data-sets,” remarks Nilesh Saratkar. “By using these tools, which are available with IBM BigInsights, our data scientists and data engineers can spend less time writing complex Hadoop MapReduce jobs, which saves a lot of development and analysis time. To put it in another way: why write code manually in a complex language like Java, when you can use the IBM tools with minimum up-front investment and no additional training cost?
“The IBM BigInsights platform also gives us access to a host of new capabilities, including BigIntegrate for ETL processing. We know that around 70 percent of the time is spent on resolving basic data quality and data integration challenges, so building data quality and data governance into the solution gives us a huge advantage—not just operationally, but also to help us ensure compliance with HIPAA regulations and other data privacy standards on the use of confidential patient health information. As early adopters, we are working closely with IBM to help them develop and refine the BigIntegrate solution.”
Because the company’s data scientists often need to explore new data-sets totaling several terabytes of data, Quest Diagnostics can also make use of IBM BigSheets to provide a spreadsheet-like view of data while it resides on the Hadoop cluster.
“With BigSheets, our data scientists can understand the structure of a new data-set, follow the relationships between data quickly, and see the lineage without any help from the IT team or from developers,” says Nilesh Saratkar. “Due to the size of the data-sets, this would be impossible with traditional desktop-based spreadsheet tools.”
Planning healthy outcomes
Quest Diagnostics’ journey into big data analytics is ultimately designed to help physicians gain new evidence-based insight into treatments and results, improving patient care. By taking real-world inputs from multiple sources in varying formats, and using BigInsights to build a structured data-flow and harness powerful analytics, Quest Diagnostics is finding new ways to deliver results.
“We estimate that the integrated capabilities of BigIntegrate could perhaps take away the majority of the data integration work required to manage and match the data inputs that produce the ‘golden records’ we need for our diagnostic analyses,” says Nilesh Saratkar.
Alongside the medical diagnostics, Quest Diagnostics also explores general business analytics, using the same powerful tools to examine its engagement with physicians and hospitals. For example, by looking at the behavior of its clients, Quest Diagnostics can predict when a group of hospitals may be about to switch to another lab services provider, and take action to retain the customer relationship. Similarly, by trawling the data to identify changing levels of demand for different types of tests, Quest Diagnostics can identify new opportunities that help to grow the business.
Nilesh Saratkar concludes: “We are all about creating a scalable big data platform, using Hadoop as a data lake, completing analyses and pushing insight back to the business. IBM best practices for big data technologies are vital to Quest Diagnostics’ continuing progress, and we know that with IBM we have a partner that listens and provides answers quickly. Additionally, IBM has deep resource and support capabilities, a significant advantage to Quest Diagnostics as we continue to grow.”
“IBM BigInsights and BigIntegrate offer robust, reliable big data analytics components on a fully integrated platform, helping Quest Diagnostics to deliver diagnostic results easily, rapidly and cost-effectively, while meeting relevant standards for patient privacy and complying with HIPAA regulations. Any time we have new use cases or business opportunities, we reach out to IBM, look at our existing technology stack, and consider whether we have the components we need. We communicate our ideas and IBM comes up with possible solutions, and that’s how we grow together.”