Azure, Apache Kafka, Apache Spark
The client is a leading U.S provider of pharmacy software solutions with a focus on long-term care. The company’s mission is to improve healthcare and strengthen the role of independent pharmaceutical organizations by providing affordable and high-quality specialized software.
The client’s flagship product is a SaaS platform for pharmacy management that allows handling patient information, orders, pharmaceutical product transfer and delivery, billing, and electronic prescriptions.
The client came to NIX United for a data analytics solution that can be used inside the company for various products and also can be provided as a SaaS for businesses.
The NIX team’s scope included the following tasks:
Create robust and secure data storage
Set up fast data processing for future analytics
The NIX team was responsible for collecting data coming from the flagship product, as well as processing and storing it for further analytics and the development of predictive models.
The flagship SaaS software has an architecture with more than 100 microservices. The two main data sources are the Apache Kafka streaming platform and Azure data lake.
On the backend, Apache Kafka uses operational PostgresDB and MongoDB as data sources. For data transfer, we used Debezium—an open-source distributed platform for change data capture. The data gets into Kafka in the form of JSON messages, and for each microservice, we create a separate topic.
As a next step, we took data from Apache Kafka as a data source for the Azure data lake. Using custom PySpark scripts, we set up a data stream from Kafka that is converted into parquet-type files and stored in the data lake.
As we mentioned above, to process streaming data from Apache Kafka, the NIX team used PySpark scripts. From Kafka, raw data goes in two directions—temporary Postgres storage and the data lake.
Data from all sources is stored in PostgreSQL tables and as parquet files in the Azure data lake. For the ETL process, we used Apache Airflow and SQL functions to extract, process, and store raw data. After that, we used SQL functions to process the raw data, remove duplicates and empty rows, and save the data to permanent storage.
The NIX team was also responsible for legacy data processing and transformation from the client’s claim processing system. The legacy database has 8 folders, each containing 100 days of data, with about 1-1.3 million claims per day.
The data was originally compressed and encrypted, so we converted it using a prebuilt Java package in Python and transferred it to the database.
The client received a robust data engineering and analytics solution that enables real-time reports.
All data generated inside the company walls now serves as a bedrock for analytics.
10 experts (Project Manager, 6 Data Engineers, 3 QA Engineers)
Azure, Apache Kafka, Apache Spark, Apache Airflow, PostgreSQL, MongoDB
Creating a platform, ARTiFACTS Verify, designed to track chain-of-custody throughout the entire drug supply chain all in one spot.
Robust data analytics solution with up-to-date predictive models that provide real-time insights and recommendations to healthcare providers.
NIX reduced testing cycles by 4 times through automation, resulting in decreased expenses associated with maintaining legacy code.
Web platform and a mobile app with verified, point-of-care information for clinicians.
NIX team established a well-curated auto testing practice for Thomson Reuters, ultimately resulting in more reliable and robust software products for the client.
SEO activities to enhance website structure and attract more organic users necessary for the company’s consistent growth.
Blockchain-powered SaaS solution designed to help medical companies work with patient data in a secure way.
Microservice-based web platform for all-in-one pharmacy management to increase efficiency and improve patient care.
Power BI analytics solution empowers the client with comprehensive analytics that boosts the productivity for the teams and individuals.
See more success stories
Our representative gets in touch with you within 24 hours.
We delve into your business needs and our expert team drafts the optimal solution for your project.
You receive a proposal with estimated effort, project timeline and recommended team structure.