Enhancing Clinical Trials with Scalable AWS Infrastructure and Automated CI/CD

NIX helped the client maintain industry leadership, providing the product with a scalable, automated, and resilient IT infrastructure.

Business Domain

Pharmaceutical
Service

Web Development, Mobile Development, Cloud Services
Technologies

AWS, Python, Terraform

Business Overview

The client is a leading provider of cloud-based software solutions for the clinical research industry that offers an all-in-one platform, both for web and mobile users, that streamlines study start-up, pipeline management, and business intelligence. Their technology connects research sites, CROs, and sponsors, facilitating efficient trial execution at scale.

Challenges

The inability to scale and automate according to business needs posed a threat to their competitive edge. Competitors with modernized, automated solutions could have overtaken their market position, and regulatory compliance risks could have risen due to inconsistencies in infrastructure management. Without proper clustering and load balancing, system failures could have disrupted ongoing trials, compromising data integrity and patient safety. This could lead to severe risks, including reduced platform reliability, slower study start-up times, and increased downtime due to inefficient deployment processes.

To maintain industry leadership and support long-term growth, the product required a scalable, automated, and resilient IT infrastructure.

Implementing clustering and CI/CD was challenging due to legacy infrastructure and manual processes. Clusters required load balancers, EFS, secure secrets management, and ElastiCache. Automating CI/CD with GitHub Actions and Terraform from scratch involved building a full pipeline and migrating data to centralized EFS while ensuring high availability.

Solution

Architecture and Clusterization

We migrated the client’s existing applications to the new infrastructure without altering the core business logic. NIX began by redesigning the client’s infrastructure for high availability and scalability. This involved migrating from individual EC2 instances to a robust cluster architecture spanning three availability zones. We introduced an Application Load Balancer to distribute traffic efficiently and integrated Amazon EFS for centralized, flexible application storage, replacing the previous instance-specific storage. To enhance security, AWS Secrets Manager was implemented for secure credential storage. For efficient session management, we integrated Amazon ElastiCache, ensuring seamless user experiences during peak loads. This architectural overhaul provided a foundation for automatic horizontal scaling, replacing the manual vertical scaling previously used.

Automation of CI/CD with Terraform and GitHub Actions

We built a complete CI/CD pipeline from scratch, using GitHub Actions for automation and Terraform for Infrastructure as Code (IaC). Terraform streamlined AWS provisioning, ensuring consistent deployments while reducing manual setup and error. GitHub Actions automated build, test, and deployment processes, enabling continuous integration and delivery. We migrated build and deploy scripts from PHP to Python for better maintainability and efficiency. Automation extended to provisioning new client environments, where GitHub Actions deployed infrastructure, applications, and databases based on predefined parameters. Leveraging AWS and Terraform Cloud, we ensured scalability, high availability, and cost-effective infrastructure management while minimizing risks from manual errors and configuration drift.

Deployment and Scaling

The transition to a cluster-based architecture and automated CI/CD enabled automatic deployment of new applications and infrastructure. We removed the previous EC2 cluster, which relied on manual instance management, and implemented autoscaling groups that automatically add or remove instances based on demand. This ensured high availability and optimal performance, even during peak loads. We also automated the deployment of new clients, streamlining the onboarding process and reducing time-to-market.

AWS Tools

The following AWS services were tailored to the client’s needs and integrated to modernize and automate the client’s cloud infrastructure, ensuring scalability, security, and efficiency:

Virtual Private Cloud (VPC): Established an isolated network environment, allowing precise control over network configurations, IP addressing, and subnets.
Subnets:
– Public Subnets: Hosted web-facing resources requiring direct internet access.
– Private Subnets: Secured internal components, including backend services and application logic.
– DB Subnets: Optimized for database security and compliance requirements.
NAT Gateway: Enabled secure outbound internet access for private subnet instances while blocking inbound traffic.
Internet Gateway: Connected the VPC to the internet, facilitating external communication.
EC2 Instances with Auto Scaling Group (ASG) in Multi-AZ: Deployed virtual servers with auto-scaling to dynamically adjust resources based on traffic load, ensuring high availability and failover protection across multiple Availability Zones.
RDS for MySQL with Read Replicas in Multi-AZ: Implemented a managed database with read replicas for performance optimization and data redundancy.
Secrets Manager: Secured and managed sensitive credentials, such as database passwords and API keys, improving compliance and security.

ECS Cluster on Fargate: Deployed and managed containerized applications without managing underlying infrastructure, simplifying scalability and maintenance.
EFS: Provided a centralized, scalable file storage system, enabling efficient data access across multiple instances.
API Gateway: Managed and secured API interactions, enabling seamless communication between services and external integrations.
ECR: Stored and managed Docker container images, ensuring efficient container deployment workflows.
ElastiCache: Implemented an in-memory caching layer to accelerate application performance by reducing database query loads.
IAM Roles: Defined precise access permissions for AWS resources, enhancing security and governance.
KMS: Managed cryptographic keys for secure data encryption, ensuring compliance with data protection regulations.
VPC Peering: Established private connectivity between VPCs, enabling secure inter-service communication without exposure to the public internet.

Outcomes

NIX’s comprehensive modernization efforts delivered significant results and tangible benefits for the client’s project:

20–25% reduction in total cost of ownership (TCO) for computing services and databases
30–40% cost reduction on databases by transitioning to Graviton (g-type) instances
3–4-times increase in load capacity, enabling substantial growth over the next 3–5 years
Significant reduction in deployment times through automated CI/CD pipelines
Enhanced high availability and reliability of the platform
Elimination of previous bottlenecks through auto scaling and cluster architecture

Lessons Learned

One of the key takeaways was the importance of gathering detailed client requirements from the very beginning. Understanding the client’s infrastructure limitations, business goals, and long-term scaling needs upfront allowed us to design a tailored solution that resolved current bottlenecks and prepared the platform for future growth.

Our team deepened its knowledge in AWS clustering, ensuring high availability and fault tolerance by distributing workloads across multiple availability zones. We mastered the design and implementation of automated scaling strategies, transitioning the client from a rigid, manual infrastructure to a fully-automated, self-scaling environment.

Our proficiency in CI/CD tools like GitHub Actions and Terraform grew significantly. By leveraging IaC, we streamlined deployments, enabling rapid provisioning of new environments while maintaining consistency across regions.