47% Ad Spend Savings: Real-time ML Optimization for Leading DSP

NIX developed an ML-powered real-time bidding system, enabling the client to save up to 47% on ad spend, optimize campaigns, and improve decision-making.

Business Domain

Marketing & Advertising
Service

Data Engineering, AI, ML
Technologies

Python, Angular, LightGBM

Business Overview

Our client is a leading demand-side platform (DSP) specializing in mobile app retargeting for global brands. Their product delivers a comprehensive marketing funnel designed to retain, re-engage, and increase conversions by prompting users who previously interacted with the client’s apps to return and become active again.

This powerful app retargeting solution is driven by a sophisticated machine learning (ML) algorithm and seamlessly integrates with major attribution partners such as Adjust, Appsflyer, Kochava, and Singular for enhanced automation.

Project Scope

The client approached the NIX team to enhance their retargeting capabilities by leveraging real-time data for optimized engagement strategies. The primary objectives were to:

Develop machine learning algorithms to maximize successful user retargeting
Expand the solution beyond basic retargeting to drive key actions like app installs, in-app payments, and overall user engagement
Ensure real-time prediction delivery within a strict 20-millisecond timeframe

This solution would directly contribute to efficient ad delivery by accurately predicting user response, ensuring optimal ad placement, and ultimately minimizing advertising spend while adhering to client budget constraints.

Solution

We’ve developed an ML pipeline that delivers real-time insights into user behavior and ad profitability. This allows for optimizing campaign spending and achieving significantly more effective targeting. The system analyzes millions of user records daily, strategically matching them with auction bids to power highly accurate predictive models, ensuring your advertising budget is maximized.

To ensure continuous performance and adaptability, we’ve also implemented an advanced machine learning operations (MLOps) platform. This platform automates critical processes like data preparation, model retraining, and seamless ML pipeline deployments with hot-swap capabilities. The result is uninterrupted efficiency, rapid scalability, and the ability to adapt instantly to market changes, all while providing comprehensive real-time and long-term performance metrics for complete transparency.

Architecture Design

Key Deliverables

Automated data preprocessing and feature engineering, including intermediate ML models for enhanced aggregations and probability-based features
Robust ML pipeline to ensure automated retraining, continuous monitoring, seamless hot-swap deployment, and high-speed, efficient, real-time inference
ML pipeline based on LightGBM’s and NN’s that provides comprehensive campaign optimization across open, win, payment, and profit targets, supporting diverse strategies.
Intuitive monitoring dashboards delivering real-time analytical insights into campaign performance
Custom MLOps platform based on Airflow and OpenSearch to automatically retrain and redeploy ML components and ML pipelines for a production infrastructure

ML-powered Ad Bidding System

01 UI App
02 API App
03 Airflow Framework
04 Spark Feature Generator
05 Metrics Calculation and Logging

01
UI App

Manages and monitors ML pipelines for price prediction and ML models used in feature generation, allowing for manual updates and deletions
02
API App

Provides access to pipeline data, enabling the addition of new pipelines and versions, including some endpoints to resolve the cold-start problem for new campaigns
03
Airflow Framework

Automates weekly/bi-weekly retraining of predictive models and updates bidder configurations, storing models on S3
04
Spark Feature Generator

Daily processes raw data into features and aggregations for ML, storing recent data in Aerospike
05
Metrics Calculation and Logging

Hourly collects prediction results, calculates metrics, and stores/visualizes performance in DataDog

The user opens the app, the request is sent to the app’s server, then to the platform with auctions, bidding, and predictions, and the winning ad is shown to the user.

On average, we analyze about 4 million user records per advertising campaign. Then we match this information with the bid at the auction and run a predictive model.

Outcome

The NIX team delivered an ML-powered real-time bidding system that can process millions of data simultaneously. Moreover, we also streamlined data management, improved bidding process and automatic report generation.

This upgrade empowered the client with new ways to save up to 47% on knowingly non-performing advertisements, shift budgets to more promising campaigns and make better predictions.

Team:

8 experts (Project Manager, 3 Data Scientists, 2 Angular Developers, 2 Data Engineers)

Tech stack:

Python, lightGBM, pytorch, pandas, pyspark, numpy, optuna, polars, seaborn, AWS S3, AWS Athena, AWS CodeArtifact, AWS SageMaker, AWS EKS, mysql, FastAPI, VueJs, Kibana, DataDog, Airflow, ElasticSearch, boto3, Bitbucket, Docker, Kubernetes