Real-time decision making using ML/AI is the holy grail of customer-facing applications. It’s no longer a long-shot dream; it’s our new reality. The real-time decision engine leverages the latest features in Apache Spark 2.3, including stream-to-stream joins and Spark ML, to directly improve the customer experience. We will discuss the architecture at length, including data source features and technical intricacies, as well as model training and serving dynamics. Critically, real-time decision engines that directly affect customer experience require production-level SLAs and/or reliable fallbacks to avoid meltdowns.

These Slides were put together for Data Platforms 2018 presented by Qubole.

Recorded Video of the talk @ BrightTalk


Avoiding Performance Potholes: Scaling Python for Data Science using Spark @ Spark + AI Summit Using new PySpark 2.3 Vectorized Pandas UDFs: Lessons

Leave a Reply

Your email address will not be published. Required fields are marked *