Apache Spark 2.0 set the architectural foundations of structure in Spark, unified high-level APIs, structured streaming, and the underlying performant components like Catalyst Optimizer and Tungsten Engine. Since then the Spark community has continued to…

Productionalizing ML models is hard. In fact, very few ML projects make it to production, and one of the hardest problems is data! Most AI platforms are disconnected from the data platform, making it challenging to keep features constantly updated and available…

Forum link to presentation: https://forum.numer.ai/t/feature-neutralisation-autocorrelation-presentation/2341

[00:50] Interview with rsmillie94 begins
[22:54​] Presentation - Feature Neutralization: Some Theoretical Concerns
[23:14] Feature Value and Prediction…