In this code-level session, we show you how to integrate your Apache Spark application with Amazon SageMaker. We'll include details on how networking, archit...

ROCm, the Radeon Open Ecosystem, is an open-source software foundation for GPU computing on Linux. ROCm supports TensorFlow and PyTorch using MIOpen, a library of highly optimized GPU routines for deep learning. In this talk, we describe how Apache Spark…

Talk of Xavier Tordoir at Scala UA Ukraine

Project Hydrogen is a major Apache Spark initiative to bring state-of-the-art AI and Big Data solutions together. It contains three major projects: 1) barrie...

Holden, an open source developer advocate at Google, discusses how the Apache Arrow is new in Spark 2.3, and offers faster interchange between Spark and Python. Apache Arrow also has connections to Tensorflow (and even without those can be fed from Pandas).…

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference…

Holden is a transgender Canadian open source developer with a focus on Apache Spark and related "big data" tools. She is the co-author of Learning Spark, High Performance Spark, and Kubeflow for Machine Learning. She is a committer and PMC on Apache Spark

The beta release of "Spark Operator" allows native execution of Spark applications on Kubernetes clusters -- no Hadoop or Mesos required.

Developing ML and deep learning applications to be deployed in production is much more than just training a model. Google has taken years of experience in de...

Apache Beam is a key technology for building scalable End-to-End ML pipelines, as it is the data preparation and model analysis engine for TensorFlow Extended (TFX), a framework for horizontally scalable Machine Learning (ML) pipelines based on TensorFlow.…

In this two-part blog series, we introduce the concepts and benefits of working with both spark-submit and the Kubernetes Operator for Spark. In Part 1, we introduce both tools and review how to get started monitoring and managing your Spark clusters on Kubernetes.…

Improving distinct performance & looking at Scala 2.12 support. Created this PR - https://github.com/apache/spark/pull/22010

The Jupyter notebook is one of the most used tools in data science projects. It’s a great tool for developing software in python and has…

Passionate about AI? wonder what TensorFrames is and how it’s related to TensorFlow? read this!

While crystal balls are notoriously unreliable, lately there has been a lot of interesting work around the planning and development of Apache Spark 3.0. In t...

In this talk, we will highlight major efforts happening in the Spark ecosystem. In particular, we will dive into the details of adaptive and static query optimizations in Spark 3.0 to make Spark easier to use and faster to run. We will also demonstrate how…

This talk will take an two existing Spark ML pipeline (Frank The Unicorn, for predicting PR comments (Scala) - https://github.com/franktheunicorn/predict-pr-...

Frameworks like Spark and TensorFlow have commoditized cluster computing and training of neural networks. However, they leave precious performance on the table, especially when used together. Flare is a new back-end for Spark SQL that yields significant speedups…

Can Apache Spark slip its earthly bounds and go serverless, clusterless? Popular cloud services are becoming more capable. AWS Lamba now runs three times lon...

For more info including classes resources and the coupon code for additional complementary training please click here: https://databricks.com/p/webinar/free-training-how-to-build-a-cloud-data-platform
In part 2 of this 4 part series you’ll learn how to create…