, complex data sets from a variety of sources. Python,PySpark,Big data Technologies... is a must. Hand-On knowledge in Map Reduce, Hadoop, PySpark, Hbase and ElasticSearch. Development and support experience in Big Data...
Data Pipelines Own end-to-end ETL/ELT pipelines using Databricks (PySpark, Spark SQL). Build streaming and batch...-on experience in: Data Engineering Core Databricks (PySpark, Spark SQL, Delta Lake) - mandatory Kafka/MSK / Streaming pipelines...