Real Time Big Data analytics: Parquet (and Spark) + bonus

Posted by Garren on

Apache Spark and Parquet (SParquet) are a match made in scalable data analytics and delivery heaven. Spark brings a wide ranging, powerful computing platform to the equation while Parquet offers a data format that is purpose-built for high-speed big data analytics. If this sounds like fluffy marketing talk, resist the temptation to close this tab,… Continue reading→

Categories
Apache Spark
Tags
, , , , , 3 Comments on Real Time Big Data analytics: Parquet (and Spark) + bonus

Connecting Apache Spark to External Data sources (e.g. Redshift, S3, MySQL)

Posted by Garren on

Pre-requisites AWS S3 Hadoop AWS Jar AWS Java SDK Jar * Note: These AWS jars should not be necessary if you’re using Amazon EMR. Amazon Redshift JDBC Driver Spark-Redshift package * * The Spark-redshift package provided by Databricks is critical particularly if you wish to WRITE to Redshift, because it does bulk file operations instead… Continue reading→

Categories
Apache Spark
Tags
, , , , Leave a Comment on Connecting Apache Spark to External Data sources (e.g. Redshift, S3, MySQL)