@ParquetFormat and Impala NYC Meetup
Tuesday, October 29, 2013 from 6:30 PM to 9:30 PM (EDT)
New York, NY
Please join us for a meetup hosted by @TwitterOSS and @ClouderaEng. The meetup will be held at Projective Space LES in NYC (see public transit directions). The meetup will include formal talks and ample time to socialize.
We have limited space for the event, and attendees must RSVP.
Doors open at 6:30, and talks begin at 7pm.
Pizza and beverages will be provided courtesy of Twitter and Cloudera.
Note: This event is closed to press.
Parquet: An Open Columnar Storage Format for Hadoop (@squarecog)
We would like to introduce Parquet, a columnar file format for Hadoop. Performance and compression benefits of using columnar storage formats for storing and processing large amounts of data are well documented in academic literature as well as several commercial analytical databases. Parquet supports deeply nested structures, efficient encoding and column compression schemes, and is designed to be compatible with a variety of higher-level type systems. It is available as a standalone library, allowing any Hadoop framework or tool to build support for it with minimal dependencies. As of this release, Parquet is supported by Apache Pig, plain Hadoop Map-Reduce, and Cloudera’s Impala, and is being put into production at Twitter. We will discuss Parquet’s design and share performance numbers.
Practical Performance Analysis and Tuning for Cloudera Impala (@ClouderaEng)
Impala brings SQL to Hadoop, but it also brings SQL performance tuning to those using the platform. This technical talk will cover several topics in Impala performance analysis, including the performance implications of using Parquet as a storage format.
Case Study: Parquet and Impala (@arahuja)
Arun will be talking about experiences using Parquet and Impala on two biomedical data sets: first, a collection of BAM files; the second, a collection of gene expression data sets from GEO.
Projective Space LES is located at 72 Allen Street, New York