Free

Apache Drill Developer Day

Event Information

Share this event

Date and Time

Location

Location

Harvest properties training room

4555 Great America Parkway

Santa Clara, CA 95054

View Map

Event description

Description

Drill Developers,

MapR Technologies has offered to host it on Nov 14th in Training room downstairs.

at

Training Room at
4555 Great America Pkwy, Suite 201, Santa Clara, CA, 95054.


Agenda for the meetup.
Lunch starts at 12:00 PM.


[12:25 - 12:40] Welcome
- Recap on last year's activities
- Preview of this year's focus

[12:40 - 1:00] Storage plugins
- Adding new storage plugins for the following:
- Netflix Iceberg, Kudu(some code already exists), Cassandra,
Elasticsearch, Carbondata, ORC/XML file formats, Spark
RDD/DataFrames/Datasets, Graph databases & more
- Improving documentation related to Storage plugins


[1:00 - 1:45] Schema discovery & Evolution
- Creation, management of schema
- Handling schema changes in certain common cases
- Handling NULL values elegantly
- Schema learning (similar to MSGpack plugin)
- Query Hints

[1:45 - 2:30] Metadata Management
- Defining an abstraction layer for various types of metadata: views,
schema, statistics, security
- Underlying storage for metadata: what are the options and their
trade-offs?
- Hive metastore
- Parquet metadata cache (parquet specific for row group metadata)
- Ease of using the parquet files generated by other engines (like Spark)


[2:30 - 2:45] Break

[2:45 - 4:00] Resource management
- Resource limits per query
- Optimal memory assignment for blocking operators based on stats
- Enhancing the blocking and exchange operators to live within the memory
limits
- Aligning with admission control/queueing (YARN concepts)
- Query scheduling based on queues using tagging and costing
- Drill on Kubernetes


[4:00 - 4:20] Apache Arrow
- Benefits of integrating Apache Drill with Apache Arrow
- Possible trade-offs & implementation hurdles

[4:20 - 4:40] Performance Improvements
- Efficient handling of Broadcast/Semi/Anti Semi join
- Drill Statistics handling
- Optimizing complex Parquet reader

Share with friends

Date and Time

Location

Harvest properties training room

4555 Great America Parkway

Santa Clara, CA 95054

View Map

Save This Event

Event Saved