#compute

Twitter Engineering

The key tenets to building and operating a compute platform are:
<OL>
<LI>Reliability &amp; Scalability: Maintain and scale a compute cluster with no downtime</LI>
<LI>Efficiency: Utilize hardware resources in an efficient and cost-effective way</LI>
<LI>Developer Agility: Enabling developers to create, deploy and manage their services in a frictionless manner.</LI>
</OL>
Come and learn how we are building and operating a compute platform at scale. Enjoy a drink, great eats and meet folks who are working on building compute platforms.
About Twitter's Compute Platform:
The platform has grown from managing a few hundred containers to hundreds of thousands of containers, across tens of thousands of hosts. Built on Apache Mesos, Apache Aurora and a suite of internal systems that manage the lifecycle of an application service — Over a thousand services that power Twitter and its associated products run on this platform.
Agenda:
Doors open // 5:30 PM
Food, drinks &amp; networking // 5:30 PM - 6:30 PM
Introduction by Chris Pinkham, VP Engineering // 6:30 PM
Tech talks with Q&A // 6:35 PM - 8:30 PM
Registration closes // 6:45 PM
End of event // 9:00 PM
Speakers:
<TABLE>
<TBODY STYLE="border-bottom: 1px solid #ccc;">
<TR>
<TD STYLE="padding: 10px 5px; vertical-align: top;"><IMG ALT="" SRC="https://cdn.evbuc.com/eventlogos/833468/ericdanielsencompute150.png"></TD>
<TD STYLE="padding: 10px 5px; vertical-align: top;">
"Server Provisioning and Management at Scale" by Eric Danielson, Tech Lead - Provisioning Engineering
Twitter operates over tens of thousands of physical servers in its datacenters. Come learn about the systems we use to provision, track, and manage a fleet of this size, and the lessons we've learned building them.
</TD>
</TR>
</TBODY>
<TBODY STYLE="border-bottom: 1px solid #ccc;">
<TR>
<TD STYLE="padding: 10px 5px; vertical-align: top;"><IMG ALT="" SRC="https://cdn.evbuc.com/eventlogos/833468/davidrobinsoncompute150.png"></TD>
<TD STYLE="padding: 10px 5px; vertical-align: top;">
"Managing a large scale compute platform" by David Robinson, Site Reliability Engineer - Compute
Twitter has used Aurora and Mesos to scale from only tens of nodes to tens of thousands. During the process of scaling up, the configuration, deployment, and operational procedures for this system have evolved and improved significantly. This talk will describe how a small team manages a large Mesos+Aurora cluster.
</TD>
</TR>
</TBODY>
<TBODY STYLE="border-bottom: 1px solid #ccc;">
<TR>
<TD STYLE="padding: 10px 5px; vertical-align: top;"><IMG ALT="" SRC="https://cdn.evbuc.com/eventlogos/833468/vinucharanyacompute150.png"><IMG ALT="" SRC="https://cdn.evbuc.com/eventlogos/833468/jessicayuencompute.png"></TD>
<TD STYLE="padding: 10px 5px; vertical-align: top;">
"Chargeback for multi-tenant Infrastructure systems" by Vinu Charanya, Software Engineer and Jessica Yuen, Software Engineer - Cloud Infrastructure Management
We built a chargeback system that incentivizes both the service owner and infrastructure owners to improve resource utilization at Twitter scale. In this talk, we will cover how we built a generic system that helps define chargeable resources, collects utilization metrics, resolves owners and generates billing and utilization reports to correct owners for all multi-tenant infrastructure systems.
</TD>
</TR>
</TBODY>
<TBODY STYLE="border-bottom: 1px solid #ccc;">
<TR>
<TD STYLE="padding: 10px 5px; vertical-align: top;"><IMG ALT="" SRC="https://cdn.evbuc.com/eventlogos/833468/davidmclaughincompute.png"></TD>
<TD STYLE="padding: 10px 5px; vertical-align: top;">
"Aurora Workflows" by David Mclaughin, Software Engineer - Compute
Workflows is a new entity in Aurora which represents a set of release targets involved in deploying an artifact to production. The goal is to reduce cognitive overhead from shepherding a deploy from development to production, as well as becoming a key building block for continuous deployment automation in the future. Come hear about how we’re building this and how it’s used within Twitter.
</TD>
</TR>
</TBODY>
<TBODY STYLE="border-bottom: 1px solid #ccc;">
<TR>
<TD STYLE="padding: 10px 5px; vertical-align: top;"><IMG ALT="" SRC="https://cdn.evbuc.com/eventlogos/833468/maosongfucompute.jpg"></TD>
<TD STYLE="padding: 10px 5px; vertical-align: top;">
"Twitter Heron on Aurora" by Maosong Fu, Tech Lead - Real-Time Compute
Heron is the next generation real time streaming analytics platform used heavily in Twitter. Heron runs diverse real time analytics applications ranging from counting to real time machine learning. Heron leverages Aurora extensively to run its topologies. In this talk, we will outline how Heron uses Aurora to distribute computation across nodes, discuss the benefits of using Aurora and suggestions on how Aurora can be improved further to support real time applications.
</TD>
</TR>
</TBODY>
<TBODY STYLE="border-bottom: 1px solid #ccc;">
<TR>
<TD STYLE="padding: 10px 5px; vertical-align: top;"><IMG ALT="" SRC="https://cdn.evbuc.com/eventlogos/833468/1432844339298.jpg"></TD>
<TD STYLE="padding: 10px 5px; vertical-align: top;">
"AWS + Mesos/Aurora at Twitter" by David Hagar, Engineering Manager - Compute (TCDC)
In 2014 TellApart pounced on Aurora as the solution to several problems we were facing in our infrastructure. Hear about our experiences, problems encountered and solved, as well as the next steps: how we are working to make Aurora within AWS an official platform for Twitter, and the challenges of setting up a system that spans public and private clouds.
</TD>
</TR>
</TBODY>
<TBODY>
<TR>
<TD STYLE="padding: 10px 5px; vertical-align: top;"><IMG ALT="" SRC="https://cdn.evbuc.com/eventlogos/833468/.png"></TD>
<TD STYLE="padding: 10px 5px; vertical-align: top;">
"Lightning Talk" by Guest Speaker
</TD>
</TR>
</TBODY>
</TABLE>

@TwitterHQ

Our {communityGuidelinesLink} describe the sort of content we prohibit on Eventbrite. If you suspect an event may be in violation, you can report it to us so we can investigate.

To report other categories of prohibited or illegal content, submit {link}

#compute

More upcoming events from Twitter Engineering

Follow organizers to get events picked for you

#compute

More upcoming events from Twitter Engineering

Follow organizers to get events picked for you

You might also like...

Sign-In for personalized recommendations