Flexible Coding for Distributed Systems

In this talk, flexible coding is presented for the ubiquitous distributed systems. Facing the fact that the failures are unpredictable in distributed storage, a framework for flexible codes to achieve the optimal latency of accessing information is proposed. Instead of accessing a fixed number of nodes as in a conventional code, a flexible code allows one to recover the entire information from a flexible number of storage nodes, and use all the available nodes efficiently. Moreover, flexible coding in distributed matrix multiplication for data analytics is presented. The proposed constructions allow a master node to efficiently obtain the computation results from a flexible number of available servers in the presence of unknown failures. Given the storage capacity of the servers, the computation load is optimized, leading to the reduction of the computing latency.