TY - GEN

T1 - Distributed Computations with Layered Resolution

AU - Esfahanizadeh, Homa

AU - Cohen, Alejandro

AU - Medard, Muriel

AU - Shitz, Shlomo Shamai

N1 - Publisher Copyright:
© 2022 IEEE.

PY - 2022/1/1

Y1 - 2022/1/1

N2 - Modern computationally-heavy applications are often time-sensitive, demanding distributed strategies to accelerate them. On the other hand, distributed computing suffers from the bottleneck of slow workers in practice. Distributed coded computing is an attractive solution that adds redundancy such that a subset of distributed computations suffices to obtain the final result. However, the final result is still either obtained within a desired time or not, and for the latter, the resources that are spent are wasted. In this paper, we introduce the novel concept of layered-resolution distributed coded computations such that lower resolutions of the final result are obtained from collective results of the workers-at an earlier stage than the final result. This innovation makes it possible to have more effective deadline-based systems, since even if a computational job is terminated because of timing, an approximated version of the final result can be released. Based on our theoretical and empirical results, the average execution delay for the first resolution is notably smaller than the one for the final resolution. Moreover, the probability of meeting a deadline is one for the first resolution in a setting where the final resolution exceeds the deadline almost all the time, reducing the success rate of the systems with no layering.

AB - Modern computationally-heavy applications are often time-sensitive, demanding distributed strategies to accelerate them. On the other hand, distributed computing suffers from the bottleneck of slow workers in practice. Distributed coded computing is an attractive solution that adds redundancy such that a subset of distributed computations suffices to obtain the final result. However, the final result is still either obtained within a desired time or not, and for the latter, the resources that are spent are wasted. In this paper, we introduce the novel concept of layered-resolution distributed coded computations such that lower resolutions of the final result are obtained from collective results of the workers-at an earlier stage than the final result. This innovation makes it possible to have more effective deadline-based systems, since even if a computational job is terminated because of timing, an approximated version of the final result can be released. Based on our theoretical and empirical results, the average execution delay for the first resolution is notably smaller than the one for the final resolution. Moreover, the probability of meeting a deadline is one for the first resolution in a setting where the final resolution exceeds the deadline almost all the time, reducing the success rate of the systems with no layering.

UR - http://www.scopus.com/inward/record.url?scp=85146121102&partnerID=8YFLogxK

U2 - 10.1109/CloudNet55617.2022.9978858

DO - 10.1109/CloudNet55617.2022.9978858

M3 - Conference contribution

AN - SCOPUS:85146121102

T3 - Proceedings of the 2022 IEEE Conference on Cloud Networking 2022, CloudNet 2022

SP - 257

EP - 261

BT - Proceedings of the 2022 IEEE Conference on Cloud Networking 2022, CloudNet 2022

A2 - Secci, Stefano

A2 - Durairajan, Ramakrishnan

A2 - Linguaglossa, Leonardo

A2 - Kamiyama, Noriaki

A2 - Nogueira, Michele

A2 - Rovedakis, Stephane

PB - Institute of Electrical and Electronics Engineers

T2 - 11th IEEE International Conference on Cloud Networking, CloudNet 2022

Y2 - 7 November 2022 through 10 November 2022

ER -