Indexing cloud data lakes within the lakes

Grisha Weintraub; Ehud Gudes; Shlomi Dolev

doi:10.1145/3456727.3463828

Indexing cloud data lakes within the lakes

Grisha Weintraub, Ehud Gudes, Shlomi Dolev

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Cloud data lakes are a modern approach for storing large amounts of data in a convenient and inexpensive way. The main idea is the separation of compute and storage layers. However, to perform analytics on the data in this architecture, the data should be moved from the storage layer to the compute layer over the network for each calculation. Obviously, that hurts calculation performance and requires huge network bandwidth. We are exploring different approaches for adding indexing to the cloud data lakes with the goal of reducing the amounts of data read from the storage, and as a result, improving query execution time.

Original language	English
Title of host publication	SYSTOR '21: The 14th ACM International Systems and Storage Conference, Haifa, Israel, June 14-16, 2021
Editors	Bruno Wassermann, Michal Malka, Vijay Chidambaram, Danny Raz
Publisher	Association for Computing Machinery, Inc
ISBN (Electronic)	9781450383981
DOIs	https://doi.org/10.1145/3456727.3463828
State	Published - 2021
Event	14th ACM International Conference on Systems and Storage, SYSTOR 2021 - Virtual, Online, Israel Duration: 14 Jun 2021 → 16 Jun 2021

Conference

Conference	14th ACM International Conference on Systems and Storage, SYSTOR 2021
Country/Territory	Israel
City	Virtual, Online
Period	14/06/21 → 16/06/21

Keywords

engineering
Electrical and Electronic Engineering
Computer Science Applications
Hardware and Architecture
Software

Access to Document

10.1145/3456727.3463828

Cite this

@inproceedings{7d7254ccf9e74731a597ef4767402cbe,

title = "Indexing cloud data lakes within the lakes",

abstract = "Cloud data lakes are a modern approach for storing large amounts of data in a convenient and inexpensive way. The main idea is the separation of compute and storage layers. However, to perform analytics on the data in this architecture, the data should be moved from the storage layer to the compute layer over the network for each calculation. Obviously, that hurts calculation performance and requires huge network bandwidth. We are exploring different approaches for adding indexing to the cloud data lakes with the goal of reducing the amounts of data read from the storage, and as a result, improving query execution time.",

keywords = "engineering, Electrical and Electronic Engineering, Computer Science Applications, Hardware and Architecture, Software",

author = "Grisha Weintraub and Ehud Gudes and Shlomi Dolev",

note = "Publisher Copyright: {\textcopyright} 2021 ACM.; 14th ACM International Conference on Systems and Storage, SYSTOR 2021 ; Conference date: 14-06-2021 Through 16-06-2021",

year = "2021",

doi = "10.1145/3456727.3463828",

language = "English",

editor = "Bruno Wassermann and Michal Malka and Vijay Chidambaram and Danny Raz",

booktitle = "SYSTOR '21: The 14th ACM International Systems and Storage Conference, Haifa, Israel, June 14-16, 2021",

publisher = "Association for Computing Machinery, Inc",

}

Weintraub, G, Gudes, E & Dolev, S 2021, Indexing cloud data lakes within the lakes. in B Wassermann, M Malka, V Chidambaram & D Raz (eds), SYSTOR '21: The 14th ACM International Systems and Storage Conference, Haifa, Israel, June 14-16, 2021., 22, Association for Computing Machinery, Inc, 14th ACM International Conference on Systems and Storage, SYSTOR 2021, Virtual, Online, Israel, 14/06/21. https://doi.org/10.1145/3456727.3463828

Indexing cloud data lakes within the lakes. / Weintraub, Grisha; Gudes, Ehud ; Dolev, Shlomi.
SYSTOR '21: The 14th ACM International Systems and Storage Conference, Haifa, Israel, June 14-16, 2021. ed. / Bruno Wassermann; Michal Malka; Vijay Chidambaram; Danny Raz. Association for Computing Machinery, Inc, 2021. 22.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Indexing cloud data lakes within the lakes

AU - Weintraub, Grisha

AU - Gudes, Ehud

AU - Dolev, Shlomi

PY - 2021

Y1 - 2021

N2 - Cloud data lakes are a modern approach for storing large amounts of data in a convenient and inexpensive way. The main idea is the separation of compute and storage layers. However, to perform analytics on the data in this architecture, the data should be moved from the storage layer to the compute layer over the network for each calculation. Obviously, that hurts calculation performance and requires huge network bandwidth. We are exploring different approaches for adding indexing to the cloud data lakes with the goal of reducing the amounts of data read from the storage, and as a result, improving query execution time.

AB - Cloud data lakes are a modern approach for storing large amounts of data in a convenient and inexpensive way. The main idea is the separation of compute and storage layers. However, to perform analytics on the data in this architecture, the data should be moved from the storage layer to the compute layer over the network for each calculation. Obviously, that hurts calculation performance and requires huge network bandwidth. We are exploring different approaches for adding indexing to the cloud data lakes with the goal of reducing the amounts of data read from the storage, and as a result, improving query execution time.

KW - engineering

KW - Electrical and Electronic Engineering

KW - Computer Science Applications

KW - Hardware and Architecture

KW - Software

UR - http://www.scopus.com/inward/record.url?scp=85108411706&partnerID=8YFLogxK

U2 - 10.1145/3456727.3463828

DO - 10.1145/3456727.3463828

M3 - Conference contribution

BT - SYSTOR '21: The 14th ACM International Systems and Storage Conference, Haifa, Israel, June 14-16, 2021

A2 - Wassermann, Bruno

A2 - Malka, Michal

A2 - Chidambaram, Vijay

A2 - Raz, Danny

PB - Association for Computing Machinery, Inc

T2 - 14th ACM International Conference on Systems and Storage, SYSTOR 2021

Y2 - 14 June 2021 through 16 June 2021

ER -

Indexing cloud data lakes within the lakes

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this