Tengu: an Experimentation Platform for Big data Applications

TitleTengu: an Experimentation Platform for Big data Applications
Publication TypeConference Paper
Year of Publication2015
AuthorsVanhove, T, Van Seghbroeck, G, Wauters, T, De Turck, F, Vermeulen, B, Demeester, P
Conference NameThe International Workshop on Computer and Networking Experimental Research Using Testbeds
Conference LocationColumbus, Ohio, USA

Big data applications have stringent service requirements for scalability and fault-tolerance and involve high volumes of data, high processing speeds and large varieties of database technologies. In order to test big data management solutions, large experimentation facilities are needed, which are expensive in terms of both resource cost and configuration time. This paper presents Tengu, an experimentation platform for big data applications that can automatically be instantiated on GENI (US federation of testbeds) and Fed4FIRE (EU federation of testbeds) compatible testbeds. Tengu allows for automatic deployments of several data processing, storage and cloud technologies, including Hadoop, Storm and OpenStack. The paper discusses the Tengu architecture, the Tengu-as-a-service approach and a demonstration of an automated instantiation of the Tengu experimentation suite on the Virtual Wall, a large-scale Emulab testbed at the iMinds research institute in Europe.