Serverless Data Pipelines for IoT Data in Edge and Cloud Environments using Microsoft Azure
Name
Martin Kisand
Abstract
Advancements in wireless connectivity and smart device technologies have contributed to the fast growth of Internet of Things (IoT) devices. Growth of IoT networks is generating increasing amounts of heterogeneous raw data that has to be analysed in real time to enable effective data based decision making. Challenges with latency, bandwidth and cost of cloud centric IoT solutions have driven adoption of edge computing paradigm to bring computing closer to data source. To harness edge and cloud computing resources together for continuous IoT data, serverless data pipelines can be designed taking advantage of Function as a Service (FaaS) paradigm. In this paper serverless data pipeline is proposed for real time IoT use case using Microsoft Azure cloud computing platform. Proposed forced audio to text alignment pipeline was tested to evaluate its performance consistency, reliability and how performance was affected by computing resources in pipeline. Pipeline proved to be mostly reliable with few failures but end-to-end execution times were rather inconsistent. Adding CPU and working memory to serverless function performing forced alignment lowered function execution times with increase of working memory from 3.5GB to 7GB. Raising working memory further to 14GB did not achieve better results compared to 7GB with current test scenario. Lowest working memory setting proved to be most cost effective because pricing is based on available computing resources and doubling or quadrupling resources did not have that much of an impact on performance as it had on cost. With proposed serverless pipeline implementation Azure did not seem to offer cost effective FaaS options considering smaller scale IoT applications that need custom functionality because premium plan can not scale to zero instances and lowest working memory setting is 3.5GB. In case of smaller scale IoT applications where functionality offered by Azure function consumption plan is not sufficient other cloud service providers could offer more suitable solutions with lower costs.
Graduation Thesis language
English
Graduation Thesis type
Bachelor - Computer Science
Supervisor(s)
Shivananda Rangappa Poojara
Defence year
2023