Abstract
Having constantly increasing amounts of data, the analysis of it is often entrusted for a MapReduce framework. The execution of an analytical workload can be cheapened by adopting cloud computing resources, and in particular by using spot instances (cheap, fluctuating price instances) offered by Amazon Web Services (AWS).
The users aiming for the spot market are presented with many instance types placed in multiple datacenters in the world, and thus it is difficult to choose the optimal deployment. In this paper, we propose the framework SpotADAPT (Spot-Aware (re-)Deployment of Analytical Processing Tasks) which is designed to help users by first, estimating the workload execution time on different AWS instance types, and, second, proposing the deployment
(i.e., specific availability zone, instance type, pricing model) aligned with user-provided optimization goals (fastest or cheapest execution within boundaries). Moreover, during the execution of the workload, SpotADAPT suggests a redeployment if the current spot instance gets terminated by Amazon or a better deployment becomes possible due to fluctuations of the spot prices.
The approach is evaluated using the actual execution times of typical analytical workloads and real spot price traces. SpotADAPT's suggested deployments are comparable to the theoretically optimal ones, and in particular, it shows good cost benefits for the budget optimization -- on average SpotADAPT is at most 0.3% more expensive than the theoretically optimal deployments.
The users aiming for the spot market are presented with many instance types placed in multiple datacenters in the world, and thus it is difficult to choose the optimal deployment. In this paper, we propose the framework SpotADAPT (Spot-Aware (re-)Deployment of Analytical Processing Tasks) which is designed to help users by first, estimating the workload execution time on different AWS instance types, and, second, proposing the deployment
(i.e., specific availability zone, instance type, pricing model) aligned with user-provided optimization goals (fastest or cheapest execution within boundaries). Moreover, during the execution of the workload, SpotADAPT suggests a redeployment if the current spot instance gets terminated by Amazon or a better deployment becomes possible due to fluctuations of the spot prices.
The approach is evaluated using the actual execution times of typical analytical workloads and real spot price traces. SpotADAPT's suggested deployments are comparable to the theoretically optimal ones, and in particular, it shows good cost benefits for the budget optimization -- on average SpotADAPT is at most 0.3% more expensive than the theoretically optimal deployments.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the ACM Eighteenth International Workshop On Data Warehousing and OLAP (DOLAP 2015) |
| Number of pages | 10 |
| Place of Publication | New York, NY, USA |
| Publisher | Association for Computing Machinery (ACM) |
| Publication date | 23 Oct 2015 |
| Pages | 59-68 |
| ISBN (Electronic) | 978-1-4503-3785-4 |
| DOIs | |
| Publication status | Published - 23 Oct 2015 |
| Event | ACM Eighteenth International Workshop On Data Warehousing and OLAP - Melbourne Convention and Exhibition Centre, Melbourne, Australia, Melbourne, Australia Duration: 23 Oct 2015 → 23 Oct 2015 Conference number: 18 |
Workshop
| Workshop | ACM Eighteenth International Workshop On Data Warehousing and OLAP |
|---|---|
| Number | 18 |
| Location | Melbourne Convention and Exhibition Centre, Melbourne, Australia |
| Country/Territory | Australia |
| City | Melbourne |
| Period | 23/10/2015 → 23/10/2015 |
Keywords
- Amazon Web Services
- EC2
- Spot instances
- Hadoop
- execution time estimation