The aim was to Hybrid cloud with Kubernetes and Cloud bursting for a sample application.
But what exactly does cloud bursting mean? The term comes from the English "to burst". It is about bursting applications in the private part of a hybrid cloud as the load increases by Public cloud resources to expand. The private cloud therefore bursts open and is - if you like - absorbed by the public cloud.
| The hybrid cloud is a solution that combines a private cloud with one or more public cloud services and enables communication between different services using proprietary software. Hybrid cloud services are powerful because they give companies better control over their private data. An organization can store confidential data in a private cloud or on-premises data center while benefiting from the robust computing resources of a managed public cloud. https://www.citrix.com/de-de/glossary/what-is-hybrid-cloud.html |
For which applications is this interesting? It is mainly interesting when the load fluctuates greatly. A certain base load is covered by the private cloud and the peaks automatically scale to the public cloud. This means that costs are only incurred there when the resources are needed.
Use Case Machine Learning
Machine Learning (ML for short) can take up quite a lot of resources and time. Neural networks for image recognition, which are used for the ImageNet project now only require a few minutes of training time. However, several nodes, each with several GPUs, must be used for acceleration1 . At the same time, the workload differs from that of a web server, for example. While the latter potentially runs indefinitely, the training of a neural network is completed at some point. Considerable fluctuations can therefore occur during machine learning.
In the example application, training data is to be uploaded to a hybrid cloud platform. The training is carried out using the Python library Keraswhich has a simple API and offers various Examples is provided. However, other frameworks or libraries can also be used. At the beginning of the training, the training data is downloaded from the platform. The actual learning is then carried out. Finally, the finished model is uploaded to the platform. Whether the training takes place in the private or public part must be decided automatically by the hybrid cloud.
Technical hurdles: Communication of individual services
Individual services can communicate with each other within a Kubernetes cluster. Each service has its own cluster-internal DNS entry. A cluster usually consists of several nodes. Kubernetes decides which container to run on. There are at least two clusters in a hybrid cloud. Ideally, these behave like one large cluster. This results in the following challenges:
- Cross-cluster communicationServices can communicate with each other across cluster boundaries as if they were all in the same cluster.
- Multicluster schedulingKubernetes pods are automatically placed in a free cluster.
Technical implementation through cross-cluster communication and multicluster scheduling
Fortunately, there are solutions for both problems from the Kubernetes community. For the Cross-cluster communication the service mesh can Istio as Multi-cluster installation can be used. This has the nice side effect that all network traffic is automatically encrypted via mutual TLS (mTLS). Other solutions such as Cilium or Left earth have similar functions.
The Multicluster scheduling can be accessed via the Multicluster Scheduler can be achieved by Admiralty. Its use is also described in the blog post Running Argo Workflows Across Multiple Kubernetes Clusters explained on their website.
What does this mean for our use case? By using Istio, the training data can be downloaded from any cluster in the hybrid cloud without the need for extra activations. The execution location of the machine learning is decided by the Multicluster Scheduler.
For bursting to work, the public cloud must have the Autoscaling can be activated for the Kubernetes cluster. This means that new accounts are automatically added to the cluster when the load increases.
Testing the sample environment
Naturally, we subjected the sample application to a test. A large number of ML jobs were created in a short time and evaluated in which cluster they were executed. The expectation was that the private cluster would be used first and then the public cluster. This worked for the most part in the test, although individual jobs were already "bussed" to the public cloud when there was still sufficient capacity available in the private cloud. This reveals a current weakness of the scheduler. Its decisions are difficult to understand or influence. The possibility Cluster weighting could provide a remedy. The private cluster could then be assigned a higher priority. However, as the project is still in the beta phase, implementation should be considered in the future.
Conclusion: How can cloud bursting be implemented?
Cloud bursting can be based on Kubernetes, Istio and the Multicluster Scheduler implement. It can help to make better use of the cloud in the event of load fluctuations. The functional scope of the scheduler currently still lacks a feature for prioritizing clusters. Several other projects for multi-cluster management are in the pipeline. These include, for example Kubernetes Federation, Submariner or Google Anthos. So it remains exciting.
Simon Mennig wrote this blog post as part of his thesis "Challenges with a hybrid cloud".
