As this instance needs to run 24/7, we will create a t2.micro linux (ubuntu) instance, which is just enough for this purpose. This instance does not run jobs, but rather peforms the role of spinning up and down AWS spot instances on demand, that in turn, runs the job. We will need to setup the runner manager instance. This, when used with spot instances, can save us up to 90% cost, improve performance with little or no downtime.įor the purpose of this guide, we will need a gitlab account of course and an aws account, as we will be creating resources.

GitLab Runner has the ability to automatically spin up and down VMs to make sure your builds get processed immediately.

(Just spin it up, run the job, and shut it down), similar to lambdas. We need a setup that will just run for the time needed to complete the job. This makes spot instances best for our purpose. AWS EC2 Spot Instances let you take advantage of unused EC2 capacities in the AWS cloud.Īccording Amazon, Spot Instances are available at up to a 90% discount compared to On-Demand prices and you can use Spot Instances for various stateless, fault-tolerant, or flexible applications such as big data, containerized workloads, CI/CD, web servers, high-performance computing (HPC), and other test & development workloads.

What are AWS EC2 Spot InstancesĪ Spot Instance is an unused EC2 instance that is available for less than the On-Demand price. To make it even worse, setting up cache keys will not help as cached files get deleted onces the docker container volume gets unmounted. Part of the problems posed by the above option is the fact that, despite the cost incurred, the clusters still perform very poor, network strength is the most annoying part of it, as some node.js projects for instance can take ages for yarn to successfully install dependencies. Your ci jobs are never going to run every minute, so why pay Google or Amazon for minutes you never used? With how easy and quick this setup is, you will most likely use it, but at the end of the tunnel lies more bills. With few clicks in the setup dashboard, you can setup and deploy gitlab runner in a Google Kubernetes cluster or Amazon. Gitlab makes the process seemless as it integrates very well with Google Cloud Platform, as well as AWS. The other option entails setting up your own gitlab runner. If this is enough for all your projects, then you are lucky, but if not, you have to scale your plan to get more minutes. The first is to use gitlab shared runners, which affords you up 2,000 free ci pipeline minutes in a month. There are two options you have when you want to setup continuous integration and deployment in gitlab.

How to install gitlab on aws