mirror of https://github.com/01-edu/public.git
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
156 lines
10 KiB
156 lines
10 KiB
1 year ago
|
## Code-Keeper
|
||
|
|
||
|
<center>
|
||
|
<img src="./resources/cloud-design.jpg?raw=true" style = "width: 600px !important; height: 600px !important;"/>
|
||
|
</center>
|
||
|
|
||
|
### Objective
|
||
|
|
||
|
In this project, you will create a complete pipeline to scan and deploy a microservices-based application using Node.js. Your challenge is to design, implement, and optimize a pipeline that incorporates industry best practices for continuous integration, continuous deployment, and security. Your mission is to:
|
||
|
|
||
|
- Set up a source control system for the Node.js microservices source code and the infrastructure configuration.
|
||
|
- Create a Pipeline to create, update, or delete the infrastructure for the staging and production environment.
|
||
|
- Create a continuous integration (CI) pipeline to build, test, and scan the source code.
|
||
|
- Create a continuous deployment (CD) pipeline to deploy the application to a staging and production environment.
|
||
|
- Ensure the security and reliability of the application throughout the pipeline stages.
|
||
|
|
||
|
### Prerequisites
|
||
|
|
||
|
To complete this project, you should have a good understanding of the following:
|
||
|
|
||
|
- Node.js and its ecosystem
|
||
|
- Gitlab and Gitlab CI
|
||
|
- Ansible as a configuration management and automation tool
|
||
|
- Docker and containerization
|
||
|
- Terraform as an Infrastructure as Code (IaC)
|
||
|
- Cloud platforms (e.g., AWS, Azure, or Google Cloud)
|
||
|
|
||
|
### Tips
|
||
|
|
||
|
- Spend time on the theory before rushing into the practice.
|
||
|
- Read the official documentation.
|
||
|
|
||
|
> Any lack of understanding of the concepts of this project may affect the difficulty of future projects, take your time to understand all concepts.
|
||
|
|
||
|
> Be curious and never stop searching!
|
||
|
|
||
|
### Roleplay
|
||
|
|
||
|
To further enhance the learning experience and assess the student's knowledge of DevOps concepts and practices, we will include a roleplay question session as part of the project. This exercise will require students to apply their knowledge in various real-life scenarios, helping them to solidify their understanding of the material and prepare for real-world situations.
|
||
|
|
||
|
The goal of the roleplay question session is to:
|
||
|
|
||
|
- Assess your understanding of the concepts and technologies used in the project.
|
||
|
- Test your ability to communicate effectively and explain your decisions.
|
||
|
- Challenge you to think critically about your solution and consider alternative approaches.
|
||
|
|
||
|
Prepare for a roleplay question session where you will assume the role of a DevOps engineer presenting your solution to your team or a stakeholder. You should be ready to answer questions and provide explanations about your decisions, architecture, and implementation.
|
||
|
|
||
|
### Deploy GitLab and Runners for Pipeline Execution
|
||
|
|
||
|
You must deploy a GitLab instance using Ansible. This hands-on exercise will help you gain a deeper understanding of Ansible as a configuration management and automation tool while also giving you experience in deploying and configuring GitLab.
|
||
|
|
||
|
1. Create an Ansible playbook to deploy and configure a GitLab instance. The playbook should automate the installation of GitLab and any required dependencies. It should also configure GitLab settings such as user authentication, project settings, and CI/CD settings.
|
||
|
|
||
|
2. Deploy a GitLab instance on a cloud platform (e.g., AWS, Azure, or Google Cloud) or in a local environment using the Ansible playbook. Ensure that the instance is accessible to all team members and is configured to support collaboration and code reviews.
|
||
|
|
||
|
3. Configure the GitLab instance to support CI/CD pipelines by setting up GitLab Runners and integrating them with your existing pipeline. Update your pipeline configuration to utilize GitLab CI/CD features and execute tasks on the deployed Runners.
|
||
|
|
||
|
4. Demonstrate the successful deployment and configuration of GitLab using Ansible.
|
||
|
|
||
|
### The pipelines
|
||
|
|
||
|
You are a DevOps engineer at a company that is transitioning to an Agile approach and wants to achieve high delivery for their microservices architecture. As the DevOps engineer, your manager has tasked you with creating a pipeline that supports Agile methodologies and enables faster, more consistent deployments of the microservices.
|
||
|
|
||
|
![code-keeper](resources/code-keeper.png)
|
||
|
|
||
|
1. You will use your `crud-master` source code and `cloud-design` infrastructure, to create a complete pipeline for the following applications:
|
||
|
|
||
|
- `Inventory application` is a Node.js server that contains your inventory-app code running and connected to the inventory database.
|
||
|
- `billing application` is a Node.js server that contains your billing-app code running and connected to the billing database and consuming the messages from the RabbitMQ queue.
|
||
|
- `api-gateway application` is a Node.js server that contains your api-gateway-app code running and forwarding the requests to the other services.
|
||
|
> Each application must exist in a single repository.
|
||
|
|
||
|
2. You must provision your `cloud-design` infrastructure for two environments on a cloud platform (e.g., AWS, Azure, or Google Cloud) using `Terraform`.
|
||
|
|
||
|
- `Production Environment`: The live infrastructure where the software is deployed and used by end-users, requires stable and thoroughly tested updates to ensure optimal performance and functionality.
|
||
|
- `Staging Environment`: A replica of the production environment used for testing and validating software updates in a controlled setting before deployment to the live system.
|
||
|
> The two environments should be similar in design, resources, and services used!
|
||
|
> Your infrastructure configuration must exist in an independent repository with a configured pipeline!
|
||
|
|
||
|
The pipeline should include the following stages:
|
||
|
|
||
|
- `Init`: Initialize the Terraform working directory and backend. This job downloads the required provider plugins and sets up the backend for storing the Terraform state.
|
||
|
|
||
|
- `Validate`: Validate the Terraform configuration files to ensure correct syntax and adherence to best practices. This helps catch any issues early in the pipeline.
|
||
|
|
||
|
- `Plan`: Generate an execution plan that shows the changes to be made to your infrastructure, including the resources that will be created, updated, or deleted. This job provides a preview of the changes and enables you to review them before applying.
|
||
|
|
||
|
- `Apply to Stagging`: Apply the Terraform configuration to create, update, or delete the resources as specified in the execution plan. This job provisions and modifies the infrastructure in the staging environment.
|
||
|
|
||
|
- `Approval`: Require manual approval to proceed with deployment to the `production environment`. This step should involve stakeholders and ensure the application is ready for production.
|
||
|
|
||
|
- `Apply to Production`: Apply the Terraform configuration to create, update, or delete the resources as specified in the execution plan. This job provisions and modifies the infrastructure in the production environment.
|
||
|
|
||
|
3. Design and implement a CI pipeline for each repository that will be triggered on every code push or pull request. The pipeline should include the following stages:
|
||
|
|
||
|
- `Build`: Compile and package the application.
|
||
|
- `Test`: Run unit and integration tests to ensure code quality and functionality.
|
||
|
- `Scan`: Analyze the source code and dependencies for security vulnerabilities and coding issues. Consider using tools such as `SonarQube`, `Snyk`, or `WhiteSource`.
|
||
|
- `Containerization`: Package the applications into Docker images using a Dockerfile, and push the images to a container registry (e.g., Docker Hub, Google Container Registry, or AWS ECR).
|
||
|
|
||
|
4. Design and implement a CD pipeline that will be triggered after the CI pipeline has been completed. The pipeline should include the following stages:
|
||
|
|
||
|
- `Deploy to Staging`: Deploy the application to a `staging environment` for further testing and validation.
|
||
|
- `Approval`: Require manual approval to proceed with deployment to the `production environment`. This step should involve stakeholders and ensure the application is ready for production.
|
||
|
- `Deploy to Production`: Deploy the application to the `production environment`, ensuring zero downtime and a smooth rollout.
|
||
|
|
||
|
> Each repository must have a pipeline!
|
||
|
|
||
|
> Any modification in the application's source code must rebuild and redeploy the new version to the `Staging Environment` and then to the `Production Environment` after manual approval.
|
||
|
|
||
|
### Security
|
||
|
|
||
|
Your pipelines and infrastructure should adhere to the following cybersecurity guidelines:
|
||
|
|
||
|
- Restrict triggers to protected branches: Prevent unauthorized users from deploying or tampering by triggering pipelines only on protected branches, controlling access, and minimizing risk.
|
||
|
|
||
|
- Separate credentials from code: Avoid storing credentials in application code or infrastructure files. Use secure methods like secret management tools or environment variables to prevent exposure or unauthorized access.
|
||
|
|
||
|
- Apply the least privilege principle: Limit user and service access to the minimum required, reducing potential damage in case of breaches or compromised credentials.
|
||
|
|
||
|
- Update dependencies and tools regularly: Minimize security vulnerabilities by keeping dependencies and pipeline tools updated. Automate updates and monitor for security advisories and patches.
|
||
|
|
||
|
### Documentation
|
||
|
|
||
|
You must push a `README.md` file containing full documentation of your solution (prerequisites, configuration, setup, usage, ...).
|
||
|
|
||
|
### Bonus
|
||
|
|
||
|
If you complete the mandatory part successfully and you still have free time, you can implement anything that you feel deserves to be a bonus, for example:
|
||
|
|
||
|
- Security scan for the infrastructure configuration using `tfsec`.
|
||
|
- Add `Infracost` in your infrastructure pipeline to estimate the infrastructure cost.
|
||
|
- Use `Terragrunt` to create multiple Environments.
|
||
|
|
||
|
Challenge yourself!
|
||
|
|
||
|
### Submission and audit
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
You must submit the `README.md` file and all files used to create and delete and manage your infrastructure and your source code manager: CI/CD files, Terraform files, Dockerfiles, Manifests,...
|
||
|
You must be started and your users and applications repository and CI/CD must be configured correctly to audit it.
|
||
|
|
||
|
> In the audit you will be asked different questions about the concepts and the practice of this project, prepare yourself!
|