Browse Source

fix(hole-in-bin) fix subject

pull/2031/head
Zouhair AMAZZAL 1 year ago committed by Zouhair AMAZZAL
parent
commit
70bd60a1cd
  1. 148
      subjects/devops/code-keeper/README.md
  2. 124
      subjects/devops/code-keeper/audit/README.md
  3. BIN
      subjects/devops/code-keeper/resources/cloud-design.jpg
  4. BIN
      subjects/devops/code-keeper/resources/code-keeper.png

148
subjects/devops/code-keeper/README.md

@ -1,148 +0,0 @@
## Code-Keeper
<center>
<img src="./resources/cloud-design.jpg?raw=true" style = "width: 400px !important; height: 400px !important;"/>
</center>
### Objective
In this project, you will create a complete pipeline to scan and deploy a microservices-based application using Node.js. Your challenge is to design, implement, and optimize a pipeline that incorporates industry best practices for continuous integration, continuous deployment, and security. Your mission is to:
- Set up a source control system for the Node.js microservices source code and the infrastructure configuration.
- Create a Pipeline to create, update, or delete the infrastructure for the staging and production environment.
- Create a continuous integration (CI) pipeline to build, test, and scan the source code.
- Create a continuous deployment (CD) pipeline to deploy the application to a staging and production environment.
- Ensure the security and reliability of the application throughout the pipeline stages.
### Prerequisites
To complete this project, you should have a good understanding of the following:
- Node.js and its ecosystem
- Gitlab and Gitlab CI
- Ansible as a configuration management and automation tool
- Docker and containerization
- Terraform as an Infrastructure as Code (IaC)
- Cloud platforms (e.g., AWS, Azure, or Google Cloud)
### Tips
- Spend time on the theory before rushing into the practice.
- Read the official documentation.
> Any lack of understanding of the concepts of this project may affect the difficulty of future projects, take your time to understand all concepts.
> Be curious and never stop searching!
### Roleplay
To further enhance the learning experience and assess the student's knowledge of DevOps concepts and practices, we will include a roleplay question session as part of the project. This exercise will require students to apply their knowledge in various real-life scenarios, helping them to solidify their understanding of the material and prepare for real-world situations.
The goal of the roleplay question session is to:
- Assess your understanding of the concepts and technologies used in the project.
- Test your ability to communicate effectively and explain your decisions.
- Challenge you to think critically about your solution and consider alternative approaches.
Prepare for a roleplay question session where you will assume the role of a DevOps engineer presenting your solution to your team or a stakeholder. You should be ready to answer questions and provide explanations about your decisions, architecture, and implementation.
### Deploy GitLab and Runners for Pipeline Execution
You must deploy a GitLab instance using Ansible. This hands-on exercise will help you gain a deeper understanding of Ansible as a configuration management and automation tool while also giving you experience in deploying and configuring GitLab.
1. Create an Ansible playbook to deploy and configure a GitLab instance. The playbook should automate the installation of GitLab and any required dependencies. It should also configure GitLab settings such as user authentication, project settings, and CI/CD settings.
2. Deploy a GitLab instance on a cloud platform (e.g., AWS, Azure, or Google Cloud) or in a local environment using the Ansible playbook. Ensure that the instance is accessible to all team members and is configured to support collaboration and code reviews.
3. Configure the GitLab instance to support CI/CD pipelines by setting up GitLab Runners and integrating them with your existing pipeline. Update your pipeline configuration to utilize GitLab CI/CD features and execute tasks on the deployed Runners.
4. Demonstrate the successful deployment and configuration of GitLab using Ansible.
### The pipelines
You are a DevOps engineer at a company that is transitioning to an Agile approach and wants to achieve high delivery for their microservices architecture. As the DevOps engineer, your manager has tasked you with creating a pipeline that supports Agile methodologies and enables faster, more consistent deployments of the microservices.
![code-keeper](resources/code-keeper.png)
1. You will use your `crud-master` source code and `cloud-design` infrastructure, to create a complete pipeline for the following applications:
- `Inventory application` is a Node.js server that contains your inventory-app code running and connected to the inventory database.
- `billing application` is a Node.js server that contains your billing-app code running and connected to the billing database and consuming the messages from the RabbitMQ queue.
- `api-gateway application` is a Node.js server that contains your api-gateway-app code running and forwarding the requests to the other services.
> Each application must exist in a single repository.
2. You must provision your `cloud-design` infrastructure for two environments on a cloud platform (e.g., AWS, Azure, or Google Cloud) using `Terraform`.
- `Production Environment`: The live infrastructure where the software is deployed and used by end-users, requires stable and thoroughly tested updates to ensure optimal performance and functionality.
- `Staging Environment`: A replica of the production environment used for testing and validating software updates in a controlled setting before deployment to the live system.
> The two environments should be similar in design, resources, and services used!
> Your infrastructure configuration must exist in an independent repository with a configured pipeline!
The pipeline should include the following stages:
- `Init`: Initialize the Terraform working directory and backend. This job downloads the required provider plugins and sets up the backend for storing the Terraform state.
- `Validate`: Validate the Terraform configuration files to ensure correct syntax and adherence to best practices. This helps catch any issues early in the pipeline.
- `Plan`: Generate an execution plan that shows the changes to be made to your infrastructure, including the resources that will be created, updated, or deleted. This job provides a preview of the changes and enables you to review them before applying.
- `Apply to Stagging`: Apply the Terraform configuration to create, update, or delete the resources as specified in the execution plan. This job provisions and modifies the infrastructure in the staging environment.
- `Approval`: Require manual approval to proceed with deployment to the `production environment`. This step should involve stakeholders and ensure the application is ready for production.
- `Apply to Production`: Apply the Terraform configuration to create, update, or delete the resources as specified in the execution plan. This job provisions and modifies the infrastructure in the production environment.
3. Design and implement a CI pipeline for each repository that will be triggered on every code push or pull request. The pipeline should include the following stages:
- `Build`: Compile and package the application.
- `Test`: Run unit and integration tests to ensure code quality and functionality.
- `Scan`: Analyze the source code and dependencies for security vulnerabilities and coding issues. Consider using tools such as `SonarQube`, `Snyk`, or `WhiteSource`.
- `Containerization`: Package the applications into Docker images using a Dockerfile, and push the images to a container registry (e.g., Docker Hub, Google Container Registry, or AWS ECR).
4. Design and implement a CD pipeline that will be triggered after the CI pipeline has been completed. The pipeline should include the following stages:
- `Deploy to Staging`: Deploy the application to a `staging environment` for further testing and validation.
- `Approval`: Require manual approval to proceed with deployment to the `production environment`. This step should involve stakeholders and ensure the application is ready for production.
- `Deploy to Production`: Deploy the application to the `production environment`, ensuring zero downtime and a smooth rollout.
> Each repository must have a pipeline!
> Any modification in the application's source code must rebuild and redeploy the new version to the `Staging Environment` and then to the `Production Environment` after manual approval.
### Cyber Security
Your pipelines and infrastructure should adhere to the following cybersecurity guidelines:
- Restrict triggers to protected branches: Prevent unauthorized users from deploying or tampering by triggering pipelines only on protected branches, controlling access, and minimizing risk.
- Separate credentials from code: Avoid storing credentials in application code or infrastructure files. Use secure methods like secret management tools or environment variables to prevent exposure or unauthorized access.
- Apply the least privilege principle: Limit user and service access to the minimum required, reducing potential damage in case of breaches or compromised credentials.
- Update dependencies and tools regularly: Minimize security vulnerabilities by keeping dependencies and pipeline tools updated. Automate updates and monitor for security advisories and patches.
### Documentation
You must push a `README.md` file containing full documentation of your solution (prerequisites, configuration, setup, usage, ...).
### Bonus
If you complete the mandatory part successfully and you still have free time, you can implement anything that you feel deserves to be a bonus, for example:
- Security scan for the infrastructure configuration using `tfsec`.
- Add `Infracost` in your infrastructure pipeline to estimate the infrastructure cost.
- Use `Terragrunt` to create multiple Environments.
Challenge yourself!
### Submission and audit
You must submit:
- CI/CD pipeline configuration files, scripts, and any other required artifacts.
- An Ansible playbook and used scripts for deploying and configuring a GitLab instance.
- A well-documented README file that explains the pipeline design, the tools used, and how to set up and use the pipeline.
Your Solution must be running and your users and applications repository and CI/CD must be configured correctly for the audit session.
> In the audit you will be asked different questions about the concepts and the practice of this project, prepare yourself!

124
subjects/devops/code-keeper/audit/README.md

@ -1,124 +0,0 @@
#### General
##### Check the Repo content:
Files that must be inside the repository:
- CI/CD pipeline configuration files, scripts, and any other required artifacts.
- An Ansible playbook and used scripts for deploying and configuring a GitLab instance.
- A well-documented README file that explains the pipeline design, the tools used, and how to set up and use the pipeline.
###### Are all the required files present?
##### Play the role of a stakeholder:
As part of the evaluation process, conduct a simulated real-world scenario where the students assume the role of a DevOps engineer and explain their solution to a team or stakeholder. Evaluate their understanding of the concepts and technologies used in the project, as well as their ability to communicate effectively and think critically about their solution.
During the roleplay, ask them the following questions:
- Can you explain the concept of DevOps and its benefits for the software development lifecycle?
- How do DevOps principles help improve collaboration between development and operations teams?
- What are some common DevOps practices, and how did you incorporate them into your project?
- How does automation play a key role in the DevOps process, and what tools did you use to automate different stages of your project?
- Can you discuss the role of continuous integration and continuous deployment (CI/CD) in a DevOps workflow, and how it helps improve the quality and speed of software delivery?
- Can you explain the importance of infrastructure as code (IaC) in a DevOps environment, and how it helps maintain consistency and reproducibility in your project?
- How do DevOps practices help improve the security of an application, and what steps did you take to integrate security into your development and deployment processes?
- What challenges did you face when implementing DevOps practices in your project, and how did you overcome them?
- How can DevOps practices help optimize resource usage and reduce costs in a cloud-based environment?
- Can you explain the purpose and benefits of using GitLab and GitLab Runners in your project, and how they improve the development and deployment processes?
- What are the advantages of using Ansible for automation in your project, and how did it help you streamline the deployment of GitLab and GitLab Runners?
- Can you explain the concept of Infrastructure as Code (IaC) and how you implemented it using Terraform in your project?
- What is the purpose of using continuous integration and continuous deployment (CI/CD) pipelines, and how did it help you automate the building, testing, and deployment of your application?
- How did you ensure the security of the application throughout the pipeline stages?
- Can you explain the continuous integration (CI) pipeline you've implemented for each repository?
- Can you explain the continuous deployment (CD) pipeline you've implemented for each repository?
###### Did the students have a good understanding of the concepts and technologies used in the project?
###### Did the students have the ability to communicate effectively and explain their decisions?
###### Are the students capable of thinking critically about their solution and considering alternative approaches?
##### Review the GitLab and Runners Deployment:
###### Was the GitLab instance deployed and configured successfully using Ansible?
###### Are the GitLab Runners integrated with the existing pipeline and executing tasks as expected for all repositories?
##### Review the Infrastructure Pipeline:
###### Does the student deploy the `cloud-design` project infrastructure for two environments (staging, prod) on a cloud platform (e.g., AWS, Azure, or Google Cloud) using `Terraform`?
###### Do the two environments are similar in design, resources, and services used?
###### Does the student's infrastructure configuration exist in an independent repository with a configured pipeline?
###### Are the Init, Validate, Plan, Apply to Staging, Approval, and Apply to production environment stages implemented correctly in the infrastructure pipeline?
###### Are the Init, Validate, Plan, Apply to Staging, Approval, and Apply to stagging environment stages implemented correctly in the infrastructure pipeline?
##### Review the CI Pipeline:
- `Build`: Compile and package the application.
- `Test`: Run unit and integration tests to ensure code quality and functionality.
- `Scan`: Analyze the source code and dependencies for security vulnerabilities and coding issues. Consider using tools such as `SonarQube`, `Snyk`, or `WhiteSource`.
- `Containerization`: Package the applications into Docker images using a Dockerfile, and push the images to a container registry (e.g., Docker Hub, Google Container Registry, or AWS ECR).
###### Are the Build, Test, Scan, and Containerization stages implemented correctly in the CI pipeline for each repository?
##### Review the CD Pipeline:
- `Deploy to Staging`: Deploy the application to a `staging environment` for further testing and validation.
- `Approval`: Require manual approval to proceed with deployment to the `production environment`. This step should involve stakeholders and ensure the application is ready for production.
- `Deploy to Production`: Deploy the application to the `production environment`, ensuring zero downtime and a smooth rollout.
###### Are the Deploy to Staging, Approval, and Deploy to Production stages implemented correctly in the CD pipeline for each repository?
##### Review the functionality of pipelines:
###### Are the pipelines working properly and updating the application and infrastructure after each modification in each repository?
##### Check whether the students have effectively implemented the following cybersecurity guidelines:
`Restrict triggers to protected branches`: Ensure that the pipelines are triggered only on protected branches, preventing unauthorized users from deploying or tampering with the application. Check that access control measures are in place to minimize risk.
`Separate credentials from code`: Confirm that the students have not stored credentials in application code or infrastructure files. Look for the use of secure methods like secret management tools or environment variables to prevent exposure or unauthorized access.
`Apply the least privilege principle`: Assess if the students have limited user and service access to the minimum required level. This reduces potential damage in case of breaches or compromised credentials.
`Update dependencies and tools regularly`: Check if the students have a process for keeping dependencies and pipeline tools updated. Verify if they have automated updates and monitored for security advisories and patches to minimize security vulnerabilities.
###### Are triggers restricted to protected branches, ensuring unauthorized users cannot deploy or tamper with the application?
###### Have the students separated credentials from code, using secure methods like secret management tools or environment variables?
###### Did the students apply the least privilege principle to limit user and service access to the minimum required level?
###### Do the students have a process for updating dependencies and tools regularly, automating updates, and monitoring for security advisories and patches?
##### Review the Documentation:
###### Does the `README.md file contain all the necessary information about the solution (prerequisites, setup, configuration, usage, ...)?
###### Is the documentation provided by the student clear and complete, including well-structured diagrams and thorough descriptions?
#### Bonus
###### +Did the student add any optional bonus?
###### +Is this project an outstanding project?

BIN
subjects/devops/code-keeper/resources/cloud-design.jpg

diff.bin_not_shown

Before

Width:  |  Height:  |  Size: 19 KiB

BIN
subjects/devops/code-keeper/resources/code-keeper.png

diff.bin_not_shown

Before

Width:  |  Height:  |  Size: 1.6 MiB

Loading…
Cancel
Save