diff --git a/subjects/devops/cloud-design-k8s/README.md b/subjects/devops/cloud-design-k8s/README.md new file mode 100644 index 000000000..a8247d627 --- /dev/null +++ b/subjects/devops/cloud-design-k8s/README.md @@ -0,0 +1,215 @@ +## Cloud-Design + +
+ +
+ +### Objective + +The objective of this project is to challenge your understanding of DevOps and +cloud technologies by providing hands-on experience in deploying and managing a +microservices-based application with a Cloud Provider platform. Your mission is +to: + +Set up and configure an Cloud Provider environment for deploying microservices. +Deploy the provided microservices' application to the defined environment. +Implement monitoring, logging, and scaling to ensure that the application runs +efficiently. Implement security measures, such as securing the databases and +making private resources accessible only from a Virtual Private Cloud (VPC). +Incorporate managed authentication for publicly accessible applications using +an Authentication management services (e.g. AWS Cognito). Optimize the +application to handle varying workloads and unexpected events. + +### Hints + +Before starting this project, you should know the following: + +- Basic DevOps concepts and practices. +- Familiarity with containerization and orchestration tools, such as Docker and + Kubernetes. +- Understanding of your chosen Cloud provider platform. +- Familiarity with Terraform as a Infrastructure as Code (IaC) tools. +- Knowledge of monitoring and logging tools, such as Prometheus, Grafana, and + ELK stack. + +> Any lack of understanding of the concepts of this project may affect the +> difficulty of future projects, take your time to understand all concepts. + +> Be curious and never stop searching! + +### Role play + +To enhance the learning experience and assess your knowledge, a role play +question session will be included as part of the `Cloud-Design-k8s` Project. +This section will involve answering a series of questions in a simulated +real-world scenario where you assume the role of a Cloud engineer explaining +your solution to a team or stakeholder. + +The goal of the role play question session is to: + +- Assess your understanding of the concepts and technologies used in the + project. +- Test your ability to communicate effectively and explain your decisions. +- Challenge you to think critically about your solution and consider + alternative approaches. + +Prepare for a role play question session where you will assume the role of a +Cloud engineer presenting your solution to your team or a stakeholder. You +should be ready to answer questions and provide explanations about your +decisions, architecture, and implementation. + +### Architecture + +By using the provided previous projects' solutusion +[`crud-master-py`](https://github.com/01-edu/crud-master-py), +[`play-with-containers`](https://github.com/01-edu/play-with-containers), and +[`orchestrator`](https://github.com/01-edu/orchestrator) you have to design and +deploy the infrastructure on your chose Cloud provider respecting the project +requirements, consisting of the following components: + +- `inventory-database container` is a PostgreSQL database server that contains + your inventory database, it must be accessible via port `5432`. +- `billing-database container` is a PostgreSQL database server that contains + your billing database, it must be accessible via port `5432`. +- `inventory-app container` is a server that contains your + inventory-app code running and connected to the inventory database and + accessible via port `8080`. +- `billing-app container` is a server that contains your billing-app + code running and connected to the billing database and consuming the messages + from the RabbitMQ queue, and it can be accessed via port `8080`. +- `RabbitMQ container` is a RabbitMQ server that contains the queue. +- `api-gateway-app container` is a server that contains your + API Gateway code running and forwarding the requests to the other + services, and it's accessible via port `3000`. + +Design the architecture for your cloud-based microservices' application. You +are free to choose the services and architectural patterns that best suit your +needs, as long as they meet the project requirements and remain within a +reasonable cost range. Consider the following when designing your architecture: + +1. `Scalability`: Ensure that your architecture can handle varying workloads + and can scale up or down as needed. AWS offers services like Auto Scaling + that can be used to achieve this. + +2. `Availability`: Design your architecture to be fault-tolerant and maintain + high availability, even in the event of component failures. + +3. `Security`: Incorporate security best practices into your architecture, such + as encrypting data at rest and in transit, using private networks, and + securing API endpoints. Also, ensure that the databases and private + resources are accessible only from the VPC and using the cloud provider + managed authentication for publicly accessible applications. + +4. `Cost-effectiveness`: Be mindful of the costs associated with the services + and resources you select. Aim to design a cost-effective architecture + without compromising performance, security, or scalability. + +5. `Simplicity`: Keep your architecture as simple as possible, while still + meeting the project requirements. Avoid overcomplicating the design with + unnecessary components or services. + +### Cost management: + +1. `Understand the pricing model`: Familiarize yourself with the pricing model + of the cloud provider and services you are using. Be aware of any free + tiers, usage limits, and pay-as-you-go pricing structures. + +2. `Monitor your usage`: Regularly check your cloud provider's billing + dashboard to keep track of your usage and spending. Set up billing alerts to + notify you when your spending exceeds a certain threshold. + +3. `Clean up resources`: Remember to delete or stop any resources that you no + longer need, such as virtual machines, storage services, and load balancers. + This will help you avoid ongoing charges for idle resources. + +4. `Optimize resource allocation`: Use the appropriate resource sizes for your + needs and experiment with different configurations to find the most + cost-effective solution. Consider using spot instances, reserved instances, + or committed use contracts to save on costs, if applicable. + +5. `Leverage cost management tools`: Many cloud providers offer cost management + tools and services to help you optimize your spending. Use these tools to + analyze your usage patterns and identify opportunities for cost savings. + +> By being aware of your cloud usage and proactively managing your resources, +> you can avoid unexpected costs and make the most of your cloud environment. +> Remember that the responsibility for cost management lies with you, and it is +> crucial to stay vigilant and proactive throughout the project. + +### Infrastructure as Code: + +Provision the necessary resources for your AWS environment using Terraform as +an Infrastructure as Code (IaC) tools. This includes setting up Cloud computing +instances, containers, networking components, and storage services (e.g. AWS +S3). + +### Containerize the microservices: + +Use Docker to build container images for each microservice. Make sure to +optimize the Dockerfile for each service to reduce the image size and build +time. + +### Deployment: + +Deploy the containerized microservices on your cloud provider using an +orchestration tool. You must use K8s. +Ensure that the services are load-balanced (consider using a load +balancer) and can communicate with each other securely. + +### Monitoring and logging: + +Set up monitoring and logging tools to track the performance and health of your +application. Use tools like Prometheus, Grafana, ELK stack or any managed +service offered by your chosen cloud provider to visualize metrics and logs. + +### Optimization: + +Implement auto-scaling policies to handle varying workloads and ensure high +availability. Test the application under different load scenarios and adjust +the resources accordingly. + +### Security: + +Implement security best practices by provisioning and managine SSL/TLS +certificates for HTTPS, securing API endpoints with an API Gateway, regularly +scanning for vulnerabilities with a proper service, and implementing managed +authentication for publicly accessible applications with an authetication +management services. Ensure that the databases and private resources are secure +and accessible only from the VPC. + +### Documentation + +Create a `README.md` file that provides comprehensive documentation for your +architecture, which must include well-structured diagrams, thorough +descriptions of components, and an explanation of your design decisions, +presented in a clear and concise manner. Make sure it contains all the +necessary information about the solution (prerequisites, setup, configuration, +usage, ...). This file must be submitted as part of the solution for the +project. + +### Bonus + +If you complete the mandatory part successfully and you still have free time, +you can implement anything that you feel deserves to be a bonus, for example: + +- Use your own `crud-master`, `play-with-containers`, and `orchestrator` + solution instead of the provided ones. + +- Use `Function as a Service (FaaS)` in your solution. + +- Use `Content Delivery Network (CDN)` to optimize your solution. + +- Implementing alert systems to ensure your application runs smoothly. + +Challenge yourself! + +### Submission and audit + +Upon completing this project, you should submit the following: + +- Your documentation in the `README.md` file. +- Source code for the microservices and any scripts used for deployment. +- Configuration files for your Infrastructure as Code (IaC), containerization, + and orchestration tools. diff --git a/subjects/devops/cloud-design-k8s/audit/README.md b/subjects/devops/cloud-design-k8s/audit/README.md new file mode 100644 index 000000000..69164088f --- /dev/null +++ b/subjects/devops/cloud-design-k8s/audit/README.md @@ -0,0 +1,99 @@ +#### General + +##### Check the Repo content. + +Files that must be inside the repository: + +- Detailed documentation in the `README.md` file. +- Source code for the microservices and scripts required for deployment. +- Configuration files for the chosen cloud provider Infrastructure as Code (IaC), containerization, and orchestration tools. + +###### Are all the required files present? + +##### Play the role of a stakeholder. + +Organize a simulated scenario where the students take on the role of Cloud engineers and explain their solution to a team or stakeholder. Evaluate their grasp of the concepts and technologies used in the project, their communication efficacy, and their critical thinking about their solution. +Suggested roleplay questions include: + +- What is the cloud and its associated benefits? +- Why is deploying the solution in the cloud preferred over on-premises? +- How would you differentiate between public, private, and hybrid cloud? +- What drove your decision to select the specific cloud provider for this project, and what factors did you consider? +- Can you describe your microservices application's cloud-based architecture and the interaction between its components? +- How did you manage and optimize the cost of your cloud solution? +- What measures did you implement to ensure application security on the cloud, and what security best practices did you adhere to? +- What monitoring and logging tools did you utilize, and how did they assist in identifying and troubleshooting application issues? +- Can you describe the auto-scaling policies you implemented and how they help your application accommodate varying workloads? +- How did you optimize Docker images for each microservice, and how did it influence build times and image sizes? +- If you had to redo this project, what modifications would you make to your approach or the technologies you used? +- How can your solution be expanded or altered to cater to future requirements like adding new microservices or migrating to a different cloud provider? +- What challenges did you face during the project and how did you address them? +- How did you ensure your documentation's clarity and completeness, and what measures did you take to make it easily understandable and maintainable? + +###### Were the students able to answer all the questions correctly? + +###### Did the students demonstrate a thorough understanding of the concepts and technologies used in the project? + +###### Were the students able to communicate effectively and justify their decisions? + +###### Could the students critically evaluate their solution and consider alternative strategies? + +##### Review the Architecture Design. + +Review the student's architecture design, ensuring that it meets the project requirements: + +1. `Scalability`: Does the architecture utilize services to manage varying workloads and scale as required? +2. `Availability`: Is the architecture designed to be fault-tolerant and maintain high availability, even during component failures? +3. `Security`: Does the architecture integrate security best practices, such as data encryption, use of VPC, and secure API endpoints with managed authentication? +4. `Cost-effectiveness`: Is the architecture designed to be cost-effective on the chosen cloud provider without compromising performance, security, or scalability? +5. `Simplicity`: Is the architecture straightforward and free of unnecessary complexity while still fulfilling project requirements? + +###### Did the architecture design and choice of services align with all the project requirements above? + +###### Were the students able to design a cost-effective architecture that meets the project requirements? + +##### Check the student documentation in the `README.md` file. + +###### Does the `README.md` file contain all the necessary information about the solution (prerequisites, setup, configuration, usage, ...)? + +###### Is the documentation provided by the student clear and complete, including well-structured diagrams and thorough descriptions? + +##### Verify the deployment. Ask the auditee **to show you**, the auditor, the use of the commands `kubectl`, the CLI tool for the chosen cloud provider and any other necessary with the right options to answer the following questions. + +###### Are all the microservices running as expected in the cloud environment, with no errors or connectivity issues? + +###### Is the load balancing configured correctly, effectively distributing traffic across the services? + +###### Are the microservices communicating with each other securely, using proper authentication and encryption methods? + +##### Evaluate the infrastructure setup. Ask the auditee **to show you**, the auditor, the use of the commands `terraform plan` and/or `terraform apply` to answer the following questions. + +###### Is `Terraform` used effectively to provision and manage resources in the cloud environment? + +###### Does the infrastructure setup follow the architecture design and the project requirements? + +##### Assess containerization and orchestration. Ask the auditee **to show you**, the auditor, the use of the commands chose cloud provider CLI, `docker ps`, and/or `kubectl` or any other necessary with the right options to answer the following questions. + +###### Are the Dockerfiles optimized for efficient container builds? + +###### Is the orchestration setup (e.g., Kubernetes manifests) configured correctly? + +##### Evaluate monitoring and logging. + +###### Are monitoring and logging dashboards providing useful insights into the application performance and health? + +##### Assess optimization efforts. + +###### Are the auto-scaling policies configured correctly to handle varying workloads? + +###### Does the application and resource allocation remain efficient under different load scenarios? + +##### Check security best practices. + +###### Has the student implemented security best practices, such as using HTTPS, securing API endpoints, and regularly scanning for vulnerabilities? + +#### Bonus + +###### +Did the student add any optional bonus? + +###### +Is this project an outstanding project? diff --git a/subjects/devops/cloud-design-k8s/resources/cloud-design.jpg b/subjects/devops/cloud-design-k8s/resources/cloud-design.jpg new file mode 100644 index 000000000..512b093f6 Binary files /dev/null and b/subjects/devops/cloud-design-k8s/resources/cloud-design.jpg differ