mirror of https://github.com/01-edu/public.git
Browse Source
- make cloud-design cloud-provider neutral - make it mandatory to use k8spull/2435/head
nprimo
8 months ago
committed by
Niccolò Primo
3 changed files with 314 additions and 0 deletions
@ -0,0 +1,215 @@ |
|||||||
|
## Cloud-Design |
||||||
|
|
||||||
|
<center> |
||||||
|
<img |
||||||
|
src="./resources/cloud-design.jpg?raw=true" style = "width: 600px |
||||||
|
!important; height: 600px !important;"/> |
||||||
|
</center> |
||||||
|
|
||||||
|
### Objective |
||||||
|
|
||||||
|
The objective of this project is to challenge your understanding of DevOps and |
||||||
|
cloud technologies by providing hands-on experience in deploying and managing a |
||||||
|
microservices-based application with a Cloud Provider platform. Your mission is |
||||||
|
to: |
||||||
|
|
||||||
|
Set up and configure an Cloud Provider environment for deploying microservices. |
||||||
|
Deploy the provided microservices' application to the defined environment. |
||||||
|
Implement monitoring, logging, and scaling to ensure that the application runs |
||||||
|
efficiently. Implement security measures, such as securing the databases and |
||||||
|
making private resources accessible only from a Virtual Private Cloud (VPC). |
||||||
|
Incorporate managed authentication for publicly accessible applications using |
||||||
|
an Authentication management services (e.g. AWS Cognito). Optimize the |
||||||
|
application to handle varying workloads and unexpected events. |
||||||
|
|
||||||
|
### Hints |
||||||
|
|
||||||
|
Before starting this project, you should know the following: |
||||||
|
|
||||||
|
- Basic DevOps concepts and practices. |
||||||
|
- Familiarity with containerization and orchestration tools, such as Docker and |
||||||
|
Kubernetes. |
||||||
|
- Understanding of your chosen Cloud provider platform. |
||||||
|
- Familiarity with Terraform as a Infrastructure as Code (IaC) tools. |
||||||
|
- Knowledge of monitoring and logging tools, such as Prometheus, Grafana, and |
||||||
|
ELK stack. |
||||||
|
|
||||||
|
> Any lack of understanding of the concepts of this project may affect the |
||||||
|
> difficulty of future projects, take your time to understand all concepts. |
||||||
|
|
||||||
|
> Be curious and never stop searching! |
||||||
|
|
||||||
|
### Role play |
||||||
|
|
||||||
|
To enhance the learning experience and assess your knowledge, a role play |
||||||
|
question session will be included as part of the `Cloud-Design-k8s` Project. |
||||||
|
This section will involve answering a series of questions in a simulated |
||||||
|
real-world scenario where you assume the role of a Cloud engineer explaining |
||||||
|
your solution to a team or stakeholder. |
||||||
|
|
||||||
|
The goal of the role play question session is to: |
||||||
|
|
||||||
|
- Assess your understanding of the concepts and technologies used in the |
||||||
|
project. |
||||||
|
- Test your ability to communicate effectively and explain your decisions. |
||||||
|
- Challenge you to think critically about your solution and consider |
||||||
|
alternative approaches. |
||||||
|
|
||||||
|
Prepare for a role play question session where you will assume the role of a |
||||||
|
Cloud engineer presenting your solution to your team or a stakeholder. You |
||||||
|
should be ready to answer questions and provide explanations about your |
||||||
|
decisions, architecture, and implementation. |
||||||
|
|
||||||
|
### Architecture |
||||||
|
|
||||||
|
By using the provided previous projects' solutusion |
||||||
|
[`crud-master-py`](https://github.com/01-edu/crud-master-py), |
||||||
|
[`play-with-containers`](https://github.com/01-edu/play-with-containers), and |
||||||
|
[`orchestrator`](https://github.com/01-edu/orchestrator) you have to design and |
||||||
|
deploy the infrastructure on your chose Cloud provider respecting the project |
||||||
|
requirements, consisting of the following components: |
||||||
|
|
||||||
|
- `inventory-database container` is a PostgreSQL database server that contains |
||||||
|
your inventory database, it must be accessible via port `5432`. |
||||||
|
- `billing-database container` is a PostgreSQL database server that contains |
||||||
|
your billing database, it must be accessible via port `5432`. |
||||||
|
- `inventory-app container` is a server that contains your |
||||||
|
inventory-app code running and connected to the inventory database and |
||||||
|
accessible via port `8080`. |
||||||
|
- `billing-app container` is a server that contains your billing-app |
||||||
|
code running and connected to the billing database and consuming the messages |
||||||
|
from the RabbitMQ queue, and it can be accessed via port `8080`. |
||||||
|
- `RabbitMQ container` is a RabbitMQ server that contains the queue. |
||||||
|
- `api-gateway-app container` is a server that contains your |
||||||
|
API Gateway code running and forwarding the requests to the other |
||||||
|
services, and it's accessible via port `3000`. |
||||||
|
|
||||||
|
Design the architecture for your cloud-based microservices' application. You |
||||||
|
are free to choose the services and architectural patterns that best suit your |
||||||
|
needs, as long as they meet the project requirements and remain within a |
||||||
|
reasonable cost range. Consider the following when designing your architecture: |
||||||
|
|
||||||
|
1. `Scalability`: Ensure that your architecture can handle varying workloads |
||||||
|
and can scale up or down as needed. AWS offers services like Auto Scaling |
||||||
|
that can be used to achieve this. |
||||||
|
|
||||||
|
2. `Availability`: Design your architecture to be fault-tolerant and maintain |
||||||
|
high availability, even in the event of component failures. |
||||||
|
|
||||||
|
3. `Security`: Incorporate security best practices into your architecture, such |
||||||
|
as encrypting data at rest and in transit, using private networks, and |
||||||
|
securing API endpoints. Also, ensure that the databases and private |
||||||
|
resources are accessible only from the VPC and using the cloud provider |
||||||
|
managed authentication for publicly accessible applications. |
||||||
|
|
||||||
|
4. `Cost-effectiveness`: Be mindful of the costs associated with the services |
||||||
|
and resources you select. Aim to design a cost-effective architecture |
||||||
|
without compromising performance, security, or scalability. |
||||||
|
|
||||||
|
5. `Simplicity`: Keep your architecture as simple as possible, while still |
||||||
|
meeting the project requirements. Avoid overcomplicating the design with |
||||||
|
unnecessary components or services. |
||||||
|
|
||||||
|
### Cost management: |
||||||
|
|
||||||
|
1. `Understand the pricing model`: Familiarize yourself with the pricing model |
||||||
|
of the cloud provider and services you are using. Be aware of any free |
||||||
|
tiers, usage limits, and pay-as-you-go pricing structures. |
||||||
|
|
||||||
|
2. `Monitor your usage`: Regularly check your cloud provider's billing |
||||||
|
dashboard to keep track of your usage and spending. Set up billing alerts to |
||||||
|
notify you when your spending exceeds a certain threshold. |
||||||
|
|
||||||
|
3. `Clean up resources`: Remember to delete or stop any resources that you no |
||||||
|
longer need, such as virtual machines, storage services, and load balancers. |
||||||
|
This will help you avoid ongoing charges for idle resources. |
||||||
|
|
||||||
|
4. `Optimize resource allocation`: Use the appropriate resource sizes for your |
||||||
|
needs and experiment with different configurations to find the most |
||||||
|
cost-effective solution. Consider using spot instances, reserved instances, |
||||||
|
or committed use contracts to save on costs, if applicable. |
||||||
|
|
||||||
|
5. `Leverage cost management tools`: Many cloud providers offer cost management |
||||||
|
tools and services to help you optimize your spending. Use these tools to |
||||||
|
analyze your usage patterns and identify opportunities for cost savings. |
||||||
|
|
||||||
|
> By being aware of your cloud usage and proactively managing your resources, |
||||||
|
> you can avoid unexpected costs and make the most of your cloud environment. |
||||||
|
> Remember that the responsibility for cost management lies with you, and it is |
||||||
|
> crucial to stay vigilant and proactive throughout the project. |
||||||
|
|
||||||
|
### Infrastructure as Code: |
||||||
|
|
||||||
|
Provision the necessary resources for your AWS environment using Terraform as |
||||||
|
an Infrastructure as Code (IaC) tools. This includes setting up Cloud computing |
||||||
|
instances, containers, networking components, and storage services (e.g. AWS |
||||||
|
S3). |
||||||
|
|
||||||
|
### Containerize the microservices: |
||||||
|
|
||||||
|
Use Docker to build container images for each microservice. Make sure to |
||||||
|
optimize the Dockerfile for each service to reduce the image size and build |
||||||
|
time. |
||||||
|
|
||||||
|
### Deployment: |
||||||
|
|
||||||
|
Deploy the containerized microservices on your cloud provider using an |
||||||
|
orchestration tool. You must use K8s. |
||||||
|
Ensure that the services are load-balanced (consider using a load |
||||||
|
balancer) and can communicate with each other securely. |
||||||
|
|
||||||
|
### Monitoring and logging: |
||||||
|
|
||||||
|
Set up monitoring and logging tools to track the performance and health of your |
||||||
|
application. Use tools like Prometheus, Grafana, ELK stack or any managed |
||||||
|
service offered by your chosen cloud provider to visualize metrics and logs. |
||||||
|
|
||||||
|
### Optimization: |
||||||
|
|
||||||
|
Implement auto-scaling policies to handle varying workloads and ensure high |
||||||
|
availability. Test the application under different load scenarios and adjust |
||||||
|
the resources accordingly. |
||||||
|
|
||||||
|
### Security: |
||||||
|
|
||||||
|
Implement security best practices by provisioning and managine SSL/TLS |
||||||
|
certificates for HTTPS, securing API endpoints with an API Gateway, regularly |
||||||
|
scanning for vulnerabilities with a proper service, and implementing managed |
||||||
|
authentication for publicly accessible applications with an authetication |
||||||
|
management services. Ensure that the databases and private resources are secure |
||||||
|
and accessible only from the VPC. |
||||||
|
|
||||||
|
### Documentation |
||||||
|
|
||||||
|
Create a `README.md` file that provides comprehensive documentation for your |
||||||
|
architecture, which must include well-structured diagrams, thorough |
||||||
|
descriptions of components, and an explanation of your design decisions, |
||||||
|
presented in a clear and concise manner. Make sure it contains all the |
||||||
|
necessary information about the solution (prerequisites, setup, configuration, |
||||||
|
usage, ...). This file must be submitted as part of the solution for the |
||||||
|
project. |
||||||
|
|
||||||
|
### Bonus |
||||||
|
|
||||||
|
If you complete the mandatory part successfully and you still have free time, |
||||||
|
you can implement anything that you feel deserves to be a bonus, for example: |
||||||
|
|
||||||
|
- Use your own `crud-master`, `play-with-containers`, and `orchestrator` |
||||||
|
solution instead of the provided ones. |
||||||
|
|
||||||
|
- Use `Function as a Service (FaaS)` in your solution. |
||||||
|
|
||||||
|
- Use `Content Delivery Network (CDN)` to optimize your solution. |
||||||
|
|
||||||
|
- Implementing alert systems to ensure your application runs smoothly. |
||||||
|
|
||||||
|
Challenge yourself! |
||||||
|
|
||||||
|
### Submission and audit |
||||||
|
|
||||||
|
Upon completing this project, you should submit the following: |
||||||
|
|
||||||
|
- Your documentation in the `README.md` file. |
||||||
|
- Source code for the microservices and any scripts used for deployment. |
||||||
|
- Configuration files for your Infrastructure as Code (IaC), containerization, |
||||||
|
and orchestration tools. |
@ -0,0 +1,99 @@ |
|||||||
|
#### General |
||||||
|
|
||||||
|
##### Check the Repo content. |
||||||
|
|
||||||
|
Files that must be inside the repository: |
||||||
|
|
||||||
|
- Detailed documentation in the `README.md` file. |
||||||
|
- Source code for the microservices and scripts required for deployment. |
||||||
|
- Configuration files for the chosen cloud provider Infrastructure as Code (IaC), containerization, and orchestration tools. |
||||||
|
|
||||||
|
###### Are all the required files present? |
||||||
|
|
||||||
|
##### Play the role of a stakeholder. |
||||||
|
|
||||||
|
Organize a simulated scenario where the students take on the role of Cloud engineers and explain their solution to a team or stakeholder. Evaluate their grasp of the concepts and technologies used in the project, their communication efficacy, and their critical thinking about their solution. |
||||||
|
Suggested roleplay questions include: |
||||||
|
|
||||||
|
- What is the cloud and its associated benefits? |
||||||
|
- Why is deploying the solution in the cloud preferred over on-premises? |
||||||
|
- How would you differentiate between public, private, and hybrid cloud? |
||||||
|
- What drove your decision to select the specific cloud provider for this project, and what factors did you consider? |
||||||
|
- Can you describe your microservices application's cloud-based architecture and the interaction between its components? |
||||||
|
- How did you manage and optimize the cost of your cloud solution? |
||||||
|
- What measures did you implement to ensure application security on the cloud, and what security best practices did you adhere to? |
||||||
|
- What monitoring and logging tools did you utilize, and how did they assist in identifying and troubleshooting application issues? |
||||||
|
- Can you describe the auto-scaling policies you implemented and how they help your application accommodate varying workloads? |
||||||
|
- How did you optimize Docker images for each microservice, and how did it influence build times and image sizes? |
||||||
|
- If you had to redo this project, what modifications would you make to your approach or the technologies you used? |
||||||
|
- How can your solution be expanded or altered to cater to future requirements like adding new microservices or migrating to a different cloud provider? |
||||||
|
- What challenges did you face during the project and how did you address them? |
||||||
|
- How did you ensure your documentation's clarity and completeness, and what measures did you take to make it easily understandable and maintainable? |
||||||
|
|
||||||
|
###### Were the students able to answer all the questions correctly? |
||||||
|
|
||||||
|
###### Did the students demonstrate a thorough understanding of the concepts and technologies used in the project? |
||||||
|
|
||||||
|
###### Were the students able to communicate effectively and justify their decisions? |
||||||
|
|
||||||
|
###### Could the students critically evaluate their solution and consider alternative strategies? |
||||||
|
|
||||||
|
##### Review the Architecture Design. |
||||||
|
|
||||||
|
Review the student's architecture design, ensuring that it meets the project requirements: |
||||||
|
|
||||||
|
1. `Scalability`: Does the architecture utilize services to manage varying workloads and scale as required? |
||||||
|
2. `Availability`: Is the architecture designed to be fault-tolerant and maintain high availability, even during component failures? |
||||||
|
3. `Security`: Does the architecture integrate security best practices, such as data encryption, use of VPC, and secure API endpoints with managed authentication? |
||||||
|
4. `Cost-effectiveness`: Is the architecture designed to be cost-effective on the chosen cloud provider without compromising performance, security, or scalability? |
||||||
|
5. `Simplicity`: Is the architecture straightforward and free of unnecessary complexity while still fulfilling project requirements? |
||||||
|
|
||||||
|
###### Did the architecture design and choice of services align with all the project requirements above? |
||||||
|
|
||||||
|
###### Were the students able to design a cost-effective architecture that meets the project requirements? |
||||||
|
|
||||||
|
##### Check the student documentation in the `README.md` file. |
||||||
|
|
||||||
|
###### Does the `README.md` file contain all the necessary information about the solution (prerequisites, setup, configuration, usage, ...)? |
||||||
|
|
||||||
|
###### Is the documentation provided by the student clear and complete, including well-structured diagrams and thorough descriptions? |
||||||
|
|
||||||
|
##### Verify the deployment. Ask the auditee **to show you**, the auditor, the use of the commands `kubectl`, the CLI tool for the chosen cloud provider and any other necessary with the right options to answer the following questions. |
||||||
|
|
||||||
|
###### Are all the microservices running as expected in the cloud environment, with no errors or connectivity issues? |
||||||
|
|
||||||
|
###### Is the load balancing configured correctly, effectively distributing traffic across the services? |
||||||
|
|
||||||
|
###### Are the microservices communicating with each other securely, using proper authentication and encryption methods? |
||||||
|
|
||||||
|
##### Evaluate the infrastructure setup. Ask the auditee **to show you**, the auditor, the use of the commands `terraform plan` and/or `terraform apply` to answer the following questions. |
||||||
|
|
||||||
|
###### Is `Terraform` used effectively to provision and manage resources in the cloud environment? |
||||||
|
|
||||||
|
###### Does the infrastructure setup follow the architecture design and the project requirements? |
||||||
|
|
||||||
|
##### Assess containerization and orchestration. Ask the auditee **to show you**, the auditor, the use of the commands chose cloud provider CLI, `docker ps`, and/or `kubectl` or any other necessary with the right options to answer the following questions. |
||||||
|
|
||||||
|
###### Are the Dockerfiles optimized for efficient container builds? |
||||||
|
|
||||||
|
###### Is the orchestration setup (e.g., Kubernetes manifests) configured correctly? |
||||||
|
|
||||||
|
##### Evaluate monitoring and logging. |
||||||
|
|
||||||
|
###### Are monitoring and logging dashboards providing useful insights into the application performance and health? |
||||||
|
|
||||||
|
##### Assess optimization efforts. |
||||||
|
|
||||||
|
###### Are the auto-scaling policies configured correctly to handle varying workloads? |
||||||
|
|
||||||
|
###### Does the application and resource allocation remain efficient under different load scenarios? |
||||||
|
|
||||||
|
##### Check security best practices. |
||||||
|
|
||||||
|
###### Has the student implemented security best practices, such as using HTTPS, securing API endpoints, and regularly scanning for vulnerabilities? |
||||||
|
|
||||||
|
#### Bonus |
||||||
|
|
||||||
|
###### +Did the student add any optional bonus? |
||||||
|
|
||||||
|
###### +Is this project an outstanding project? |
After Width: | Height: | Size: 116 KiB |
Loading…
Reference in new issue