From f048c0fa87767573b5cd28064f63a4fffd0af170 Mon Sep 17 00:00:00 2001 From: mikysett Date: Wed, 28 Sep 2022 15:21:49 +0100 Subject: [PATCH] feat(localhost): subject and audit ready for review due to git wrong pull/merge previous commits have been lost. commit messages relevant to this project are transcripted here as reference: commit 05f8f75d05f1f26315f96e60fa5c8e2f1e85f4d3 Author: mikysett Date: Mon Sep 26 16:53:53 2022 +0100 feat(localhost): complete subject and audit audit completely refactored and improved (more clear, more questions, irrelevant question removed, bonuses updated) small improvements on the subject commit fd3a2a00aec6c58c9c34b1c78c6d3d0bbad11b74 Author: mikysett Date: Thu Sep 22 17:45:38 2022 +0100 feat(localhost): change bonuses php file with mysql is redundant (a CGI must already be implemented). to suggest rewriting it with a different language looks like a nice challenge, maybe a bit big for a bonus commit 6b6f2b9fb2c6c42421b7636c9870707787d847a0 Author: mikysett Date: Thu Sep 22 17:41:32 2022 +0100 feat(localhost): add details for CGI and config file commit 9506a2b8bb0ca17e8dc72ede8826ea227ab14800 Author: mikysett Date: Thu Sep 22 16:39:31 2022 +0100 refactor(localhost): improve first paragraph style commit 0c4e6300ee3513d35b508d49e55fe7f5db431cf6 Author: mikysett Date: Thu Sep 22 16:35:36 2022 +0100 feat(localhost): add clarity for I/O multiplexing commit de46cebb8b58c1fb0887c9dce8830937eff6c444 Author: mikysett Date: Thu Sep 22 16:14:31 2022 +0100 fix(localhost): remove error code 311 and add 500 error 311 doesn't exist. error 500 seems relevant and should be implemented by students. commit a82f254a9a5c20e4c1ce7e480490f554758c47da Author: Michele Sessa Date: Wed Sep 21 18:37:36 2022 +0100 refactor(localhost): change subject structure overall structure modified to have more modulary and clarity. this is still a work in progress and far to be complete. at the moment few parts were removed/replaced, focus being in reorganizing what already exists. future commits should focus on adding restrictions/information for clarity and to better define the work to be done by the student. commit 2244f72d4519ce446db079ebe09d0ecae6491484 Author: Zainab Dnaya Date: Wed Sep 21 13:19:48 2022 +0100 docs(audit/localhost) : Fix Many po commit 3e7946086a01b79531c7fe1afff50f4788ce2613 Author: Zainab Dnaya Date: Tue Sep 20 22:02:17 2022 +0100 Update README.md commit 7b2b9865ede21697286b2c16604cdab40df8b0ab Author: zainabdnaya Date: Wed Jul 27 11:25:41 2022 +0100 feat: subject commit 6b198c4a8524cab92d5c623eb5d8892adee4af68 Author: zainabdnaya Date: Wed Jul 27 09:57:00 2022 +0100 feat: subject commit 856819bfe03b80ad7e966e4e970778170f46210e Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 20:03:38 2022 +0100 Update README.md commit 336e6a5dfb8b2f5c36aa6ea3bdd44a214f2e3179 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 20:02:10 2022 +0100 Update README.md commit 812f0c9a21dc7769858c18c02287ad1b282ae8d4 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 20:00:18 2022 +0100 add condition in bonus part commit 88cea7e2450c61651be771452369bbea446b5c13 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 19:58:48 2022 +0100 Update README.md commit ea50617445b706e92bbfc0cef98399cded003679 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 19:57:24 2022 +0100 Add bonus part commit 5799df1bd954aec24583be32f9b7b615d2ab6cc2 Author: zainabdnaya Date: Mon Jul 25 18:56:40 2022 +0100 feat: Update the audit commit ebefd1dd87864f085ee8258d3950bbc69a3a32da Author: zainabdnaya Date: Mon Jul 25 18:17:46 2022 +0100 feat: Update the audir commit 5824b1e8359ba3afb880ee476959ec7b9d44a104 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Fri Jul 22 15:58:57 2022 +0100 Update README.md commit 2af3808b9ac1867202ac79bf553b5c428824281d Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Fri Jul 22 15:40:40 2022 +0100 add condition of http code and redirections commit 0b4d91409333e7087cbb83dfcbc6e2bdaba8f823 Author: zainabdnaya Date: Mon Jul 25 11:12:19 2022 +0100 Localhost Subject commit e67bc965ed2d422783dbf77af0999092b78a5156 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Thu Jul 21 12:51:02 2022 +0100 add cgi condition commit 5ff06919be80b1bb67f2e1642561c41708f6809e Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Thu Jul 21 12:43:21 2022 +0100 Update README.md commit 20dd21f24db9dc16957f96cc4e71f88f2436a4a5 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Thu Jul 21 12:27:30 2022 +0100 add hints && add conditions commit 456e875a2e246e69251051afcb6e92b7cc013ccb Author: hamza Date: Wed Jul 20 11:57:00 2022 +0100 fix the name commit 51ff541d7e731104aedefccf5f52ae9618049cca Author: hamza Date: Wed Jul 20 11:56:28 2022 +0100 add subject for localhost commit 71aec7298b8bf4f0a7a6b1643ac0b830a3daaba3 Author: Michele Sessa Date: Wed Sep 21 18:37:36 2022 +0100 refactor(localhost): change subject structure overall structure modified to have more modulary and clarity. this is still a work in progress and far to be complete. at the moment few parts were removed/replaced, focus being in reorganizing what already exists. future commits should focus on adding restrictions/information for clarity and to better define the work to be done by the student. commit d914a302ce32c8836fb5f0fae9dc58ea0bc94ed6 Author: Zainab Dnaya Date: Wed Sep 21 13:19:48 2022 +0100 docs(audit/localhost) : Fix Many po commit 2ddf32ff5c64b6bab5cc588df8840fe5596850a7 Author: Zainab Dnaya Date: Tue Sep 20 22:02:17 2022 +0100 Update README.md commit 6f6b410fbf57812806a91327a423aa6e184f189c Author: zainabdnaya Date: Wed Jul 27 11:25:41 2022 +0100 feat: subject commit 789f9496f9c297b9e6f38338ef955e8e2932836b Author: zainabdnaya Date: Wed Jul 27 09:57:00 2022 +0100 feat: subject commit 8aba27a2ff05cd2ae5a24d6036693279ae161ca8 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 20:03:38 2022 +0100 Update README.md commit 31def6fdb23e9ab967f76d0ee475c5a6d903a6c4 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 20:02:10 2022 +0100 Update README.md commit 7b104f5e7e648b99f95edb8a38a38a40e67ddd68 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 20:00:18 2022 +0100 add condition in bonus part commit 1856eaa5b0b183ab3f9da4649922bfc21d4bd724 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 19:58:48 2022 +0100 Update README.md commit 3d5c2807bd692686f5088942b8411edca9de092f Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Tue Jul 26 19:57:24 2022 +0100 Add bonus part commit 3c3f1663a7ce9c58a21c94aabe5ee94ae1b62b8c Author: zainabdnaya Date: Mon Jul 25 18:56:40 2022 +0100 feat: Update the audit commit c1818288c822e58ce27e687df0b0161a8f6bf4fe Author: zainabdnaya Date: Mon Jul 25 18:17:46 2022 +0100 feat: Update the audir commit 1f79b2261fbad7f606dd3899d4c6e517bee01f33 Author: zainabdnaya Date: Mon Jul 25 11:12:19 2022 +0100 Localhost Subject commit 6f3c37ef1afdc77bbcf7b0fad7d4e546b5b04ced Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Fri Jul 22 15:58:57 2022 +0100 Update README.md commit 47628970b2e3a5a175ee5807bd03bf9377272043 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Fri Jul 22 15:40:40 2022 +0100 add condition of http code and redirections commit 234e09311e74b4c88655b8de1c7145aa1ce56800 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Thu Jul 21 12:51:02 2022 +0100 add cgi condition commit a41ec15a3aa3179f096724b782449ad825dd5e6f Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Thu Jul 21 12:43:21 2022 +0100 Update README.md commit 37d29e27cf63346a1f3199ad994654eef5914762 Author: Hamza elkhatri <40549481+Hamzaelkhatri@users.noreply.github.com> Date: Thu Jul 21 12:27:30 2022 +0100 add hints && add conditions commit 3ab59cd27e0e28a6a5815831b5a520edcbb0d935 Author: hamza Date: Wed Jul 20 11:57:00 2022 +0100 fix the name commit cb9e0859453ab060af7780b8846f83b9c138654a Author: hamza Date: Wed Jul 20 11:56:28 2022 +0100 add subject for localhost --- subjects/localhost/README.md | 76 ++++++++++++++++++++++++++++++ subjects/localhost/audit/README.md | 65 +++++++++++++++++++++++++ 2 files changed, 141 insertions(+) create mode 100644 subjects/localhost/README.md create mode 100644 subjects/localhost/audit/README.md diff --git a/subjects/localhost/README.md b/subjects/localhost/README.md new file mode 100644 index 00000000..2c2c8bb6 --- /dev/null +++ b/subjects/localhost/README.md @@ -0,0 +1,76 @@ +## Localhost + +Finally you are going to understand how internet works from the server side. The Hypertext Transfer Protocol was created in order to ensure a reliable way to communicate on a request/response base. + +This protocol is used by servers and clients (usually browsers) to serve content and it is the backbone of the World Wide Web, still it is also used in many other cases that are far beyond the scope of this exercise. + +Here you will learn the basics of the protocol and a good place to start could be the [HTTP/1.1 RFC](https://www.rfc-editor.org/rfc/rfc9112.html). + + +### Instructions + +- The project can be written in one of these languages [`Rust`, `C++`, `C`]. + +#### The Server + +- Your server should **never** crash. +- All requests should timeout if they are taking too long. +- Your server should be able to listen on multiple ports and instantiate multiple servers at the same time. +- You must use only one process and one thread. +- Your server must receive a request from the browser/client and send a response using the `HTTP` header and body. +- Your server should be compatible with `HTTP/1.1` protocol. +- You can compare your results with `NGINX` which will be used as the reference. +- Your server should be compatible with the last version of your chosen browser. +- Your server should manage at least [`GET`, `POST`, `DELETE`] methods. +- Your server should be able to receive file uploads made by the client. +- Your server should handle cookies and sessions. +- You should create default error pages for at least the following error codes [400,403,404,405,413,500]. +- Your server should call `select` function (or `poll` or equivalent) only once for each client/server communication. +- All reads and writes should pass by `select` or equivalent API. +- All I/O operations should be non-blocking. +- You should manage chunked and unchunked requests. +- You should set the right status for each response. + +#### The CGI +- Based on the file extension the server will execute the corresponding `CGI` (for example `.php` or `.py`). +- You need to implement only one `CGI` of your choice. +- You are allowed to fork a new process to run the `CGI`. +- `CGI` expects the file to process as first argument and `EOF` as end of the body. +- Pay attention to the directory where the `CGI` will run for correct relative paths handling. +- The `CGI` will check `PATH_INFO` environment variable to define the full path. + +#### Configuration File + +In the file you should be able to specify the following: + +- The host (server_address) and one or multiple ports for each server. +- The first server for a host:port will be the default if the "server_name" didn't match any other server. +- Path to custom error pages. +- Limit client body size for uploads. +- Setup routes with one or multiple of the following settings: + - Define a list of accepted HTTP methods for the route. + - Define HTTP redirections. + - Define a directory or a file from where the file should be searched (for example, if `/test` is rooted to `/usr/Desktop`, the URL `/test/my_page.html` will route to `/usr/Desktop/my_page.html`). + - Define a default file for the route if the URL is a directory. + - Specify a `CGI` to use for a certain file extension. + - Turn on or off directory listing. + - Set a default file to answer if the request is a directory. +- No need to manage comments "(#)". + +> Routes won't need to support regular expressions. + +> There is no need to pass through `poll` when reading the configuration file. + +#### Testing your server +- Do stress tests (for example with `siege -b [IP]:[PORT]`), it must stay available at all costs (availability should be up to 99.5). +- Create tests for as many cases as you can (redirections, bad configuration files, static and dynamic pages, default error pages and so on). +- You will be requested to provide and explain your tests during the audits. +- You can use the language you prefer to write tests, as long as they are exhaustive and the auditor can check their behavior. +- Test possible memory leaks before to submit the project. +- Once again, the server should never crash and never leak memory. + +### Bonus +- Handle at least one more `CGI`. +- Write the project in two different programming languages. + +> If the two languages are C and C++ the provided solution for C++ should heavily rely on C++ specific features. diff --git a/subjects/localhost/audit/README.md b/subjects/localhost/audit/README.md new file mode 100644 index 00000000..5f6d1ee4 --- /dev/null +++ b/subjects/localhost/audit/README.md @@ -0,0 +1,65 @@ +#### Functional + +#### Localhost is about creating your own HTTP server and test it with an actual browser. +#### Take the necessary time to understand the project and to test it, looking into the source code will help a lot. + +### Basic server mechanics + +#### The student should be able to justify his choices and explain the following: +###### How does an HTTP server works? +###### Which function was used for I/O Multiplexing and how does it works? +###### Is the server using only one select (or equivalent) to read the client requests and write answers? +###### Why is it important to use only one select and how was it achieved? +###### Read the code that goes from the select (or equivalent) to the read and write of a client, is there only one read or write per client per select (or equivalent)? +###### Are the return values for I/O functions [read,recv,write,send] checked properly? (checking only -1 or 0 is not enough, both should be checked). +###### If an error is returned by the previous functions on a socket, is the client removed? +###### Is writing and reading ALWAYS done through a select (or equivalent)? + +### Configuration file + +#### Check the configuration file and ensure the following configs are working: +##### Setup a single server with a single port. +##### Setup multiple servers with different port. +##### Setup multiple servers with different hostnames (for example: curl --resolve test.com:80:127.0.0.1 http://test.com/). +##### Setup custom error pages. +##### Limit the client body (for example: curl -X POST -H "Content-Type: plain/text" --data "BODY with something shorter or longer than body limit"). +##### Setup routes and ensure they are taken into account. +##### Setup a default file in case the path is a directory. +##### Setup a list of accepted methods for a route (for example: try to DELETE something with and without permission). + +### Methods and cookies + +#### For each method be sure to check the status code (200, 404 etc): +###### Are the GET requests working properly? +###### Are the POST requests working properly? +###### Are the DELETE requests working properly? +###### Test a WRONG request, is the server still working properly? +###### Upload some files to the server and get them back to test they were not corrupted. +###### A working session and cookies system is present on the server? + +### Interaction with the browser + +#### Open the browser used by the team during tests and its developer tools panel to help you with tests. +###### Is te browser connecting with the server with no issues? +###### Are the request and response headers correct? (It should serve a full static website without any problem). +###### Try a wrong URL on the server, is it handled properly? +###### Try to list a directory, is it handled properly? +###### Try a redirected URL, is it handled properly? +###### Check the implemented CGI, does it works properly with chunked and unchunked data? + +### Port issues + +###### Configure multiple ports and websites and ensure it is working as expected. +###### Configure the same port multiple times. The server should find the error. +###### Configure multiple servers at the same time with different configurations but with common ports. Ask why the server should work if one of the configurations isn't working. + +### Siege & stress test + +##### Use siege with a GET method on an empty page, availability should be at least 99.5% with the command `siege -b [IP]:[PORT]`. +##### Check if there is no memory leak (you could use some tools like top). +##### Check if there is no hanging connection. + +### Bonus Part + +##### +There's more than one CGI system such as [Python,C++,Perl]. +##### +There is a second implementation of the server in a different language (repeat practical tests on it before to validate).