last of the lectures done
This commit is contained in:
Zed A. Shaw
parent
c0ed9f0dcd
commit
73716db4e4
|
|
@ -14,40 +14,70 @@ Project Description
|
|||
The Plan
|
||||
====
|
||||
|
||||
Make the *statsserver* do something using a simple protocol.
|
||||
|
||||
|
||||
|
||||
The Purpose
|
||||
====
|
||||
|
||||
Learn the first steps in create a server that answers a protocol.
|
||||
|
||||
|
||||
|
||||
The Requirements
|
||||
====
|
||||
|
||||
Create this protocol:
|
||||
|
||||
create Create a new statistic.
|
||||
mean Get the current mean of a statistic.
|
||||
sample Add a new sample to a statistics.
|
||||
dump Get all of the elements of a statistic (sum, sumsq, n, min, and max).
|
||||
|
||||
|
||||
|
||||
The Requirements
|
||||
====
|
||||
|
||||
1. You'll need to allow people to name these statistics, which means using one of the map style data structures to map names to ``Stats`` structs.
|
||||
2. You'll need to add the ``CRUD`` standard operations for each name. CRUD stands for create read update delete. Currently, the list of commands above has create, mean, and dump for reading; and sample for updating. You need a delete command now.
|
||||
3. You may also need to have a ``list`` command for listing out all of the available statistics in the server.
|
||||
|
||||
|
||||
Pause!
|
||||
====
|
||||
|
||||
I'm going to give you clues to solve this, so if you want to try on your own pause now!
|
||||
|
||||
|
||||
|
||||
The Clues
|
||||
====
|
||||
|
||||
* Create the data structures first for holding the information for each of these commands.
|
||||
* Then write a protocol parser to handle it and fill in the data.
|
||||
* Then pass that data to a function that knows how to do that command.
|
||||
* You can just store the stats in a Hashmap, BSTree, or TSTree for now.
|
||||
* KEEP IT SIMPLE!
|
||||
|
||||
|
||||
|
||||
Important References
|
||||
====
|
||||
|
||||
* You'll want to refer to the bstring documentation as much as possible to know what functions to use.
|
||||
|
||||
|
||||
|
||||
Encouragement
|
||||
====
|
||||
|
||||
* Remember that this is *supposed* to be hard.
|
||||
* You are *supposed* to struggle with this.
|
||||
* This could take you a while, but keep up the struggle, do it bit by bit, and test little pieces as you go.
|
||||
* Automate your tests!
|
||||
|
||||
|
||||
|
||||
End Of Lecture 49a
|
||||
|
|
|
|||
|
|
@ -14,25 +14,25 @@ Solution
|
|||
The Plan
|
||||
====
|
||||
|
||||
I'll show you how I implemented the protocol in the smallest code possible.
|
||||
|
||||
|
||||
|
||||
The Purpose
|
||||
====
|
||||
|
||||
I won't implement all of the CRUD operations, so you can go look at the
|
||||
git repo for this project to see a full implementation.
|
||||
|
||||
|
||||
|
||||
The Setup
|
||||
====
|
||||
|
||||
|
||||
First I setup the data, then the protocol parser, then the handlers.
|
||||
|
||||
|
||||
|
||||
The Final Code
|
||||
====
|
||||
|
||||
The last thing I would do is add better tests and round out the protocol with CRUD operations.
|
||||
|
||||
|
||||
|
||||
End Of Lecture 49b
|
||||
|
|
|
|||
|
|
@ -14,36 +14,59 @@ Project Description
|
|||
The Plan
|
||||
====
|
||||
|
||||
You are now given vague instructions and have to "solve" as best you can.
|
||||
|
||||
|
||||
|
||||
The Purpose
|
||||
====
|
||||
|
||||
To give you freedom to be creative, and also taste a real project with vague
|
||||
specifications.
|
||||
|
||||
|
||||
The Requirements
|
||||
====
|
||||
Many times all you get is a single sentence in a bug tracker. Oh well.
|
||||
|
||||
|
||||
|
||||
The Requirements
|
||||
====
|
||||
|
||||
Allow people to work with statistics at arbitrary URLs in the server.
|
||||
You get to define what that means, but think "web application".
|
||||
|
||||
|
||||
|
||||
Pause!
|
||||
====
|
||||
|
||||
Try to solve it on your own then continue.
|
||||
|
||||
|
||||
|
||||
The Clues
|
||||
====
|
||||
|
||||
Answer these questions:
|
||||
|
||||
1. What happens when I have a statistics "under" another, as in /age/northamerica/ is under /age/.
|
||||
2. Could you do the summary statistics we talked about? A mean of means and mean of standard deviations that are rolled up the tree?
|
||||
3. What data structures do you need? Starting with data is key here too. Data data data.
|
||||
4. Are your tests good enough? Before you start you might want to get good tests that use the protocol.
|
||||
|
||||
|
||||
|
||||
Important References
|
||||
====
|
||||
|
||||
* Definitely look at the statistics code you built in liblcthw if you do the summary statistics.
|
||||
|
||||
|
||||
|
||||
Encouragement
|
||||
====
|
||||
|
||||
This is hard, as I've said all along, however it is all doable. It's simply a matter of breaking the problems down and tackling each little piece.
|
||||
|
||||
|
||||
|
||||
End Of Lecture 50a
|
||||
|
|
|
|||
|
|
@ -14,24 +14,21 @@ Solution
|
|||
The Plan
|
||||
====
|
||||
|
||||
|
||||
|
||||
The Purpose
|
||||
====
|
||||
Show you how I solved the problem of routing the names of statistics as URLs.
|
||||
|
||||
|
||||
|
||||
The Setup
|
||||
====
|
||||
|
||||
|
||||
|
||||
The Server
|
||||
====
|
||||
1. First thing I did was make sure my tests were really good.
|
||||
2. Then I designed the data structures I'd need.
|
||||
3. Then I did the work to make them functions.
|
||||
4. The protocol shouldn't need to change.
|
||||
|
||||
|
||||
|
||||
The Echo
|
||||
The Code
|
||||
====
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -14,36 +14,55 @@ Project Description
|
|||
The Plan
|
||||
====
|
||||
|
||||
Learn to store the statistics to the hard disk.
|
||||
|
||||
There are meany issues with this.
|
||||
|
||||
|
||||
|
||||
The Purpose
|
||||
====
|
||||
|
||||
To teach you about various problems related to securely storing files.
|
||||
|
||||
|
||||
|
||||
The Requirements
|
||||
====
|
||||
|
||||
For this exercise, you'll add two commands for storing to and loading statistics
|
||||
from a hard drive:
|
||||
|
||||
store
|
||||
If there's a URL, store it to a hard drive.
|
||||
|
||||
load
|
||||
If there are two URLs, load the statistic from the hard drive based on the first URL, and then put it into the second URL that's in memory.
|
||||
|
||||
|
||||
|
||||
The Requirements
|
||||
====
|
||||
|
||||
1. If URLs have ``/`` characters in them, then that conflicts with the filesystem's use of slashes. How will you solve this?
|
||||
2. If URLs have ``/`` characters in them, then someone can use your server to overwrite files on a hard drive by giving paths to them. How will you solve this?
|
||||
3. If you choose to use deeply nested directories, then traversing directories to find files will be very slow. What will you do here?
|
||||
|
||||
|
||||
|
||||
The Requirements
|
||||
====
|
||||
|
||||
4. If you choose to use one directory and hash URLs (oops, I gave a hint), then directories with too many files in them are slow. How will you solve this?
|
||||
5. What happens when someone loads a statistic from a hard drive into a URL that already exists?
|
||||
6. How will someone running ``statserve`` know where the storage should be?
|
||||
|
||||
|
||||
|
||||
The Clues
|
||||
====
|
||||
|
||||
|
||||
|
||||
Important References
|
||||
====
|
||||
|
||||
|
||||
|
||||
Encouragement
|
||||
====
|
||||
There are no clues. You can do this.
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -14,11 +14,7 @@ Solution
|
|||
The Plan
|
||||
====
|
||||
|
||||
|
||||
|
||||
|
||||
The Purpose
|
||||
====
|
||||
Show you how I solved the problem of storing the statistics to disk.
|
||||
|
||||
|
||||
|
||||
|
|
@ -26,6 +22,9 @@ The Purpose
|
|||
The Setup
|
||||
====
|
||||
|
||||
1. I first wrote a simple storage backend.
|
||||
2. Then I tested it really well.
|
||||
3. Then I worked it into the server at key points.
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -14,36 +14,73 @@ Project Description
|
|||
The Plan
|
||||
====
|
||||
|
||||
Learn to improve your server by hacking it.
|
||||
|
||||
|
||||
|
||||
The Purpose
|
||||
====
|
||||
|
||||
Turn on defensive mode and destroy what you've created to make it better.
|
||||
|
||||
|
||||
The Requirements
|
||||
|
||||
Your Protocol Is Junk
|
||||
====
|
||||
|
||||
"What?! But you told me to write it!"
|
||||
|
||||
|
||||
The Requirements
|
||||
|
||||
Hacking The Protocol
|
||||
====
|
||||
|
||||
* Fuzzing the names.
|
||||
* Thrashing the size of strings.
|
||||
* Putting '\0' in the data.
|
||||
* Feeding crazy large numbers in.
|
||||
|
||||
|
||||
The Clues
|
||||
|
||||
The Rule
|
||||
====
|
||||
|
||||
Protocols without exact grammars *or* fixed size elements will be hacked.
|
||||
|
||||
|
||||
Important References
|
||||
|
||||
Your Disk Storage Is Junk
|
||||
====
|
||||
|
||||
"What?! C'mon!"
|
||||
|
||||
|
||||
Encouragement
|
||||
|
||||
Hacking Your Disk
|
||||
====
|
||||
|
||||
* Fuzzing the file names.
|
||||
* Filling the disk.
|
||||
* Rampant create/delete operations.
|
||||
* Doing insanely long directories.
|
||||
* Trying to access arbitrary files.
|
||||
|
||||
|
||||
|
||||
The Rule
|
||||
====
|
||||
|
||||
The disk works, but you need a way to monitor storage or just have
|
||||
tons of it.
|
||||
|
||||
Watch for arbitrary paths and *always* sanitize them.
|
||||
|
||||
|
||||
|
||||
Other Ways To Break It
|
||||
====
|
||||
|
||||
Review your own code using our defensive techniques and see if you can find more holes. How many can you find?
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -14,19 +14,40 @@ Solution
|
|||
The Plan
|
||||
====
|
||||
|
||||
Hack the server with a few simple scripts to demonstrate problems.
|
||||
|
||||
Show you my improvements to the server to solve the problems I discussed.
|
||||
|
||||
|
||||
|
||||
The Purpose
|
||||
====
|
||||
|
||||
Teach you that you *never* assume your code is good.
|
||||
|
||||
You're a scientist not a god.
|
||||
|
||||
Always assume there's a chance you're wrong.
|
||||
|
||||
|
||||
|
||||
|
||||
The Final Code
|
||||
Hacking Demo
|
||||
====
|
||||
|
||||
This is an ultra light hacking demo.
|
||||
|
||||
It could get much worse if I had more time.
|
||||
|
||||
|
||||
|
||||
Fixing These Flaws
|
||||
====
|
||||
|
||||
Now I'll fix a few of these flaws to show you how it's done, and leave the
|
||||
rest to you.
|
||||
|
||||
Look at the github repository to see all of the things I fixed.
|
||||
|
||||
|
||||
|
||||
End Of Lecture 52b
|
||||
|
|
|
|||
|
|
@ -0,0 +1,59 @@
|
|||
Learn C The Hard Way
|
||||
=======
|
||||
|
||||
Exercise 52
|
||||
----
|
||||
|
||||
Hacking and Improving Your Server
|
||||
|
||||
Solution
|
||||
----
|
||||
|
||||
|
||||
|
||||
The Plan
|
||||
====
|
||||
|
||||
Discuss the final things I'd do to make a project complete, by showing you
|
||||
my final project as it lives on github.
|
||||
|
||||
|
||||
|
||||
Getting It Online
|
||||
====
|
||||
|
||||
|
||||
Get it online and accessible to people.
|
||||
|
||||
|
||||
|
||||
Documenting It
|
||||
====
|
||||
|
||||
Document it and improve the usability to make sure that the documents are easy to read.
|
||||
|
||||
|
||||
|
||||
Test Coverage
|
||||
====
|
||||
|
||||
Do as much test coverage as possible.
|
||||
|
||||
|
||||
|
||||
Handling Corner Cases
|
||||
====
|
||||
|
||||
Improve any corner cases and add defenses against any attacks that I can find.
|
||||
|
||||
|
||||
|
||||
The Final Code
|
||||
====
|
||||
|
||||
|
||||
|
||||
End Of Lecture 52c
|
||||
=====
|
||||
|
||||
|
||||
Loading…
Reference in New Issue