diff --git a/p3.md b/p3.md new file mode 100644 index 0000000000000000000000000000000000000000..839a69a38678d89ef1a84f3206df05e25c37a8b6 --- /dev/null +++ b/p3.md @@ -0,0 +1,376 @@ +# Project 3: Crypto +**Due: Sept 28, 2024, 11:59:59 pm, v1.0** + +### Overview + +This project has the following tasks: +1. Generate public/private RSA key pairs pete. +1. Use public-key crypto to sign and validate signatures +1. Build signed *bindings* of permanent names to changing values. +1. Add a new `dump` command to see the raw bindings. +1. Add a new `verifybinding` command to verify bindings. + + + +## New Packages +The new crypto packages we will be using include: +- **pem**: PEM (Privacy Enhanced Mail) encoding is the de facto format + for storing and sending cryptographic keys and certificates. This is + just encoding, like `base32`, and is not encryption. + - **x509**: Widespread standard for public key certificate + formats. We will not be using certificates, but we will be using + routines from this package. +- **rand**: random numbers +- **rsa**: RSA (Rivest-Shamir-Adleman) public keys. + + +## Task 1: Generate a public/private keypair +Signing, verifying, and using JSON blobs (including recipes) is going to be the +centerpiece of this project. Public key crypto is asymmetric: they +come in pairs with a *public* key and a *private* key. For this +project, we will use the private key just to sign JSON blobs, and the +public key to verify the signatures. As you might expect, public keys +are not secret, in fact they need to be widely disseminated in order +to be used. Private keys are secret. + +- `go run blob.go genkeys`: Should generate a 2048-bit RSA private key using the `rsa` routine + `GenerateKey`, and write to output file `key.private`. Write the corresponding public key + to `key.public`. + +To write the public key: +- create a `pem.Block` specifying `Type` "RSA PUBLIC KEY" and passing + the results of `x509.MarshalPKCS1PublicKey` to `Bytes`. +- encode this using `pem.EncodeToMemory` +- write the results to a file. + +The resulting private key should look something like this: +``` +-----BEGIN RSA PRIVATE KEY----- +MIIEpAIBAAKCAQEAqxFEoBIumwA9h+t9xWPNA9gikUyB0ACgBW39MEwNLskg0TNx +o/HG9fPbNbLA5rrRNsp9NzGTC7DbcZiguMvraxGIsX3i0zjHPzPUYZoyIoSUtx2m +OoAn8G6E1gE3jSImp1GZar6CtD7uzTSHJbJ21CHJAc723LIvpBa1wZ/62zuuzkts +Wav9lvCaXx+7w3gbwMp5E4vLCxthKx3GWKCJzg0SLwTiBQDSJnqbodXric/KE/cg +rWcT1Og2McHWaxT0uY7RPpYQYIX170KE69QiYbZ1PpE6o464/YjwjNhfKe5CfFci +tt1cL9lCwBCZqPDjtQsO4cRX6FPVS5RvAk/D4wIDAQABAoIBABdp86SdGHfqnoFe +AAbmVAc0q/aLLDFWBJD5ru/PWPaQXMeFbQZtbzf2uogtAS4TX9NJ/71wMZomCMMR +it4AOyaabcUtX8BdQxgpdeYt/rKuxtQRYNEc/VpxJglMfVf51qSMDJ2JmcYl3vWD +PQQx2wXJ4gM3Wp52XQLugM4RoXokBc2MMM0zbOLkyzLbLfLKlHQWcUiSKPTFKEuH +9NH5Oc9aEnWdEUI0vElXBXCX1dgtjLjgbjEeU62ZN9hnRAMnp0sxz233jc6vVU0L +4yuVpRX/4m7r6S9mVYf3KMD8D4EYSLHXy0FzV7Ac7qnyximMLwNQDwbtp++eNqjE +C055jZECgYEAz+GBZ0ZfN5rzGJ9NTEYxxOszxO93ysxlrDTjposz08vM5GJlFM1t +UUi7KYhTShEqQFClhwkqW2FdE9LoOCq4BhCGU5pbnq3BHWV9uO6cHMuhMD93S9XD +hozbZk9r8MrGd89Z2X/tK+KzBSLzayYduNcYT5XPJP3HtSHIin7qBw0CgYEA0qpK +x8osznri58J+cvjWB2sNrLEUW1+yw/Q428Qek2b/r1IlV4pl8K6exjAw/rz81/MB +B+5s32Gm8psmqcT3qTpTPyVtbBkFsCwgAM1rmixm4V7QLvsDnyrZsuBN4T0zCtQ+ +8+cf39XWFFIDjMvu8vGcJgLR5jj4SGBJdvasOq8CgYEAgHvGeUhbbYjNm3hKVExG +Uol0s9G2Xpe6d5cw5SzAWbVq6/WMuDDH0id71o21vN+jF6FAzZdyoIwq9Sez85Mj +rkvkWiPbYNXPuBWUgQqpXnrVI3b2it6SPMUujauk8WzDAiYcSHvy4N76+r/BZ4Zl +dGstUXMsVpasKl25DzCmALkCgYEAu8qMEN9b49BNtxV8zRafDEvVC81q/S0o2V86 +1EVWkEWvxWSv3wKDbvLqnHdXJa3oosR/dceHi/Wr8fZ2l736nANfNBo0GbmQhYRA +Hxb/RZcxOtPfNxISH2/+UmN7aT654nxjhd7RXiJrzP9zJK6iWjUg4g1/eP/t6+7R +blfkHgcCgYAu3oReQNNFHuz8j8fJ2jWWA7E0+E25MUaBwpvfMQnxOG5RA0ExlNYL +66ju0Teb1s2KmAnPZyOceQUhKg04NJTJot/2wyaa1jhXiuAfbVg6xdXjFjcA+Be3 +iqNNVTfiJPDPPa5MjgZqNaPZ7S0eIzxe8pRHVGoR7C8f02msYswrOQ== +-----END RSA PRIVATE KEY----- +``` + +Write the private key out similarly, substituting "public" for +"private" where appropriate: +``` +-----BEGIN RSA PUBLIC KEY----- +MIIBCgKCAQEAqxFEoBIumwA9h+t9xWPNA9gikUyB0ACgBW39MEwNLskg0TNxo/HG +9fPbNbLA5rrRNsp9NzGTC7DbcZiguMvraxGIsX3i0zjHPzPUYZoyIoSUtx2mOoAn +8G6E1gE3jSImp1GZar6CtD7uzTSHJbJ21CHJAc723LIvpBa1wZ/62zuuzktsWav9 +lvCaXx+7w3gbwMp5E4vLCxthKx3GWKCJzg0SLwTiBQDSJnqbodXric/KE/cgrWcT +1Og2McHWaxT0uY7RPpYQYIX170KE69QiYbZ1PpE6o464/YjwjNhfKe5CfFcitt1c +L9lCwBCZqPDjtQsO4cRX6FPVS5RvAk/D4wIDAQAB +-----END RSA PUBLIC KEY----- +``` + +Reading the keys back in requires: +- reading data from the file +- decoding the pem encoding +- x509 parsing (rather than marshalling) + + +## Task 2: Sign and Verify + + +### Signing +Cryptographic signing is accomplished by computing a *hash* of the +material to be signed, and encrypting that hash with a private key. We +encrypt a hash rather than the actual text because public key crypto +works only over small amounts of data, limited by key size. + +For this project we want to include our signatures in the JSON text, +meaning that we have a recursion problem: we can not sign something +that includes the signature resulting from the signing. The answer is +is that we will sign everything *except* the signature, and then just +add the signature on the end. + +We define signing on anything in JSON. Let's start with an example file recipe: +``` +io> go run blob.go put blob.go +sha256_32_TFBQWYJ7DGDPXPMYUQI7XJCF73FYCB27DP75EPMLBAHXT4JYATEA==== +io> go run blob.go desc last +{ + "Name": "blob.go", + "Size": 3228, + "Mode": 420, + "ModTime": "2023-09-15T21:46:37.753398955-04:00", + "IsDir": false, + "Version": 1, + "PrevSig": "", + "ChildSigs": null, + "ChildNames": null, + "DataBlockSigs": [ + "sha256_32_ZXF7E5X2VWUXNMY2ERBBEVG2KEKCB6KYMVBYFDQ5EQTTQBOMEUUA====" + ] +} +``` +Signing this blob gives us: +``` +io> go run blob.go -q key.private sign last +sha256_32_63W7EPA6FACLUPU6N2NPVE2IQBUXV7UKIYNXVHRZEKGZJG7EX3EA==== +io> go run blob.go desc last +{ + "Name": "blob.go", + "Size": 3228, + "Mode": 420, + "ModTime": "2023-09-15T21:46:37.753398955-04:00", + "IsDir": false, + "Version": 1, + "PrevSig": "", + "ChildSigs": null, + "ChildNames": null, + "DataBlockSigs": [ + "sha256_32_ZXF7E5X2VWUXNMY2ERBBEVG2KEKCB6KYMVBYFDQ5EQTTQBOMEUUA====" + ] +, +"signature": "RMNDXPYJUASFBSYJ4XQDLLXG64SOEH32CTNMZE3VRXSRJXXHTZO74MRO3MBHRUJKSYMWRCFH22RB3PAGVU4PARQK223II2BFGDUBCMHRD5R4BOAOH2MK7LO446HRKVQHQX3ZT542DSTVSB6ZRA7ASSZLNXPS6WR3VKOED5OEGB7YG45X3LFTZPJUUBG7YRGT7NEXECCOPUNSETU2B2V45KGRSE3MNAU2E3WT62F36AH7YJR2IIVFECPBDJHMMQ7LW6EPQ5PHG4UXPRQJZ4GC37KHB4F44VQIOIMSMRFHQNILRM3DYW3KYQQUH2Y64TKCQAK6NSWCRGWAJO5RP32USYU7AJCDLKS6U6IW5GGKSPJLN55Y2ON2WKAVZVTEOUDMRPPZFVBTDQX63IIETRI53QZPKA======"} +``` + +The signed blob differs only at the end. The +procedure for generating the signature should be followed exactly: +1. read in the private key +1. trim all whitespace off the end of the JSON +1. remove the last character (which should be '}') +1. take the SHA256 hash +1. call `rsa.SignPKCS1v15` on the hash result: this gives the raw signature +1. encode the signature using base32 +2. add ",\n\"signature\": \"", the encoded signature, and "\"}" to the + end of the truncated original blob + + +**Verification** is mostly the same, but in reverse. Left as an +exercise for the reader: + +``` +io> go run blob.go -p key.public verify last +Verification of "sha256_32_63W7EPA6FACLUPU6N2NPVE2IQBUXV7UKIYNXVHRZEKGZJG7EX3EA====" w/ key.public succeeded +``` + + + +## Task 3: Mutable signed *bindings* + +Our series of projects so far describes a write-only store. Blobs can +be removed, but the blob corresponding to a hash can not be changed. +We'd like to support permanent names, called *bindings*, that +can point to different content over time. + +For example, you might define a `root` binding that points to the current +root of a file system stored in your blobstore. You might define a +`status` binding that shows the current status of your system. In +either case the binding binds the *Name* to a *Value*, where *Value* is the +sig containing the current content. +``` +type Binding struct { + Name "status" + Value "sha256_32_63W7f...." + PublicKeyHash ".....something....." // discussed below +} +``` + +Ordinary file recipes are protected in the sense that their names are +derived from their content; there is no way to modify either without +detection. + +Since a binding is designed to evolve over time, we need to have a +different way of securing it. We therefore *sign* the JSON Binding, +and include the public key in the JSON. Including the public key has +two uses: +- First, public keys are publicly bound to individual principals, thereby + identifying the binding's creator. +- Second, the binding is signed by the included public key's private + key counterpart. Therefore a binding can be validated by using the included public key +to verify the signature. + +The following command creates a binding: +- `-p <publickey.fname> -q <privatekey.fname> binding <raw binding sig>`: create a bindings + + +Say, for example, that we wish a permanent link to our server's status page. We can create an initial page as follows: +``` +io:p3> go run blob.go put status.html +sha256_32_ZQOZMQ7KQ3LHR3OCHSBIUIUITUM3HJRNGG6WMTXHSENUQN54PIZA==== +io:p3> go run blob.go desc last +{ + "Name": "status.html", + "Size": 75, + "Mode": 420, + "ModTime": "2023-09-15T21:46:37.754750286-04:00", + "IsDir": false, + "Version": 1, + "PrevSig": "", + "ChildSigs": null, + "ChildNames": null, + "DataBlockSigs": [ + "sha256_32_7JXCJT52ARJ7UWE5WOYUEHOP2LZOTSC6KFBZDOHIH7SIX37BQ7VA====" + ] +} +``` +This can also be retrieved at: +``` +http://localhost:8000/sha256_32_7JXCJT52ARJ7UWE5WOYUEHOP2LZOTSC6KFBZDOHIH7SIX37BQ7VA==== +``` + +The above work (w/ the http server listening to port 8000), but it's +unwieldy and immutable. +We can use a *binding* to make the status mutable and easier to type/remember: +``` +io:p3> go run blob.go -p key.public -q key.private binding status last +status +io:p3> go run blob.go desc status +{ + "Name": "status.html", + "Size": 75, + "Mode": 420, + "ModTime": "2023-09-15T21:46:37.754750286-04:00", + "IsDir": false, + "Version": 1, + "PrevSig": "", + "ChildSigs": null, + "ChildNames": null, + "DataBlockSigs": [ + "sha256_32_7JXCJT52ARJ7UWE5WOYUEHOP2LZOTSC6KFBZDOHIH7SIX37BQ7VA====" + ] +} +``` +Note that **the binding is automatically resolved** to it's `Value`. This +means that `http://localhost:8000/status` now shows the current status page. +``` +io:p3> curl 'localhost:8000/status' +<center> +<h1>Things are looking pretty Bleak.</h1> +``` +We do not (yet) have a way to view an existing binding. + +### The public key property +The `PublicKey` property of the binding is the hash of the +public key corresponding to the private key used to create the +signature. A hash can be used to verify something, but it can not be +reversed to get back the public key on which it is based. + +Luckily, we have a means of mapping hashes back to hashed data: +our store. Your code binding creation code, therefore, should also: +- write the public key to the store +- put the resulting hash in the binding as the value for + "PublicKeyHash" + +### New command: `dump` +Recall we can not view the binding we inserted using `desc`, as it +will automatically +resolve to the binding's value. + +Create a new function `dump`. `dump` is identical to `desc` except that it +retrieves blobs in a *raw* mode: returning the raw binding text rather than resolving +through to the binding `Value`. + +Implementation of the dump command requires an extra `raw` parameter to be added to +`BlobGetRequest`. +When `raw` is `false`, the server's local blob read should +check to see if the blob is a `Binding` (format is JSON, and can successfully +be unmarshalled into a `Binding` struct) and *resolve* this binding by +returning the blob pointed to by the binding's `Value` property, +rather than the binding itself. + +If `raw` is `true`, the binding itself should be returned. + +``` +io:~/818/projects/p3/solution> go run blob.go dump status +{ + "Name": "status", + "Value": "sha256_32_7WT7PVM6OMK5LMKZRTBRO3L4ECCFGSZI5OL7VV6ZYTG3QUDMDIYA====", + "PublicKeyHash": "sha256_32_MC5S74TZMM7HG7ZK25QMTA5SHBCFEBUGZQF2IX24NSRRL5NWOC3A====" +, +"signature": "BYPHD3JGZLGJVWGEJHDU5KN7G6JDKU2GQKQ575DPAMDCR77B6ZBMYNEABVOSLGJVKXTXNK7NYNAKMTRKXJ3GW5YJUUVAATNDVF7XBFH27H4EZPDQULQRC4PFZABBJQIT7FFWX4E73VHNQZSJRVLYM3LB243FJD46532JNK243HKIPKNUFGUB2PLENV2AD72HV7CR3ORRTBRHFRRDSLYOCB7M7LXTFG26FXGM2VG3ZKMPPXO7MII7FIWOIN77D75D5IMBJ6RMM3KQWP3TPPEVMECOXEMI7ITKA3QKWWLI7KCR2TXHWTEOU3YY2TE4N7FHR7TALCERFPPTH5LDRFKB3YHZLCQATMBDFSETCWV63TVXWDCB2ZQTMLYFW4LTBOOUSMVGVYMXGYBUJZMYKRWYJADZCI======"} +``` + + +### Updating a binding +A binding is updated by replacing it with a new binding using the same name. +We can update the current status shown above by overwriting the `status` binding with a new blob name: +``` +io:p3> go run blob.go put status2.html +sha256_32_XDZOUHLABSFPCFT456UMXZKKGODPOS4YKOMIGQZOKLQREYTOHJMQ==== +io:p3> go run blob.go -p key.public -q key.private binding status last +status +io:p3> curl 'localhost:8000/status' +<center> +<h1>Things are all better now. </h1> +``` + + +## Task 4: Verify bindings +A binding can be verified by using `PublicKeyHash` property to retrieve +the public key from the blob store, and then using that public key to +verify the binding's signature. + +Do this by creating a new command: `verifybinding <name>`. +Assuming the public key is in the store, as described above, you will +not need to pass either key to the blob client at the command line. + +``` +io:~/818/projects/p3/solution> go run blob.go verifybinding status +Verification of "status" w/ key.public succeeded +``` + +More pedantically, verify a binding as follows: +- retrieve the actual binding blob (not what it resolves to) +- unmarshal a binding object from the blob +- grab the object's `PublicKeyHash` property and retrieve the associated blob +- write that blob to a file in /tmp +- verify the signature on the binding JSON w/ the saved public key + + +## Servers and Trust +Most people of servers as *trusted*, and with good reason. Trusted +servers make life easier. However, even if you trust your server +provider's intent, no security is unbreakable. + +Our server in this project might be categorized as "trust but +verify". Anything we get back from the server can be verified or +checked for tampering. Simple blobs must match their hashes. Signed +JSON recipes or bindings can be verified. + +However, we have no way of checking the server is not throwing blobs +away, or returning "blob not found" which it is actually there. + +In this course we will be discussing a couple papers ("SUNDr", +"Spork") that develop completely untrusted servers. But not trusting +your server has consequences, both in terms of complexity and +performance. In practice, most systems are somewhere in the +middle. For example, we could require clients to authenticate with the +server, or require servers to validate signed blobs before accepting them. + +## Video walkthrough + + + +## Grading +I will give roughly even credit for each of the tasks. + +## Submit +Commit to the repository. + diff --git a/p3.md~ b/p3.md~ new file mode 100644 index 0000000000000000000000000000000000000000..f21dd2f25c3ccf52c6312aec9994f6c8df7c3fb1 --- /dev/null +++ b/p3.md~ @@ -0,0 +1,372 @@ +# Project 3: Crypto +**Due: Sept 28, 2024, 11:59:59 pm, v1.0** + +### Overview + +This project has the following tasks: +1. Generate public/private RSA key pairs pete. +1. Use public-key crypto to sign and validate signatures +1. Build signed *bindings* of permanent names to changing values. +1. Add a new `dump` command to see the raw bindings. +1. Add a new `verifybinding` command to verify bindings. + + + +## New Packages +The new crypto packages we will be using include: +- **pem**: PEM (Privacy Enhanced Mail) encoding is the de facto format + for storing and sending cryptographic keys and certificates. This is + just encoding, like `base32`, and is not encryption. + - **x509**: Widespread standard for public key certificate + formats. We will not be using certificates, but we will be using + routines from this package. +- **rand**: random numbers +- **rsa**: RSA (Rivest-Shamir-Adleman) public keys. + + +## Task 1: Generate a public/private keypair +Signing, verifying, and using JSON blobs (including recipes) is going to be the +centerpiece of this project. Public key crypto is asymmetric: they +come in pairs with a *public* key and a *private* key. For this +project, we will use the private key just to sign JSON blobs, and the +public key to verify the signatures. As you might expect, public keys +are not secret, in fact they need to be widely disseminated in order +to be used. Private keys are secret. + +- `go run blob.go genkeys`: Should generate a 2048-bit RSA private key using the `rsa` routine + `GenerateKey`, and write to output file `key.private`. Write the corresponding public key + to `key.public`. + +To write the public key: +- create a `pem.Block` specifying `Type` "RSA PUBLIC KEY" and passing + the results of `x509.MarshalPKCS1PublicKey` to `Bytes`. +- encode this using `pem.EncodeToMemory` +- write the results to a file. + +The resulting private key should look something like this: +``` +-----BEGIN RSA PRIVATE KEY----- +MIIEpAIBAAKCAQEAqxFEoBIumwA9h+t9xWPNA9gikUyB0ACgBW39MEwNLskg0TNx +o/HG9fPbNbLA5rrRNsp9NzGTC7DbcZiguMvraxGIsX3i0zjHPzPUYZoyIoSUtx2m +OoAn8G6E1gE3jSImp1GZar6CtD7uzTSHJbJ21CHJAc723LIvpBa1wZ/62zuuzkts +Wav9lvCaXx+7w3gbwMp5E4vLCxthKx3GWKCJzg0SLwTiBQDSJnqbodXric/KE/cg +rWcT1Og2McHWaxT0uY7RPpYQYIX170KE69QiYbZ1PpE6o464/YjwjNhfKe5CfFci +tt1cL9lCwBCZqPDjtQsO4cRX6FPVS5RvAk/D4wIDAQABAoIBABdp86SdGHfqnoFe +AAbmVAc0q/aLLDFWBJD5ru/PWPaQXMeFbQZtbzf2uogtAS4TX9NJ/71wMZomCMMR +it4AOyaabcUtX8BdQxgpdeYt/rKuxtQRYNEc/VpxJglMfVf51qSMDJ2JmcYl3vWD +PQQx2wXJ4gM3Wp52XQLugM4RoXokBc2MMM0zbOLkyzLbLfLKlHQWcUiSKPTFKEuH +9NH5Oc9aEnWdEUI0vElXBXCX1dgtjLjgbjEeU62ZN9hnRAMnp0sxz233jc6vVU0L +4yuVpRX/4m7r6S9mVYf3KMD8D4EYSLHXy0FzV7Ac7qnyximMLwNQDwbtp++eNqjE +C055jZECgYEAz+GBZ0ZfN5rzGJ9NTEYxxOszxO93ysxlrDTjposz08vM5GJlFM1t +UUi7KYhTShEqQFClhwkqW2FdE9LoOCq4BhCGU5pbnq3BHWV9uO6cHMuhMD93S9XD +hozbZk9r8MrGd89Z2X/tK+KzBSLzayYduNcYT5XPJP3HtSHIin7qBw0CgYEA0qpK +x8osznri58J+cvjWB2sNrLEUW1+yw/Q428Qek2b/r1IlV4pl8K6exjAw/rz81/MB +B+5s32Gm8psmqcT3qTpTPyVtbBkFsCwgAM1rmixm4V7QLvsDnyrZsuBN4T0zCtQ+ +8+cf39XWFFIDjMvu8vGcJgLR5jj4SGBJdvasOq8CgYEAgHvGeUhbbYjNm3hKVExG +Uol0s9G2Xpe6d5cw5SzAWbVq6/WMuDDH0id71o21vN+jF6FAzZdyoIwq9Sez85Mj +rkvkWiPbYNXPuBWUgQqpXnrVI3b2it6SPMUujauk8WzDAiYcSHvy4N76+r/BZ4Zl +dGstUXMsVpasKl25DzCmALkCgYEAu8qMEN9b49BNtxV8zRafDEvVC81q/S0o2V86 +1EVWkEWvxWSv3wKDbvLqnHdXJa3oosR/dceHi/Wr8fZ2l736nANfNBo0GbmQhYRA +Hxb/RZcxOtPfNxISH2/+UmN7aT654nxjhd7RXiJrzP9zJK6iWjUg4g1/eP/t6+7R +blfkHgcCgYAu3oReQNNFHuz8j8fJ2jWWA7E0+E25MUaBwpvfMQnxOG5RA0ExlNYL +66ju0Teb1s2KmAnPZyOceQUhKg04NJTJot/2wyaa1jhXiuAfbVg6xdXjFjcA+Be3 +iqNNVTfiJPDPPa5MjgZqNaPZ7S0eIzxe8pRHVGoR7C8f02msYswrOQ== +-----END RSA PRIVATE KEY----- +``` + +Write the private key out similarly, substituting "public" for +"private" where appropriate: +``` +-----BEGIN RSA PUBLIC KEY----- +MIIBCgKCAQEAqxFEoBIumwA9h+t9xWPNA9gikUyB0ACgBW39MEwNLskg0TNxo/HG +9fPbNbLA5rrRNsp9NzGTC7DbcZiguMvraxGIsX3i0zjHPzPUYZoyIoSUtx2mOoAn +8G6E1gE3jSImp1GZar6CtD7uzTSHJbJ21CHJAc723LIvpBa1wZ/62zuuzktsWav9 +lvCaXx+7w3gbwMp5E4vLCxthKx3GWKCJzg0SLwTiBQDSJnqbodXric/KE/cgrWcT +1Og2McHWaxT0uY7RPpYQYIX170KE69QiYbZ1PpE6o464/YjwjNhfKe5CfFcitt1c +L9lCwBCZqPDjtQsO4cRX6FPVS5RvAk/D4wIDAQAB +-----END RSA PUBLIC KEY----- +``` + +Reading the keys back in requires: +- reading data from the file +- decoding the pem encoding +- x509 parsing (rather than marshalling) + + +## Task 2: Sign and Verify + + +### Signing +Cryptographic signing is accomplished by computing a *hash* of the +material to be signed, and encrypting that hash with a private key. We +encrypt a hash rather than the actual text because public key crypto +works only over small amounts of data, limited by key size. + +For this project we want to include our signatures in the JSON text, +meaning that we have a recursion problem: we can not sign something +that includes the signature resulting from the signing. The answer is +is that we will sign everything *except* the signature, and then just +add the signature on the end. + +We define signing on anything in JSON. Let's start with an example file recipe: +``` +io> go run blob.go put blob.go +sha256_32_TFBQWYJ7DGDPXPMYUQI7XJCF73FYCB27DP75EPMLBAHXT4JYATEA==== +io> go run blob.go desc last +{ + "Name": "blob.go", + "Size": 3228, + "Mode": 420, + "ModTime": "2023-09-15T21:46:37.753398955-04:00", + "IsDir": false, + "Version": 1, + "PrevSig": "", + "ChildSigs": null, + "ChildNames": null, + "DataBlockSigs": [ + "sha256_32_ZXF7E5X2VWUXNMY2ERBBEVG2KEKCB6KYMVBYFDQ5EQTTQBOMEUUA====" + ] +} +``` +Signing this blob gives us: +``` +io> go run blob.go -q key.private sign last +sha256_32_63W7EPA6FACLUPU6N2NPVE2IQBUXV7UKIYNXVHRZEKGZJG7EX3EA==== +io> go run blob.go desc last +{ + "Name": "blob.go", + "Size": 3228, + "Mode": 420, + "ModTime": "2023-09-15T21:46:37.753398955-04:00", + "IsDir": false, + "Version": 1, + "PrevSig": "", + "ChildSigs": null, + "ChildNames": null, + "DataBlockSigs": [ + "sha256_32_ZXF7E5X2VWUXNMY2ERBBEVG2KEKCB6KYMVBYFDQ5EQTTQBOMEUUA====" + ] +, +"signature": "RMNDXPYJUASFBSYJ4XQDLLXG64SOEH32CTNMZE3VRXSRJXXHTZO74MRO3MBHRUJKSYMWRCFH22RB3PAGVU4PARQK223II2BFGDUBCMHRD5R4BOAOH2MK7LO446HRKVQHQX3ZT542DSTVSB6ZRA7ASSZLNXPS6WR3VKOED5OEGB7YG45X3LFTZPJUUBG7YRGT7NEXECCOPUNSETU2B2V45KGRSE3MNAU2E3WT62F36AH7YJR2IIVFECPBDJHMMQ7LW6EPQ5PHG4UXPRQJZ4GC37KHB4F44VQIOIMSMRFHQNILRM3DYW3KYQQUH2Y64TKCQAK6NSWCRGWAJO5RP32USYU7AJCDLKS6U6IW5GGKSPJLN55Y2ON2WKAVZVTEOUDMRPPZFVBTDQX63IIETRI53QZPKA======"} +``` + +The signed blob differs only at the end. The +procedure for generating the signature should be followed exactly: +1. read in the private key +1. trim all whitespace off the end of the JSON +1. remove the last character (which should be '}') +1. take the SHA256 hash +1. call `rsa.SignPKCS1v15` on the hash result: this gives the raw signature +1. encode the signature using base32 +2. add ",\n\"signature\": \"", the encoded signature, and "\"}" to the + end of the truncated original blob + + +**Verification** is mostly the same, but in reverse. Left as an +exercise for the reader: + +``` +io> go run blob.go -p key.public verify last +Verification of "sha256_32_63W7EPA6FACLUPU6N2NPVE2IQBUXV7UKIYNXVHRZEKGZJG7EX3EA====" w/ key.public succeeded +``` + + + +## Task 3: Mutable signed *bindings* + +Our series of projects so far describes a write-only store. Blobs can +be removed, but the blob corresponding to a hash can not be changed. +We'd like to support permanent names, called *bindings*, that +can point to different content over time. + +For example, you might define a `root` binding that points to the current +root of a file system stored in your blobstore. You might define a +`status` binding that shows the current status of your system. In +either case the binding binds the *Name* to a *Value*, where *Value* is the +sig containing the current content. +``` +type Binding struct { + Name "status" + Value "sha256_32_63W7f...." + PublicKeyHash ".....something....." // discussed below +} +``` + +Ordinary file recipes are protected in the sense that their names are +derived from their content; there is no way to modify either without +detection. + +Since a binding is designed to evolve over time, we need to have a +different way of securing it. We therefore *sign* the JSON Binding, +and include the public key in the JSON. Including the public key has +two uses: +- First, public keys are publicly bound to individual principals, thereby + identifying the binding's creator. +- Second, the binding is signed by the included public key's private + key counterpart. Therefore a binding can be validated by using the included public key +to verify the signature. + +The following command creates a binding: +- `-p <publickey.fname> -q <privatekey.fname> binding <raw binding sig>`: create a bindings + + +Say, for example, that we wish a permanent link to our server's status page. We can create an initial page as follows: +``` +io:p3> go run blob.go put status.html +sha256_32_ZQOZMQ7KQ3LHR3OCHSBIUIUITUM3HJRNGG6WMTXHSENUQN54PIZA==== +io:p3> go run blob.go desc last +{ + "Name": "status.html", + "Size": 75, + "Mode": 420, + "ModTime": "2023-09-15T21:46:37.754750286-04:00", + "IsDir": false, + "Version": 1, + "PrevSig": "", + "ChildSigs": null, + "ChildNames": null, + "DataBlockSigs": [ + "sha256_32_7JXCJT52ARJ7UWE5WOYUEHOP2LZOTSC6KFBZDOHIH7SIX37BQ7VA====" + ] +} +``` +This can also be retrieved at: +``` +http://localhost:8000/sha256_32_7JXCJT52ARJ7UWE5WOYUEHOP2LZOTSC6KFBZDOHIH7SIX37BQ7VA==== +``` + +The above work (w/ the http server listening to port 8000), but it's +unwieldy and immutable. +We can use a *binding* to make the status mutable and easier to type/remember: +``` +io:p3> go run blob.go -p key.public -q key.private binding status last +status +io:p3> go run blob.go desc status +{ + "Name": "status.html", + "Size": 75, + "Mode": 420, + "ModTime": "2023-09-15T21:46:37.754750286-04:00", + "IsDir": false, + "Version": 1, + "PrevSig": "", + "ChildSigs": null, + "ChildNames": null, + "DataBlockSigs": [ + "sha256_32_7JXCJT52ARJ7UWE5WOYUEHOP2LZOTSC6KFBZDOHIH7SIX37BQ7VA====" + ] +} +``` +Note that **the binding is automatically resolved** to it's `Value`. This +means that `http://localhost:8000/status` now shows the current status page. +``` +io:p3> curl 'localhost:8000/status' +<center> +<h1>Things are looking pretty Bleak.</h1> +``` +We do not (yet) have a way to view an existing binding. + +### The public key property +The `PublicKey` property of the binding is the hash of the +public key corresponding to the private key used to create the +signature. A hash can be used to verify something, but it can not be +reversed to get back the public key on which it is based. + +Luckily, we have a means of mapping hashes back to hashed data: +our store. Your code binding creation code, therefore, should also: +- write the public key to the store +- put the resulting hash in the binding as the value for + "PublicKeyHash" + +### New command: `dump` +Recall we can not view the binding we inserted using `desc`, as it +will automatically +resolve to the binding's value. + +Create a new function `dump`. `dump` is identical to `desc` except that it +retrieves blobs in a *raw* mode: returning the raw binding text rather than resolving +through to the binding `Value`. + +Implementation of the dump command requires an extra `raw` parameter to be added to +`BlobGetRequest`. +When `raw` is `false`, the server's local blob read should +check to see if the blob is a `Binding` (format is JSON, and can successfully +be unmarshalled into a `Binding` struct) and *resolve* this binding by +returning the blob pointed to by the binding's `Value` property, +rather than the binding itself. + +If `raw` is `true`, the binding itself should be returned. + +``` +io:~/818/projects/p3/solution> go run blob.go dump status +{ + "Name": "status", + "Value": "sha256_32_7WT7PVM6OMK5LMKZRTBRO3L4ECCFGSZI5OL7VV6ZYTG3QUDMDIYA====", + "PublicKeyHash": "sha256_32_MC5S74TZMM7HG7ZK25QMTA5SHBCFEBUGZQF2IX24NSRRL5NWOC3A====" +, +"signature": "BYPHD3JGZLGJVWGEJHDU5KN7G6JDKU2GQKQ575DPAMDCR77B6ZBMYNEABVOSLGJVKXTXNK7NYNAKMTRKXJ3GW5YJUUVAATNDVF7XBFH27H4EZPDQULQRC4PFZABBJQIT7FFWX4E73VHNQZSJRVLYM3LB243FJD46532JNK243HKIPKNUFGUB2PLENV2AD72HV7CR3ORRTBRHFRRDSLYOCB7M7LXTFG26FXGM2VG3ZKMPPXO7MII7FIWOIN77D75D5IMBJ6RMM3KQWP3TPPEVMECOXEMI7ITKA3QKWWLI7KCR2TXHWTEOU3YY2TE4N7FHR7TALCERFPPTH5LDRFKB3YHZLCQATMBDFSETCWV63TVXWDCB2ZQTMLYFW4LTBOOUSMVGVYMXGYBUJZMYKRWYJADZCI======"} +``` + + +### Updating a binding +A binding is updated by replacing it with a new binding using the same name. +We can update the current status shown above by overwriting the `status` binding with a new blob name: +``` +io:p3> go run blob.go put status2.html +sha256_32_XDZOUHLABSFPCFT456UMXZKKGODPOS4YKOMIGQZOKLQREYTOHJMQ==== +io:p3> go run blob.go -p key.public -q key.private binding status last +status +io:p3> curl 'localhost:8000/status' +<center> +<h1>Things are all better now. </h1> +``` + + +## Task 4: Verify bindings +A binding can be verified by using `PublicKeyHash` property to retrieve +the public key from the blob store, and then using that public key to +verify the binding's signature. + +Do this by creating a new command: `verifybinding <name>`. +Assuming the public key is in the store, as described above, you will +not need to pass either key to the blob client at the command line. + +``` +io:~/818/projects/p3/solution> go run blob.go verifybinding status +Verification of "status" w/ key.public succeeded +``` + +More pedantically, verify a binding as follows: +- retrieve the actual binding blob (not what it resolves to) +- unmarshal a binding object from the blob +- grab the object's `PublicKeyHash` property and retrieve the associated blob +- write that blob to a file in /tmp +- verify the signature on the binding JSON w/ the saved public key + + +## Servers and Trust +Most people of servers as *trusted*, and with good reason. Trusted +servers make life easier. However, even if you trust your server +provider's intent, no security is unbreakable. + +Our server in this project might be categorized as "trust but +verify". Anything we get back from the server can be verified or +checked for tampering. Simple blobs must match their hashes. Signed +JSON recipes or bindings can be verified. + +However, we have no way of checking the server is not throwing blobs +away, or returning "blob not found" which it is actually there. + +In this course we will be discussing a couple papers ("SUNDr", +"Spork") that develop completely untrusted servers. But not trusting +your server has consequences, both in terms of complexity and +performance. In practice, most systems are somewhere in the +middle. For example, we could require clients to authenticate with the +server, or require servers to validate signed blobs before accepting them. + +## Grading +I will give roughly even credit for each of the tasks. + +## Submit +Commit to the repository. + diff --git a/p3.mp4 b/p3.mp4 new file mode 100644 index 0000000000000000000000000000000000000000..f571be575c699b88d389629106e5b7b62aeb2628 Binary files /dev/null and b/p3.mp4 differ