Lab 1: Getting started: VMs, JSON, and Hashing
Today’s lab is getting started on an exciting 8-lab sequence where we will eventually create our own fake cryptocurrency, run by mining nodes that you write the software for!
How labs will work in this class
Labs and projects are the main work for this course. In the first part of the semester, we will be working to create and run our own fake cryptocurrency network in a series of labs.
Collaboration
The goal is for each of you to write the software for and run your own node in the network. It is crucial that you write your own software, not just for the “no cheating” aspect, but because we want our network to be robust. If everyone is running the same software, it may have the same small mistakes everywhere, which can make the network vulnerable and economically useless.
That being said, you are encouraged to help each other in understanding and debugging. There are not really many great tricks or mysteries that you need to figure out here, but rather a lot of understanding and skills with different programming technologies. So, it’s okay if you:
- Chat with classmates on how to tackle some issue
- Find helpful resources on the web
- Share links to resources that you find
- Ask a classmate for help on a small, specific debugging problem. (That would be, asking a classmate to look at your code that doesn’t work, not asking to look at their code which already works.)
All of the above must still be clearly documented in comments in your code. The more specific, the better!
However, it’s still not allowed to:
- Share your working code with anyone else
- Ask to look at someone else’s working code
- Ask someone to do your work for you
- Get help from anyone not in this class this semester
When in doubt, just email me and ask about it rather than risk violating your integrity.
Google forms
Most labs will be accompanied by a Google Form you have to fill out with some specifics on what you did for the lab. Often there will need to be some kind of “proof” that your code work, like the address of a coin that you mined. You can always submit out the form and revise it again and again until the deadline, and are encouraged to do so as you progress through the lab.
Gitlab
You will write your code in a gitlab repository shared with your instructor. We will not use the submit system, so this is the only way you will share your work. Typically, you will indicate you’re finished with the lab by pushing a commit to the main
branch and along with a certain tag. For example, for this lab, when you’re finished you should run some commands like:
git add all.py of.py your.py code.py files.py and.py README.txt
git commit -am "done with lab 1"
git tag lab01
git push --tags
Reference
Some of this might be review for some of you, but most of it is probably new. Feel free to skip and come back to parts as you need them later on.
Python
You can use any programming language you want in this class as long as you can get things to work. But I will use Python3 for everything and give the most help for that, so that’s what I recommend at least to get started.
Python has a few basic types which we will become very familiar with:
Numbers:
int
andfloat
an turn a string into a number.Strings:
str
is the name of the class for text stringsBytes: A
bytes
object is like a string, except that it is raw data instead of text. The equivalent is an array ofunsigned char
in C or C++.Bytes objects are the inputs and outputs to most of the cryptography functions we will use.
To convert between strings and bytes, you call
encode
anddecode
. For example:s = "money" # s is a str s.encode() # this returns a bytes object s.encode().decode() # brings you back to the original string
Lists: denoted with square brackets, like
[1, 2, 10]
.Python lists act like ArrayList in Java: it’s very fast to add or remove from the end of the list, and to index anywhere in the list.
Dicts: denoted with curly bracketts, like
{'cow': 'moo', 'dog': 'bark'}
.This is a map data structure, specifically a hash map, associating keys to values. Note that the keys are not stored in order.
Python has lots of helpful built-in libraries, and very complete and pretty great documentation at https://docs.python.org/3/.
JSON
JSON stands for “JavaScript Object Notation”, but it’s really just a de-facto standard for text-based web communication these days. It is a way to encode (or serialize) nested structures in any programming language.
The built-in json library is pretty great. The two most important methods are json.dumps
, which turns a Python object into a JSON-encoded string, and json.loads
, which does the opposite.
JSON types include numbers, strings, booleans, dictionaries, lists, and any of these things nested inside dictionaries and lists. Notably, JSON does not natively include raw bytes, which is why we will be consistently encoding bytes to hes for this class.
Cryptographic hashes
There are many cryptographic hash functions available in Python’s hashlib library. We will mostly use the sha3_256
hash function. Look at the documentation to figure out how to use it - remember that it will always deal with bytes
so conversion to/from strings is often necessary.
Hex strings
Whenever we store raw binary data, like a hash digest or signature key, the “right way” to store it is as a bytes
object. But those can be pretty annoying to look at and compare.
So to make our lives easier, in this class we will always store such things as hex strings. To turn a bytes
into a hex string, you just call the hex()
method. To go the other way, you call the bytes.fromhex(your_string_here)
function.
HTTP
You will need to communicate with servers using the HTTP protocol in your code. Later, you will run a server yourself. But don’t worry; Python has lots of help with this!
First recall some terminology from your networks class:
- Host: can be a name like example.com or an IP address
- Port: default is 80 for HTTP, but it can be on any port if you put a colon and the port number after the host in a URL
- GET request: normal HTTP page load; we will use it to fetch information like the current blockchain head or the contents of some block.
- POST request: form submission. We will use it to submit blocks and transactions to miners.
For example, if you visit the URL
http://some.website.net:1234/some/page?what=info&when=now
this causes a GET request to host some.website.net
on port 1234 with the path /some/page
and two GET query parameters.
To make GET and POST requests we will use the excellent Python requests library. The quickstart guide on their homepage is a much clearer and more concise overview than what I could write here, so go read it!
Setup: VM access and Gitlab repo
We will do all of our work in a virtual environment so as not to run afoul of regulations on the mission network. Remember, mining actual cryptocurrency (as opposed to our fake coins for this class) is not allowed on any Navy network. There, I said it!
To access any of these resources, you must be connected to the USNA mission network (or the VPN).
VMWare Vcenter
Click on “launch vsphere client (html)”
Use your standard USNA login name and password
Click the box after very carefully reading the DoD terms and conditions
Click through to see a scary window telling you that you can’t do anything. This is normal.
Click on the second little icon thingy at the top that has some boxes and a folded corner:
Expand Datacenter/si486i. You should see a VM name with your username in it.
Click on that and go to “Launch web console” (big blue button).
Your default password is “myvm”, which you should be forced to change (to whatever you want) when you first login.
SSH access to VM
Click on “launch vsphere client (html)”
Use your standard USNA login name and password
Click the box after very carefully reading the DoD terms and conditions
Click through to see a scary window telling you that you can’t do anything. This is normal.
Click on the second little icon thingy at the top that has some boxes and a folded corner:
Expand Datacenter/si486i. You should see a VM name with your username in it.
Click on that and go to “Launch web console” (big blue button).
Your default password is “myvm”, which you should be forced to change (to whatever you want) when you first login.
(Optional)
If you don’t want to access the VM by clicking a bunch of things, your kind instructor has set up a way for you to ssh directly (well, through a proxy). You should have instructions in an email.
Gitlab repo
We will use the (USNA Intranet only) gitlab to save your code and share it with your instructor.
Create a gitlab account if you don’t have one already.
You must use your normal USNA username
mXXXXXX
and USNA email or else I won’t know it’s you!But the password can be anything you want.
Go to menu / create new project. Click to make a new blank project.
Use the following settings:
- Project name:
goatcoin
(all lowercase, no spaces). - Visibility: Private
- Initialize with a README Then click the button to create the new repo.
- Project name:
This should open your repo page. From there, go to Project Information / Members
Find your instructor and invite them with a Maintainer role.
Now log into your VM and clone your repository. There are two ways to do this. If you want to avoid having to type your gitlab password all the time, follow these steps:
- On your VM, run the command
ssh-keygen -t ed25519
and follow the prompts. - Still on the VM, type
cat ~/.ssh/id_ed25519.pub
- That should give a single line of output.
- Now go to your gitlab page
- Under your user icon (top-right), go to Preferences
- Select SSH Keys on the left panel
- Paste that entire line (from step 3) into the box and click Add Key
- Now go back to your
goatcoin
project page - Click the blue “Clone” button and copy/paste the SSH option.
If you don’t feel like setting up the ssh keys, then just do steps 8 and 9 except use the HTTPS option.
The actual lab
For this lab, you will write a series of small python programs to do some hashing, create a block, download and send blocks to a server.
Hash a string
Create a program hash_string.py
that prompts the user for a single-line message and, then encodes this string to bytes, hashes it using the sha3_256
algorithm, turns the output into a hex string, and displays that hex string of the hash.
Hash a block
Here is an example of a block for the first couple labs, in JSON form:
{
"prev_hash": "d6f5cf480f944cf7bc3bbdb16fc430db076b45536ed83a94f9eb0d577427761e",
"payload": {
"chat": "Hello, world!"
},
"version": 0
}
Notice that:
- It is a dictionary with three keys
prev_hash
,payload
, andversion
. - The
prev_hash
should be a length-64 hex string - The
payload
is itself another dictionary, with a single keychat
that can map to any (short) message. - The
version
should be 0.
Create a program hash_block.py
that:
- Reads a hash digest and chat message from the user with
input()
- Constructs a block in Python according to the structure above
- Encodes that block with
json.dumps
- Computes a
sha3_256
hash of the encoded json string - Prints out that resulting hash digest.
Get the current blockchain head
There is a blockchain server running in the VM environment on a machine called cat
, port 5000. Write a program show_chat.py
that uses the requests library to:
- Make a request to
http://cat:5000/head
- Read the text of the response, which should be a length-64 hex string
- Make a request to
http://cat:5000/fetch/HASH
, whereHASH
is replaced with the hash from step 2 - Read the text of this second response, which should be a JSON string encoding a block
- Decode the JSON and get the chat message out of payload of the resulting block.
- Print that chat message - the most recent one posted to the blockchain - to the terminal.
Add a message to the chat
Don’t you want your voice to be heard? Create a program send_chat.py
that:
- Prompts the user for a desired short message string
- Gets the current hash head as in steps 1 and 2 above.
- Creates a new block with the desired chat message in a payload, the previous hash fetched from the server, and version 0.
- Submits that block (in JSON-encoded form) as the value for key
block
in a POST request to urlhttp://cat:5000/push
- Checks that the response is HTTP 200 OK
- Prints a congratulatory message to the terminal
(You will probably want to use your show_chat.py
program to make sure the message really went through!)
Hint: you should find this page from the Python requests library documentation helpful in how to send data to the POST request.
Submission
Complete the form and push to gitlab with tag lab01
. You can submit the form over and over again until the deadline, and should definitely push to gitlab frequently, but should only push that lab01
tag when you are really finished.