SI 486I Spring 2022 / Labs


This is the archived website of SI 486I from the Spring 2022 semester. Feel free to browse around; you may also find more recent offerings at my teaching page.

Lab 1: Getting started: VMs, JSON, and Hashing

Today’s lab is getting started on an exciting 8-lab sequence where we will eventually create our own fake cryptocurrency, run by mining nodes that you write the software for!

How labs will work in this class

Labs and projects are the main work for this course. In the first part of the semester, we will be working to create and run our own fake cryptocurrency network in a series of labs.

Collaboration

The goal is for each of you to write the software for and run your own node in the network. It is crucial that you write your own software, not just for the “no cheating” aspect, but because we want our network to be robust. If everyone is running the same software, it may have the same small mistakes everywhere, which can make the network vulnerable and economically useless.

That being said, you are encouraged to help each other in understanding and debugging. There are not really many great tricks or mysteries that you need to figure out here, but rather a lot of understanding and skills with different programming technologies. So, it’s okay if you:

  • Chat with classmates on how to tackle some issue
  • Find helpful resources on the web
  • Share links to resources that you find
  • Ask a classmate for help on a small, specific debugging problem. (That would be, asking a classmate to look at your code that doesn’t work, not asking to look at their code which already works.)

All of the above must still be clearly documented in comments in your code. The more specific, the better!

However, it’s still not allowed to:

  • Share your working code with anyone else
  • Ask to look at someone else’s working code
  • Ask someone to do your work for you
  • Get help from anyone not in this class this semester

When in doubt, just email me and ask about it rather than risk violating your integrity.

Google forms

Most labs will be accompanied by a Google Form you have to fill out with some specifics on what you did for the lab. Often there will need to be some kind of “proof” that your code work, like the address of a coin that you mined. You can always submit out the form and revise it again and again until the deadline, and are encouraged to do so as you progress through the lab.

Gitlab

You will write your code in a gitlab repository shared with your instructor. We will not use the submit system, so this is the only way you will share your work. Typically, you will indicate you’re finished with the lab by pushing a commit to the main branch and along with a certain tag. For example, for this lab, when you’re finished you should run some commands like:

git add all.py of.py your.py code.py files.py and.py README.txt
git commit -am "done with lab 1"
git tag lab01
git push --tags

Reference

Some of this might be review for some of you, but most of it is probably new. Feel free to skip and come back to parts as you need them later on.

Python

You can use any programming language you want in this class as long as you can get things to work. But I will use Python3 for everything and give the most help for that, so that’s what I recommend at least to get started.

Python has a few basic types which we will become very familiar with:

  • Numbers: int and float an turn a string into a number.

  • Strings: str is the name of the class for text strings

  • Bytes: A bytes object is like a string, except that it is raw data instead of text. The equivalent is an array of unsigned char in C or C++.

    Bytes objects are the inputs and outputs to most of the cryptography functions we will use.

    To convert between strings and bytes, you call encode and decode. For example:

    s = "money" # s is a str
    s.encode() # this returns a bytes object
    s.encode().decode() # brings you back to the original string
  • Lists: denoted with square brackets, like [1, 2, 10].

    Python lists act like ArrayList in Java: it’s very fast to add or remove from the end of the list, and to index anywhere in the list.

  • Dicts: denoted with curly bracketts, like {'cow': 'moo', 'dog': 'bark'}.

    This is a map data structure, specifically a hash map, associating keys to values. Note that the keys are not stored in order.

Python has lots of helpful built-in libraries, and very complete and pretty great documentation at https://docs.python.org/3/.

JSON

JSON stands for “JavaScript Object Notation”, but it’s really just a de-facto standard for text-based web communication these days. It is a way to encode (or serialize) nested structures in any programming language.

The built-in json library is pretty great. The two most important methods are json.dumps, which turns a Python object into a JSON-encoded string, and json.loads, which does the opposite.

JSON types include numbers, strings, booleans, dictionaries, lists, and any of these things nested inside dictionaries and lists. Notably, JSON does not natively include raw bytes, which is why we will be consistently encoding bytes to hes for this class.

Cryptographic hashes

There are many cryptographic hash functions available in Python’s hashlib library. We will mostly use the sha3_256 hash function. Look at the documentation to figure out how to use it - remember that it will always deal with bytes so conversion to/from strings is often necessary.

Hex strings

Whenever we store raw binary data, like a hash digest or signature key, the “right way” to store it is as a bytes object. But those can be pretty annoying to look at and compare.

So to make our lives easier, in this class we will always store such things as hex strings. To turn a bytes into a hex string, you just call the hex() method. To go the other way, you call the bytes.fromhex(your_string_here) function.

HTTP

You will need to communicate with servers using the HTTP protocol in your code. Later, you will run a server yourself. But don’t worry; Python has lots of help with this!

First recall some terminology from your networks class:

  • Host: can be a name like example.com or an IP address
  • Port: default is 80 for HTTP, but it can be on any port if you put a colon and the port number after the host in a URL
  • GET request: normal HTTP page load; we will use it to fetch information like the current blockchain head or the contents of some block.
  • POST request: form submission. We will use it to submit blocks and transactions to miners.

For example, if you visit the URL

http://some.website.net:1234/some/page?what=info&when=now

this causes a GET request to host some.website.neton port 1234 with the path /some/page and two GET query parameters.

To make GET and POST requests we will use the excellent Python requests library. The quickstart guide on their homepage is a much clearer and more concise overview than what I could write here, so go read it!

Setup: VM access and Gitlab repo

We will do all of our work in a virtual environment so as not to run afoul of regulations on the mission network. Remember, mining actual cryptocurrency (as opposed to our fake coins for this class) is not allowed on any Navy network. There, I said it!

To access any of these resources, you must be connected to the USNA mission network (or the VPN).

VMWare Vcenter

  1. Go to https://vcenter.cs.usna.edu

  2. Click on “launch vsphere client (html)”

  3. Use your standard USNA login name and password

  4. Click the box after very carefully reading the DoD terms and conditions

  5. Click through to see a scary window telling you that you can’t do anything. This is normal.

  6. Click on the second little icon thingy at the top that has some boxes and a folded corner:

  7. Expand Datacenter/si486i. You should see a VM name with your username in it.

  8. Click on that and go to “Launch web console” (big blue button).

  9. Your default password is “myvm”, which you should be forced to change (to whatever you want) when you first login.

SSH access to VM

(Optional)

If you don’t want to access the VM by clicking a bunch of things, your kind instructor has set up a way for you to ssh directly (well, through a proxy). You should have instructions in an email.

Gitlab repo

We will use the (USNA Intranet only) gitlab to save your code and share it with your instructor.

  1. Create a gitlab account if you don’t have one already.

    You must use your normal USNA username mXXXXXX and USNA email or else I won’t know it’s you!

    But the password can be anything you want.

  2. Go to menu / create new project. Click to make a new blank project.

  3. Use the following settings:

    • Project name: goatcoin (all lowercase, no spaces).
    • Visibility: Private
    • Initialize with a README Then click the button to create the new repo.
  4. This should open your repo page. From there, go to Project Information / Members

  5. Find your instructor and invite them with a Maintainer role.

Now log into your VM and clone your repository. There are two ways to do this. If you want to avoid having to type your gitlab password all the time, follow these steps:

  1. On your VM, run the command ssh-keygen -t ed25519 and follow the prompts.
  2. Still on the VM, type cat ~/.ssh/id_ed25519.pub
  3. That should give a single line of output.
  4. Now go to your gitlab page
  5. Under your user icon (top-right), go to Preferences
  6. Select SSH Keys on the left panel
  7. Paste that entire line (from step 3) into the box and click Add Key
  8. Now go back to your goatcoin project page
  9. Click the blue “Clone” button and copy/paste the SSH option.

If you don’t feel like setting up the ssh keys, then just do steps 8 and 9 except use the HTTPS option.

The actual lab

For this lab, you will write a series of small python programs to do some hashing, create a block, download and send blocks to a server.

Hash a string

Create a program hash_string.py that prompts the user for a single-line message and, then encodes this string to bytes, hashes it using the sha3_256 algorithm, turns the output into a hex string, and displays that hex string of the hash.

Hash a block

Here is an example of a block for the first couple labs, in JSON form:

{
  "prev_hash": "d6f5cf480f944cf7bc3bbdb16fc430db076b45536ed83a94f9eb0d577427761e",
  "payload": {
    "chat": "Hello, world!"
  },
  "version": 0
}

Notice that:

  • It is a dictionary with three keys prev_hash, payload, and version.
  • The prev_hash should be a length-64 hex string
  • The payload is itself another dictionary, with a single key chat that can map to any (short) message.
  • The version should be 0.

Create a program hash_block.py that:

  1. Reads a hash digest and chat message from the user with input()
  2. Constructs a block in Python according to the structure above
  3. Encodes that block with json.dumps
  4. Computes a sha3_256 hash of the encoded json string
  5. Prints out that resulting hash digest.

Get the current blockchain head

There is a blockchain server running in the VM environment on a machine called cat, port 5000. Write a program show_chat.py that uses the requests library to:

  1. Make a request to http://cat:5000/head
  2. Read the text of the response, which should be a length-64 hex string
  3. Make a request to http://cat:5000/fetch/HASH, where HASH is replaced with the hash from step 2
  4. Read the text of this second response, which should be a JSON string encoding a block
  5. Decode the JSON and get the chat message out of payload of the resulting block.
  6. Print that chat message - the most recent one posted to the blockchain - to the terminal.

Add a message to the chat

Don’t you want your voice to be heard? Create a program send_chat.py that:

  1. Prompts the user for a desired short message string
  2. Gets the current hash head as in steps 1 and 2 above.
  3. Creates a new block with the desired chat message in a payload, the previous hash fetched from the server, and version 0.
  4. Submits that block (in JSON-encoded form) as the value for key block in a POST request to url http://cat:5000/push
  5. Checks that the response is HTTP 200 OK
  6. Prints a congratulatory message to the terminal

(You will probably want to use your show_chat.py program to make sure the message really went through!)

Hint: you should find this page from the Python requests library documentation helpful in how to send data to the POST request.

Submission

Complete the form and push to gitlab with tag lab01. You can submit the form over and over again until the deadline, and should definitely push to gitlab frequently, but should only push that lab01 tag when you are really finished.