2.2.8 Cycle 8 - Node Refactor

Design

Objectives

After beginning to plan out the slightly more advanced inter nodal communication (which should be in a later cycle not too much later than this one) I have come to the realisation that housing the cryptography and configuration module in the same place is probably not a good idea.

This is because after a few experiments I realised that without editing and recompiling it, the V-Web module that I'm using to host the api routes for the nodes currently prevents the loading of custom non-public variables into the web object to be used within those routes. What that means is that I can't setup the node's cryptography keys to be passed around the node whenever it receives a new api request without making them public and accessible by all parts of the program (which isn't great for security) and hence need to find a work around.

There are two main options here:

1. Reload the configuration object every time the program needs the cryptography keys to do anything.

2. Store the keys as a separate object and just reload them when they're needed.

Since separating the configuration and key objects helps reduce the amount of data that needs to be loaded and saved per time unit and is better for security, I will be going with the second option.

• Separate the keys into their own object.

• Allow the private key to remain private and unusable from any non-direct functionality

• Refactor any sections of code that need to be updated to support this new method

• Fix any newly generated errors and get back to at least what was working before this refactor

Usability Features

• Keys are stored in a seperate file so that users can copy and paste just their key file to run additional nodes without having to manually enter their key data.

• Ensure that all files for this project are stored within one "monochain" folder so as to keep them contained and not overrun the user's file system.

Key Variables

Variable Name	Use
Keys	This is the keys object that holds the private and public keys as-well as some functions for using these keys (sign, verify, etc).
key_path	This is a string that represents the path of the file where the keys object is stored and loaded from.

Pseudocode

The keys object

This is the object that stores both the private, public key and the two key functions that need access to private key. The first being the sign function which signs data using the private key, and the validate function that ensures the public and private key within the object are matching and work together.

// Psuedocode
IMPORT cryptography AS dsa 
// dsa is used as the name since that is the cryptography algorithm being used.

OBJECT Keys:
	priv_key 
	pub_key
	
	FUNCTION validate_keys(this):
		// defines some random data to check a key pair with
		data = "Hello, world!"	
		
		// signs that data using the private key
		signature = this.sign(data)	
		
		// validate that data with that signature and key pair
		verified = verify(this.pub_key, data, signature)	
		
		IF (verified == false):	// if verification failed
			OUTPUT "Signature verification failed"
			EXIT	// exit the program
		ELSE:
			// verification succeeded
			OUTPUT "Signature verified"
			RETURN true	// valid so return true
		END IF
	END FUNCTION

	FUNCTION sign(this, data) {
		// wrap the sign function to prevent having conditional data throughout program.
		TRY:
			signature = dsa.sign(this.priv_key, data)
			RETURN signature
		CATCH:
			OUTPUT "Error signing data"
			EXIT
		END TRY
	END FUNCTION
END OBJECT

FUNCTION verify(public_key, data, signature):
	// wrap verify function to prevent having conditional data throughout program.
	TRY:
		verified = dsa.verify(public_key, data, signature)
		RETURN verified
	CATCH:
		OUTPUT "Error verifying data"
		RETURN false
	END TRY
END FUNCTION

The keys handler

This is what allows the keys to be stored, loaded and generated into the keys object.

// Psuedocode
IMPORT utils
IMPORT cryptography AS dsa 
IMPORT json

FUNCTION failed_to_get_keys(key_path):
	OUTPUT "Could not load keys from file, would you like to generate a new pair?"
	IF utils.ask_for_bool():
		new_keys = gen_keys()
		failed = utils.save_file(key_path, json.encode(new_keys))
		
		IF failed {
			println("Cannot continue, exiting...")
			EXIT
		END IF
		
		RERTURN new_keys
	ELSE
		OUTPUT "Cannot operate without a keypair, Exiting..."
		EXIT 
	END IF
END FUNCTION

FUNCTION get_keys(key_path):
	raw = utils.read_file(key_path)

	IF (!raw.loaded):
		RETURN failed_to_get_keys(key_path)
	END IF

	TRY:
		keys = json.decode(Keys, raw.data)
		OUTPUT "Keys loaded from file."
		RETURN keys
	CATCH:
		RETURN failed_to_get_keys(key_path)
	END TRY
	
END FUNCTION

FUNCTION gen_keys():
	TRY:
		public_key, private_key = dsa.generate_key()
		
		keys = {
			pub_key: public_key,
			priv_key: private_key,
		}
		keys.validate_keys()

		RETURN keys
	CATCH:
		OUTPUT "Error generating keys"
		EXIT
	END TRY
END FUNCTION

Development

The end result for this code involves two files, one of which handles the creation and functions of the keys object, and the other handles the loading, saving and creation of that object.

The benefit of this is that it allows the private key to remain as a private property so that the only functions that can access it are those that are a part of the keys object and used for signing data. Therefore although if part of the program was infected with some kind of malware it would allow that malware to sign data/transactions which is obviously not good, it wouldn't give them access to the keys directly hence making it harder for the bad actor responsible for the malware to copy the keys and gain complete control of the wallet.

Outcome

The keys object.

// /packages/node/src/modules/cryptography/main.v
module cryptography	// this module
import crypto.ed25519 as dsa	// an external module used to handle all the dsa stuff - too complicated for the time scope of this project.

pub struct Keys {
	priv_key []u8	//dsa.PrivateKey
	pub: 
		pub_key []u8	//dsa.PublicKey
}

type KeysType = Keys

pub fn (this Keys) validate_keys() bool {
	data := "Hello, world!".bytes()		// defines some random data to check a key pair with
	signature := this.sign(data)	// signs that data using tthe private key
	verified := verify(this.pub_key, data, signature)		// validate that data with that signature and key pair
	if verified == false {	// if verification failed
		eprintln("Signature verification failed")
		exit(130)	// exit with error code 130
	} else {	// verification succeeded
		println("Signature verified")
		return true	// valid so return true
	}
}

pub fn (this Keys) sign(data []u8) []u8 {
	// wrap the sign function to prevent having conditional data throughout program.
	signature := dsa.sign(this.priv_key, data) or {
		eprintln("Error signing data")
		exit(140)
	}
	return signature
}

pub fn verify(public_key dsa.PublicKey, data []u8, signature []u8) bool {
	// wrap verify function to prevent having conditional data throughout program.
	verified := dsa.verify(public_key, data, signature) or {
		eprintln("Error verifying data")
		return false
	}
	return verified
}	

The keys handler.

// V code from /packages/node/src/modules/cryptography/keyHandling.v
module cryptography
import utils
import crypto.ed25519 as dsa
import json

fn failed_to_get_keys(key_path string) (Keys){
	println("Could not load keys from file, would you like to generate a new pair?")
	if utils.ask_for_bool(0) {
		new_keys := gen_keys()
		failed := utils.save_file(key_path, json.encode(new_keys), 0)
		if failed {
			println("Cannot continue, exiting...")
			exit(215)
		}
		return new_keys
	} else {
		eprintln("Cannot operate without a keypair, Exiting...")
		exit(1)
	}
}

pub fn get_keys(key_path string) (Keys) {
	raw := utils.read_file(key_path, true)

	if !raw.loaded {
		return failed_to_get_keys(key_path)
	}

	keys := json.decode(Keys, raw.data) or {
		return failed_to_get_keys(key_path)
	}

	println("Keys loaded from file.")
	return keys
}

pub fn gen_keys() (Keys) {
	// This is just a wrapper function to prevent the rest of the code having to deal with keys being incorrectly generated
	// throughout the rest of the program

	public_key, private_key := dsa.generate_key() or {
		eprintln("Error generating keys")
		exit(150)
	}
	keys := Keys{
		pub_key: public_key,
		priv_key: private_key,
	}

	keys.validate_keys()

	return keys
}v

Challenges

There were two key challenges to overcome during this development phase, the first one being a challenge I knew I'd have to overcome and the second being a very annoying bug. I've split these challenges into two separate sections for the sake of ease of reading.

Managing the storage of multiple files

Since the configuration settings are already saved in their own file and I am now going to be saving the keys in a separate file and I don't want to keep dumping files into the user's home directory, it's probably a good idea to start storing files within a folder to keep everything in one place.

Hence all files will now be stored within the /monochain/ folder so as to improve organisation. As a byproduct of this, I needed to write some form of script to check if the 'monochain' folder actually exists before anything was saved to it and if not, create the folder and then save a file to it.

This resulted in the adaptation of the current save_file function (which previously just wrapped the os.write_file function so as to reduce error handling) to the following:

// This is within the 'utils' module
pub fn save_file(path string, data string, recursion_depth int) (bool) {
	mut failed := false

	// splits the requested path into an array of sub-directories
	dirs := path.split("/")
	if dirs.len > 1 {
		// this means the file is in a folder

		// assemble the directory path
		mut dir := ""
		mut i := 0
		for i < dirs.len - 1 {
			dir += dirs[i] + "/"

			// check if path exists
			if !os.exists(dir) {
				// the directory doesn't exist, so we need to create it
				os.mkdir(dir) or {
					// something went wrong creating the directory, return without saving
					eprintln("[utils] Failed to create the directory at path $dir, error $err")
					failed = true
					return true
				}
				println("[utils] Created directory: " + dir)
			}
		
			//increment to the next sub-directory in the path
			i++
		} 
	}

	os.write_file(path, data) or { // try and write data to file
		// if it failed, run the recursion depth checker to ensure there hasn't been too many failed attempts
		eprintln('[utils] Failed to save file, error ${err}\nTrying again.')
		if recursion_depth >= 5{
			eprintln("[utils] Failed to save file too many times.")
			failed = true
		} else {
			// if the recursion depth is less than 5, just try again.
			failed = save_file(path, path, recursion_depth + 1)
		}
	}

	return failed
}

This code also allows for the creation of sub-directories within the main folder as it simply loops through all sections of the file path. An example of this would be the creation of a file at path /monochain/subfolder1/subfolder2/file.txt which is supported with this code.

Overcoming a tricky SIGSEGV error

One of the major issues with Vlang that I have been dealing with over the last few development cycles is the SIGSEGV error. This is an error that occurs when a program tries to access a memory address that it has not been given permission to access by the operating system and one that occurs for a variety of different reasons.

The main problem with this error is that because of how the Vlang compiler deals with these errors is to just tell the developer that one has occurred without any hints of why or where it could've happened, meaning that as soon as a SIGSEGV error occurs the developer is completely on their own to solve it.

Although I have encountered a few of these errors before, they have all been after changing only a few lines of code and hence I can simply tweak those few lines of code until the error is fixed. However in this case the error occurred after I had written an entire file of code and included it to be compiled, so I had to spend a few hours checking each line of code individually until eventually I discovered that the error wasn't even directly in that file at all and was instead caused by a type declaration loop across two separate files.

This type declaration loop was caused due to a struct being defined based upon a type, then that type being defined based upon the struct.

Struct A {TypeB}

Type B = Struct A

This was annoying because in most other languages if this was to occur, the compiler would give you a nice error message due to it being fairly easy to catch prior to compilation, but in Vlang that was not the case.

Solving this error after discovering this was pretty simple, the challenge was just finding out the cause of the error in the first place.

Testing

To assist in the testing of this module I created a batch of automated tests

Tests

Test	Instructions	What I expect	What actually happens	Passed?
1	Generate a key pair.	The program not to exit, which will happen if the key pair cannot be generated.	As Expected
2	Generate a key pair and sign some data.	The program not to exit, which will happen if the key pair cannot be generated or sign the data.	As Expected
3	Generate a key pair, sign and then verify some data.	The program not to exit and to verify that the message matches the signature.	As Expected

Evidence

Automation Code