4.1 Evaluation of Success Criteria
Node Software & Protocol
Individual blocks
1
Blocks must refer to the previous block in the chain.
Not met.
2
Blocks must be detected as invalid and disregarded by the node software.
Not met.
3
Blocks must be detected as valid and approved by node software.
Not met.
Purely in terms of the actual software aspect none of these three criterion were met since I did not reach the stage in development of the project which would allow for the addition and translation of protocol rules into the actual software networking to allow the blockchain to run and send theorised data through it, however in the protocol form this criterions where all laid out as discussed in each individual one.
Criterion 1
Blocks must refer to the previous block in the chain.
This criterion was not implemented in software due to time constraints as discussed in summary paragraph above. However in terms of the protocol, the framework for how this would work and hence would be implemented into the node software can be found in the initial project analysis here.
Criterion 2 and 3
2. Blocks must be detected as invalid and disregarded by the node software. 3. Blocks must be detected as valid and approved by node software.
Both of these Cycles relate to the same thing: validating blocks and acting accordingly; with criterion 2 being in relation to disregarding invalid blocks and cycle 3 approving valid blocks.
Sadly neither of these criterion were implemented in the node software due to time constraints as discussed in summary paragraph above.
Although this was not implemented on the scale of blocks, this criterion was partially met in the fashion of the node software detecting and disregarding transactions and messages that were invalid. This was first implemented in cycle 10 where it was made that the software should only sign specific types of data that it receieved and was then expanded in more detail throughout cycles 12 and 14.
// V code - within the "pong" route in /src/modules/server/main.v
// with this version of the node software all messages should be time objects
time := time.parse(req_parsed.message) or {
eprintln("Incorrect time format supplied to /pong/:req by node claiming to be $req_parsed.ping_key")
return app.server_error(403)
}
// time was okay, so store a slight positive grudge
println("Time parsed correctly as: $time")Therefore it can be seen that there is evidence that this criterion would've been met if the project's development continued over a longer time span.
Network Communication
4
Nodes should have an open api server running using http that contains enough pathways to complete all the needed types of communication.
5
Node should have web-socket connection abilities for server or client connections.
Partially met in Cycle 14.
6
Nodes should be able to add and remove each other to their communication lists which should be stored locally.
7
Nodes should be able to store a local "grudge" object that just remembers who the node has a good or bad experience with to change their trust level when they create their next block.
Not met.
Criterion 4
Nodes should have an open api server running using http that contains enough pathways to complete all the needed types of communication.
The above image shows the home route of the http api server running on the node software, since there are quite a few api routes the rest are shown below:
These routes not only allow for the http server to be used for communication between nodes but also allow for the user to interact and send messages over the network. This can be seen through the categorisation supplied with the 'User Pages' being used to show the user some data or allow them to interact with their node through a nicer ui than having to use the terminal window; the 'Internal Api' is then used to ensure the person accessing the node does actually have control of the node and let them send messages using the web interface; and finally the 'External Api' are the routes that will be called by other nodes looking to interact with or send a message to the receipient node.
Criterion 5
Node should have web-socket connection abilities for server or client connections.
This criterion was partially met in cycle 14 where websockets were implemented using VLang's net.websockets module and the functionality tested well initially but had some serious issues upon continued use.
The above screenshot shows a message being successfully received and then the software crashing after attempting to forward it onto the other nodes it was aware of. This occurred approximately 20 messages into a test where a message was sent every 0.5 seconds as such showing that the websockets partially work but cannot handle continuted use very well.
Sadly this is primarily an issue with VLang and although improved code on my half would most likely expand the length of time that the software runs for before crashing there is a high probability that these sorts of issues would continue to happen. This is because the panic shown above and similar errors that were encountered occoured upon the websocket module calling other vlang standard library modules through my middleware meaning that the issues were within the modules supplied to the vlang standard library which I cannot easily fix.
Criterion 6
Nodes should be able to add and remove each other to their communication lists which should be stored locally.
In order to prove that this functionality works quickly and easily after changes I built the following automatic testing functions:
// This tests the creation of a node reference
fn test_create_ref() {
database.launch()
db := database.connect(false, pg.Config{})
create_test_ref(db)
database.stop()
// this test will crash if it fails
assert true
}
// This tests to check if the node will correctly state if it is aware of a node
fn test_aware_ref() {
database.launch()
db := database.connect(false, pg.Config{})
create_test_ref(db)
aware := db.aware_of('test')
database.stop()
assert aware
}
// This tests to see if the node will collect all of it's references properly
fn test_refs() {
database.launch()
db := database.connect(false, pg.Config{})
create_test_ref(db) // create a ref if the test one doesn't already exist
refs := db.get_refs()
database.stop()
assert refs.len > 0
}
// This is used to generate a testing reference object when needed
fn create_test_ref(db database.DatabaseConnection) {
mut aware := db.aware_of('test')
if !aware {
// just in case the test ref doesn't exist
db.create_ref('test', 'test'.bytes())
aware = db.aware_of('test')
}
}These functions are part of the larger 'database_test.v' file which I designed in cycle 13 but are the only tests necessary to prove this criterion has been met. Upon running these tests the following is output to the terminal by VLang's built in testing functionality.
As shown above, not only did the three tests discussed pass, but so did the other 5 in this testing module, hence proving that this criterion is met.
Criterion 7
Nodes should be able to store a local "grudge" object that just remembers who the node has a good or bad experience with to change their trust level when they create their next block.
Although I did begin work on this criterion while introducing the sql database in cycle 13, I quickly realised that there would be no way of actually using grudges until block creation was introduced later on in the project so removed it to be reintroduced later. Sadly however this later point was never reached so this criterion has not been met, and it should be introduced shortly after block creation has been implemented and criterion 1,2 and 3 have been met.
Criterion 8
Nodes should send through a digital signature with each communication.
As of cycle 12, all messages and methods of communication sent between nodes contains a signature except for some specific exceptions in which it is not required such as when one node request the initiation of a handshake from another node, since the signatures are then sent within the handshake. Therefore this criterion is currently met, although it would need to be continued to be met throughout future development and the introduciton of more communication methods and usecases between nodes.
Criterion 9
Nodes should be able to remember each other through a combination of their wallet address and their ips/domains.
As of cycle 10 all messages between nodes contain the sending node's ip address or dns domain (acts the same as an ip address in all forms of communication used within the project) and whenever a node encounters a message from a node that it does not recognise the ip address/domain of it will send it a handshake request in order to check it's validity and then assuming it is a valid monochain node will store the data it learns about that node locally.
Initially this data was stored in JSON files but since I realised this could get slow and complicated quickly in cycle 13 this was improved to be done through a postgreSQL sql database which the node software queries and talks to as often is required. By default this database will be automatically setup and controlled by the node software through the use of a docker container but it can be hosted manually by the user if they wished to have more control over the database, for example to give multiple nodes access to the same database.
General features
11
The configuration handler should be usable and accessible to technical users with non-technical users being able to use it with additional help.
Met in Usability testing.
13
The node dashboard should be easily accessible and usable by all forms of users.
Partially met in Usability testing.
Criterion 10
A configuration handler.
The configuration handler for the node software was originally introduced in cycle 3 but it has been continued to be improved, had more settings added and restructured for easier use throughout many various points of the project and should be continue into the future as development proceeds.
This handler controls the generation of new configurations, alongside the loading, saving and use of those configuration settings as is needed by other parts of the project.
Criterion 11
The configuration handler should be usable and accessible to technical users with non-technical users being able to use it with additional help.
In usability testing, three of the questions in the user survey were examined to show that all users had atleast heard of the configuration file existing, with the majority knowing what it could be used for and some of the more technical users within that pool knowing how to use it without additional help but the majority of users requiring assistance.
This meets the criterion since the majority of users are non-technical users and those users who did not require assistance were the technical users, however in the future this could still be improved by including some basic documentation to assist users and hopefully allow more users to be able to use the configuration handler.
Criterion 12
Nodes should have some kind of web based access dashboard.
This criterion was first met in cycle 12 and uses a simple token based system to ensure that users who gain access to the dashboard are in fact in control of the node which hence meets the criterion.
Although this could be improved by adding more functionality to the dashboard to give users more of a reason to use it alongside making the way in which users both login to and the dashboard itself look nicer and contain more context to explain how to login and use it so as to make the dashboard simpler to use as some users did not understand how to login without assistance.
Criterion 13
The node dashboard should be easily accessible and usable by all forms of users.
This criterion was proven to be partially met in usability testing as the majority of users seemed to know how to access the dashboard but due to the mis-communication during testing some users did not realise they were navigating to and using the dashboard which would probably increase the number of users who stated they were able to navigated and use it.
I suggest this as I discovered more users said they knew how to access the dashboard than the amount who said they knew what it was and after the survey I talked to all the users who did not agree to either question relating to this topic in the survey they all seemed to realise what it was after a very very brief explanation, hence suggesting some form of basic documentation or tutorial explaining the essentials to them could bring this up to being met.
Although alongside this I believe that the login page to access the dashboard should be a lot more clear about where to find the token that was generated to login with (it is output within the node's terminal output) as most users who tested this had to search around for a few minutes to find it if they did so without assistance and in reality this should be a very simple task. Hence in the future it would be a good idea to add a basic tutorial/summary on how to login so as to speed this up.
Non-functional
15
The software should be capable of receiving multiple messages per second.
Not met, as of Function and Robustness testing.
16
The software should attempt to retry any failed processes (such as loading a file) and should only exit after multiple failed attempts.
Met, as of Function and Robustness testing.
Criterion 14
The node software should not crash.
This criterion was tested in function and robustness testing and it was found that the node software will not crash under casual use on Windows and MacOS systems or in specific scenarios on Linux systems, but it will crash if put under constant/heavy load or communication involving websockets (which are on by default) with a Linux system.
Since the websocket functionality can be disabled using the configuration handler, all three system types can be setup such that the software does not crash under casual use but they will all crash under consistant heavy use. Example of such consistant heavy use is the sending of a message more than once per second for atleast 10 messages in a row which has a high probability of causing a crash, with that probability seemingly increasing as more messages are sent.
Some parts of this are due to my own code, such as the number of messages sent in a row before the software crashes, which can very likley be increased as to increase stability through the refactoring of networking code. However, other parts are due to the limitations of the language that the software is written in itself, such as the Linux systems behaving differently to the Windows and MacOS systems whilst running the same code.
Criterion 15
The software should be capable of receiving multiple messages per second.
As summarised in the criterion above, the software cannot handle multiple messages per second for longer than about 10 seconds. While sending one message every 0.5 seconds I found the software would regularly crash after about 10 seconds which proves this criterion has not been met. This should then be worked on in the future.
Criterion 16
The software should attempt to retry any failed processes (such as loading a file) and should only exit after multiple failed attempts.
As shown in detail in testing for function and robustness the node software will attempt to retry any failed processes that do not exit the program completely (such as the websocket failures which crash the entire program) and will only exit after atleast 3 reattempts.
Any scenario in which a process is not completed after failing is due to the entire program crashing which is out of my control as programs compiled by Vlang should theroetically not do that, but as they do these failures are out of my control.
Webportal
Page types
19
A "downloads" page that lets the user download the node software.
20
A "wallet" page that allows the user to see any data or items stored within their digital wallet.
Created in Cycle 1, has not been met it's functionality.
Criterion 17
A "home" page that summarises what the project is about.
The webportal has a home page that was introduced in cycle 1 to give visitors an understanding of the basic summary of the project and was then improved significantly in cycle 7 to be easier and simpler to use whilst also making it look nicer to visitors.
Criterion 18
An "info" page that explains the project in more detail.
The info page was originally introduced in cycle 1 and was simply a large block of text explaining some of the technical side of the project but after cycle 7 it was refactored to explain more concepts relevant to the project under general questions which act as headers and give the project a lot more detail.
Criterion 19
A "downloads" page that lets the user download the node software.
The downloads page for the webportal was originally introduced in cycle 1 and was simply a button which allowed a user to download the first versions of the node software but throughout continued development it has gained a nicer look, a basic tutorial on how to setup a node and automated configuration which ensures the node software being downloaded is the newest compiled version.
Criterion 20
A "wallet" page that allows the user to see any data or items stored within their digital wallet.
This page was created in cycle 1 and simply contained a message to the user that this page would be introduced in the future, the message has changed slghtly since then but as this page requires functionality from the node software that does not currently exist it has not progressed past this form.
In order to get this page to a point at which it would meet this criterion transactions, blocks and blockchain state would need to be in working order such that the contents of a specific wallet could actually be viewed and shown on this page.
Non-functional
21
Non-technical users must be able to identify what the point of the project is just from the homepage of the website.
Met as of usability testing.
22
Users should be able to get to the majority of what they would want to get to within 3 clicks.
Met in Cycle 7.
23
The web-portal should be available and working on a variety of device sizes including mobile and desktop.
Partially met in Cycle 7.
24
The web portal should run without crashing under any circumstance.
Met as of testing for function and robustness.
Criterion 21
Non-technical users must be able to identify what the point of the project is just from the homepage of the website.
As examined in usability testing the vast majority of users understood the idea of the project after being given free range on the webportal and since not all of the users were likely to have read all of the information on the info page it can be assumed that the home page was the primary force supporting this. Therefore it can be stated that users could identify the point of the project from the homepage of the website.
Criterion 22
Users should be able to get to the majority of what they would want to get to within 3 clicks.
As examined in usability testing, all pages can be visited within an absolute maximum of 2 clicks of the homepage which including the 1 click it takes to get back to the home page from any other page it can be stated that all pages can be navigated to within 3 clicks from any other page.
Criterion 23
The web-portal should be available and working on a variety of device sizes including mobile and desktop.
This was partially met in cycle 7 as the webportal originally only worked on desktop but in this cycle it was updated to work with almost all mobile devices, however the tablet screen sizes were overlooked and as such some specific elements (the home page buttons, tutorial on how to setup the node software) go off of the screen which I would classify as not working. Although as technically the site is available on all device types and it only 'breaks' on devices that aren't as commonly used this criterion is partially met but this could be fixed in the future.
Criterion 24
The web portal should run without crashing under any circumstance.
This criterion was met in testing for funcition and robustness and as the webportal has still not crashed since it's initial public debut it has met this criterion.
It is worth noting here that client sided exceptions such as a client going offline while using the website do not count as "crashing" the webportal since they can be fixed by refreshing the page and do not effect anyone except that client/computer.
Last updated