NEAR Live Contract Review | Part 1: Voting Contracts

Introduction

Hey, my name is Eugene and I’m here with NEAR and today we’ll try something new. We’ll go through one of our contracts in Rust and we will review how it’s all done.  This is the first session, so let me introduce myself. I work at NEAR, I work on runtime and contracts. Today we’re going to discuss a voting contract and this is the contract that was used by the mainnet to move from phase one to phase two. Validators had to vote to enable transfers which essentially made the network fully decentralized. Here is the original video that the guide is based on.

Structural Overview

I’m using the CLI on Rust as it is one of the greatest tools to work on rust and on the left you can see a repository for core contracts. This is open source on our page on GitHub and there are a bunch of contracts and this one in particular is the voting contract. So let me first go through the quick structure of how this all works. In Rust you have the src and lib. Lib is the main file, it’s a library because we’re not really compiling the binary but we’re compiling a library into WebAssembly. 

There is a cargo tunnel: this is the way to describe dependencies. So we’re saying we’re gonna build a voting contract and we will use the NEAR SDK version 2.0.0 and there’s a bunch of flags that we use as instructions on how to compile with the release build. 

There are a bunch of optimizations that improve the size of the code. There is no debugging for any release and also panic is equal to abort, this is important for WebAssembly. Finally we have overflow checks equals true, this means when integer overflow happens you can subtract a number. So let’s say you subtract 2-1 and this goes negative and if your integer is unsigned this will raise a panic. This operation in usually silently rolls over in C++  so on rust you can enable a flag that will make it panic on any overflow or if you exceed the limit of the integer in either direction. So if you go beyond u32 it will raise a panic.  When you develop contracts for any smart contract platform usually overflows and  silent overflows are not the intended behavior and can lead to abuse so ideally your contract will panic and you’ll just handle it. So that’s just for safety. 

Lib.rs

Then we have other files and we’ll go there later, but first let’s start with the main contract. This is a single file and it is fairly simple, yet at the same time it has some interesting stuff. The way contracts are done on NEAR Protocol in Rust is that we have a main structure. In this case we’re using a structure called the voting contract structure which is similar to a class in other languages and we have a few macros here. 

So near_bindgen is a macro from NEAR SDK that turns the structure into something that we use for contracts and I’ll go into the details later. BorshDeserialize and BorshSerialize are two macros that allow this structure and all fields in it to be serialized into our binary format called borsh and also to get deserialized. This is needed to be able to store this entire contract in a state. The way contracts work is they have some persistent state from which they can read information on every invocation of the contract and they also have the local memory which is only available when the contract has started. It is going to be better explained later. So we have a structure and in Rust we have the implementation of  structure which is just basically the methods of the class. 

Constructor

So one of the methods is new, this is a constructor. It doesn’t take any arguments for this particular contract and it returns self. Self is the equivalent of a voting contract, basically it’s saying that it constructs a new instance of this class. We have a decorator macro called init and this init macro allows us to initialize a state of the contract without having a previous state. The way in which contracts are usually deployed on NEAR Protocol is that you deploy the code of the contract. Essentially you deployed a class and then you need to initialize an instance by calling a constructor. That will write a state, in our case this particular structure, into the persistent storage of the contract. So it will be available every time you call this contract.  When you first initialize the voting contract it doesn’t take any arguments because it doesn’t need to know anything. It will only rely on the current context of the blockchain and this context comes from a module called env which is short for the environment which is from the NEAR SDK. 

This is where we’ll need information down the line and we’ll use arguments that other contracts or users call in this contract. There’s two things happening in the constructor. First, we check that the state doesn’t exist, meaning that the contract has not yet been initialized. 

This init macro allows to for the contract to be called multiple times so it doesn’t check if the existing state already, it allows us to do this. We are working to add another macro which is called init_once that will allow us to only initialize the contract once and it will do this check automatically. The third (assert!(!env::state_exists)); is similar to a unit test it just verifies that the given expression is true and if it’s not true it then raises a panic which just stops the current function call and sends a “the contract is already initialized” message as an error. So it’s saying if the state doesn’t exist or if the state already exists, basically inverse of that, then the contract was already initialized and we cannot initialize it again. If we didn’t have this then we would be able to override whatever was there with a new state. State_exists actually comes from NEAR Protocol and we probably need to look at how NEAR SDK works in Rust  to better explain it. Every time a method that is not a constructor is called, let’s use the ping method as an example,  it takes the mutable self argument and it creates specific code from this contract that does a bunch of operations by using the decorator near_bindgen before getting into the execution phase. 

One of the operations is if you take the mutable self or self it first tries to deserialize the current state. The way it works is on a low level api is that we have a state key. This is the key with which we store the data of the main structure. 

So let’s say we constructed this state in the constructor basically this is the return address of the last value. 

You return this new structure which we initialized the way we wanted to and then the votingContract structure gets serialized into the borsh binary format by the near_bindgen macro and it is also getting written into the persistent storage under a key state. The value is going to be the current value of the votingContract structure so next time a method like the ping method  is called, it will first read the binary part out of storage, deserialize it and create the pub struct VotingContract structure. 

So then, only after that, the ping part will start getting executed. So the state_exists part comes from the NEAR SDK. It’s basically a wrapper around all it does is it checks if the given storage has this particular key and if it has a key then it knows that the state already exists.

What does init actually do? Init expects that you return a new class or a new instance and it will write it to the storage.

It is basically just saying, “I don’t expect any arguments but you will return me a new state that I will write into the storage”. We can also have a default method on the contract and we don’t want our contract to be initialized with the default argument without calling the initialization. We would say if a default method is called (this happens when the state doesn’t exist, it was not initialized) then this method is called by default. We don’t want this to happen so we’re saying VotingContract should be initialized first. So it basically just panics with an error. NEAR SDK basically tries to read the state from the storage from the state key first and if it doesn’t exist then it will call default but if it exists then it’s not going to call default.

 Interface of the Contract

Let’s see the interface of the contract first. We have a few methods and Rust has a pub decorator which means visibility is public and if there is no visibility it means that it’s private. 

So the NEAR bunch of decorators will only turn public methods into the methods that you can call on the contract. Which means the check_result method is not exposed to the contract it cannot be called by someone else, while all of the rest of these methods can be called by essentially anyone. One more important thing about the NEAR SDK is a difference in the arguments that you can pass. So if you pass the mut self argument, it means the method will be allowed to modify the state. We call this type of method a regular function call, and they can only come through a transaction. If self is mutable they can be called change methods. In contrast without the mut self the state of this structure will be immutable. The regular self argument basically creates a read only state, you cannot modify it. This type of method can be called using a view call and they are not going to modify the state. If you don’t have mut self then it will not write a state attempt. The important part is that we should treat them as non-changeable methods so they are not modifying the state. Before we dive into any particular method let’s look into the vote method, it has an argument in addition to state. So by default near_bingen expects arguments to be passed in JSON. The default deserializer of arguments is  JSON, so the body of the method expects a method to be called with something like the following: ‘{“is_vote”: true}’. So method name woould be vote and the argument is going to be the JSON informative is_vote equals true. This is important because you cannot pass arguments using a positional call into a contract but internally within this implementation we can call vote with positional arguments. And within unit tests we can also use positional arguments. If within Rust you are not calling the WebAssembly binary before the near_bingen and does its magic to turn it into a contract then you should use positional arguments. But if you are calling it either from simulation tests or on the blockchain then you are already calling that pre-compiled WebAssembly library binary contract and it will expect it in the JSON format. The raw WebAssembly doesn’t know any type inputs and the parse inputs are completely different.

Finally, the return arguments. This view call returns an Option<WrappedTimestamp>. The return types after the near_bingen contract macro expansion are also JSON. This type is a specific custom u64 type which is coming from the NEAR SDK JSON types. It’s a structure that has a raw u64 type but when the JSON deserialization or serialization is called on it it turns it into a string. The reason for this is that the JSON standard doesn’t support all u64 types. The limit is 2^53 because javascript originally had this limitation on numbers. The default JSON standard that comes from the frontend will not serialize large u64 numbers into a proper number. Instead we wrap the u64 integers (unsigned 64 bits) u128 integers into strings so that method will return you a JSON. Something like a string if where the actual number is “1000”, for example, or if the result doesn’t  exist it will return you null. This is a high level overview of the NEAR SDK.

Voting Call

We can now get into one of the methods. The main method that we have in this contract is a voting call. When you’re an active validator, you have an account that has at least one seat in the NEAR Protocol, so you must have a pretty large stake to be an active validator. Then, you can call this method before phase two in order to cast your vote.

If is_vote is true it means that you want to vote in the move to phase two, if you said is_vote is false it means you are withdrawing your vote. Let’s say you first tested your vote, and then you want to cancel it for some reason, for example, you feel that you didn’t know something and you want to retract it. Let’s see how it works. Vote is a change method and it automatically gets an argument deserialized by near_bindgen. As we discussed before even though it was expecting the JSON format, internally it is just a boolean variable that you can normally use in Rust. The method is not about how the state has changed or actions you did, so you don’t need to manually save the state. It will be automatically saved if the method completes. So the first thing it does is it calls the ping method. We’ll go back to that method later, but it does update the internal state by checking if the epoch has changed and who the active validator is in order to update the data. Then the vote method checks if we have already reached a result: if self.result.is_some(). In our case the result is a field that says if we have already reached 2 thirds of the validators, just a timestamp at to start each phase. 

This timestamp is in nanoseconds because this is how we expose the timestamp within a context. Option means that it either exists or doesn’t exist, if it exists, it’s given a timestamp. It means that voting already happened and we reached the phase 2 at the given timestamp in nanoseconds, if it’s none,  it doesn’t exist. It means the voting was not reached. So we just checked if we had already reached a vote. If we reached phase 2 already then we don’t need to do anything, your vote doesn’t matter because it’s already phase 2 and the timestamp is already there. Rust does not have null instead, it has option types which can return some value or none and you have to explicitly deal with it where it yells at you. But when it gets encoded for the outside world it is getting converted to JSON, for example it is getting stored and sent as null. Next is the query, the context of the NEAR SDK where we check who called us.

There’s three types of accounts that are available for context, one of them is a predecessor this is the account id of immediate predecessor of who called vote. NEAR is a synchronous environment and you can have a chain of cross-contract calls, so let’s say Alice calls Bob’s contract, Bob calls Cheryl’s contract, and in this case the predecessor is going to be Bob. Via Cheryl’s contract, the predecessor will be the immediate account who called us in this case, Bob. In contrast, there are also signer account id’s. This is the account who signed the original transaction. Let’s say it was Alice who called Bob’s contract and then so the signer account id will be Alice. The current account id is the account of the contract so in this case it’s going to be Cheryl, the account that holds the contract. On NEAR that contract always belongs to some account in which it has a state and balance, and has code deployed. For security reasons it’s very important to always not rely on the signer of the transaction, but on the immediate predecessor.  Let’s say Bob has a malicious contract and if he were to rely on the signer Bob can call some token transfer contract and withdraw a balance from Alice to themselves or their owner by acting on Alice’s behalf. If Bob would try to do this then the token contract will get the predecessor who is Bob so the Bob’s contract can only spend balance of their own. So it has some limitations but this guarantees that if you call in some contract this contract cannot really act on your behalf, except on itself.

The next thing we do is check the argument that was passed. If it is the vote then it means we want to add this our stake from the caller to a set of active validators who voted for phase two, so otherwise the value is zero. It means we want to set our stake to zero so we call a special method: validator_stake on the environment which is the close a host function on the NEAR Protocol runtime to get the active stake from the current epoch of the given account id. It will return how much stake they have for a given validator and it will be done immediately. So assert!(stake>0, ‘ {} is not a validator’, account_id); is a synchronous call where we verify that your stake is 0 or above 0 to make sure that we don’t pollute the state with a non-active validator. This way we can limit how many validators can get into our state, because we only had 100 seats when this contract was used. It means you can limit the number of validators in the state of this contract to a hundred. So basically if an assert triggers the term of the take is zero then we throw a panic and everything we did before this all changes to the state. It will be rolled back to the beginning of the function call only for this contract. So in this case coming to the example of Alice, Bob and Cheryl, only the state of Cheryl’s contract is going to be reverted but the state of Bob is not because of the synchronous environment. Next we check if we already voted before. Votes is a hashmap. 

Hashmaps

The hashmap is in the memory map, meaning every time it’s serialized, or deserialize all values have to be serialized into one piece, and then deserialized back. Whenever we call a method on this contract it will deserialize all fields of the hashmap, so all hundred potential accounts will be deserialized at once, but it will do this in a single storage read instead of a hundred storage reads. Even though the overall size of this is large because each account can be up to 64 characters and the balance is 16 bytes, the overall size of this is not as large, so it’s 100 times 80 so it’s like 8 kilobytes. 

What we need in this ping method is to iterate on every single vote in order to update the stake of the validator, and if we would have to iterate on every single key from a persistent collection. For example, for an ordered map then we will have to do something in the order of 100 reads, and in this case we we actually need to override them, which means we need a 100 writes and 100 reads, which is much more expensive than one-time bulk rate of 100 accounts, because we know that the structure is limited to 100. You don’t need to introduce the pagination that you might need to do with a persistent map. This is because the try was stored in the storage, it’s kept as a try, and this makes it expensive to do a lot of reads. So in this case it was beneficial to have one map to be able to iterate. So the guideline here would be if you need to iterate on every single key you probably want to have a hashmap or a vector but it’s also that you know that it’s bound to 100. If NEAR were fully sharded then this will need to be modified potentially with a potentially persistent map or maybe you can do something like a sharded map. We don’t have it right now, it’s not like a bulk map. It will be a persistent map of bulk objects, because you still need to iterate on all of them. Maybe you can split your accounts into 20 buckets, and then every bucket will be a vector, but you will store them separately into a persistent map so when you need to return it, you only need to read 20 keys or you introduce pagination for the ping method. Currently our storage is probably one of the most expensive operations that we face by default, and if you do like overwriting things then it will potentially be more expensive than a hashmap. That’s why we picked a hashmap but it’s not dramatically worse. But to be safe we can pretty much estimate 100 feet into our current gas limits, and it makes it a good choice for this particular contract. 

Voting Call (continued)

So at let voted_stake we remove the old stake, this is just a safety check. It’s an invariant, which means it’s unlikely to trigger in a real environment. After that, it verifies that the current stake is less than the total stake, then we subtract the old stake and introduce a new stake. I would argue that a better order would be to first subtract the old stake and then introduce a new stake in order to avoid overflows. Even though with our limit on the total supply it’s not possible to overflow, but you always want to be careful. Finally if this was an active vote if it was not a withdrawal then we introduce this into the storage. At the end we will check the result again. 

Let’s go to check_result first. We verify that when check_result is called we check that the result is not a sum. That’s another way of checking it, and it tells us it’s an invariant. We see what the total validator stake is. The let total_stake is another method that is returned from the context, and it’s saying we have 100 million total stake. Then internally total_voted_stake is the active stake encompassing all validators who voted on this. Let’s say if the active stake is more than two thirds rounded down of the total stake, then we reach the result and that’s pretty much the end of this contract. So voting has completed, this is a fairly simple method.

Ping 

Now we want to go to ping. This method happens on the boundary of epochs. After an epoch switches, it means a new set of validators became active within the epoch on NEAR Protocol. Validators do not change so even if someone got kicked out they will only be inactive in the next epoch. We verify that we didn’t reach the result in the self.result.is_none(). Then we take the current epoch height, the method suggests an integer that always is increasing, and after that we verify that it doesn’t match the previous epoch height. So we started with the epoch height 0 but every time a ping is called it can update the epoch. So if the epoch has switched, it can pass one epoch and five epochs, it doesn’t matter; what we’re going to do is we’re gonna go through all previous active votes that voted for phase 2. You reset the total stake to 0, and this is the last way of taking all of the data out of a fresh map and setting it to default. So basically we got a copy of the map from the internal structure and now the internal structure contains an empty hashmap. It’s a new thing that it was recently introduced in Rust. The it is called the default hashmap, which is just an empty map. It gave us data from our structure and now this contract structure contains an empty map. This is so that we can modify the current map or the new map based on the data from the old map in one go. We basically just need to call account_id, because we don’t need to know what the previous stake was. So next we check if someone was removed from the map in line 9, or removed from the active validators. We get their stake if their stake is positive we add them to the total stake and we keep their vote but if they were kicked out they have to vote again. Basically we update the total stake by recomputing what the new total stake is. Then we store all the results back to our contract state and at the end we do two things. We check the result after epoch has switched, if validators acquired more take or the total stake decreased then the result might have happened already. We remember at which epoch we did so next time someone calls ping or vote we don’t need to go through this if the epoch hasn’t switched. 

Final Rundown

That’s pretty much the voting contract. Basically every time someone calls a vote we first update data from the previous validators and write it into internal structure. Most importantly, we get the total_voted_stake updated and then we call check_result twice occasionally within the one call. What’s important is even though we can call ping here from one method, but also we can call ping directly by a transaction. You don’t need to reload, you can just call ping. 

There’s a bunch of view calls. The first just returns the result, the second returns how many stakes go into the pool, and the third is a functional way of returning all the votes. It basically the iterates on the map and then wraps it into the specific type of the balance using into. There is also a unit test in the same code and this just the rust way of doing this and the unit test can validate some logic within a contract by faking context and other things.