
Abstractions can be very powerful tools to express complex concepts succinctly. Declarative programming is all about abstracting irrelevant implementation details away, allowing engineers to focus more on logic than on code. Functional programming is a type of declarative programming, where we can express our logic in terms of functions.

In this blog post, I'll be moving from Go to increasingly abstract Scala to solve a problem.

## The Problem

Let's say we have a database of counts of page impressions, and we want to get total page impressions and clicks over any finite, discrete interval of time.

We have a table that looks like the following:

```sql
CREATE TABLE event_sums (
  -- The timestamp of the minute that these sums were collected for.
  timestamp int,

  -- The account of the events, e.g. "Asuna Inc."
  account string,

  -- The source of the event, e.g. "Marketing website 1".
  source string,

  sum blob,

  PRIMARY KEY((timestamp, account, source))
);
```

Sum will be a serializable type that looks like this:

```go
type Sum struct {
  PageImpressions uint64
  Clicks          uint64
}
```

Thus, we can create a type like this that maps one-to-one to sum.

```go
type Query struct {
  Timestamp int
  Account   string
  Source    string
}
```

Let's assume our database cannot add the serialized sum blobs. We're going to have to code this ourselves.

## The Go solution

Here's our first try at a solution, without an implementation of `fetch`.

```go
func add(a Sum, b Sum) {
  return Sum{
    PageImpressions: a.PageImpressions + b.PageImpressions,
    Clicks: a.Clicks + b.Clicks
  }
}

func fetch(query Query) Sum {
  // ...
}

func combine(queries []Query) Sum {
  var sum Sum
  for _, query := range queries {
    sum = add(sum, fetch(query))
  }
  return sum
}
```

This is really simple, but it is unperformant, as we run the database calls in serial. Let's try running things in parallel!

```go
func combine(queries []Query) Sum {
  var sum Sum

  var wg sync.WaitGroup
  sumsChan := make(chan Sum)

  // Fetch sum for each query
  for _, query := range queries {
    wg.Add(1)
    go func(query Query) {
      // Push to the sums channel
      sumsChan <- fetch(query)
      wg.Done()
    }(sum)
  }

  // Add the sums as they are fetched
  go func() {
    for sumEl := range sumsChan {
      sum = add(sum, sumEl)
    }
  }()

  // Wait for all of the wg to be done being fetched
  wg.Wait()
  close(sum)

  return sum
}
```

That's quite a bit more complicated, but still readable. However, what if we don't want to fire all of the database requests at once? We can use a struct token channel to do this, as follows:

```go
func combine(queries []Query) Sum {
  var sum Sum

  var wg sync.WaitGroup
  sumsChan := make(chan Sum)

  // Limit to 10 concurrent requests
  limiter := make(chan struct{}, 10)

  // Fetch sum for each query
  for _, query := range queries {
    wg.Add(1)
    go func(query Query) {
      // Rate limiting
      limiter <- struct{}{}

      // Push to the sums channel
      sumsChan <- fetch(query)

      wg.Done()
      <-limiter
    }(sum)
  }

  // Add the sums as they are fetched
  go func() {
    for sumEl := range sumsChan {
      sum = add(sum, sumEl)
    }
  }()

  // Wait for all of the wg to be done being fetched
  wg.Wait()
  close(sum)
  close(limiter)

  return sum
}
```

This is getting kind of long&#x2026; now, what if fetch returns an error?

```go
func fetch(query Query) (Sum, error) {
  // ...
}
```

We'll have to make quite a few changes to the code:

- Create an error channel to send errors to
- Somehow signal the goroutines to not proceed if an error has been encountered
- Handle the error channel so the error can be returned
- Close the error channel

Dammit, Go error handling!

This can easily cause our little `combine` function to be over 50 lines of code, which can quickly become an unreadable mess. Furthermore, if we do any of these steps wrong, we could run into bugs.

As software engineers, we shouldn't have to worry about problems that have been solved countless times by others, such as error handling on a parallelized function.

## The Scala solution

Before you read this section, keep in mind that Scala is a _very_ complex language, and explaining the exact mechanism for everything that works here would be very lengthy and out of scope of this blog post.

First, we'll redefine a few methods.

```scala
def add(a: Sum, b: Sum): Sum = {
  Sum(
    pageImpressions = a.pageImpressions + b.pageImpressions,
    clicks = a.clicks + b.clicks
  )
}

def fetch(query: Query): Option[Sum] = ???
```

There's one big difference here: we're using an `Option[Sum]` rather than a `Sum`. An `Option[Sum]` can have two values: a `Some[Sum]`, which means that it contains a `Sum`, and a `None`, which means the `Option` is empty. This lets us not have to deal with nil values, which also allows for some very powerful abstractions in Scala.

Now behold: the functional way of doing things.

```scala
def sum(queries: List[Query]): Future[Sum] = {
  val futures = Future.sequence(queries.map(q => fetch(q)))
  futures.map { list =>
    list.foldLeft(MatchSum(0, 0)) {
      case (acc, v) => v match {
        case Some(x) => acc + x
        case None => acc
      }
    }
  }
}
```

This code does all of the above. It:

- handles errors naturally using [Future](http://docs.scala-lang.org/overviews/core/futures.html)'s built-in error handling mechanism
- limits concurrent requests to a predictable 10 using a thread pool implicitly used by the Future construct (see [ExecutionContext](http://docs.scala-lang.org/overviews/core/futures.html))
- adds all of the match sums, with the `+` being homologous to the `add` method in Go and the `fold` being homologous to the for-loop and accumulator variable re-assigning.

The Scala is almost _one-fifth_ the length of the Go code in number of lines. The one major difference in implementation is that we are storing all of the `Sum`s in memory before we add them together, as `Future.sequence(...)` returns a `Future[List[Sum]]`, but we will get to that later.

We can still do better by borrowing a concept from abstract algebra: the [monoid](https://en.wikipedia.org/wiki/Monoid). Using the [Cats library](http://typelevel.org/cats), we define a monoid as follows:

```scala
object SumMonoid extends Monoid[Sum] {

  // Our combine function
  def combine(a: Sum, b: Sum): Sum = a + b

  // The identity element
  def empty = Sum(0, 0)

}
```

Using the `Monoid`, the `fold` in the above function may be rewritten as so:

```scala
def sum(queries: List[Query]): Future[Sum] = {
  val futures = Future.sequence(queries.map(q => fetch(q)))
  futures.map { list =>
    list.foldLeft(SumMonoid.empty) {
      case (acc, v) => v match {
        case Some(x) => SumMonoid.combine(acc, x)
        case None => acc
      }
    }
  }
}
```

Furthermore, for every `Monoid[T]`, there exists a `Monoid[Option[T]]`, so we can write things like this:

```scala
def sum(queries: List[Query]): Future[Sum] = {
  val futures = Future.sequence(queries.map(q => fetch(q)))
  futures.map { list =>
    list.foldLeft(Monoid[Option[Sum]].empty) {
      case (acc, v) => Monoid[Option[Sum]].combine(acc, x)
    } match {
      case Some(x) => x
      case None => Monoid[Sum].empty
    }
  }
}
```

Or simply:

```scala
def sum(queries: List[Query]): Future[Sum] = {
  val futures = Future.sequence(queries.map(q => fetch(q)))
  futures.map { list =>
    list.foldLeft(Monoid[Option[Sum]].empty)(Monoid[Option[Sum]].combine) match {
      case Some(x) => x
      case None => Monoid[Sum].empty
    }
  }
}
```

It turns out, folding over a `Monoid` is a common use case. Cats gives us a [Foldable](http://eed3si9n.com/herding-cats/using-monoids-to-fold.html) typeclass that allows us to write `list.combineAll` to perform the fold.

```scala
def sum(queries: List[Query]): Future[Sum] = {
  val futures = Future.sequence(queries.map(q => fetch(q)))
  futures.map { list =>
    list.combineAll match {
      case Some(x) => x
      case None => Monoid[Sum].empty
    }
  }
}
```

Unwrapping an `Option[T]` to a `T`, using the empty value of the `Monoid` is also a common operation. Cats gives us the `.orEmpty` method for this.

```scala
def sum(queries: List[Query]): Future[Sum] = {
  val futures = Future.sequence(queries.map(q => fetch(q)))
  futures.map { list =>
    list.combineAll.orEmpty
  }
}
```

Rewriting things yet again to use the `_` syntax, we can write this:

```scala
def sum(queries: List[Query]): Future[Sum] = {
  val futures = Future.sequence(queries.map(q => fetch(q)))
  futures.map(_.combineAll.orEmpty)
}
```

Also, the `q => fetch(q)` can simply be written as `fetch`.

```scala
def sum(queries: List[Query]): Future[Sum] = {
  val futures = Future.sequence(queries.map(fetch))
  futures.map(_.combineAll.orEmpty)
}
```

Using Cats's [Traverse](http://typelevel.org/cats/typeclasses/traverse.html) typeclass defined on `List[_]`, we can write things like this:

```scala
def sum(queries: List[Query]): Future[Sum] = {
  val futures = queries.map(fetch).sequence
  futures.map(_.combineAll.orEmpty)
}
```

Or simply:

```scala
def sum(queries: List[Query]): Future[Sum] = {
  queries.map(fetch).sequence.map(_.combineAll.orEmpty)
}
```

But wait: we're still storing all of the `Sum`s in memory! Fortunately, `Future` can also be treated as a `Monoid`, so like `Option`, we can fold over it. Thus, we can omit the `.sequence` and `.combineAll` it.

```scala
def sum(queries: List[Query]): Future[Sum] = {
  queries.map(fetch).map(_.combineAll.orEmpty)
}
```

With comments:

```scala
def sum(queries: List[Query]): Future[Sum] = {
  // Get a list of queries
  queries
    // Get the Sum of each Query
    .map(fetch)
    // Aggregate using the Sum monoid
    .map(_.combineAll.orEmpty)
}
```

We've now turned a 50 line function into a one-liner (sans the Monoid). Damn!

If you haven't been exposed to this kind of programming before, you may be thinking that the Scala one-liner is much harder to read than the Go. However, I argue that once you learn and fully understand the abstractions, the Scala code will be better tested, more concise, and easier to understand. In a future blog post, I will be discussing exactly why I believe this.

Thanks to [Pradyuman Vig](https://github.com/pradyuman) for reading drafts of this.


The Power of High Level Abstractions


Let's say you have a set of keys, and you want to lock on a per-key level. A naive solution could look like this:

```scala
class LockManager[K] {

  private[this] val logger = getLogger

  val locks: Atomic[Map[K, Lock]] = Atomic(Map[K, Lock]())

  def run[T](key: K)(fn: => T): T = {
    locks.getAndTransform { locksMap =>
      locksMap.get(key) match {
        case Some(lock) => {
          lock.lock()
          locksMap
        }
        case None => locksMap + (key -> new ReentrantLock())
      }
    }
    try {
      fn
    } finally {
      locks.getAndTransform { locksMap =>
        locksMap.get(key) match {
          case Some(lock) => {
            lock.unlock()
            locksMap - key
          }
          case None => {
            logger.warn(s"Lock for ${key} suspiciously missing")
            locksMap  // wtf?? this shoudnt happen
          }
        }
      }
    }
  }

}
```

However, there's quite a few things wrong with this:

- It has the potential to use a LOT of memory. (if we have high key cardinality)
- We need to lock BOTH the map AND the key lock. (If we had mutability a la `ConcurrentHashMap`, we could get around this, but that'd be un-Scala-like.)
- We keep creating keys and destroying keys. If we didn't destroy keys and keys weren't often reused, we'd OOM.
- Code is inevitably messy due to all of the above.

The thing is, we usually don't _need_ all resources to be accessible simultaneously. People rarely do. Thus, we can create a locking system using the mod of a hash, similar to a hash table, where each "bucket" is simply a lock. This is also known as **lock striping**.

With this strategy in mind, we can rewrite the above class like so:

```scala
class LockManager[K](power: Int) {

  private[this] val logger = getLogger

  val locks: Vector[Lock] = Vector.fill[Lock](1 << power)(new ReentrantLock())
  val modMask = locks.length - 1

  def get(key: K): Lock = {
    locks(key.hashCode() & modMask)
  }

  def run[T](key: K)(fn: => T): T = {
    get(key).lock()
    try {
      fn
    } finally {
      get(key).unlock()
    }
  }

}
```

Note that we've fixed quite a few problems:

- We are fixed to a known amount of memory.
- We don't need to lock the list of locks, as we're only performing reads. (Also Scala enforces immutability on `Vector`, yay!)
- We can "get" a lock, since locks are never deleted.
- We allocate all of the locks upfront, so GC can't screw us over. This also hugely simplifies our code.

The only downside to this is that each lock could possess multiple keys, though you can probably tweak the numbers to find something with a good balance of throughput and memory usage.

### Further reading

- [A good StackOverflow post](http://stackoverflow.com/questions/8354758/designing-a-key-based-lock-or-lock-map)
- [Guava's implementation](http://google.github.io/guava/releases/21.0/api/docs/com/google/common/util/concurrent/Striped.html)


Lock Striping in Scala


Consider the following code:

```scala
import cats._

case class MyClass(a: Int)

object Main {
  implicit val myMonoid = Monoid[MyClass]

  def main(args: Array[String]): Unit = {
    println(myMonoid)
  }
}
```

For those unfamiliar with Cats/Scalaz, `Monoid` is a simulacrum-generated typeclass, so its `apply` method simply resolves the implicit in scope. It looks something like this:

```scala
object Monoid {
  def apply[T](implicit m: Monoid[T]): Monoid[T] = m
  // ...
}
```

This code will not compile because the `Monoid` cannot be implicitly resolved. However:

```scala
import cats._

case class MyClass(a: Int)

object Main {
  implicit val myMonoid: Monoid[MyClass] = Monoid[MyClass]

  def main(args: Array[String]): Unit = {
    println(myMonoid)
  }
}
```

This will compile and print `null`. It resolves the implicit using the default empty `null` value!

Be careful when exporting implicits in your objects.


A Pitfall in Scala Implicits


Amazon ECR is Amazon's container registry service and is a convenient place to store Docker images. If you're already using AWS, it's pretty nice to use as it has free data transfer and it requires minimal setup.

However, using it with Jenkins is deceptively complicated. You may think the [Amazon ECR Jenkins Plugin][amazon-ecr-plugin] all you need, but if you're not using Jenkins Pipelines, you're on the wrong track-- it doesn't work! You will get an error that looks something like this:

```
[myservice] $ docker push 123456789.dkr.ecr.us-east-1.amazonaws.com/myservice:latest
The push refers to a repository [123456789.dkr.ecr.us-east-1.amazonaws.com/myservice]
62807a93cfba: Preparing
35c20f26d188: Preparing
c3fe59dd9556: Preparing
6ed1a81ba5b6: Preparing
a3483ce177ce: Preparing
ce6c8756685b: Preparing
30339f20ced0: Preparing
0eb22bfb707d: Preparing
a2ae92ffcd29: Preparing
ce6c8756685b: Waiting
30339f20ced0: Waiting
0eb22bfb707d: Waiting
a2ae92ffcd29: Waiting
denied: Your Authorization Token has expired. Please run 'aws ecr get-login' to fetch a new one.
Build step 'Docker Build and Publish' marked build as failure
Finished: FAILURE
```

The easiest solution I've found is to set up the [Amazon ECR Credential Helper][amazon-ecr-credential-helper], of which the documentation is pretty straightforward to follow to set up.

When doing this, make sure your credential file is named `~/.aws/credentials`. If you use `config` it won't work.

Note that you don't need to use the Cloudbees Docker Build And Push plugin anymore if you use this. Just use plain ol' `docker build` and `docker push`!

This blog post didn't really go over any of the details, but if you run into this issue, this is how I solved it!

[amazon-ecr-plugin]: https://wiki.jenkins-ci.org/display/JENKINS/Amazon+ECR
[amazon-ecr-credential-helper]: https://github.com/awslabs/amazon-ecr-credential-helper


Setting up Amazon ECR with Jenkins


When developing Legends.ai, we highly followed the advice of Tufte's [The Visual Display of Quantitative Information](http://amzn.to/2zi7a3l) and optimized for data density. Our designs use stacked bar graphs whenever displaying relative and absolute magnitude, as opposed to more ubiquitous but much less dense pie charts.

After building our frontend, we noticed many things looked misaligned: squares looked like they had random margins, and the bars on our stacked bar charts simply didn't line up. We first thought this was a Chrome rendering error, but it turned out to be much worse: a fundamental problem in the way LCD's display pixels.

Below is a CSS rendition of the problem:

<style dangerouslySetInnerHTML={{ __html: `
  .buggybg {
    background-color: #000;
    padding: 5px;
    width: 110px;
  }
  .buggycss {
    position: relative;
    height: 5px;
    width: 100px;
    background-color: red;
  }
  .buggycss.large {
    height: 15px;
    width: 300px;
  }
  .buggycss .green {
    width: 50%;
    position: absolute;
    left: 0;
    top: 0;
    height: 100%;
    background-color: green;
  }
  .vbar {
    position: relative;
    width: 5px;
    height: 50px;
  }
  .bartop {
    background-color: red;
  }
  .barbottom {
    background-color: green;
  }
  .vbarbg {
    background-color: black;
    padding: 5px;
    width: 15px;
  }
  .vbar.large {
    width: 15px;
    height: 150px;
  }
`}} />

<div class="buggycss"><span class="green"></span></div>

This happens due to the way pixels are implemented on a display.

![(from Wikipedia)][https://upload.wikimedia.org/wikipedia/commons/d/de/Closeup_of_pixels.JPG]

If you don't see anything off, you are probably on a high DPI display. But for my large 1440p display and my friend's Retina Macbook, the problem is very clear.

Many effects occur when altering the shapes and colors. If you put a dark background behind the bar, the offset seems to invert.

<div class="buggybg"><div class="buggycss"><span class="green"></span></div></div>

Changing the size of the bar doesn't change the magnitude of the offset, since the problem is still only in one pixel.

<div class="buggycss large"><span class="green"></span></div>

Orienting the bar vertically seems to still have the same problem, but shifted in a different direction.

<div class="vbar bartop"></div><div class="vbar barbottom"></div>

With a black background, again the direction is inverted.

<div class="vbarbg"><div class="vbar bartop"></div><div class="vbar barbottom"></div></div>

The reason this happens is unclear to me. If a pixel consists of parallel strips of color channels, why would a 90 degree turn also have the same problem?

And again, the problem is no more apparent when the figure is enlarged.

<div class="vbar bartop large"></div><div class="vbar barbottom large"></div>


Beware of Colored Rectangles on LCDs


I've always been a huge fan of Ethereum; however, it has some shortcomings, as recently made evident by [CryptoKitties](http://www.bbc.com/news/technology-42237162). In this blog post, I will introduce EOS and explain how it will solve many of Ethereum's current problems.

## What is EOS and DPoS?

Similar to Ethereum, EOS is a blockchain platform supporting smart contracts. It differs from Ethereum mainly by its consensus algorithm: DPoS.

Ethereum blocks are generated just like Bitcoin in that it requires miners to compute hashes to compete in finding a block (proof of work). Similar to Bitcoin, there is a limit to how big a block can be, which allows for anyone to be able to run a full node.

EOS, on the other hand, uses nodes that are voted on called "block producers" which are in charge of processing transactions into blocks. There is no block size limit, and block producers can be massive datacenters processing many transactions in parallel. Due to this, not everyone is intended to be able to run a full node; however, the system has much greater performance as a result.

![DPoS under normal operation](https://steemitimages.com/0x0/https://steemitimages.com/DQmUnFg35XqTednrwUiZo7AHX7SHQ7hAUTytA8BLwk64EN7/image.png)

[Find more information on DPoS here.](https://steemit.com/dpos/@dantheman/dpos-consensus-algorithm-this-missing-white-paper)

## Ethereum vs EOS

There are many problems with Ethereum **in its current state** that EOS has solutions for, which I will list some of below.

### Scalability/Speed

Ethereum can only handle ~23 transactions per second (TPS) on average. Larger transactions require more gas, so this could be even less. This is due to block size limits: larger blocks cannot be processed by all nodes, so Ethereum resorts to using smaller blocks in favor of decentralization.

Furthermore, the Ethereum block time is around 15 seconds, which will be reduced to around 5-7 seconds when PoS is introduced. Taking confirmations into account, this causes a lot of lag between performing a transaction and the transaction being recorded on chain. This prohibits many types of applications that require more real-time feedback, e.g. decentralized asset exchange.

EOS has a projected TPS of 100k/sec and a block time of 2 seconds on the testnet with a target time of 500ms on the mainnet. DPoS enables EOS to do things that the Ethereum network never will be able to do: higher TPS per node and lower block times, since all nodes are known.

### Contract execution fees

In order for one to use a dApp, one must pay gas in order to perform an action. This requires people to buy Ether, which can be a barrier to entry for those that do not already have Ether. For example, if I decided to make a decentralized Wikipedia where every edit appended some sort of pointer to the blockchain, I would have to pay a fee in order to make my edit. Although the fee may be small, it is still a fee that needs to be dealt with.

However, with EOS, there are no such fees. The "fees" (which are not actually transaction fees, explained in next section) are paid by the dApp, so the user does not need to own any EOS in order to be able to use an application. This is great for having new, non-crypto people adopt the platform.

As someone with no knowledge of crypto other than the word "Bitcoin", would you rather register an account on [Steemit](https://steemit.com) **just like you do on other websites** and post your content **just like you do on other websites**, or would you rather download Metamask, buy Ether on Coinbase, transfer said Ether to your wallet 10 days after you bought it, and then pay a few cents to post your content, waiting 30 seconds before it shows up?

### The "hodl" factor

As an investor, it is generally better to own things that people will not want to sell. There is an analogy where Bitcoin is gold, Ethereum (Ether) is oil, and EOS is real estate.

Bitcoin is gold because you want to buy it once and keep it for a long time. There is no inherent value in it other than scarcity, as it is not even viable as a payments platform. Transaction fees are increasing dramatically.

Ether is oil because you use it to buy gas. If you ever need to execute a transaction on the Ethereum network, whether that is token transfer or contract execution, you must pay Ether for it.

EOS is real estate because it works in a completely different way from Bitcoin or Ethereum: token holders get a percentage of the bandwidth of the EOS network relative to their total share of the tokens. If EOS can handle 100k TPS and you hold 1% of tokens, you are reserved 1k TPS. The only cost to developing your dApp is purchasing tokens in the first place. Furthermore, if you are not a dApp developer, you can rent your stake in the network, allowing you to collect interest on your stake.

EOS encourages people to buy and hold their tokens-- not just for speculation, but because they need to in order to reserve network bandwidth.

### Other factors

Other less important but still notable factors include:

- **Named account system** Use a username rather than a long hexadecimal string.
- **Storage** IPFS-based storage works by replicating files across all 21 delegates. This is an amazing level of redundancy, and these nodes are trusted and voted on by the network to hold your data. Ethereum has no file storage built in. One could argue that file storage should not be part of the blockchain platform, but having an all-inclusive solution could make development easier. [Read the EOS Storage whitepaper here.](https://github.com/EOSIO/Documentation/raw/master/EOS.IO%20Storage.pdf)
- **WASM** Rather than writing Solidity, you write C++ (officially supported) or any language that compiles to WASM.

## EOS "Shortcomings"

Now, EOS is not a complete upgrade over Ethereum. From the [Ethereum wiki](https://github.com/ethereum/wiki/wiki/Sharding-FAQ), Vitalik's scalability trilemma states that blockchain systems can only at most have two of the following three properties: decentralization, scalability, and security.

EOS is scalable and secure, but it sacrifices decentralization, which Vitalik defines as "the system being able to run in a scenario where each participant only has access to O(c) resources, ie. a regular laptop or small VPS". The question is: is decentralization really necessary? Is it really a core value of blockchain technology?

The Wikipedia definition of decentralization is quite different:

> Decentralization is the process of distributing or dispersing functions, powers, people or things away from a central location or authority.

Having massive block producers is not decentralized in Vitalik's definition, but it is definitely decentralized by the Wikipedia definition. Block producers can be and will be anyone token holders decide to vote for.

Note that Ethereum in its current state is much more centralized than Bitshares and Steem, both based on the same DPOS technology as EOS.

![Distribution of block validators in various blockchains](https://i.imgur.com/0eT9SAE.jpg)

_If you'd like to read the debate between Vitalik and Dan, see [Vitalik's criticism](https://np.reddit.com/r/ethereum/comments/6qm0y2/is_the_ethereum_team_defending_their_ground/dkyk94c/) and [Dan's response](https://steemit.com/eos/@dan/reponse-to-vitalik-s-written-remarks)._

There is also the looming factor that EOS may not deliver on its promises. Unlike most of the other major cryptocurrencies currently on the market, they do not have a mainnet deployed-- EOS is currently an ERC20 token that you can later redeem for "real" EOS in the future. Although the testnet is out, they have a way to go before building the 100K TPS platform they promised. Also, their ICO is uncapped, meaning that they are collecting as much money as they can for their ICO-- many consider this to be a "scammy" play.

However, the lead developer of EOS, Daniel Larimer, has built two other extremely successful DPOS platforms (Bitshares and Steem) with proven technology, and I am confident that he and his team will be able to execute on their vision, especially with \$1B worth of funding.

## Ethereum has its place

I believe Ethereum makes sense for things that do not require mainstream adoption and/or high TPS, i.e. as a "settlement layer". It is more decentralized than EOS and suited for things that need that.

Ethereum is aware of its performance problems, with Casper, Plasma, and sharding being solutions currently in the ideation phase. However, it will be a while before these are implemented in production. The real world performance of these solutions is currently unknown.

For applications such as social networks or Uber, I believe EOS will find significant traction due to its performance. DApp developers are already running into Ethereum performance bottlenecks. I believe average web users-- arguably a large portion of the market for dApp users-- are willing to tradeoff decentralization for usabilty, and they probably care more about the blockchain being public and auditable rather than it being forgeable on any random laptop.

If EOS can fulfill its promises before Ethereum can solve its scaling dilemma, EOS is likely to have a strong advantage in finding traction among common internet users, and thus overtake Ethereum in high performance smart contracts/dApps.

[Read the EOS whitepaper here.](https://github.com/EOSIO/Documentation/blob/master/TechnicalWhitePaper.md)

Also, a fun video: [Vitalik reveals Ethereum 2.0.](https://www.youtube.com/watch?v=pSNv1mBb3Qs)

> First they ignore you, then they laugh at you, then they fight you, then you win. ~Gandhi

Thanks to Dylan Macalinao, Michael Huang, Tejas Manohar, and Justin Park for reading drafts of this.

_(If you liked this post, [join our crypto Discord!](https://discord.gg/5AkBWSY))_


Why EOS will overtake Ethereum in high performance smart contracts


As mentioned [in a previous post](https://ian.pw/posts/2017-12-08-why-eos-will-overtake-ethereum-in-high-performance-smart-contracts.html), I believe EOS will overtake Ethereum as the smart contract platform used by the mainstream audience. Part of this is due to its economic model which encourages people to use the platform and to not sell their currency. In this post, I will argue why EOS's economic model offers more benefits to token holders than Ethereum's.

## Transaction fees

Ethereum, Bitcoin, and most other cryptocurrencies charge small costs per transaction called **transaction fees**. Why do these fees exist?

Imagine a world where there were no transaction fees. In Bitcoin and Ethereum, there is a limit to the number of transactions that can be included in a block. Thus, if I were a malicious individual who wanted the network to fail, I could make a lot of transactions between my own wallets without losing any money. However, due to the transaction limit, I would render the blockchain useless for anyone else, and it would be difficult to determine what transactions are legitimate.

However, with transaction fees, miners have an incentive to prioritize certain transactions over others, as they will make more money if they choose transactions with higher fees. In Ethereum specifically, each transaction has a certain amount of "gas" associated with it and a gas limit per block. Thus in order to raise your chance of your transaction getting accepted, you increase the gas price of your transaction.

As seen in this scenario, transaction fees exist to prevent spam. These fees are acceptable and almost expected for transferring currency. However, smart contracts do not solely encompass currency: they exist to provide a way for anyone to access a decentralized, immutable ledger of anything. It is unreasonable to have transaction fees for many applications of blockchain technology.

Consider a dApp where one can share links and text, similar to Reddit. If you had to pay a bit of currency every time you posted something, would you be motivated to add more content to the website, or would you keep your content to yourself? Let's say you had to pay a fee every time you upvoted or downvoted content-- again, would you vote if you had to pay a few cents for them?

![Ethereum gas prices. A few cents per transaction adds up. Source: ethgasstation.info](https://i.imgur.com/wQlH7pa.png)

## Proof-of-stake bandwidth allocation

EOS on the other hand has no transaction fees. This then begs the question: how does EOS prevent spam?

EOS works differently in that it allocates blockchain bandwidth based on the amount of EOS tokens one holds. That is, if you hold 1% of all tokens, you are entitled to 1% of all bandwidth of the network.

Thus if a spammer wanted to cause network congestion, they cannot exceed their total reserved bandwidth if the network is at full capacity. That is, if they hold 1% of all EOS tokens, they cannot use up more than 1% of the EOS network's total possible bandwidth. Any EOS token holder can perform operations on the blockchain since they have reserved bandwidth.

Bandwidth allocation can work two ways: using your own bandwidth or delegating bandwidth to someone else. Holding tokens gives you the ability to perform operations on the network. In the case that you do not hold any tokens, a dApp can allocate some tokens to give you bandwidth.

This model of transaction "fees" encourages hold tokens, propping prices up. In order to use the network without someone else delegating bandwidth to you, you must own tokens. In case you are not using your tokens, a bandwidth marketplace will exist: token holders can rent out their excess bandwidth to get more tokens. The only hodl factor Ethereum has is that it is a deflationary currency.

Note that in Ethereum's future, Casper proof-of-stake will encourage holding for those who would like to validate blocks. However, those that do not care about running a node will not be affected by this benefit.

Furthermore, unlike Ethereum, tokens are not spent in order to perform actions on the network. Due to this, EOS users will not be discouraged from using the network to perform various tasks, such as sending out a tweet or posting a classified. Ethereum users on the other hand must spend tokens for actions, causing users to be more reluctant to use dApps for non-financial tasks.

## But what about the miners?

One may wonder why a miner (in EOS, block producer) would process transactions without transaction fees. In Ethereum and Bitcoin, miners are incentivized to create blocks for two reasons: block reward and transaction fees.

The block reward is a guaranteed amount of tokens given to the miner of the block. This reward is specified in the rules of the blockchain. For example, in Bitcoin, a block grants the miner 12.5 bitcoins. Ethereum is a bit more complex than this but follows a similar structure.

Miners are incentivized to include transactions in blocks due to transaction fees. They include the highest fee transactions in their blocks to make more money.

Without transaction fees, why would an EOS block producer include as many transactions as possible into blocks? The answer is _voting_.

EOS has 21 block producers which process transactions into blocks on the EOS blockchain. These block producers are voted on by token holders, of which token holder voting power is based on the amount of tokens they hold.

A token holder wants the network to continue functioning, so if a block producer is not performing up to par, they would be voted off the block producer list. Token holders are incentivized to pick the best performing block producers for the task in order to increase the value of their tokens.

Block producers are paid very well for their performance (EOS has an initial 5% annual inflation), and if voted off, they lose access to all block rewards. Thus they are highly incentivized to not underperform.

## Conclusion

Compared to traditional token models, EOS encourages:

- Hodling -- EOS users want to hold tokens to rent out or gain access to more bandwidth.
- Usage of dApps -- EOS users will use the platform more because there are no transaction fees.

Its economic model is better poised to encourage activity on the platform and thus make it the dominant platform in the mainstream.

Thanks to Michael Huang, Grace Jiang, Tejas Manohar, and Andrew Tian for reading drafts of this.

_(If you liked this post, [join our crypto Discord!](https://discord.gg/5AkBWSY))_


An analysis on the economic models of EOS and Ethereum and the advantages and disadvantages to both.

The Cryptoeconomics of EOS vs Ethereum


First off, this article assumes familiarity with [Maker](https://makerdao.com/) CDP's. (If not, I highly suggest reading up on it!)

Recall that the liquidation ratio of a CDP is the collateral-to-debt ratio of the CDP. For example, if the liquidation ratio is 150% on ETH/USD, if ETH is $100 and you have 1 ETH in the CDP, you may accrue up to $66 of debt (that is, generate up to \$66 Dai) as your collateral is worth 150% of your debt. This gives us a leverage ratio of 1.6x. But can we go higher?

Assuming enough liquidity exists, we can then buy 0.66 ETH with this Dai.

![The amount of additional leverage drops off each round.](/images/20180102_additional-leverage-per-round.png)

We can then put this ETH back into our CDP to draw more Dai. We can draw $\$66.66 * 1/1.5 = \$44.44$ more Dai from our CDP. We can keep doing this forever to generate more Dai and thus more leverage; however, we get less Dai each time. But how much?

![Cumulative leverage converges to a finite value.](/images/20180102_cumulative-leverage-per-round.png)

Let $\lambda$ represent the liquidation ratio and $L$ represent our maximum leverage ratio. We compute $L$ to be the following

$$
L = \frac{1}{l} + \frac{1}{\lambda^2} + \frac{1}{\lambda^3} + \ldots
$$

But this is just a [geometric series](https://en.wikipedia.org/wiki/Geometric_series) with $a = 1$ and $r = 1/\lambda$. Using the infinite sum of geometric series formula, we get:

$$
L = \frac{1}{1 - r} = \frac{1}{1 - \frac{1}{\lambda}}
$$

which in the case of $\lambda = 1.5$, 3x leverage.

![Riskier assets will have higher liquidation ratios.](/images/20180102_leverage_liquidation.png)

## Theoretical vs Actual Leverage

The above calculation accounts for the maximum leverage in an infinitely liquid ETH/DAI market with an instant ability to fund a CDP. However, there are several factors that come into play:

- Finite liquidity on ETH/DAI
- CDP funding and trade delays

### Finite liquidity

The amount of ETH one can buy with Dai is currently very limited, and the spread is significantly wide. The above calculations make the assumption that one can continue to buy more ETH at the market price, which is very far from the truth.

However, this liquidity will improve over time, as the future of the Maker system requires this liquidity to exist for its auto-liquidation system to work.

### CDP funding and trade delays

There are several actions one must perform to increase the collateral locked in the CDP contract, all taking one block each:

1. Generating Dai from the CDP
2. Buying W-ETH from the WETH/DAI market
3. Converting W-ETH to PETH
4. Locking the newly generated PETH into the CDP.

One should note that step 3 will not exist in the final version of Maker -- pooled Ether is a workaround to the MKR token generation system not being in place. However, this means that at a minimum, three blocks will pass before the CDP can be refunded.

If one generates the maximum amount of Dai from the CDP, they are at immediate risk of liquidation if the price of Ether drops even one cent. Since the process of refunding the CDP is not atomic, there is a very real risk of this liquidation taking place.

In theory, one could write a smart contract that performs this entire 4-step process in one transaction with a specified minimum accepted price. Before this exists, however, one is subject to the aformentioned risks.

_(If you liked this post, [join our crypto Discord!](https://discord.gg/5AkBWSY))_


A summary of how to compute and obtain maximum leverage on MakerDAO.

Maximum Leverage on Maker


Since March, I've been using NixOS as my primary operating system and Nix as my primary package manager. Everything is configured in my dotfiles which are synchronized via Git between three machines:

- Personal Laptop (OSX)
- Work Laptop (OSX)
- Work Desktop (Nixos)

In this post, I'll go over some of the advantages and disadvantages of switching my setup over from Bash scripts, brew, and Arch's `pacman` to Nix/NixOS.

## Why did I switch to Nix/NixOS?

My old setup had many lot of problems, some of which included:

- **Package synchronization.** I'd have to manually install new packages across my different systems, especially cross-OS. This was especially annoying for CLI tools that didn't have proper installers, such as [the Jenkins X CLI](https://jenkins-x.io/). Nix would allow me to define cross-platform build scripts for these kinds of applications, so I'd only have to worry about this once.
- **Package version synchronization.** Even more annoying than the above was making sure I had the same version of the tools installed across all of my systems. Jenkins X in particular changes drastically across different versions, so this was especially important to me.
- **Dotfile synchronization.** I had this [janky Bash script that I wrote in 2014](https://github.com/macalinao/dotfiles/blob/09e43c5a8f38b1a6662fd73ed026e47ccef12734/install.sh) that would create symlinks from my dotfiles to my home directory. It required constant updating and it would often break and only link half of my files. I really liked how Nix generations were atomic so that a bad script I wrote wouldn't cause my system to get upgraded halfway.
- **Mistakes had permanent consequences.** I'm sure every Arch user has screwed up something in their system causing random bugs to happen. Since it's hard to uninstall things cleanly or remember what exactly you did, I would just live with the results of my mistake. Nix has simple rollbacks which are really good for if you're hacking around with random packages that you plan on uninstalling later.

I had been hearing a lot about the virtues of Nix since late last year (2018), and in March 2019 I finally decided to pull the trigger.

The main reasons why I transitioned over were:

- **Code as configuration.** I loved the idea of having configuration files deterministically install all of the applications on my computer.
- **Simple reverts of my environment.** As mentioned above, I screwed up my Arch install many times.
- **Broad and active ecosystem.** I came from Arch, so I was used to having every Linux package under the sun available at my disposal. There was a pretty decent Nix wiki (or so I thought at the time) so I felt like I was in decent hands.
- **Functional.** Scala was my favorite language at the time, and I was starting to look more into Haskell development. Pretty much everyone I ran into that used Nix also wrote Haskell, so I thought that it would get me more plugged into that programming community.

After trying out NixOS in a VM for about a week, I backed up and reformatted my Arch SSD and started from scratch.

## What went well

Aside from the above advantages, there are many things I've found to be really useful that I didn't know of when I first installed NixOS.

### General service/program configuration

There are several things I never set up on my Arch computer because I thought they would be too complicated to set up in my dotfiles, and if I broke things I wouldn't easily be able to replicate my issues.

One thing I implemented was [an nginx ingress for my HTTP services](https://github.com/macalinao/dotfiles/blob/master/nixos/services/nginx.nix). I have a few HTTP services on my machine that I'd like to expose to anyone on the local network (all configured and installed through Nix):

- Transmission (for torrents)
- A file server, for my aforementioned torrents
- A few servers (frontend and backend repos of my company's product)
- Plex, for media viewing

I wanted a way to be able to share certain services with other people and devices on my network without having to expose more than port 80, so I set up an nginx proxy. This was really easy to setup, and I did not need to write any actual nginx config. I was also easily able to add the `fancyindex` extension to nginx without having to recompile or install anything extra. Setting up htpasswd was also very easy as I just had to read an htaccess file from my secrets repo.

Other things I've configured via Nix include VSCode (including packages), Private Internet Access, and Jupyter notebook. They were all much easier to set up than if I had done the manual way.

### XOrg configuration

One of the big reasons I got NixOS was that my XMonad installation didn't work properly on Arch out of the box. Nix would let me remove all of the confounding factors that might have been interfering with a proper installation of the window manager.

Here's my configuration:

In my `configuration.nix`:

```
services.xserver = {
  enable = true;
  # displayManager.startx.enable = true;
  dpi = 96;

  desktopManager = {
    default = "xfce";
    xterm.enable = false;
    xfce = {
    enable = true;
  };
};
```

In my `home-manager` config:

```
xsession.enable = true;

xsession.windowManager.xmonad = {
  enable = true;
  enableContribAndExtras = true;
  config = ../dotfiles/xmonad/xmonad.hs;
};
```

Through this, I'm able to set up an XMonad config and get it working properly on startup. Also, since it's code as configuration, I'm able to transfer this between machines or to friends I'm evangelizing XMonad to. It was super easy to set up, and debugging was pretty painless since I'd just have to `nixos-rebuild switch` whenever I made changes.

## What went poorly

Nix/NixOS are not for the faint of heart. You have to be reasonably proficient with a lot of things to use NixOS at all. While most things are made 10x easier by Nix, some things are extremely difficult, and you don't know whether or not you're even able to do what you're trying to do.

If you already don't like writing custom logic in your build scripts, you will have an even worse time just trying to get your system to behave the way you want it to if you diverge even slightly from defaults. Think debugging webpack configuration, but instead of just worrying about JavaScript you have to worry about your _entire system_.

There's several UX issues that make me not recommend this to beginners or people that have tight deadlines.

### Sometimes nixpkgs-unstable just breaks.

A package I wanted to install uses Wine on the backend, but when I tried to install it, it started to complain that [qtbase could not build](https://github.com/NixOS/nixpkgs/issues/63829). There are several solutions to this problem, including:

- **Forking nixpkgs and fixing it yourself.** Nix makes it really easy to do this, though with this you'll have to maintain your fork and your build cache is more limited. I haven't done this because of those two reasons.
- **Overriding the derivation.** You can replace any package in Nixpkgs (or anywhere for that matter) with a custom derivation. However, sometimes this doesn't work because the problem exists across multiple packages (as was the case with the above).

This is all really annoying to do, and you can waste a lot of time if you don't know Nix very well (like me). I often just don't install the package and wait for it to get fixed in Nixpkgs. There's also a simple workaround: to install via Brew/installer script on my Mac.

### Sometimes you get a random build error with no error message.

While writing this blog post, I was trying to upgrade my Hakyll installation (what I use to generate my blog). I was getting an error that `digest-0.0.1.2` failed to build with a cryptic error message. The solution was to add `zlib` to the `nix` section of my `stack.yaml`, but _nowhere_ in the error message did it say that I had missing zlib headers, that I needed to install zlib, etc.

I found out what I needed to do via Google, but the fix was still a guess. It would have been much better if I knew what I had to do from a compile error or equivalent. There is probably a way to find the error, but I couldn't find out how.

### You need to write a derivation for everything not in Nixpkgs.

Technically, you can use installer scripts that install to `/usr/local/bin`, but it's not correct. You should be managing everything through Nix to reap its rewards. Fortunately, there's a lot of helper functions that make installing things like Go CLIs very easy, but it still requires running a bunch of commands and writing code.

### Many packages don't exist.

One example is a Linux [League of Legends](https://github.com/Nefelim4ag/League-Of-Legends) installer for Arch. Now I'm forced to only use my Mac for League, which is a shame because I have a 1080Ti. I also tried creating a Nix derivation, but something in `nixpkgs-unstable` was broken.

### Documentation can be really old.

If you're configuring a package, most tutorials tell you to use `packageOverrides`; however, [this is deprecated](https://github.com/NixOS/nixpkgs/issues/43266). There are plenty of other things in Nix that have new and better ways of doing them but they are largely undocumented, probably due to the small community relative to Nix's surface area.

## Next steps

I've had a really great experience with Nix so far, despite its high learning curve and some complications. There's a few things I want to try over the next few months:

- **Creating a shared Nix config for everyone at my company, [Abacus](https://abacusfi.com).** We have a lot of random tools that need to be installed, and it would be great to onboard new engineers with one import.
- **Building some of our services at Abacus in Nix.** Some low hanging fruit could be our CLI tools.
- **Building our Docker images in Nix.** It's supposed to also save a lot of space as Nix only downloads what's necessary.
- **Managing my config with overlays.** I followed what I was taught on Google, but there are definitely better ways of doing what I'm doing.
- **The Hydra build server.** I've been using Jenkins X so there really isn't a reason for me to do this right now, but it's supposedly much faster and easier than any other CI tool out there. After seeing the virtues of Nix over the past few months, I believe it.

I highly recommend Nix/NixOS to anyone adept at \*nix (no pun intended), and I also recommend it for anyone trying to provision a suite of applications on Macs (like an IT person). It's truly the best OS and package manager I've used.

You can see my current configuration [in my dotfiles repo on GitHub](https://github.com/macalinao/dotfiles). Feel free to fork it and [reach out](https://ian.pw/contact.html) if you have any questions!


Nix and NixOS: first impressions


The [Compound protocol](https://compound.finance) is an unprecedented technological advancement in the history of finance: for the first time in history, one can borrow money and earn interest with no humans, governments, or credit involved.

Rates vary frequently, though, and it is important to understand how the protocol determines interest rates.

## The utilization rate

All interest rates in Compound are determined as a function of a metric known as the **utilization rate**. The utilization rate $U_a$ for a money market $a$ is defined[^1] as:

$$U_a = Borrows_a / (Cash_a + Borrows_a - Reserves_a)$$

- $Borrows_a$ refers to the amount of $a$ borrowed.
- $Cash_a$ refers to the amount of $a$ left in the system.
- $Reserves_a$ refers to the amount of $a$ that Compound keeps as profit.

Intuitively speaking, this is the percentage of money borrowed out of the total money supplied.

For example: given that reserves are 0, if Alice supplies \$500 <TokenIcon token="USDC" /> and Bob supplies \$500 USDC, but Charles borrows \$100 USDC, the total borrows is \$100 and the total cash is $500 + 500 - 100 = 900$, so the utilization rate is $100 / (900 + 100) = 10\%$.

A high ratio signifies that a lot of borrowing is taking place, so interest rates go up to get more people to inject cash into the system. A low ratio signifies that demand for borrowing is low, so interest rates go down to encourage more people to borrow cash from the system. This follows economic theory's idea of price (the "price" of money is its interest rate) relative to supply and demand.

## Borrow and supply rates

Borrow and supply rates are calculated using the utilization rate and several arbitrary constants.

The supply rate is calculated as follows:

$$\text{Supply Interest Rate}_a = \text{Borrowing Interest Rate}_a * U_a * (1 - \text{Reserve Factor}_a)$$

where

- $U_a$ is the utilization rate of $a$
- $\text{Reserve Factor}_a$ is the percentage of the spread between the supply and borrow rates that the protocol keeps as profit[^2]
- $\text{Borrowing Interest Rate}_a$ is the interest rate that borrowers should pay for $a$.

### The Compound Standard Interest Rate Model

The borrowing rate's calculation depends on something called an **interest rate model** -- the algorithmic model to determine a money market's borrow and supply rates. In this post, I'll go over the interest rate model used for USDC and most other coins: the **Compound Standard Interest Rate Model**.

This interest rate model takes in two parameters:

- Base rate per year, the minimum borrowing rate
- Multiplier per year, the rate of increase in interest rate with respect to utilization

The graph is linear:

$$\text{Borrow Interest Rate} = \text{Multiplier} * \text{Utilization Rate} + \text{Base Rate}$$

### Example: WBTC rates

<figure>
    <a href="https://compound.finance/markets/WBTC">
        <NextImage src="https://static.ian.pw/images/2020-12-20-wbtc-rate-model.png" width="512" height="467" alt="The WBTC rate model." />
    </a>
    <figcaption>The WBTC rate model.</figcaption>
</figure>

Let's say the protocol has 10,000 <TokenIcon token="WBTC" /> supplied to it and users are borrowing 1,000 WBTC. The utilization rate is thus 10%. What should the rates be?

The cWBTC market uses a model where the base rate is 2% and the multiplier is 30%. To calculate our **borrow interest rate**:

$$\text{Borrow Interest Rate} = 30\% * 10\% + 2\% = 5.0\%$$

Next, let's assume the **reserve factor** of the WBTC market is 20%. We can now calculate the **supply interest rate**:

$$\text{Supply Interest Rate}_a = 5.0\% * 10\% * (1 - 20\%) = 0.4\%$$

For a sanity check, let's make sure that there is a net positive cash flow; that is, the protocol is not losing money:

$$
\begin{aligned}
\text{Supply} * \text{Supply Interest Rate} &\leq \text{Borrows} * \text{Borrow Interest Rate} \\
10,000 * 0.4\% &\leq 1,000 * 5.0\% \\
40 &\leq 50
\end{aligned}
$$

You can view the effect of utilization rate on the supply using this interactive graph on [the Compound WBTC market's summary page](https://compound.finance/markets/WBTC). Note that the numbers are slightly different due to the graph on the website including <TokenIcon token="COMP" /> rewards APY.

[Here is the underlying smart contract of the rate model.](https://etherscan.io/address/0xbae04cbf96391086dc643e842b517734e214d698)

### Example: Effects of large withdraws of WBTC on interest rates

Let's say that with the previous example, a whale decided to withdraw an additional 8,000 WBTC from the protocol. What happens to rates?

The new utilization factor is $(8,000+1,000)/10,000 = 90\%$, up 9x from 10%.

$$\text{Borrow Interest Rate} = 30\% * 90\% + 2\% = 29\%$$

$$\text{Supply Interest Rate}_a = 29\% * 90\% * (1 - 20\%) = 20.88\%$$

An interesting thing to note here is that the supply interest rate increased dramatically (up 5120%), but the borrow rate only increased by 480%. This is because the supply interest rate is proportional to the square of the utilization rate, whereas the borrow interest rate is only linearly proportional to the utilization rate.

### The Jump Rate model

Some markets follow what is known as the "Jump Rate" model. This model has the standard parameters:

- Base rate per year, the minimum borrowing rate
- Multiplier per year, the rate of increase in interest rate with respect to utilization

but it also introduces two new parameters:

- Kink, the point in the model in which the model follows the jump multiplier
- Jump Multiplier per year, the rate of increase in interest rate with respect to utilization after the "kink"

The borrow rate of the jump rate model is defined as follows:

$$
\begin{aligned}
\text{Borrow Interest Rate} &= \text{Multiplier} * min(U_a, \text{Kink}) \\
    &+ \text{Jump Multiplier} * max(0, U_a - \text{Kink}) \\
    &+ \text{Base Rate}
\end{aligned}
$$

### Example: USDC rate model

<figure>
    <a href="https://compound.finance/markets/USDC">
        <NextImage src="https://static.ian.pw/images/2020-12-20-usdc-rate-model.png" width="512" height="467" alt="The WBTC rate model." />
    </a>
    <figcaption>The USDC rate model is a jump rate model.</figcaption>
</figure>

The USDC rate model is a jump rate model with the following parameters:

- Base rate: 0%/yr
- Multiplier: 5%/yr
- Kink: 80%
- Jump multiplier: 109%/yr

The USDC market also has a reserve factor of 7%.

Let's say that the market has the following status:

- \$180M in borrows
- \$20M in cash

What is the borrow rate and supply rate?

Doing the math:

$$U_a = \$180M/(\$180M + \$20M) = 90\%$$

$$\text{Borrow Interest Rate} = 5\% * 80\% + 109\% * (90\% - 80\%) + 0\% = 14.9\%$$

$$\text{Supply Interest Rate}_a = 14.9\% * 90\% * (1 - 7\%) = 12.5\%$$

## Interest rate spikes

In Compound, interest rate is not locked at the price of borrowing: it continuously fluctuates based on changes in the utilization rate.

Another way interest rates could spike is if the Chief Economist decides that the interest rates should go up. This has already happened in the case of MakerDAO, where the stability fee has ranged between 0% and 8%. Fortunately, both Compound and MakerDAO have transparent processes when changing interest rates with beautiful governance dashboards and decisions voted on by governance token holders.

This is in stark contrast to the current quarterly speculation on the Fed/FOMC's decisions. Compound's decision making process on the other hand is transparent and decentralized, protecting the interests of financiers (pun intended).[^3]

### Predicting accrued interest

One can compute the total interest they will pay on a principal balance $P$ for a duration in days $t$ with the following equation:

$$\text{Total Interest} = P(1 + \frac{r}{B_y})^{B_y / 365 * t}$$

where

- $B_y = 2102400$, the number of blocks in the year, and
- $r$ is the expected value of the APR of the interest over the given period.

The number 2,102,400 assumes 15 second blocks.[^4]

### Hedging against rate hikes

Since your interest rate isn't locked in at the time of borrowing, you are be vulnerable to interest rate changes. It is important to understand how the interest rate model of your selected currency works to ensure that there are no surprise massive interest rate changes.

There are several protocols attempting to build interest rate swaps for Compound, e.g. [swaprate.finance](https://swaprate.finance/) and [Opium Protocol](https://opium.network). Interest rate swaps are essentially a way to lock in a borrowing interest rate (usually higher than spot) for a fixed period of time. This is useful if you want to remove more variables from a trading strategy.

[Aave](https://aave.com/), another pool-based lending protocol, has implemented pseudo-fixed rate swaps using an oracle to determine a likely upper bound for what average interest rates could look like over the term of the loan. This has its own problems though and an independent swap market is likely the best solution to this problem.

## Closing thoughts

Compound is a very powerful building block of the Ethereum DeFi ecosystem. Understanding the ways rates change is important in evaluating the potential performance of any leveraged position.

If you enjoyed this post and/or would like to hear more, please leave a comment below!

[^1]: Source: [Compound whitepaper](https://compound.finance)
[^2]: According to [proposal 31](https://compound.finance/governance/proposals/31), the Reserve Factor is a percentage of the borrowers paid interest which can be used by the governance or act as an insurance against borrower default which protects all the suppliers.
[^3]: Technically, this system is not immune to insider trading. A member of governance can trade before voting. Also since COMP can be borrowed on Compound, one can stake a lot of ETH, borrow COMP, place a large vote, then repay all of the COMP.
[^4]: At the time of writing Ethereum has been producing ~13 second blocks, so all annualized rates in Compound should be multiplied by approximately 15/13. Although [this number is hardcoded into the smart contract](https://etherscan.io/address/0xbae04cbf96391086dc643e842b517734e214d698#code), the rate model may be updated in the future via governance.


A deep dive into the factors that calculate Compound's interest rates.

Understanding Compound protocol's interest rates


Algorithmic, undercollateralized stablecoins have popped up everywhere in the past few months. They've been touted to be everything from the cure to inflation to solving world poverty. However, the current pricing mechanism used by these stablecoins is flawed: while expansions are usually able to bring the price sufficiently down, contractionary mechanisms are not able to bring the price back to peg.

While some may say that the underlying notion that an undercollateralized stablecoin could possibly work is the problem, the current problem lies in the Uniswap constant product model.

## A brief review of Uniswap

Uniswap is built on one equation: `xy = k`, where:

- $x$ is the total amount of token 0 staked in the Uniswap pool,
- $y$ is the total amount of token 1 staked in the Uniswap pool, and
- $k$ is a constant supplied at the time of creating the pool.

Due to this, $x$ and $y$ must always have the same ratio to each other in the pool in terms of relative value: $y/k = x$ and $x/k = y$. Thus a pool with a 10:1 ratio of DAI to ETH (in terms of dollar value, e.g. \$10M DAI and \$1M ETH) will always have a ratio of 10:1 DAI to ETH, no matter how expensive the price of ETH becomes.[^1]

However, the price of $x$ relative to $y$ is determined by $x/y$: if the pool is initialized with \$10M of DAI and \$1M of ETH, if the price of ETH is 1000 DAI, then that means there are 1000 DAI in the pool for each ETH. This price is able to change while keeping the dollar value ratio of the two tokens the same.

While this model is very simple (and thus cheap in terms of transaction fees), it has several problems which we will uncover later in this article.

### Example: buying tokens

Let's say the pool has the price of 1 ETH = \$1000 DAI, and the pool has reserves of \$2M USD. Due to the constant product, this means there is \$1M of ETH in the pool and \$1M of DAI, so there are 1000 ETH and 1M DAI in the pool.

Recall that $xy = k$ is the Uniswap invariant. Let the initial ETH reserves be $x_0$ and the initial DAI reserves be $y_0$, so $x_0 * y_0 = k$.

Now let's say I want to buy 500 ETH. Thus the new ETH reserves $x_1 = 500 + 1000 = 1500$. Since $k$ is constant, we can compute that the new DAI reserves should be $y_1 = k/x_1 = x_0 * y_0 / x_1$ = 666,666.66 DAI\$.

Thus we will sell 500 ETH to buy 333,333.33 DAI.

Note that the new reserves are now 500 ETH and 666,666 DAI, so the new price of ETH is $666666/500 = 1333$ DAI. However, there are now only \$666,666 ETH and \$666,666 DAI in the pool.

### Acceleration of price drift

The above solution seems innocuous at first, but it causes hidden forces in the market: with the same amount of liquidity and trade volume, as the price drifts from the initial price, the price begins to drift faster.

Recall the constant product formula:

$$
xy = k
$$

Let $x$ be the number of TOKX in the pool and $y$ be the number of TOKY in the pool. Given that the current price of TOKX relative to TOKY $p = x/y$, the change in price with respect to the change in reserves $x$ of TOKX is given by the derivative:

$$
\frac{dp}{dx} = \frac{x}{y} \frac{d}{dx} = \frac{x}{k/x} \frac{d}{dx} = \frac{x^2}{k} \frac{d}{dx} = \frac{2x}{k}
$$

This means that for every $x$ TOKX tokens sold into the pool for TOKY, the change in the price of TOKY increases at a rate of $2x/k$. Since $x$ will continue to grow as more TOKX are sold for TOKY, the slippage in price for the same number of TOKY purchased is magnified the lower the price goes.

### The black hole effect

At some point, the supply of $x$ relative to $y$ is so great such that overcoming the massive amount of liquidity is economically not viable. Even if people unstake their tokens from the pool at low prices, the imbalance may be so great that the pool will never revert to \$1.

Consider an extreme where 1 DAI is equivalent to 1 BAC: i.e. BAC is \$0.01. Let's say the total amount of liquidity is merely \$50k, a fraction of the \$60M at the time of writing.

Since the pool has a 100:1 token ratio but 1:1 constant ratio, the reserves can be computed as follows: let $x$ be the total BAC reserves and $y$ be the total DAI reserves. So $x = 100y$ due to the liquidity ratio and $y = 50000/2 = 25000$ due to k, where the pool has half of its dollar value in $y$ and half of its dollar value in $x$. Solving this system of equations, we get that there are **2.5M BAC** in the pool and **25,000 DAI** in the pool.

Now, to get the price of BAC to double in this pool, the new ratio of BAC:DAI needs to be 100:2. So $x' = 50y'$. We can solve the following system of equations to compute the change in reserves needed:

$$
\begin{aligned}
xy &= 2500000 * 25000 \\
x/100 &= y/2
\end{aligned}
$$

[This gives us](https://www.wolframalpha.com/input/?i=xy+%3D+2500000*25000%3B+x%2F100%3Dy%2F2) the new reserves of **1.7M BAC** and **35355 DAI**. The new liquidity is 2x DAI $\simeq 70k$.

So \$10k of DAI had to be spent in order to rais the price 1 cent. _Not terrible,_ you might be thinking-- unfortunately; the effects amplify as the ratio equalizes.

To get to 10 cents per BAC, the pool requires 79K DAI reserves-- 160K of liquidity.

To get to 1 dollar per BAC, the pool requires a whopping 250K DAI of reserves-- 500K of liquidity.

<figure>
  <NextImage src="https://static.ian.pw/images/2021-01-17-bac-dai-repeg.png" width="640" height="480" alt="BAC/DAI reweight" />
  <figcaption>
    Amount of DAI required to bring BAC from \$0.01 to \$1 with \$50K liquidity
  </figcaption>
</figure>

So you see, despite the pool only having \$50K of liquidity in it, the pool requires $\$250K - \$25K = \$225K$ in order to recover back to the peg, or **9x the current DAI in the pool**. It is unlikely for this recovery to happen to a dead project and the problem only worsens as the price continues to cause more selling of BAC, hence the term: **black hole effect**.

## Solutions

### Using a flatter curve

A flatter curve would prevent the price from slipping until faith in the token has completely dropped off. [Curve.fi](https://curve.fi) is one such example of a curve that is optimized for stablecoins:

<figure>
    <NextImage src="https://static.ian.pw/images/2021-01-17-stableswap.png" width="625" height="389" alt="Stableswap" />
    <figcaption>
      <a href="https://en.cryptonomist.ch/2020/09/26/guide-curve-finance/">A Guide to Curve Finance — The Cryptonomist</a>
    </figcaption>
</figure>

[Basis.cash](https://medium.com/basis-cash/stableswap-pools-and-algorithmic-stablecoins-b21ab5cb555f) is already planning on implementing this.

However, this is not a panacea -- since there is more liquidity in the middle of the curve, if the stablecoin is problematic for long enough, the price will fall rapidly at the extremes of the curve. This curve simply deplays the inevitable black hole.

### Reweighting the pair back to 1:1

By modifying the $k$ in the Uniswap constant product, one can avoid this black hole effect.

One solution is described by the [Fei Protocol](https://fei.money/) as follows[^2]:

> Peg Reweights — In the event of extended periods below the peg, the Fei Protocol can reweight the Uniswap price back to the peg. It achieves this by executing the following atomic trade: 1. Withdraw all protocol owned liquidity, 2. Buy FEI with the withdrawn ETH to bring price up to peg. 3. Resupply remaining PCV as liquidity. 4. Burn the excess FEI.

Since Basis.cash and its forks rely on a "BAC/DAI LP" pool, the pool could theoretically be upgraded in a later version of the protocol to implement this algorithm.[^3]

However, this may not always work since not all liquidity is staked into the BAC/DAI pool-- one may run into the effects mentioned above where there isn't enough money in the pool. A naive solution to this could be to have the protocol run the exchange used to determine the peg; however BAC that exists outside of the pool could in theory be used to drive the prices back down despite a peg reweighting occurring. This is okay though since the intent of BAC is to be worth \$1.

There is another downside to using such a reweighting system in BAC, where stakers in the BAC/DAI pool may be hesitant to stake as their BAC continues to be chipped off every time there is a dip in the market. This may be okay though since it is desirable to not have liquidity in the pools when prices are low again due to the black hole effect.

## Conclusion

One may think that undercollateralized seigniorage share tokens have been proven to be unreliable due to recent events. However, the current depression of three-token stablecoins is only due to the constant product model and the contraction mechanism used. The algorithmic stablecoin space is developing rapidly and I am really looking forward to what projects propose as solutions to this problem.

[^1]: The pool ratio can change over time due to pool fees collected: each transaction takes a 0.3% cut of the total input tokens transacted. The fees are kept in the pool's reserves and LP token holders accrue these fees in their pool tokens.
[^2]: Source: https://medium.com/fei-protocol/introducing-fei-protocol-2db79bd7a82b
[^3]: Most algorithmic stablecoins have some version of this pool.


Uniswap is a suboptimal place for stablecoins that rely on mean reversion to $1 to trade, as Uniswap encourages price crashes to stay low.

The Black Hole Effect of Uniswap on Algorithmic Stablecoins


Solana is fast. Why can't the EVM get to the same speed?

## Non-native instructions

### Native vs virtual instruction sets

When a computer runs programs, it does not look at code and know what to do. Instead, the code must be compiled into a series of instructions which are processed by the central processing unit, or CPU. If you've recently migrated to an [M1 Mac](https://en.wikipedia.org/wiki/Apple_M1), you'll quickly notice that there are several programs that do not run on your new computer. This is because the M1 Mac uses the [ARM](https://en.wikipedia.org/wiki/ARM_architecture) instruction set, whereas the Intel-based Macs use [x86](https://en.wikipedia.org/wiki/X86).

Instruction sets may be specialized to a specific purpose. For example, [here's a graphics card](http://developer.amd.com/wordpress/media/2012/10/R700-Family_Instruction_Set_Architecture.pdf)'s instruction set. It contains special operations for loading textures and vertices, since the hardware is specialized for 3D rendering.

However, there are instruction sets that do not correspond to instructions that the CPU hardware can directly interpret. These are generally referred to as _virtual machines_, because unlike physical machines, their instruction set architecture does not correspond directly to hardware.

Despite the indirection from machine code to virtual code, virtual machines are not always slow. The Java Runtime Environment, which processes Java Virtual Machine (or JVM) instructions, is able to operate at near native speeds due to a technique known as _just-in-time compilation_, or JIT. What the JRE does is it takes the opcodes of a JVM program and converts them to native machine instructions "just in time" for execution. When the program or function is executed, there is no longer overhead in processing individual instructions.

Runtimes not based around instruction set architectures do this as well, such as [the V8 JavaScript runtime](https://v8.dev/docs/turbofan) found in Google Chrome and Node.js.

### Shortcomings of the EVM

The Ethereum virtual machine processes [instructions](https://ethereum.github.io/yellowpaper/paper.pdf) for each Ethereum transaction. These instructions are not executed natively on a computer; instead, they are [interpreted](https://github.com/ethereum/go-ethereum/blob/94451c2788295901c302c9bf5fa2f7b021c924e2/core/vm/interpreter.go) by block producers. On Ethereum, this is miners, and for other EVM chains, these are validators.

However, Ethereum cannot easily benefit from JIT compilation.

Each opcode is associated with a [cost](https://github.com/ethereum/go-ethereum/blob/97ce6dfa6d425ba4b76278627a91c7285d6d00b2/params/protocol_params.go#L21), meaning that

On the EVM, transactions are metered by a virtual machine.

Possible optimizations:

- JIT compilation
- A new instruction set. This however means that we are no longer using the EVM.

## Serial processing

Blockchains are linear finite state machines; that is, they have a state, and for each transaction they process, the state is modified.

Solana is able to process transactions in parallel due to [Sealevel](https://medium.com/solana-labs/sealevel-parallel-processing-thousands-of-smart-contracts-d814b378192): a runtime that knows what storage will be mutated and which ones will merely be read.

The EVM, on the other hand, is not as elegant: it d

## Cross-shard communication

There is a hack to have infinite TPS: to shard.

Imagine if

In this post, I explore the different design decisions Solana has made to be the fasted blockchain in the world.


In this post, I explore the different design decisions Solana has made to be the fasted blockchain in the world.

Why EVM chains can never be as fast as Solana


Setting DNS records improperly can cause unauthorized parties to masquerade as your
email domain. In this post, I show all of the DNS records that must be added.

## Overview

- SPF
  - TXT `@`: `v=spf1 include:_spf.google.com -all`
  - TXT `*`: `v=spf1 -all`
- DMARC
  - TXT `@`: `v=DMARC1; p=quarantine; sp=reject; rua=mailto:youremail@yourdomain.com`
- DKIM
  - For Google Workspace, follow [this guide](https://support.google.com/a/answer/180504?product_name=UnuFlow&visit_id=637805508785814602-3757498782&rd=1&src=supportwidget0).

## Breakdown

### SPF HARDFAIL for unauthenticated email being sent from your root domain

- TXT `@`: `v=spf1 include:_spf.google.com -all`

This record ensures that all email being sent from your root domain comes from Google Workspace.

### SPF HARDFAIL for all email being sent from your subdomains

- TXT `*`: `v=spf1 -all`

If a subdomain does not exist, this rule ensures that all email being sent from the subdomain is flagged and/or rejected.

### DMARC

- TXT `@`: `v=DMARC1; p=quarantine; sp=reject; rua=mailto:youremail@yourdomain.com`

This rule marks all unauthenticated email from your root domain as spam (`quarantine`)
and instructs recipients of unauthenticated email to reject all email from your subdomains (`reject`).

### DKIM

- For Google Workspace, follow [this guide](https://support.google.com/a/answer/180504?product_name=UnuFlow&visit_id=637805508785814602-3757498782&rd=1&src=supportwidget0).

DKIM ensures that Google is cryptographically signing the contents of your emails. This ensures
that your email contents aren't being tampered with.


A list of all DNS records to set on new Google Workspace email subdomains, and why.

The Google Workspace Email Security Checklist: SPF, DKIM, and DMARC


The most intuitive reason for taxes to exist is to help pay for common services
and public goods that everyone uses. This is why many people love to tax the
rich: it seems fair to ask the people who can afford it to pay more for the
things that everyone uses.

However, not all taxes are created equal. A tax that is only designed to
maximize revenue can have detrimental effects on the economy. For example: let's
say that sales tax was increased to 50%. While on paper it may seem that revenue
may increase by X%, in reality the amount of sales in the economy may decrease
making the total amount of revenue lower after the tax is instated.

Now, let's say that our goal as a government is to reduce the amount of carbon
emissions from cars on the road. We could apply a _selective sales tax_ to gas,
which would increase the price of gas and reduce the amount of gas that people
buy. The direct effect of this would be that people would use less gas.

- For city dwellers, this could result in people using public transportation
  more often, since both personal and ride sharing would be more expensive.
- For rural/suburban commuters, this could result in either moving to cities
  (which reduces the amount of cars on the road) or using electric cars.
- For businesses which rely on gas to operate, e.g. tractors or delivery trucks,
  this could result in them switching to electric vehicles. This can hurt
  businesses in the short term, but the government can aid the transition via
  [subsidies](https://www.rd.usda.gov/programs-services/energy-programs/rural-energy-america-program-renewable-energy-systems-energy-efficiency-improvement-guaranteed-loans).

The point is that taxes can be used to influence behavior in addition to raising
revenue. This is why it's important to understand the effects of a tax before
implementing and/or voting for it.

## How this applies to protocols

Protocols are a form of governance, and as such they can use taxes to influence
user behavior. This is already implemented in many layer one blockchains:

- A tax on transactions to discourage spam. This already exists in the form of
  transaction fees.
- A tax on storage, to discourage state bloat. This exists in Solana in the form
  of rent, and on Ethereum in the form of storage/storage refunds.
- A tax on CPU resources, to encourage developers to optimize their code.

These also exist in the DeFi space:

- Automated market makers give fees to liquidity providers from traders crossing
  a protocol-defined spread between the bid and the ask, which is usually
  referred to as the "trade fee". For example, Uniswap V2 has a 0.3% spread on
  every pool.
  - 1/6 of this may be taken by the Uniswap protocol.
- Borrowers of borrow-lend markets pay interest to depositors, usually based on
  the [utilization
  rate](./2020-12-20-understanding-compound-protocols-interest-rates) of the
  lending market.
  - A percentage of the spread goes to the DAO.

Understanding the effects of taxes is important for protocol designers, since
they can be used to influence user behavior. For example, if a protocol wants to
encourage users to use a specific token, it can give a tax rebate to users who
use that token-- e.g., a stablecoin might buy a lot of governance power to make
it cheaper to use their stablecoin.

## Other Thoughts/Conclusion

I was inspired to write this post by [a short YouTube video about Singapore's
public transit system](https://youtu.be/vyfJgJBB3Vk). They impose a very high
tax on cars: a Honda Civic costs over $100k USD equivalent. This has resulted in
a lot of people using public transit, which has reduced traffic and pollution
and has increased the utilization/ROI of investing in the public transportation
system.

Taxes are an age-old tool for governments to influence behavior. We may be able
to rely on tested economics to help us design better systems for capturing
value.


Taxes serve two purposes: to raise revenue and to influence behavior.

Writing