Author: Nico Preti
Source: https://btctranscripts.com/bitcoinplusplus/layer-2/exploring-p2p-hashrate-markets-v2
The original text is a transcript of the author's speech at the Bitcoin++ Layer 2 conference, transcribed by sahil-tgs via review.btctranscripts.com.
The video of the speech is available at: https://www.youtube.com/watch?v=IvfmfcAX9wU
introduction
Hello everyone, my name is Nico, and I'm the co-founder of Rigly, a peer-to-peer hashrate marketplace. This presentation will be structured as follows: First, we'll discuss what "hashrate" is and the protocols that coordinate the distribution of mining tasks across the network; then, we'll briefly introduce the history of the hashrate marketplace; finally, we'll discuss how we can leverage Layer 2 to build a hashrate marketplace, which is why we're here at this conference.
Hash function and hash rate
So, what is "hash rate"?
As you can see, this is the output of a hash function. A hash function is simply a function that takes a preimage as input and outputs a corresponding image (hash value).
When you view a block in Bitcoin's block explorer, you'll see a hash value at the top, its hexadecimal form starting with a series of zeros. This design originates from Adam Back's "Hashcash" project. Bitcoin's hash rate is also based on this hashcash algorithm.
To understand how it works, you can resort to simple math: we have 0 raised to the power of k ( 0^k ). The output (or image) on the right side of the screen is something we want to start with a certain number of zeros. Therefore, 0^k is essentially the difficulty, measured in how many zeros are at the beginning.
If we represent this algebraically, we can write it as a function representing the hash value of X ( H(x) ); taking x as input, we want the output to start with a string of zeros. A better way to represent this function is to have a hash function ( H(s, c) ) whose "service string" is s , and c is a counter. This counter increments. The service string is what gives this hash function practical meaning; otherwise, you're just hashing a bunch of random stuff. You can prove that this number will produce this output after entering the hash function, but it has no practical significance. So the service string is the utility; in this case, it's the Bitcoin block. This service string is a block header that we're preparing to confirm throughout the network.
The Hashcash project uses SHA-1, with an output of 160 bits. I recall setting k to 20, meaning you'd need a million attempts to find a correct preimage. Bitcoin uses 256 bits, resulting in a much larger output and requiring far more attempts. A major difference between Hashcash and Bitcoin is that Hashcash uses 2 2^k to adjust difficulty, meaning the difficulty either doubles or halves within a difficulty cycle. In Bitcoin, we need more precise control over the difficulty for each cycle (2016 blocks). Therefore, we convert k to a floating-point number for more accurate difficulty adjustments. Every 2016 blocks, we adjust the difficulty for the next cycle, ultimately changing this floating-point number.
So what are we ultimately trying to achieve? The complete mathematical expression is this: First, you take s as the service string and x as your random starting point (for most miners, this effectively means you can influence the Merkle coinbase transaction, which we'll discuss later, but that can also be considered a random starting point because each miner is constantly changing the coinbase transaction). Then, instead of making the hash value equal to 0, you make it less than a certain difficulty target ( H(s,x,c) < 2^(nk) ). The target is equal to 2^(nk) ... This is some of the mathematical description behind how it works.
Mining process overview
In Bitcoin, we use the block header as the thing we're hashing (the service string). As a miner, you just need to give me a block header, and I can start mining. So what's in the block header? This is actually a very important question.
Typically, people learn about Bitcoin when they first encounter it, but then gradually forget about it. It gets buried deep in your mind; you might not even remember what was in the blocks. But it's important:
First, we have
Version, which is the version number of the software you are running;Then there is the hash value of the previous block (
hashPrevBlock). It is because this information is preserved that the blocks are connected one after another to form a blockchain.Next is
hashMerkleRoot, which, as I just mentioned, hashes all transactions within a block into a Merkle root in the form of a tree.There's also the timestamp (
Time) of the current block, which changes every few seconds; so there are a few bits of randomness here as well.There are also
Bits, which is the difficulty level expressed in hexadecimal;Finally, there's
Nonce, the counter. This nonce field is actually very, very small, only 32 bits. With current mining rigs, you can quickly traverse this space. Therefore, in order to successfully mine a block, we have to resort to the so-calledextranoncefield, which is placed in the coinbase transaction; changing the coinbase transaction will affect Merklegen.
As you may not know, the metric we use today to measure mining machine performance is "TH (terahashes)," which stands for 10 to the power of 12 (thousands of hash operations). If we're talking about a group of mining machines, such as a mining farm, we use "PH (petahashes)" (10 to the power of 15). Finally, if we're discussing the total hash rate of the entire network, we usually use "EH (exahashes)" (10 to the power of 18). Currently, I believe the total hash rate of the entire network has reached 350 EH. Currently, a standard mining machine operates at 100 TH per second, while a medium-sized mining farm operates at 10 PH per second.
(Translator's note: The author only truly discusses metrics for measuring hash rate (hash speed) here, such as 100 TH/s for a typical mining rig, meaning it performs 100 T hash operations per second.)
Mining Protocol
Next is the mining protocol. Earlier, we explained how hash rate, or mining, works, but how do we organize so many machines and mining farms in a network? That's where there's more to say.
Currently, we use the "Stratum V1" protocol. I'm sure you've all heard of "Stratum V2," which is a very hot topic right now, but that's a bit beyond the scope of this presentation. I'll add some information here and there, telling you the main differences between the two, but for now, just remember that we're discussing Stratum V1.
Miners and mining pools typically only use TCP connections. Therefore, in a mining pool, we open a TCP port for miners to connect, and the communication format is JSON RPC (instructions expressed in JSON format). This is very straightforward. The message types can be seen on the screen: mining.subscribe , mining.authorize , mining.set_target , mining.notify , and mining.submit . This is almost all there is to the Stratum protocol. It's very simple. I've also added some background information here.
The first thing both parties need to do is "subscribe." It's like the miner saying, "I'm here, I'm ready to mine. Please register me in your mining pool and give me a job ID." Simultaneously, it starts sending extranonce information. This is a crucial part of the Stratum protocol, truly an innovation when it first appeared in 2013 and 2014. (Incidentally, one of the authors of the Stratum protocol was also in the audience). With extranonce information, we can localize most of the work that miners need to do with their mining rigs, eliminating the need for back-and-forth communication with the mining pool. This saves a lot of overhead; if you had to communicate with the mining pool and get information for every iteration of the PoW operation, it would be enormous. So, the mining pool sends back extranonce information and says, "Start here, and then iterate and compute on your own." This is the back-and-forth communication between the two parties when the miner starts subscribing.
Another thing is "authorization." You register a "worker" with the mining pool, and this worker has a mining pool account. But you may have many workers, each corresponding to multiple machines. So, when you authorize a worker, you are essentially saying, "My worker is ready to accept tasks." This happens between the client and the server—the client being the actual mining machine—and then the mining pool sends back a shared difficulty target.
In a mining pool, we organize mining based on "shares," which is fundamentally about setting the difficulty level to match the network's requirement for mining new blocks. Essentially, a different difficulty is set for each machine so the pool receives a share from each machine every 2-5 seconds. The mining pool is basically telling the miners, "What's your hash rate? Okay, you have about this much, so I'll set the difficulty for you, maybe 1 or 2, so I can get some information from you every 2-5 seconds." It adjusts, right? Is this useful? Of course it is.
Once the mining rig knows the difficulty level it needs to mine, it will need to obtain the actual information used for mining. This information is what we explained earlier—the information that needs to be placed in the block header.
The main task a miner needs to handle independently is mining the extranonce field. The main information it obtains, as I mentioned in the "Subscription" section, is extranonce1 and extranonce2_size . extranonce1 is a fixed variable, while extranonce2_size is the length of the extranonce field, which can be iterated over by a specific miner. Then, the miner obtains the beginning of the coinbase transaction, which is the address where the mining pool receives mining rewards; and the end of the coinbase transaction, which contains some additional information (or padding). The miner iterates over the Extranonce2 field until it finds a target hash value, or an output hash value, that is lower than the shared difficulty target. Specifically, this is done by concatenating these fields ( coinbase1 + extranonce1 + extranonce2 + coinbase2 ) and then hashing them (along with other transactions) layer by layer into a Merkle root. In each block, its coinbase transaction is the leftmost leaf of this Merkle tree. To calculate the Merkle Merkle root —which is the main difference between Stratum V1 and Stratum V2—in V1, you don't get the complete tree, you only get branches, and with these branches, you can calculate the root. You already have a coinbase transaction, so the mining pool will give you that transaction, allowing you to calculate its hash; then you get the hash of the adjacent node, allowing you to calculate the hash of the next higher level; and so on, until finally you calculate Merkle root , which is obviously the most important part of your service string.
In this way, you calculate a Merkle root , and you continuously iterate over it (by modifying coinbase transactions), adding the remaining elements to form a block header. When you find a hash value that meets the share requirement, your software sends it to the mining pool. Then you wait to receive payment based on these shares. The cycle repeats.
Understanding Mining Pools
Next is the mining pool. I won't go into the whole diagram, but it does summarize all aspects of mining pools. This diagram will be posted on Revlit, and you are welcome to use it if you wish.
[A listener questioned the use of the term "KAFKA" in the image's processing center. The author explained that it's simply a software term, and different mining pools may use different ones. (Not translated)]
Mining pool settlement model
Mining pools offer various settlement models. The two main models used are: PPLNS , which settles accounts with miners based on a recent, fixed number of shares; FPPS , where every share is fully paid out.
The main difference is that in PPLNS , you (the miner) are paid out each time the pool finds a block. For example, Braiins (formerly known as "Slush Pool") uses this model. Whenever a pool finds a block, it measures the hash rate contributed by miners to finding that block based on the last few shares distributed to them before the block was found, and then pays them accordingly. Part of the reason for considering only the most recent batch of shares is to discourage miners from switching pools—only miners who were mining with the pool during the period between finding the block receive rewards.
As for FPPS , you are guaranteed that, basically, as long as the mining pool doesn't go bankrupt, your shares will be paid out. So, no matter how many shares you produce in a day, you can receive payments from the liquidity pool based on the shares you produce (or your contribution to the mining pool). This is unrelated to whether the mining pool finds a block.
Risk of clearing accounts
You're probably already wondering: what happens if a mining pool doesn't find any blocks for a day? What if it doesn't for two, three, or even longer? How does clearing out happen in an FPPS mining pool? Well, that's the risk of running an FPPS mining pool—you might have to clear out the miners first, without immediately receiving enough Bitcoin to cover your losses. In contrast, with PPLNS pools, you essentially shift the risk to the miners. It doesn't matter if you don't mine blocks for several days in a row; the pool only pays the miners when a block is mined. The pool doesn't actually bear the risk; the miners bear the risk of not receiving payment for several days.
Cases of mining pool risks
[Audience]: Are there any mining pools that have actually gone bankrupt?
Yes. However, they were later acquired by a larger mining pool. Or rather, a financial institution stepped in, essentially took over, and paid off the bills. I'm referring to "Poolin," which mixed some of its mining pool funds with some of its own DeFi investments. So when Three Arrows Capital collapsed, much of their mining pool funds were tied to funds invested in DeFi protocols. Overnight, all of Poolin's miners' due earnings became Bitcoin IOUs—that's from the mining pool's perspective. From the user's perspective, it's like saying, "I want to withdraw my Bitcoin," and the other party says, "No, no, no, now you can only get Bitcoin IOUs." This is a classic case. I also worked in a mining pool, and I'm not sharing this case with any malice.
Mining Market
Okay, now for the main course of this presentation: how do these markets surrounding cryptocurrency mining operate? With the emergence of mining pools, they essentially become a market. You possess hash rate, and you sell it to the mining pool; the pool, in turn, uses the aggregated hash rate to mine, and then distributes the profits to miners according to the proportion of hash rate – it's like a market. But can we take it a step further?
Advancements in mining hardware
This idea is original, from the Bitcoin white paper: PoW is essentially a single CPU casting a vote. Satoshi Nakamoto initially thought so. But the days of mining with CPUs are long gone. We quickly moved from CPUs to GPUs (graphics computing chips), then almost simultaneously FPGA (field-programmable gate array) mining appeared, and finally, ASIC (application-specific integrated circuits) appeared. I think ASIC mining appeared around 2012 or 2013.
Challenges of the Modern Mining Industry
From that point on, you needed to buy very specialized, expensive hardware to mine. You couldn't mine with home hardware anymore. A few years later, even if you bought such specialized hardware, you still needed to find very cheap electricity to make money. Even if you bought an ASIC and left it at home, you wouldn't make much money. And it was practically unfeasible, or rather, difficult to sustain in the long run. Now, even if you get very cheap electricity, you still need a certain scale and a lot of infrastructure to reduce operating costs. So now, mining is really difficult. Even for large miners, it's extremely challenging.
Clearly, the mining industry has always had a very strong tendency towards centralization. We see this news on Twitter, and I've heard a lot of criticism within the Bitcoin community because mining pools like Foundry and Bitmain can easily control 70-80% of the entire network's hashrate. Just two companies, right?
Of course, you can switch mining pools, and it doesn't necessarily pose an existential threat to Bitcoin, but it's always better if we can try to diversify this risk, right? And a large part of the risk I'm talking about is actually capital risk. Today, to run a mining business and achieve a certain scale, you need to invest a tremendous amount of cash upfront. You can look at the balance sheet of any publicly available large mining company, and you'll find that they are all very large, major buyers of equipment and capital goods related to these projects. This inevitably results in a lot of debt, which is very vulnerable in the current market because nobody wants to lend money to Bitcoin businesses, considering it too risky, so ultimately, loan interest rates will be very high. Perhaps you can find some new loan options, such as using ASICs as collateral. Khnighted offered such loans, allowing them to take all the mining rigs after the mining company went bankrupt; however, they had nowhere to resell these rigs because the companies had all gone bankrupt. This happened a lot last year, and I'm sure you've all heard about it.
Essentially, miners have no good way to hedge against any kind of risk. No miner can try and lock in a liquidity market for their profits. Given the nature of the mining business and the price of electricity, these profits accumulate gradually over the next six months, a year, two years, or even three years. If they predicted that the price of Bitcoin would rise and the difficulty might increase, thus causing the price of hashrate to fall, they could have entered into a financial contract to lock in a price on a decreasing curve, paying (let's say) a 2% or 3% option premium each month, and then exercising their contractual rights when the difficulty adjustment exceeds 1%. That would lock in their profits. Such contracts exist for all other major commodities in the world, but not in the Bitcoin mining industry. Moreover, as a miner, especially with this kind of capital structure, it's a very heavy burden, resulting in a high debt ratio, making it a very difficult business to succeed in.
We've been thinking about this a lot, and one of our ideas is: maybe we're looking for liquidity in the wrong places, diversifying risk in the wrong places. Why are we going to traditional financial institutions? They just want to extract as much money as possible from lending. Why don't we go directly to people with restricted funds—I mean people with Bitcoin—who are clearly very concerned about network security and interested in the decentralization of network security? Why don't we use these restricted funds to diversify this type of security risk?
You might think that "hosted mining" and "cloud mining" are similar attempts; not at all, but we can let them slide, in the most forgiving sense. To explain to those unfamiliar, hosting mining means you buy a mining rig, and someone else places it in their mining farm to run it, paying you or allocating computing power to a mining pool you specify. Cloud mining, on the other hand, involves buying a hashrate contract with a time limit. As far as I know, these hashrates are usually not actually delivered; however, I've heard of exceptions. Essentially, the difference is that in cloud mining, you don't buy a machine, but rather a contract that supposedly allows you to mine. But usually, this is quite abstract for you.
So the main problem is that all of this fundamentally contradicts the principles we uphold in Bitcoin, right? You get a completely asymmetrical trust model. You have to completely trust this person: first, you have to shell out a large sum of money to buy mining rigs from someone claiming to be a service provider; you have to trust that this person won't inflate the price of the rigs because they're essentially "agents"; you have to trust that the mine operators will keep it running; and finally, you have to trust that even if prices drop or something unexpected happens, your rigs will still work for you. You have no choice, right? Your rigs might be located in some country you've never even heard of; only God knows where you're mining. So, usually, it doesn't take long for this whole thing to fall apart. It's either a scam or a complete fraud.
Cloud mining is almost always a losing proposition. I've never heard of a cloud mining contract that's profitable. On the contrary, you usually lose everything. As for mining hosting, like I said, it's an asymmetric trust model where everyone can rip you off, and not only that, you have to pay a premium. They can suck your money at every level.
Therefore, the way we can try to avoid these situations is to establish a direct connection between miners and buyers. We should try to create a marketplace where both parties can negotiate directly.
Before that, I should add something else: we already have some hashrate futures markets. FTX launched one last year. Luxor launched a more compliant market, a non-deliverable futures market—you could call it a swap, which is much more traditional. The problem with them is that we haven't seen much liquidity. And again, I think they've gone back a bit. Like I said, why borrow money from these traditional financial institutions? They usually don't understand Bitcoin at all, and they have no incentive to understand and try to take the risks of Bitcoin. They just want to exploit the fact that you need money, and no one else is willing to lend you money, so I can lend you money at a very high interest rate.
Of course, it also created a situation where nobody really tries to buy these hashrate option contracts on traditional financial markets. Another consideration is: Okay, what if you actually demand delivery? There's no way to actually deliver it to you. And what about the contract duration? Most of the time, if you look at markets like NiceHash and Mining Regret tools, you'll find that the contract durations are very short. So, if you just want to bet on whether it will go up or down, that's possible, but the duration is too short to allow you to buy something and leave it there to run on its own. As for non-deliverable futures or swap contracts, you know, you're just buying a traditional financial instrument, not something secured by code.
The speaker, after being reminded, said he would speed up his speech.
Point-to-point hash rate
Therefore, our model involves establishing an auction mechanism to discover prices. We use a multi-signature script to pay the seller of the hash rate, and the payment is only sent after the hash rate has been delivered; we deliver the hash rate through a proxy. Thus, we can see how much hash rate was ultimately delivered.
We've drawn some diagrams to explain the whole model. So, we help establish these small contracts in the middle; the buyers are highlighted in orange at the bottom, and their funds are held in a multi-signature wallet on the chain; delivery is handled through mining pool accounts. So, hashrate from miners is put into a mining pool, and eventually, when that pool mines a block, the payment goes into a mining pool account. Then, the contract comes into play. Finally, you can actually transfer funds from your mining pool account to another location. This way, we can truly guarantee that delivery will occur, and we have a proxy that can read how much hashrate has been delivered to the mining pool account.
We already have a sample running, and you can register using this QR code; I'll provide a link later. This is basically the workflow of our model. However, we have an event-driven system where, as soon as you send us payment, we allocate a portion of the hashrate currently being mined elsewhere to your mining pool account (you can register your mining pool account in advance).
Lightning Network
The last page is about the Lightning Network. We hope to be able to send mining pool payments via the Lightning Network. The problem is—I don't know if you've heard of Jade (hardware signature generator), which can also perform hash calculations. The idea here is that (the current price of hash rate is) 10 satoshis per TH/s. Can we send such small payment codes via the Lightning Network? Can we turn every joule of heat, every wasted electricity, into hash calculations? Currently, this is not possible, but with the Lightning Network, perhaps it is.
The problem, as Lisa points out, is that if mining pools pay miners via the Lightning Network, there will be very serious liquidity issues because funds are constantly flowing out in one direction. You'll quickly run out of liquidity in these channels, which isn't the ideal working model for Lightning channels. We need payments to flow in both directions. Now, using our model, liquidity does indeed move bidirectionally. We push the payment for hashrate in one direction all at once, and then you receive hashrate deliveries and pool payments daily, with funds flowing in the opposite direction. This balances the channels.
So the question becomes, where are these mining pools located, and how do we incorporate them into new Coinbase transactions? This is a problem that "channel splicing" technology promises to solve; we can splice a new UTXO into an existing channel balance. The final closed state will only differ slightly from the state predicted in the contract terms. So, let's assume you're a buyer who wants to bet x% of your margin on winning; the channel's closed state will reflect whether you won or your opponent (the miner) won.
Finally, another potentially easier way to do this is to use a Fedipool instead of the Lightning Network. I won't go into detail here. Meanwhile, we also hope to join the Nostr protocol. We believe that using static IDs to issue payments is very promising. You can try out a sample; here's the QR code. Hope to see you there.
(over)
