In Part 1 of this series, we examined DoubleZero’s Sentinel and Passport programs and how the DoubleZero Foundation, through those programs, maintains centralized control over the system in ways that may be inconsistent with their SEC no-action relief.
In Part 2, we examine the rewards distribution process and demonstrate how it also operates with a monopoly central service provider. This analysis suggests that DoubleZero’s representations to the SEC may not accurately reflect the actual system architecture. This is not about a single point of central control. Part 2 addresses a second subsystem that appears to revolve around centralized authority.
Several possible explanations exist for this discrepancy:
- The DoubleZero team may not have fully communicated the system’s architecture to their legal counsel
- The SEC filing may have omitted material facts about the product’s operation
- The team may be citing no-action relief in a manner that does not accurately reflect the relief’s applicability
- There may have been miscommunication in the SEC review process
We present these findings for consideration and await further developments.
The 2Z Rewards Distribution Scheme
DoubleZero’s token rewards distribution scheme is complex. However, a simplified overview will suffice for this analysis. As in Part 1, we begin with their architecture diagram.
Several components are relevant here:
- Telemetry: tracks contributor activity
- Reward calculations and finalization: converts usage data into reward amounts
- Contributor rewards distribution: distributes the rewards
These components are managed through a combination of offchain “DZ Resource Contributors” (the top-left blue section) and smart contracts on Solana (the bottom-left white section). This setup is similar to Part 1, and as we will demonstrate, these system components raise questions about decentralization.
AWS Private Keys
A significant centralization concern involves AWS credentials. The issue is not that the system runs on AWS, but rather that portions of the code require an AWS access key ID and secret access key pair to function. Without these secret keys, the code cannot operate. Therefore, whoever controls these keys controls access to the software.
In the doublezero-offchain code, we find this configuration:
impl S3Config {
async fn new() -> Result<Self> {
let bucket = env::var("VALIDATOR_DEBT_S3_BUCKET")
.unwrap_or_else(|_| "malbeclabs-data-metrics-dev".to_string());
let max_consecutive_failures = env::var("VALIDATOR_DEBT_S3_MAX_CONSECUTIVE_FAILURES")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(12);
// Load AWS credentials from environment variables
let access_key_id = env::var("VALIDATOR_DEBT_AWS_ACCESS_KEY_ID")
.context("VALIDATOR_DEBT_AWS_ACCESS_KEY_ID environment variable not set")?;
let secret_access_key = env::var("VALIDATOR_DEBT_AWS_SECRET_ACCESS_KEY")
.context("VALIDATOR_DEBT_AWS_SECRET_ACCESS_KEY environment variable not set")?;
let region =
env::var("VALIDATOR_DEBT_AWS_REGION").unwrap_or_else(|_| "us-east-1".to_string());This code is part of the “validator-debt” subsystem and requires AWS configuration credentials to function. The code suggests this is controlled by Malbec Labs rather than the DoubleZero Foundation. Regardless of which party holds the keys, their existence raises questions about the applicability of the no-action relief.
While it may be theoretically possible for someone to rewrite this code without AWS dependencies, there is no specification for the software’s expected behavior beyond this code itself. This situation is comparable to proprietary file formats where understanding requires reverse engineering.
Alternatively, a third party could attempt to run this code using their own AWS keys. However, this would require operating numerous parts of the DZ Resource Contributors section and addressing the access control concerns discussed below. The ability to copy and run code multiple times does not inherently make a system decentralized.
Questions About Protocol Documentation
This implementation raises questions about whether DoubleZero is operating within the parameters of their SEC no-action relief. Releasing code for a purportedly decentralized and permissionless system that references VALIDATOR_DEBT_AWS_SECRET_ACCESS_KEY warrants scrutiny.
This is not the only concern with this software architecture. Malbec Labs may contend that this does not constitute centralization because it is merely their implementation of the protocol and others could implement it differently. We discussed this type of claim above. This raises two questions for Malbec Labs:
- Would you be willing to publish these keys publicly?
- Where is the protocol documentation? We have not found any referenced specification, only an existing system.
When a system’s implementation is conflated with the protocol definition itself, this raises questions about whether meaningful decentralization is possible. In software engineering, the terms “system” and “protocol” have distinct meanings that should not be conflated.
Validator Debt & Contributor Rewards
The telemetry infrastructure exists to compute rewards paid to contributors. This work is performed in an offchain subsystem called validator-debt. The validator-debt system uses the AWS wrapper code described above to write validator information to an AWS S3 bucket for later use in rewards calculations. A separate contributor-rewards offchain subsystem reads this data to perform rewards calculations.
These two systems communicate via the AWS private keys discussed above. The system uses private AWS credentials for internal communication. This is not blockchain-mediated but rather represents two software components communicating via a private internal channel, similar to traditional corporate systems rather than open protocols.
Additionally, DoubleZero operates a separate Solana-fork chain for various purposes with limited public participation. The internal telemetry communication is even less accessible than that chain. These two components, validator-debt and contributor-rewards, calculate the rewards that are central to DoubleZero's SEC request. However, these components are offchain and depend on private credentials to function, which appears inconsistent with DoubleZero's letter to the SEC.
While we can locate the Shapley calculation code and various instrumentation and statistics in these systems, and they do perform mathematical operations, including publishing merkle roots onchain, as demonstrated in Part 1, the system remains controlled by private keys. The cryptography ensures that calculations performed by different software components agree.
However, the same organization wrote all this software, and it communicates via private channels controlled by that organization. If that organization wishes to modify the calculation or verification methodology, the cryptography provides limited protection against such changes. While the software is not entirely unrestricted, as we will see, it is configured in a manner that resembles traditional corporate systems more than decentralized protocols.
Rewards Distribution
A revenue distribution smart contract exists on Solana that contains a “rewards accountant.” This is the onchain component of the rewards distribution process — the code that distributes the rewards DoubleZero asked the SEC to classify as part of a DePIN product that does not qualify as a security.
The onchain component appears to function as a dependent system to the offchain component controlled by a central party. It only accepts inputs from designated sources. It is not permissionless or open access but rather a controlled system that utilizes blockchain technology.
In the offchain contributor-rewards program, we find this code:
match try_build_instruction(
&ID,
FinalizeDistributionDebtAccounts::new(&self.pubkey(), dz_epoch, &self.pubkey()),
&RevenueDistributionInstructionData::FinalizeDistributionDebt,
) {
Ok(instruction) => {
let recent_blockhash = solana_rpc_client.get_latest_blockhash().await?;
let message = Message::try_compile(
&self.signer.pubkey(),
&[instruction],
&[],
recent_blockhash,
)
.unwrap();
let finalized_transaction =
VersionedTransaction::try_new(VersionedMessage::V0(message), &[&self.signer])
.unwrap();
Ok(finalized_transaction)
}
Err(err) => Err(anyhow!(
"Failed to build finalize distribution instruction: {err:?}"
)),
}
This function initiates a call to a Solana smart contract to execute RevenueDistributionInstructionData::FinalizeDistributionDebt. This is one of many steps in the process of finalizing reward distribution. We will not detail all steps here for brevity.
On the Solana side, we see how that rewards finalization message is handled:
fn try_finalize_distribution_debt(accounts: &[AccountInfo]) -> ProgramResult {
msg!("Finalize distribution debt");
// We expect the following accounts for this instruction:
// - 0: Program config.
// - 1: Debt accountant.
// - 2: Distribution.
// - 3: Payer (funder of realloc lamports).
// - 4: System program.
let mut accounts_iter = accounts.iter().enumerate();
// Account 0 must be the program config.
// Account 1 must be the debt accountant.
//
// This call ensures that the debt accountant is a signer and is the same
// debt accountant encoded in the program config.
let authorized_use =
VerifiedProgramAuthority::try_next_accounts(&mut accounts_iter, Authority::DebtAccountant)?;
// Make sure the program is not paused.
authorized_use.program_config.try_require_unpaused()?;As in Part 1, this function only operates when called by the designated “debt accountant.” Similar to the passport program and sentinel, there is a blockchain present, but the functions only execute when called by approved parties.
If this were entirely onchain and internal to a purely decentralized system, there might be room for debate. However, we have offchain components (in a repository called doublezero-offchain) calling in with privileged access. While merkle roots and various forms of cryptography are involved, the team could potentially submit alternative telemetry and modify rewards distribution without participants having recourse.
While a blockchain is present and does restrict certain aspects of reward distribution, ensuring persistence and consistency for recordkeeping, DoubleZero’s own letter states:
A set of smart contracts that controls the protocol token economics (e.g., minting of newly inflated 2Z), has been deployed to the Solana blockchain. Resource Providers are tasked with invoking these smart contracts so that the smart contracts execute, but they do not have any control over the economics or ability to upgrade such smart contracts. It is anticipated that upgrade authority for such smart contracts will be controlled by the Foundation at Launch, though the Foundation intends to decentralize control over time.
The Foundation acknowledges that it can modify this process. It can change the validation rules within this system to confirm any token distribution scheme it chooses.
Additionally, we have offchain code using private AWS keys to perform these functions.
This raises questions about whether such an architecture should qualify for the securities law treatment DoubleZero received.
Anticipated Responses
One likely response is that we misunderstand how Solana operates. The argument may be made that the authorized_use checks, both here and in Part 1, represent standard good design practices. The claim may follow that because the smart contracts run on Solana, and Solana is decentralized, the smart contracts must be decentralized. However, their own letter acknowledges they expect to retain control over the system for an indefinite period.
Another likely argument is that because the validator rewards process uses cryptography and publishes merkle roots, even if these programs have privileged access, that access only permits paying out the correct amounts.
However, the telemetry is collected via offchain processes. Since some combination of the Foundation (which, per Part 1, controls access to participate in the system) and Malbec Labs operates all offchain telemetry infrastructure, they could potentially observe different results and compute the “correct” rewards for alternative telemetry data and distribute them. No apparent dispute resolution process exists, nor any feedback mechanism for participants who believe the system operates improperly.
Running an RPC node is permissioned according to their documentation, preventing independent interface setup. The documentation does not even permit participants to exit the system freely, as it states: “Important: Please discuss with either DZF and/or Malbec Labs before deleting an existing production link.”
The team may also contend that we incorrectly view this as a system when it is actually a protocol. A protocol is a communication method — a collection of message formats and meanings assigned to properly formatted messages. To be “just a protocol,” two things are necessary: First, a protocol specification to satisfy the “a protocol” requirement. DoubleZero does not appear to have published such a specification. Second, only a protocol specification should exist to satisfy the “just” requirement.
A protocol specification with a reference implementation is acceptable. If the reference implementation contains centralized control points, that may also be acceptable in some contexts. OAuth, for example, is a complex set of protocols, many involving central authorities. However, when there is an existing deployment where one entity controls multiple central authorities, it is no longer “just a protocol.”
Discussion
We have attempted to make this analysis as accessible as possible while providing clear evidence. As discussed in Part 1, we acknowledge it is not the SEC’s responsibility to verify whether a project’s no-action relief request matches their software implementation details.
However, DoubleZero’s claims to be a blockchain project warrant examination. Using git’s git-sizer tool for basic analysis reveals interesting proportions. Among the three Foundation-owned repositories: the doublezero-solana repository (all onchain components) has 123 files and is approximately 1.23 MiB in size. The doublezero-offchain repository has 225 files and is 24.5 MiB. The doublezero-ledger code, for the second private blockchain the system uses, has 3,230 files and is 67 MiB. Additionally, the doublezero repository from Malbec Labs, which handles networking, has 1,270 files and is 30.5 MiB.
While code length does not correlate perfectly with importance, the vast majority of this system exists outside a public blockchain. We know that components residing on the public Solana blockchain are configured to work only with designated offchain software built by affiliated entities.
Recommendations
The SEC should clarify that no-action relief predicated on features a system does not possess is:
- Inapplicable to the actual system
- A waste of the commission’s resources
- Potentially an aggravating factor during enforcement actions
Failing to issue a basic statement reaffirming that no-action relief only applies to projects with the characteristics described in their relief requests creates uncertainty in the market.
Commissioners who issued statements praising such no-action relief should consider whether clarification is warranted to maintain the integrity of the no-action relief process.
DoubleZero Is Centralized And Misled The SEC: Part 2 of N was originally published in ChainArgos on Medium, where people are continuing the conversation by highlighting and responding to this story.
