ZephyrFS Protocol Definitions

Protocol buffer definitions for ZephyrFS peer-to-peer distributed storage system.

Overview

This repository contains the protocol definitions that enable communication between ZephyrFS components:

• Node-to-Node Communication: Direct peer-to-peer messaging for chunk storage and retrieval
• Coordinator Services: Centralized coordination for peer discovery and metadata management
• Security Protocols: Capability-based access control and proof-of-storage challenges

Protocol Files

Architecture

�����������������    �����������������    �����������������
  ZephyrFS Node  ����  Coordinator    ����  ZephyrFS Node  
                                                           
 " P2P Storage        " Peer Registry      " P2P Storage   
 " Chunk Ops          " Metadata Mgmt      " Chunk Ops     
 " Security           " Network Stats      " Security      
�����������������    �����������������    �����������������
                                                       
         �����������������������<�����������������������
                                 
                   Direct P2P Communication
                      (node.proto messages)

Code Generation

Prerequisites

Install protocol buffer compiler and plugins:

# Install dependencies
make install-deps

# Or manually:
# - protoc (Protocol Buffer compiler)
# - protoc-gen-rust (Rust plugin)
# - protoc-gen-go + protoc-gen-go-grpc (Go plugins)
# - protoc-gen-ts (TypeScript plugin)

Generate Code

# Generate all languages
make all

# Generate specific language
make rust       # Rust bindings
make go         # Go bindings
make typescript # TypeScript bindings

# Clean generated files
make clean

Validate Protocols

# Validate all .proto files
make validate

# Generate documentation
make docs

Protocol Usage

Node Service (node.proto)

Core P2P Operations:

service NodeService {
  rpc StoreChunk(StoreChunkRequest) returns (StoreChunkResponse);
  rpc RetrieveChunk(RetrieveChunkRequest) returns (RetrieveChunkResponse);
  rpc RegisterPeer(RegisterPeerRequest) returns (RegisterPeerResponse);
}

Example Usage (Rust):

use zephyrfs_proto::node::{NodeServiceClient, StoreChunkRequest};

let mut client = NodeServiceClient::connect("http://peer:8080").await?;
let request = StoreChunkRequest {
    chunk_id: "chunk_123".to_string(),
    data: chunk_data,
    hash: sha256_hash,
    size: chunk_data.len() as i64,
    metadata: HashMap::new(),
};
let response = client.store_chunk(request).await?;

Coordinator Service (coordinator.proto)

Network Coordination:

service CoordinatorService {
  rpc RegisterNode(RegisterNodeRequest) returns (RegisterNodeResponse);
  rpc GetActiveNodes(GetActiveNodesRequest) returns (GetActiveNodesResponse);
  rpc FindChunkLocations(FindChunkLocationsRequest) returns (FindChunkLocationsResponse);
}

Example Usage (Go):

import "github.com/ZephyrFS/zephyrfs-proto/gen/go/coordinator"

client := coordinator.NewCoordinatorServiceClient(conn)
req := &coordinator.RegisterNodeRequest{
    NodeId: nodeID,
    Addresses: []string{"127.0.0.1:8080"},
    StorageCapacity: 1000000000, // 1GB
}
resp, err := client.RegisterNode(ctx, req)

Security Service (security.proto)

Zero-Knowledge Security:

service SecurityService {
  rpc ValidateCapability(ValidateCapabilityRequest) returns (ValidateCapabilityResponse);
  rpc IssueChallenge(IssueChallengeRequest) returns (IssueChallengeResponse);
  rpc UpdateReputation(UpdateReputationRequest) returns (UpdateReputationResponse);
}

Integration Guide

Adding to Rust Project (zephyrfs-node)

# Cargo.toml
[dependencies]
tonic = "0.10"
prost = "0.12"
tokio = { version = "1.0", features = ["full"] }

[build-dependencies]
tonic-build = "0.10"

// build.rs
fn main() -> Result<(), Box<dyn std::error::Error>> {
    tonic_build::configure()
        .build_server(true)
        .build_client(true)
        .compile(
            &[
                "proto/node.proto",
                "proto/coordinator.proto",
                "proto/security.proto"
            ],
            &["proto"],
        )?;
    Ok(())
}

Adding to Go Project (zephyrfs-coordinator)

// go.mod
module github.com/ZephyrFS/zephyrfs-coordinator

require (
    google.golang.org/grpc v1.58.0
    google.golang.org/protobuf v1.31.0
    github.com/ZephyrFS/zephyrfs-proto v0.1.0
)

Adding to TypeScript Project (zephyrfs-web)

// package.json
{
  "dependencies": {
    "@grpc/grpc-js": "^1.9.0",
    "@grpc/proto-loader": "^0.7.0",
    "google-protobuf": "^3.21.0"
  }
}

Protocol Versioning

• Backward Compatibility: New fields are always optional
• Version Numbers: Each service includes version in package name
• Migration Path: Deprecated fields marked with comments
• Breaking Changes: Major version bump for incompatible changes

Development

Adding New Protocols

1. Create .proto file in protobuff/ directory
2. Follow existing naming conventions
3. Add comprehensive documentation
4. Update this README
5. Run make validate to check syntax
6. Generate code with make all

Protocol Design Guidelines

• Use meaningful field names (not generic data or info)
• Include timestamps for debugging and monitoring
• Add optional metadata maps for extensibility
• Use enums for status fields instead of strings
• Include pagination for list operations
• Add request/response IDs for correlation

Testing

# Validate protocol syntax
make validate

# Generate test code
make all

# Run integration tests (requires running services)
cd ../zephyrfs-node && cargo test proto_integration
cd ../zephyrfs-coordinator && go test ./... -tags=integration

Contributing

1. Fork the repository
2. Create feature branch: git checkout -b proto/new-feature
3. Add/modify protocol definitions
4. Run validation: make validate
5. Update documentation
6. Submit pull request

License

MIT License - see LICENSE file for details.