High-performance gRPC microservice for robot path planning using ONNX-Runtime inference.
- 🚀 High-performance gRPC API with batch inference support
- 🧠 ONNX Runtime integration for ML model inference
- 📊 Prometheus metrics with gRPC latency histograms
- 🔍 OpenTelemetry tracing for distributed observability
- 🏥 Health checks (HTTP + gRPC) for Kubernetes deployments
- 🔧 Flexible configuration via flags, environment variables, or YAML
- 🎯 Request ID tracking for debugging and correlation
- 📦 Helm chart for Kubernetes deployment
- 🧪 Comprehensive tests with mock inference engine
- Go 1.22+
- Docker (for containerized deployment)
- Redis (optional, for pose caching)
- ONNX model file (
policy_cpu.onnx) protocwith Go plugins (for regenerating protobuf code)
policy-service/
├── cmd/server/main.go # gRPC server entry point
├── internal/
│ ├── cache/redis.go # Redis client
│ ├── config/config.go # Viper configuration
│ ├── handler/ # gRPC handlers
│ │ ├── handler.go
│ │ ├── handler_test.go
│ │ └── errors.go
│ ├── inference/ # ONNX inference
│ │ ├── interface.go # InferenceEngine interface
│ │ ├── inference.go # Real ONNX implementation
│ │ ├── mock.go # Mock for testing
│ │ └── inference_test.go
│ ├── metrics/metrics.go # Prometheus metrics
│ └── middleware/ # gRPC interceptors
│ ├── metrics.go
│ ├── request_id.go
│ └── middleware_test.go
├── proto/
│ ├── planner.proto # Protobuf definitions
│ └── plannerpb/ # Generated code
├── helm/ # Helm chart
├── .github/workflows/ci.yml # CI/CD pipeline
├── config.yaml # Example config file
├── Dockerfile
└── go.mod
# Build the server
go build -o server ./cmd/server/main.go
# Run with mock inference (no ONNX required)
./server -mock
# Run with real ONNX model
./server -model policy_cpu.onnx
# Run with all options
./server -port 50051 -metrics 9100 -model policy_cpu.onnx -redis localhost:6379# Build the image
docker build -t policy-service:latest .
# Run with mock inference
docker run -p 50051:50051 -p 9100:9100 policy-service:latest -mock
# Run with model
docker run -p 50051:50051 -p 9100:9100 \
-v $(pwd)/policy_cpu.onnx:/app/policy_cpu.onnx \
policy-service:latest -model /app/policy_cpu.onnxThe service supports configuration from multiple sources (in order of precedence):
- Command-line flags (highest priority)
- Environment variables
- Config file (
config.yaml) - Defaults (lowest priority)
| Variable | Description | Default |
|---|---|---|
POLICY_SERVICE_PORT |
gRPC server port | 50051 |
POLICY_SERVICE_METRICS_PORT |
Prometheus metrics port | 9100 |
POLICY_SERVICE_MODEL |
Path to ONNX model | policy_cpu.onnx |
POLICY_SERVICE_REDIS |
Redis address | localhost:6379 |
POLICY_SERVICE_OTEL_ENABLED |
Enable OpenTelemetry | false |
OTEL_EXPORTER_OTLP_ENDPOINT |
OTLP exporter endpoint | `` |
POLICY_SERVICE_USE_MOCK |
Use mock inference | false |
./server \
-port 50051 \
-metrics 9100 \
-model policy_cpu.onnx \
-redis localhost:6379 \
-config /path/to/config.yaml \
-mock # Use mock inferenceport: 50051
metrics_port: 9100
model: "policy_cpu.onnx"
redis: "localhost:6379"
otel_enabled: false
otel_endpoint: ""
use_mock_inference: falseMetrics are exposed at http://localhost:9100/metrics:
| Metric | Type | Labels | Description |
|---|---|---|---|
grpc_server_handling_seconds |
Histogram | method, code |
gRPC request latency |
inference_batch_size |
Histogram | - | Batch sizes for inference |
inference_latency_seconds |
Histogram | - | Inference-only latency |
health_status |
Gauge | - | Service health (1=healthy) |
Every request is assigned a unique request ID:
- Extract from
x-request-idheader if present - Generate UUID if not provided
- Included in response headers
- Logged with each request
Enable distributed tracing by setting:
export POLICY_SERVICE_OTEL_ENABLED=true
export OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317
./serverOr in config.yaml:
otel_enabled: true
otel_endpoint: "http://otel-collector:4317"| Endpoint | Description | Response |
|---|---|---|
GET /healthz |
Liveness check | 200 OK or 503 Service Unavailable |
GET /readyz |
Readiness check | 200 Ready or 503 Not Ready |
GET /metrics |
Prometheus metrics | Metrics in Prometheus format |
The service implements the standard gRPC health checking protocol:
grpcurl -plaintext localhost:50051 grpc.health.v1.Health/Check# Run all tests (uses mock inference, no ONNX required)
go test -v ./...
# Run with race detector
go test -v -race ./...
# Run with coverage
go test -v -coverprofile=coverage.out ./...
go tool cover -html=coverage.out# Install Python dependencies
pip install grpcio grpcio-tools numpy
# Generate Python protobuf code
python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. proto/planner.proto
# Run load test (2000 requests, 50 concurrent)
python client_load.py --host localhost --port 50051 --requests 2000 --concurrent 50| RPC | Request | Response | Description |
|---|---|---|---|
Plan |
PlanRequest |
PlanResponse |
Single robot planning |
BatchPlan |
BatchPlanRequest |
BatchPlanResponse |
Batch robot planning |
# List services
grpcurl -plaintext localhost:50051 list
# Single plan request
grpcurl -plaintext -d '{
"robot_id": 1,
"obs": {
"data": [0.1, 0.2, 0.3, 0.4],
"channels": 1,
"height": 2,
"width": 2
}
}' localhost:50051 planner.PathPlanner/Plan
# Batch plan request
grpcurl -plaintext -d '{
"requests": [
{"robot_id": 1, "obs": {"data": [0.1, 0.2, 0.3, 0.4], "channels": 1, "height": 2, "width": 2}},
{"robot_id": 2, "obs": {"data": [0.5, 0.6, 0.7, 0.8], "channels": 1, "height": 2, "width": 2}}
]
}' localhost:50051 planner.PathPlanner/BatchPlan
# Health check
grpcurl -plaintext localhost:50051 grpc.health.v1.Health/Check# Install protoc plugins
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
# Generate Go code
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
proto/planner.proto# Install with defaults
helm install policy-service ./helm
# Install with custom values
helm install policy-service ./helm \
--set image.repository=ghcr.io/SyedDaiam9101/policy-service \
--set image.tag=v1.0.0 \
--set config.modelPath=/models/policy_cpu.onnx \
--set otel.enabled=true \
--set otel.endpoint=http://otel-collector:4317# Replicas
replicaCount: 3
# Image
image:
repository: ghcr.io/SyedDaiam9101/policy-service
tag: "v1.0.0"
# Resources
resources:
requests:
memory: "256Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "500m"
# Health probes (uses /healthz endpoint)
health:
path: /healthz
port: 9100
initialDelaySeconds: 10
periodSeconds: 10
# OpenTelemetry
otel:
enabled: true
endpoint: "http://otel-collector:4317"
# Service config
config:
port: 50051
metricsPort: 9100
modelPath: "/models/policy_cpu.onnx"
useMock: falsehelm upgrade policy-service ./helm --reuse-values --set image.tag=v1.1.0┌─────────────────────────────────────────────────────────────┐
│ gRPC Server │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Interceptor Chain │ │
│ │ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ │
│ │ │RequestID │→ │ Metrics │→ │ OpenTelemetry │ │ │
│ │ └──────────┘ └──────────┘ └──────────────────┘ │ │
│ └─────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Handler │ │
│ │ ┌──────────────────┐ ┌──────────────────────┐ │ │
│ │ │ Plan() │ │ BatchPlan() │ │ │
│ │ └──────────────────┘ └──────────────────────┘ │ │
│ └─────────────────────────────────────────────────────┘ │
│ ↓ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ InferenceEngine (interface) │ │
│ │ ┌──────────────────┐ ┌──────────────────────┐ │ │
│ │ │ ONNX Inference │ │ Mock Inference │ │ │
│ │ └──────────────────┘ └──────────────────────┘ │ │
│ └─────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
↓ ↓
┌─────────┐ ┌─────────┐
│ Redis │ │ ONNX │
│ (cache) │ │ Model │
└─────────┘ └─────────┘
MIT License