Walkthrough
PATH
Authorization & Rate Limiting
Table of Contents â
- Quickstart
- Overview
- Envoy Proxy
- Service ID Specification
- Specifying the Gateway Endpoint ID
- Gateway Endpoint Authorization
- External Auth Server
- Rate Limiter
Quickstartâ
Shannon Quickstartâ
See the PATH Quickstart Cheat Sheet for instructions on how to get started with a local PATH instance on Shannon.
âšī¸ After copying config files by running make init_envoy
, which will create the required files in the local/path/envoy
directory, please update the following files:
.allowed-services.lua
with the service IDs allowed by your PATH instance.gateway-endpoints.yaml
with your own data
Overviewâ
This folder contains everything necessary for managing authorization and rate limiting in the PATH service. Specifically, this is split into two logical parts:
- The
Envoy Proxy configuration
- The
External Auth Server
Componentsâ
A Tiltfile is provided to run all of these services locally.
- PATH Service: The service that handles requests after they have been authorized.
- Envoy Proxy: A proxy server that handles incoming requests, performs auth checks, and routes authorized requests to the
PATH
service. - External Auth Server: A Go/gRPC server that evaluates whether incoming requests are authorized to access the
PATH
service. - Rate Limiter: A service that coordinates all rate limiting.
- Redis: A key-value store used by the rate limiter to share state and coordinate rate limiting across any number of PATH instances behind the same Envoy Proxy.
- Remote gRPC Server: A server that provides the external authorization server with data on which endpoints are authorized to use the PATH service.
- PADS (PATH Auth Data Server) is provided as a functional implementation of the remote gRPC server that loads data from a YAML file or simple Postgres database.
- See PATH Auth Data Server for more information.
Architecture Diagramâ
Envoy Proxyâ
PATH uses Envoy Proxy to handle authorization and rate limiting.
The /envoy
directory houses the configuration files and settings for Envoy Proxy.
Envoy acts as a gateway, handling incoming requests, determining allowed services, performing auth checks, and routing authorized requests to the PATH service.
Envoy Configurationâ
See the Envoy Config docs for a simplified explanation of the Envoy Proxy configuration files and how they work together.
Envoy HTTP Filtersâ
The PATH Auth Server uses the following Envoy HTTP filters to handle authorization:
- lua: Extracts the Service ID from the subdomain of the request's host field and attaches it to the request as the
target-service-id
header. - header_mutation: Ensures the request does not have the
jwt-user-id
header set before it is forwarded upstream.header_mutation
is used only if the PATH instance has JWT auth enabled.
- jwt_authn: Performs JWT verification and sets the
jwt-user-id
header.jwt_authn
is used only if the PATH instance has JWT auth enabled.
- ext_authz: Performs authorization checks using the PATH Auth Server external authorization server.
- ratelimit: Performs rate limiting checks using the Rate Limiter service.
Request Lifecycleâ
Service ID Specificationâ
The target-service-id
header is used to specify the Service ID in the request.
There are two methods for specifying this header in the request:
- Target Service ID Header (e.g. -H "target-service-id: anvil")
- URL Subdomain (e.g. http://anvil.localhost:3070/v1)
Allowed Services Fileâ
The file local/path/envoy/.allowed-services.lua
defines the mapping of service IDs to the service IDs used by the PATH service.
All service IDs (and optional service aliases) used by the PATH service must be defined in this file.
.allowed-services.lua
formatreturn {
-- 1. Shannon Service IDs
["anvil"] = "anvil", -- Anvil (Authoritative ID)
-- 2. Morse Service IDs
["F000"] = "F000", -- Pocket (Authoritative ID)
["pocket"] = "F000", -- Pocket (Alias)
}
Target Service ID Headerâ
The service ID (or a configured alias) may be specified in the target-service-id
header.
curl http://localhost:3070/v1 \
-X POST \
-H "Content-Type: application/json" \
-H "target-service-id: anvil" \
-H "endpoint-id: endpoint_3_no_auth" \
-d '{"jsonrpc": "2.0", "id": 1, "method": "eth_blockNumber" }'
URL Subdomainâ
The service ID (or a configured alias) may be specified in the URL subdomain.
eg. host = "anvil.path.grove.city" -> Header: "target-service-id: anvil"
curl http://anvil.localhost:3070/v1 \
-X POST \
-H "Content-Type: application/json" \
-H "endpoint-id: endpoint_3_no_auth" \
-d '{"jsonrpc": "2.0", "id": 1, "method": "eth_blockNumber" }'
Specifying the Gateway Endpoint IDâ
The Auth Server may extract the Gateway Endpoint ID from the request in one of two ways:
This is determined by the auth_server.endpoint_id_extractor
field in the PATH .config.yaml
file.
One of:
url_path
(default)header
Requests are rejected if either of the following are true:
- The
<GATEWAY_ENDPOINT_ID>
is missing - ID is not present in the
External Auth Server
'sGateway Endpoint Store
Regardless of which extractor is used, the Gateway Endpoint ID will always be set in the endpoint-id
header if the request is forwarded to the PATH Service.
URL Path Endpoint ID Extractorâ
When using the url_path
extractor, the Gateway Endpoint ID must be specified in the URL path.
https://<SERVICE_NAME>.<PATH_DOMAIN>/v1/<GATEWAY_ENDPOINT_ID>
For example, if the SERVICE_NAME
is eth
and the GATEWAY_ENDPOINT_ID
is endpoint_3_no_auth
:
curl http://anvil.localhost:3070/v1/endpoint_3_no_auth \
-X POST \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "id": 1, "method": "eth_blockNumber" }'
Header Endpoint ID Extractorâ
When using the header
extractor, the Gateway Endpoint ID must be specified in the endpoint-id
header.
-H "endpoint-id: <GATEWAY_ENDPOINT_ID>"
For example, if the endpoint-id
header is set to endpoint_3_no_auth
:
curl http://anvil.localhost:3070/v1 \
-X POST \
-H "Content-Type: application/json" \
-H "endpoint-id: endpoint_3_no_auth" \
-d '{"jsonrpc": "2.0", "id": 1, "method": "eth_blockNumber" }'
A variety of example cURL requests to the PATH service can be found in the test_requests.mk
file.
endpoint_3_no_auth
is the endpoint from the example .gateway-endpoints.yaml
file that requires no authorization.
See the Gateway Endpoint YAML File section for more information on the GatewayEndpoint
data structure.
Gateway Endpoint Authorizationâ
The External Auth Server
evaluates whether incoming requests are authorized to access the PATH service based on the AuthType
field of the GatewayEndpoint
proto struct.
Three authorization types are supported:
JSON Web Token (JWT) Authorizationâ
For GatewayEndpoints with the AuthType
field set to JWT_AUTH
, a valid JWT issued by the auth provider specified in the envoy.yaml
file is required to access the PATH service.
The make init_envoy
command will prompt you about whether you wish to use JWT authorization.
If you do wish to use it, you will be asked to enter your auth provider's domain and audience.
Auth0 is an example of a JWT issuer that can be used with PATH.
Their docs page on JWTs gives a good overview of the JWT format and how to issue JWTs to your users:
Example Request Header:
-H "Authorization: Bearer <JWT>"
The jwt_authn
filter will verify the JWT and, if valid, set the jwt-user-id
header from the sub
claim of the JWT. An invalid JWT will result in an error.
The External Auth Server
will use the jwt-user-id
header to make an authorization decision; if the GatewayEndpoint
's Auth.AuthorizedUsers
field contains the jwt-user-id
value, the request will be authorized.
Example auth provider user ID header:
jwt-user-id: auth0|a12b3c4d5e6f7g8h9
For more information, see the Envoy JWT Authn Docs
API Key Authorizationâ
For GatewayEndpoints with the AuthType
field set to API_KEY_AUTH
, a static API key is required to access the PATH service.
Example Request Header:
-H "Authorization: <API_KEY>"
The External Auth Server
will use the authorization
header to make an authorization decision; if the GatewayEndpoint
's Auth.ApiKey
field matches the API_KEY
value, the request will be authorized.
No Authorizationâ
For GatewayEndpoints with the AuthType
field set to NO_AUTH
, no authorization is required to access the PATH service.
All requests for GatewayEndpoints with the AuthType
field set to NO_AUTH
will be authorized by the External Auth Server
.
External Auth Serverâ
The envoy/auth_server
directory contains the External Auth Server
called by the Envoy ext_authz
filter. It evaluates whether incoming requests are authorized to access the PATH service.
This server communicates with a Remote gRPC Server
to populate its in-memory Gateway Endpoint Store
, which provides data on which endpoints are authorized to use the PATH service.
External Auth Server Sequence Diagramâ
External Auth Server Configurationâ
The external authorization server loads its config from the auth_server_config
section of PATH's .config.yaml
file. For the local Tilt stack, this file is located at local/path/config/.config.yaml
.
.config.yaml auth_server_config
example:
auth_server_config:
grpc_host_port: path-auth-data-server:50051
grpc_use_insecure_credentials: true
endpoint_id_extractor_type: url_path
See the PATH Configuration section for more information on the .config.yaml
file.
For more information on external authorization using Envoy Proxy, see the Envoy documentation:
gateway_endpoint.proto
fileâ
Both the External Auth Server
and the Remote gRPC Server
use the gRPC service and types defined in the gateway_endpoint.proto
file.
This service defines two main methods for populating the External Auth Server
's Gateway Endpoint Store
:
service GatewayEndpoints {
// GetInitialData requests the initial set of GatewayEndpoints from the remote gRPC server.
rpc GetInitialData(InitialDataRequest) returns (InitialDataResponse);
// StreamUpdates listens for updates from the remote gRPC server and streams them to the client.
rpc StreamUpdates(UpdatesRequest) returns (stream Update);
}
Remote gRPC Auth Serverâ
The Remote gRPC Server
is responsible for providing the External Auth Server
with data on which endpoints are authorized to use the PATH service.
The PATH Auth Data Server (PADS) is a pre-built implementation of the Remote gRPC Server
that should be sufficient for most users.
PATH Auth Data Serverâ
The PADS repo provides a pre-built implementation of the remote gRPC server.
See the PADS Documentation for more information on the PATH Auth Data Server.
This service is available as a Docker image and may be configured to load data from a YAML file or using an opinionated Postgres database driver.
ghcr.io/buildwithgrove/path-auth-data-server:latest
This Docker image is loaded by default in the Tiltfile file at the root of the PATH repo.
Implementing a Custom Remote gRPC Serverâ
If the Gateway operator wishes to implement a custom remote gRPC server, the implementation must import the Go github.com/buildwithgrove/path/envoy/auth_server/proto
package, which is autogenerated from the gateway_endpoint.proto
file.
The custom implementation must use the methods defined in the GatewayEndpoints
service:
FetchAuthDataSync
StreamAuthDataUpdates
If you wish to implement your own custom database driver, forking the PADS repo is the easiest way to get started, though any gRPC server implementation that adheres to the gateway_endpoint.proto
service definition should suffice.
Rate Limiterâ
Rate Limit Configurationâ
-
The
External Auth Server
sets therl-endpoint-id
andrl-throughput
headers if theGatewayEndpoint
for the request should be rate limited. -
Envoy Proxy is configured to forward the
rl-endpoint-id
andrl-throughput
headers to the rate limiter service as descriptors..envoy.yaml
rate_limits:
- actions:
- request_headers:
header_name: "rl-endpoint-id"
descriptor_key: "rl-endpoint-id"
- request_headers:
header_name: "rl-throughput"
descriptor_key: "rl-throughput" -
Rate limiting is configured through the
.ratelimit.yaml
file..ratelimit.yaml
domain: rl
descriptors:
- key: rl-endpoint-id
descriptors:
- key: rl-throughput
value: "PLAN_FREE"
rate_limit:
unit: second
requests_per_unit: 30
The default throughput limit is 30 requests per second for GatewayEndpoints with the PLAN_FREE
plan type based on the rl-endpoint-id
and rl-throughput
descriptors.
The rate limiting configuration may be configured to suit the needs of the Gateway Operator in the .ratelimit.yaml
file.
Documentation and Examplesâ
As Envoy's rate limiting configuration is fairly complex, this blog article provides a good overview of the configuration options:
For more advanced configuration options, refer to the Envoy documentation: