Skip to main content
Geneva supports various environment variables that start with GENEVA_ to configure advanced behavior and fine-tune system settings.
All GENEVA_ environment variables are optional and have sensible defaults. Only set them if you need to override the default behavior.

​
Admission Control

Admission control validates cluster resources before starting jobs to prevent failures due to insufficient resources.
VariableDefaultDescription
GENEVA_ADMISSION__CHECKtrueEnable admission control checks. Set to false to skip all checks.
GENEVA_ADMISSION__STRICTtrueIf true, reject the job with ResourcesUnavailableError when resources are insufficient. If false, log a warning but allow the job to proceed.
GENEVA_ADMISSION__TIMEOUT3.0Timeout in seconds for Ray API calls during admission control checks. Prevents hanging when the cluster is in a bad state.

​
Commit and Retry Configuration

Control retry behavior for commits and version conflicts.
VariableDefaultDescription
GENEVA_COMMIT_MAX_RETRIES12Maximum number of retries for commit operations. With exponential backoff (1s, 2s, 4s, 8s, 16s, then 16s capped), 12 retries gives ~2.5 minutes total wait time before giving up.
GENEVA_VERSION_CONFLICT_MAX_RETRIES10Maximum number of retries for version conflicts during commit. Version conflicts occur when concurrent backfills commit to the same fragments. Prevents infinite loops when concurrent commits keep conflicting.
GENEVA_WRITER_STALL_IDLE_ROUNDS6Number of idle rounds (5s each) before considering a writer stalled during drain. With many concurrent backfills, resource contention can slow writers without them being truly stalled.

​
Lance Retry Configuration

This section configures retry logic for Lance I/O operations. Retries occur on OSError, ValueError, and RuntimeError("Too many concurrent writers") exceptions, and are retried with exponential backoff with jitter.
VariableDefaultDescription
GENEVA_RETRY_LANCE_ATTEMPTS7Maximum number of retry attempts for Lance I/O operations.
GENEVA_RETRY_LANCE_INITIAL_SECS0.5Initial wait time in seconds for exponential backoff when retrying Lance I/O operations.
GENEVA_RETRY_LANCE_MAX_SECS120.0Maximum wait time in seconds for exponential backoff when retrying Lance I/O operations.

​
Checkpoint Storage

Checkpoint storage configuration is experimental. The environment variable names and behavior may change in a future release.
Configure where Geneva stores checkpoint data during job execution. Checkpoints enable fault-tolerant processing by saving intermediate results so that failed jobs can resume without reprocessing completed work. By default, Geneva stores checkpoints in a _ckp/ subdirectory inside the table’s own storage location. This means checkpoints share the same bucket and IOPS budget as the table data. You can override this to store checkpoints in a separate location.
VariableDefaultDescription
JOB__CHECKPOINT__OBJECT_STORE__PATH(table dir)/_ckp/URI where checkpoint data is stored. When set, overrides the default in-table checkpoint location. Accepts any URI supported by Lance (e.g., gs://bucket/path/checkpoints, s3://bucket/checkpoints).
This variable maps to the config path job.checkpoint.object_store.path. It can also be set via config files in .config/ or pyproject.toml under the [geneva] section.

​
Why use a separate checkpoint path?

At scale, checkpoint I/O and data I/O compete for the same object store IOPS budget when they share a bucket prefix. Setting JOB__CHECKPOINT__OBJECT_STORE__PATH to a different bucket or prefix decouples checkpoint I/O from data I/O, giving each its own IOPS budget and preventing shared-prefix rate limiting. 
# Example: separate checkpoint bucket from dataset storage
JOB__CHECKPOINT__OBJECT_STORE__PATH=gs://my-checkpoints-bucket/ckpts
Equivalent programmatic configuration: 
from geneva.config import override_config_kv

override_config_kv({
    "job.checkpoint.object_store.path": "gs://my-checkpoints-bucket/ckpts",
})

​
Other Configuration

VariableDefaultDescription
GENEVA_RAY_INIT_MAX_RETRIES5Maximum number of retry attempts for ray.init() connection failures. Useful when connecting to Ray clusters that may be temporarily unavailable.
GENEVA_K8S_AUTH_MAX_RETRIES3Maximum number of retries for Kubernetes authentication operations. Must be at least 1.
GENEVA_CONFIG_DIR./.configDirectory path where Geneva looks for configuration files (.yaml, .json, .toml). Can be an absolute or relative path.