Access to Storage

S3 storage currently requires manual configuration for bucket and policy creation along with granting access to users and applications. Please contact us (Neal Feierabend or Kenny Strickler) to help get setup.

Minio

Web UI

Minio comes with a built-in web console that can be accessed by visiting https://minio.cloud.vtti.vt.edu.

Info

The web console is one of the unique features of Minio that will not translate over to Ceph. There are other web interfaces we're exploring that would run on top of Ceph, but it will be different from the Minio one. That said, Minio's version is subject to updates and changes as they upgrade it frequently. There are a number of other (open-source) tools and applications that are built to work with S3 and will usually work with Minio when configured correctly with the info from the API below.

API Access

Setting	Value
Endpoint	https://minio-api.cloud.vtti.vt.edu
Region	us-east-1
Access style	Path (not always required)
Access key	See authentication below
Secret key	See authentication below

Endpoint

The current endpoint for accessing the primary Minio instance is https://minio-api.cloud.vtti.vt.edu. This is the address that should be used for anything that isn't you accessing it from a web browser (a browser will automatically redirect from this URL to the web ui above).

Region

In addition to the endpoint, you will frequently be required to provide a region when configuring your applications which should always be us-east-1

Path style

Minio is currently configured to use "path-style" bucket access. This is sometimes automatically detected by applications and libraries, but sometimes requires explicit configuration.

Note

AWS S3 switched to a domain name based bucket system a number of years ago. This means if you had a bucket called my-vtti-bucket-on-aws the application or library would acccess it by connecting to something like https://my-vtti-bucket-on-aws.s3.us-east-1.amazonaws.com. Minio's configuration with path-style (the original AWS method still supported by most libraries) means it accesses it at https://minio-api.cloud.vtti.vt.edu/my-vtti-bucket-on-aws.

Authentication

The traditional way of accessing S3 storage is via a set of credentials, the ACCESS_KEY_ID and SECRET_ACCESS_KEY. However, our systems (Minio and Ceph) are configured to use OIDC authentication with Keycloak as the upstream Identity Provider (IdP). You'll notice this when you go to the web UI you are redirected to Keycloak for sign in. This is convenient for human access via the browser, but how do we get credentials for applications to talk to our S3 storage?

Danger

Access credentials should never be committed to a code repository. Your application should leverage environment variables or config files instead.

STS and AssumeRole

The S3 standard includes a process called Secure Token Service, and for our use case, the AssumeRoleWithWebIdentity flow. This provides a mechanism to authenticate to Keycloak first and receive a temporary set of credentials via a JWT (Javascript Web Token) that can then be used to authenticate to Minio/Ceph to receive credentials that can be used for subsequent requests. The Minio documentation provides an explanation of what this process entails, but a simple diagram of the process looks like:

sequenceDiagram
  autonumber
  Client->>Keycloak: Authenticate
  Keycloak->>Client: Access Token
  Client->>Minio: JWT Auth
  Minio->>Client: Access Key and Secret

Minio Access Keys

While STS is the "right" way to do things, and might be the only method we will support with Ceph, Minio provides a method of generating static access keys via their web UI. You can look at the Minio docs for more detail, but from within the web ui you can find the "Access Keys" menu item and use it to create and manage keys. If you are going to use this, we recommend that each set of keys you create be scoped to a specific set of needs, like accessing one particular bucket. If deploying an application, we will generate a separate set of keys for the application that will be used, injected as env vars or config file at runtime.

Authorization/Policies

By default when we create a bucket for a user or application, we will also create a read/write policy that has full access to the new bucket. We can also create a read-only policy that can be assigned to users or applications that only need to read data in the bucket but should not have the ability to add, update, or delete any data.

These roles are assigned to users and applications via the Keycloak system. We plan to make this more automated and give users more control over this process, but that's not quite ready.

Pre-prod and dev environments

At your request, we can create additional buckets for different environments like pre-prod or dev.

Note

In the future we plan to offer a separate storage system to use for testing where applications could have dev buckets for ensuring things still work when we upgrade things on the storage or identity side.