Project data via S3

We have a Minio instance that provides S3 compatible object storage. VTTI uses Minio to provide an S3 endpoint for our project folders. Each project is exposed as a seperate S3 bucket.

Generating credentials

Temporary credentials can be generated to access VTTI's S3 project folders by going to s3-login.cloud.vtti.vt.edu and logging using your VTTI credentials. After sign in you will be presented with an AccessKeyID, SecretAccessKey, and a SessionToken which can be used to authenticate to VTTI's Minio instance.

Accessing project data

After generating temporary credentials, project data can be accessed via one of Minio's SDKS or via CLI using Minio's mc application. Minio provides documentation for both the mc command and several SDKS. In all cases you will need the temporary credentials (AccessKeyID, SecretAccessKey, SessionToken) as well as the VTTI project number and the S3 endpoint URL minio-frontend.gateway.cloud.vtti.vt.edu

Python SDK

Minio provides an API Reference for Python clients. This example program connects to the VTTI Minio instance and uploads a text file to a bucket.

from minio import Minio
from minio.error import S3Error


def main():
    # Create a client with S3 credentials.
    client = Minio(
            "minio-frontend.gateway.cloud.vtti.vt.edu",
        access_key="Q81BCVLQ59K5I5QMI8S1",
        secret_key="bBxEu5d+UxkxY0FNMbgB563P+kXVQSo9txlZ4gpY",
        session_token="eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhY2Nlc3NLZXkiOiJRODFCQ1ZMUTU5SzVJNVFNSThTMSIsImFjciI6IjAiLCJhdF9oYXNoIjoiNHN3RUdpd1UwODBpd0VULTdTNFVydyIsImF1ZCI6Im1pbmlvIiwiYXV0aF90aW1lIjoxNjQ2MTY4MDA4LCJhenAiOiJtaW5pbyIsImVtYWlsIjoia3N0cmlja2xlckB2dHRpLnZ0LmVkdSIsImVtYWlsX3ZlcmlmaWVkIjp0cnVlLCJleHAiOjE2NDYxNzE3ODUsImZhbWlseV9uYW1lIjoiU3RyaWNrbGVyIiwiZ2l2ZW5fbmFtZSI6Iktlbm55IiwiaWF0IjoxNjQ2MTY4MTg2LCJpc3MiOiJodHRwczovL2tleWNsb2FrLmNsb3VkLnZ0dGkudnQuZWR1L2F1dGgvcmVhbG1zL21hc3RlciIsImp0aSI6IjRjNDA4Yzk0LTEzYWMtNGE5NS05MDAwLTQ5YzBmMmQ5NjhmNiIsIm5hbWUiOiJLZW5ueSBTdHJpY2tsZXIiLCJwb2xpY3kiOiJjb25zb2xlQWRtaW4iLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJrc3RyaWNrbGVyIiwic2Vzc2lvbl9zdGF0ZSI6IjU1MzA1ZjlkLTBmNmMtNGJjMy1iNjFhLWExNWJiYTU0YTMxOCIsInNpZCI6IjU1MzA1ZjlkLTBmNmMtNGJjMy1iNjFhLWExNWJiYTU0YTMxOCIsInN1YiI6IjViZmRlNWZmLWI4NTctNDAzNy1hOWQ1LWU5MzgxM2MyMzQ3NyIsInR5cCI6IklEIn0.jXYbQezdzfq5b9096CQrFcpq3dwD4aOknxEHMOXTXgsjJV7eDdWDbbCpbbkkHbieBNer1C0_EzWnCtSBhbbA3A",
        secure=True
    )

    # Upload test file
    client.fput_object(
        "my_project_number", "test.txt", "/home/user/python-minio/test.txt",
    )
    print(
        "'/home/user/python-minio/test.txt' is successfully uploaded as "
        "object 'test.txt' to bucket 'my_project_number'."
    )


if __name__ == "__main__":
    try:
        main()
    except S3Error as exc:
        print("error occurred.", exc)