Skip to main content
Join our research panel and help shape the future of Teradata.Sign up.

Create Parquet files in object storage

Overview​

Native Object Storage (NOS) is a feature of Teradata that allows you to query data stored in external object storage, including CSV, JSON, and Parquet files.

These datasets can be located in external object storage such as Amazon S3, Google Cloud Storage, Azure Blob Storage, or on-premises object storage.

NOS is useful when you want to explore external data without first building a data pipeline to load it into Teradata. This tutorial demonstrates how to export data from Teradata to object storage in Parquet format.

Prerequisites​

You need access to a Teradata instance. NOS is available in Teradata editions starting from version 17.10.

info

This tutorial uses Amazon S3 object storage. To complete the tutorial, you need an S3 bucket with write permissions.

Note

You can now get a hosted instance of Vantage for free at https://www.teradata.com/try.

Create a Parquet file with WRITE_NOS function​

WRITE_NOS allows you to extract selected or all columns from a database table or query results and write them to external object storage, such as Amazon S3, Azure Blob Storage, Azure Data Lake Storage Gen2, and Google Cloud Storage. This functionality writes data in Parquet format.

You can find more documentation about the WRITE_NOS functionality in the NOS documentation.

You need access to a database where you can execute the WRITE_NOS function. If you don't have such a database, run the following commands:

Note

If you would like to learn more about setting up users and their privileges, check out the NOS documentation.

  1. First create a table on your Teradata instance:
  1. Populate your table with example data:

Your table should now look like this:

  1. Create the Parquet file with WRITE_NOS. Replace <BUCKET_NAME> with the name of your S3 bucket. Also, replace <YOUR-ACCESS-KEY-ID> and <YOUR-SECRET-ACCESS-KEY> with your access key and secret.
Note

Check your cloud provider documentation to learn how to create credentials to access object storage. For example, for AWS check out How do I create an AWS access key?

Note

If you are using temporary AWS credentials, include the session token in the AUTHORIZATION string:

Now you have created a Parquet files in your object storage bucket. To query the files, follow step 4.

  1. Create an authorization object. Replace <YOUR-ACCESS-KEY-ID> and <YOUR-SECRET-ACCESS-KEY> with your access key and secret:
  1. Create a NOS-backed foreign table. Replace <BUCKET_NAME> with the name of your S3 bucket:
  1. Query the Parquet files on NOS:

The data returned from the query should look something like this:

Summary​

In this tutorial, you learned how to export data from Teradata to object storage in Parquet format using Native Object Storage (NOS). NOS supports reading data stored in CSV, JSON, and Parquet formats. NOS can also export data from Teradata to object storage.

Further reading​

Also of interest