Skip to main content
Join our research panel and help shape the future of Teradata.Sign up.

Use Amazon SageMaker AI with Teradata

Overview​

This how-to will help you to integrate Amazon SageMaker AI with Teradata. The approach this guide explains is one of many potential approaches to integrate with the service.

Amazon SageMaker AI provides a fully managed Machine Learning Platform. There are two use cases for Amazon SageMaker AI and Teradata:

  1. Data resides on Teradata and Amazon SageMaker AI will be used for both the Model definition and subsequent scoring. Under this use case Teradata will provide data into an Amazon S3 bucket so that Amazon SageMaker AI can consume training and test data sets for the purpose of model development. Teradata would further make data available via Amazon S3 for subsequent scoring by Amazon SageMaker AI. Under this model Teradata is a data repository only.

  2. Data resides on Teradata and Amazon SageMaker AI will be used for the Model definition, and Teradata for the subsequent scoring. Under this use case Teradata will provide data into an Amazon S3 bucket so that Amazon SageMaker AI can consume training and test data sets for the purpose of model development. Teradata will need to import the Amazon SageMaker AI model into a Teradata table for subsequent scoring. Under this model Teradata is a data repository and a scoring engine.

The first use case is discussed in this document.

Amazon SageMaker AI consumes training and test data from an Amazon S3 bucket. This article describes how you can load Teradata analytics data sets into an Amazon S3 bucket. The data can then be available to Amazon SageMaker AI to build and train machine learning models and deploy them into a production environment.

Prerequisites​

  • Access to a Teradata instance.
    Note

    If you need a test instance of Teradata, you can provision one for free at https://www.teradata.com/try

  • IAM permissions to access the Amazon S3 bucket and use Amazon SageMaker AI.
  • An Amazon S3 bucket to store training data.

Load data​

Amazon SageMaker AI trains machine learning models using data stored in an Amazon S3 bucket. Follow these steps to load training data from Teradata into an Amazon S3 bucket:

  1. In the Amazon SageMaker AI console, select Applications and IDEs -> Notebooks -> Notebook instances, and then select Create notebook instance. See the Amazon SageMaker AI Developer Guide for instructions on how to create a notebook instance:

Create notebook instance

  1. Open your notebook instance: Open notebook instance

  2. Start a new file by clicking on New -> conda_python3: Start new file

  3. Install Teradata Python library:

  4. In a new cell and import additional libraries:

  5. In a new cell, connect to Teradata. Replace <hostname>, <database user name>, <database password> to match your Teradata environment:

  6. Retrieve data from the table where the training dataset resides using TeradataML DataFrame API:

    Note

    For the XGBoost training job used in this how-to, place the target column first and encode categorical values as numeric values before exporting the data. The CSV file must not contain a header row.

  7. Write data to a local file:

  8. Upload the file to Amazon S3. Replace <s3_bucket_name> with the name of your Amazon S3 bucket:

Train the model​

  1. In the Amazon SageMaker AI console, select Model training & customization -> Training & tuning jobs -> Training jobs from the left menu, and then select Create training job: Create training job

  2. In the Create training job window, enter a job name, such as xgboost-bank. For the IAM role, select an existing SageMaker execution role with access to the Amazon S3 bucket, or select Create a new role. When creating a new role, choose Any S3 bucket, and then select Create role: Create IAM role

  3. For the algorithm, select Tabular – XGBoost : v1.3: Choose an algorithm

  4. Under Resource configuration, use the following values:

    • Instance type: ml.m4.xlarge
    • Instance count: 1
    • Storage volume: 30 GB
    • Maximum runtime: 1 hour
    • Keep alive period: 0 seconds

    This is a short training job and should typically complete within 10 minutes. Resource configuration

  5. Enter the following hyperparameters and leave the remaining settings at their default values:

  6. Under Input data configuration, use the following values:

    • Channel name: train
    • Input mode: File
    • S3 data type: S3Prefix
    • Distribution: FullyReplicated
    • Data format: csv
    • S3 location: The Amazon S3 prefix containing train.csv Input data configuration

  7. Under Output data configuration, enter the Amazon S3 location where Amazon SageMaker AI will store the model artifacts:

Output data configuration

  1. Leave the remaining settings at their default values and select Create training job. For detailed instructions on configuring a training job, see the Amazon SageMaker AI Developer Guide.

After the training job is created, Amazon SageMaker AI launches the ML instance and trains the model. When the training job is complete, Amazon SageMaker AI stores the resulting model artifact in the configured Amazon S3 output location:

<output-path>/<training-job-name>/output/model.tar.gz

Deploy the model​

After you train the model, deploy it to a real-time endpoint.

Create a model​

  1. In the Amazon SageMaker AI console, select Deployments & inference -> Deployable models from the left menu, and then select Create model.

  2. Enter a model name, such as xgboost-bank, and select the IAM role created or used for the training job.

  3. Under Container definition 1, configure the following:

    • For Container input options, select Provide model artifacts and inference image location.

    • Select Use a single model.

    • For Location of inference code image, enter the XGBoost 1.3-1 inference image URI for your AWS Region. For information about retrieving an XGBoost image URI, see How to use SageMaker AI XGBoost.

    • For Location of model artifacts, enter the complete Amazon S3 path to the model artifact generated by the training job:

    • For Model compression type, select CompressedModel.

Container definition 1

  1. Leave the remaining settings at their default values and select Create model.

Create an endpoint configuration​

  1. Select the model you just created, and then select Create endpoint configuration:

Create endpoint configuration

  1. Enter a name, such as xgboost-bank, and leave the remaining settings at their default values. The model is automatically populated under Variants. Select Create endpoint configuration.

Create endpoint​

  1. In the Amazon SageMaker AI console, select Deployments & inference -> Deployable models from the left menu. Select the model again, and then select Create endpoint:

Create endpoint

  1. Enter a name, such as xgboost-bank, and select Use an existing endpoint configuration:

Attach endpoint configuration

  1. Select the endpoint configuration created in the previous step, and then select Select endpoint configuration:

Select endpoint configuration

  1. Leave the remaining settings at their default values and select Create endpoint.

Wait until the endpoint status changes to InService. The model is then deployed to the endpoint and can be used by client applications.

caution

A real-time endpoint incurs charges while it is running. Delete the endpoint when you no longer need it.

Summary​

This how-to demonstrated how to extract training data from Teradata and use it to train a model in Amazon SageMaker AI. The solution used a Jupyter notebook to extract data from Teradata and upload it to an Amazon S3 bucket. An Amazon SageMaker AI training job read the data from the Amazon S3 bucket and produced a model. The model was then deployed to an Amazon SageMaker AI real-time endpoint.

Further reading​

Also of interest