To deploy the serverless endpoint in SageMaker we need to create our own container and models.

Follow the below steps to create a serverless endpoint in SageMaker.

Clone the repository https://github.com/episyche/AWS_SageMaker_serverless_with_custom_model.git in your local directory. Then open terminal and move into the repository.

Step 1: Push the docker image to AWS ECR

We need to create a Private repositories in AWS ECR.

Go to https://aws.amazon.com/ecr/

Proceed to Private registry → Repositories → Create repository as in the below image.

Now enter the Repository name in the Create repository form which you’re shown.

Click the Create repository button at the bottom of the screen.

Now you can see that the repository is created.

Now click on the repository you’ve just created, and you’ll see the View push commands button on the top right.

Now, a pop named Push commands for custom_sagemaker_container will be shown. We need to run these commands to build and push the container to the ECR.

Make sure to run these commands inside our local git repository.

Refresh the AWS ECR page, and you should see that the image has been pushed to the ECR repository.

Now click the Repositories on the left panel and copy the image URI we created. It should be like 365492719930.dkr.ecr.ap-south-1.amazonaws.com/custom_sagemaker_container

Step 3: Create model in SageMaker

Go to AWS SageMaker, and scroll to the bottom through the left pane and go to Inference → Models → Create model

Enter the Model name.

Now paste the ECR image URI we copied in the step 1 into the Location of inference code image.

Now click the Create model button on the bottom of the page and you can see the Model is created as below.

Now click on the Endpoint configurations below the Inference on the left panel and then click the Create endpoint configuration button.

Enter the name for Endpoint configuration and choose the Type of endpoint as Serverless.

Then click on the Create production variant on the Variants dialog.

Choose the Custom-Model and click the save button.

Now click on the Create endpoint configuration button on the button of the page and you can see the endpoint configuration is created as below.

Now click on the Create endpoint button shown on the above image.

In the Create and configure endpoint dialog enter the name for the endpoint.

Then choose Use an existing endpoint configuration in Attach endpoint configuration

Now choose the custom-endpoint-configuration we created before in Endpoint configuration and click the Select endpoint configuration button.

Next, click the 'Create endpoint' button at the bottom of the page, and you will observe that the endpoint has been successfully created as shown.

AWS SageMaker serverless with custom model