Deploying Models

Hyperstack Gen AI Platform supports open-source models that are deployed and could be queried per token.

Deploying models in Hyperstack Gen AI Platform is a straightforward process designed to make the transition from training to production seamless. Once you have successfully completed the training part of your AI models, they will be listed on the [Deploy page](https://www.Hyperstack Gen AI Platform.aideploy) in a 'Ready to Deploy' state.

Deploy with API

You can deploy a model by making a POST request to the deployment endpoint:

MODEL_NAME="your-model-name"
curl -X POST https://api.genai.hyperstack.cloud/tailor/v1/deploy_model \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "'$MODEL_NAME'"
  }'

Required Parameters:

• model_name: The name of the model you want to deploy

Check deployment status with API

To check the deployment status, you can make a GET request to the models endpoint:

MODEL_NAME="your-model-name"
curl -X GET "https://api.genai.hyperstack.cloud/tailor/v1/models/by_name/$MODEL_NAME" \
  -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json"

The expected response will look like this:

{
  "status": "success",
  "message": {
    "model_name": "your-model-name",
    "state": "deployed", // Check this value to confirm deployment status
    "base_model_id": 1,
    "created_at": "Tue, 29 Apr 2025",
    "last_used": "Wed, 30 Apr 2025",
    "base_model_data": {
      "display_name": "Mistral 7B Instruct (v0.3)",
      "model_type": "language_model",
      "supported_context_len": 8192,
      "supported_locations": ["default"]
    },
    "model_config": {
      "base_model": "mistral-7b-instruct-v0.3",
      "batch_size": 2,
      "learning_rate": 0.0002,
      "tags": ["v1"]
    },
    "deployment_records": [
      ["2025-04-29T10:32:03.354867+00:00", "2025-05-01T07:28:44.419329+00:00"]
      // ... additional deployment records ...
    ],
    "usage_data": [
      {
        "date": "2025-04-29T10:00:00Z",
        "tokens": 300
      }
      // ... additional usage records ...
    ]
  }
}

Check the state field in the response to confirm if your model is "deployed", "deploying", or in another state.

Undeploying with API

To undeploy a model that is currently deployed, you can make a POST request to the undeploy endpoint:

MODEL_NAME="your-model-name"
curl -X POST https://api.genai.hyperstack.cloud/tailor/v1/undeploy_model \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model_name": "'$MODEL_NAME'"
  }'

Required Parameters:

• model_name: The name of the model you want to undeploy

Response

A successful undeploy request will return:

{
  "status": "success",
  "message": "Undeploying model"
}

Deploy with UI

Viewing Trained Models

When you navigate to the "Deploy" page, you will see a list of all the models that you have trained. Each model is represented by a card that provides key information at a glance:

• Model Name: Identifies the model.
• Data Type: Describes the type of data used in the fine-tuning process.
• Base Model: Lists the base model that was fine-tuned.
• Training Date: Shows when the model was last trained.
• Deployment Status: Indicates whether the model is deployed (e.g., "Deployed" or "Ready to Deploy").
• Last Deployment Date: The most recent date when the model was deployed, if applicable.
• Last Used Date: The most recent date the model was used, if applicable.

Usage Analytics

Clicking on a model card provides detailed usage analytics for the selected model. If the model is or has been deployed, you will see a comprehensive usage chart displaying:

• Monthly Token Usage: A chart showing the number of tokens used per day for the current month.
• Token Usage Comparison: A comparison of token usage this month versus last month.
• Cost Calculation: The cost incurred for the tokens used during the month.

Deployment Actions

Deploying a Model

To deploy a dormant model, follow these steps:

1. Click the Toggle: Locate the deployment toggle switch on the model card and click it.
2. API Key Reminder: You will be reminded that you need an API key to use the deployed model.

Deleting a Model

If a model is not currently deployed, you have the option to delete it. This action is irreversible and will remove the model from your list.