model-registry/DOCS.md

jrjModelRegistry documentation

Tutorial youtube

Introducing jrjModelRegistry – a streamlined Python module designed for data scientists and developers who seek a straightforward solution for model saving and serving.

While numerous tools are available for training machine learning models, many lightweight statistical models lack simple, efficient saving mechanisms. Existing enterprise solutions like MLflow are robust but come with considerable complexity. Based on my experience, I saw the need for an abstract model registry concept that simplifies this process.

jrjModelRegistry fills this gap, offering an intuitive way to save machine learning models and transformers. It facilitates seamless integration with frameworks like FastAPI (Examples), Flask, and Django, enabling easy deployment and serving of models in production environments. Empower your machine learning workflow with jrjModelRegistry – the easy and efficient tool for model management.

Demo

curl --location 'https://ml.jasonjafari.com/models/list'

result

[
    "logisticRegYFromX1AndX2ModelFit",
    "salaryBasedOnGpaMisStatistics_Transfoms_misXStatisticsFit",
    "modelPredictSaleByTemperatureAdvertisingDiscountFit",
    "wageEducAgePower2ModelFit",
    "logRentWithBedsandLogSqftFit",
    "spamBasedOnRecipientsHyperlinksCharactersLogitModelFit",
    ...
]

demo example 1

modelPredictSaleByTemperatureAdvertisingDiscountFit train notebook

Model info

curl --location 'https://ml.jasonjafari.com/model/info/modelPredictSaleByTemperatureAdvertisingDiscountFit'

result

{
    "modelName": "modelPredictSaleByTemperatureAdvertisingDiscountFit",
    "description": "modelPredictSaleByTemperatureAdvertisingDiscountFit",
    "modelType": "sm.OLS",
    "inputs": [
        {
            "name": "Temperature",
            "type": "float"
        },
        {
            "name": "Advertising",
            "type": "float"
        },
        {
            "name": "Discount",
            "type": "float"
        }
    ],
    "outputs": [
        {
            "name": "Sales",
            "type": "float"
        }
    ]
}

predict

curl --location 'https://ml.jasonjafari.com/model/predict' \
--header 'Content-Type: application/json' \
--data '{
    "name": "modelPredictSaleByTemperatureAdvertisingDiscountFit",
    "inputs": [
        {
            "Temperature": 42,
            "Advertising": 15,
            "Discount": 5
        }
    ]
}'

result

[
    {
        "Sales": 19590.467270313893
    }
]

demo example2 train Notebook * interaction transformer

salaryBasedOnGpaMisStatistics_Transfoms_misXStatisticsFit

# info
curl --location 'https://ml.jasonjafari.com/model/info/salaryBasedOnGpaMisStatistics_Transfoms_misXStatisticsFit'

# predict
curl --location 'https://ml.jasonjafari.com/model/predict' \
--header 'Content-Type: application/json' \
--data '{
    "name": "salaryBasedOnGpaMisStatistics_Transfoms_misXStatisticsFit",
    "inputs": [
        {
            "GPA": 3.53,
            "MIS": 1,
            "Statistics": 0
        }
    ]
}'

demo example3 train Notebook * quadratic eq transformer

wageEducAgePower2ModelFit

# info
curl --location 'https://ml.jasonjafari.com/model/info/wageEducAgePower2ModelFit'

# predict
curl --location 'https://ml.jasonjafari.com/model/predict' \
--header 'Content-Type: application/json' \
--data '{
    "name": "wageEducAgePower2ModelFit",
    "inputs": [
        {
            "Educ": 12,
            "Age": 76
        }
    ]
}'

demo example4 train Notebook * log eq transformer for dependent adn independent attributes

logRentWithBedsandLogSqftFit

# info
curl --location 'https://ml.jasonjafari.com/model/info/logRentWithBedsandLogSqftFit'

# predict
curl --location 'https://ml.jasonjafari.com/model/predict' \
--header 'Content-Type: application/json' \
--data '{
    "name": "logRentWithBedsandLogSqftFit",
    "inputs": [
        {
            "Beds": 2,
            "Sqft": 900
        }
    ]
}'

demo example5 train Notebook * Logistic regression

logisticRegYFromX1AndX2ModelFit

# info
curl --location 'https://ml.jasonjafari.com/model/info/logisticRegYFromX1AndX2ModelFit'

# predict
curl --location 'https://ml.jasonjafari.com/model/predict' \
--header 'Content-Type: application/json' \
--data '{
    "name": "logisticRegYFromX1AndX2ModelFit",
    "inputs": [
        {
            "x1": 16.35,
            "x2": 49.44
        }
    ]
}'

demo example6 train Notebook * KNN

gymEnrollAgeIncomeHoursDfKnnFit

# info
curl --location 'https://ml.jasonjafari.com/model/info/gymEnrollAgeIncomeHoursDfKnnFit'

# predict
curl --location 'https://ml.jasonjafari.com/model/predict' \
--header 'Content-Type: application/json' \
--data '{
    "name": "gymEnrollAgeIncomeHoursDfKnnFit",
    "inputs": [
        {
            "Age": 26,
            "Income": 18000,
            "Hours": 14
        },
        {
            "Age": 55,
            "Income": 42000,
            "Hours": 16
        }
    ]
}'

Installation

You can install jrjModelRegistry via pip:

pip install jrjModelRegistry

jrjModelRegistry Example: Simple Linear Regression

In this example, we demonstrate how to use jrjModelRegistry to export a simple linear regression model based on a notebook from ml_models_deployments.

Example Description

This example builds a simple linear regression model to predict sales based on temperature, advertising, and discount factors. Once the model is fitted and satisfactory, jrjModelRegistry allows you to easily save and deploy the model for use in production environments.

Example Code - notebook available here

def add_constant_columnTransformer(df):
    # example transformer
    df_with_const = df.copy()
    df_with_const.insert(0, 'const', 1)
    return df_with_const

from jrjModelRegistry import jrjModelRegistry
jrjModelRegistryInstance = jrjModelRegistry({
    "baseRelativePath": "..",
    "modelsFolder": "models"
})

# Export the model using jrjModelRegistry
loadedModel = jrjModelRegistryInstance.exportModel(
     # the models is fitted and ready for usage
    simpleLinearRegressionFittedModel,
    {
        "modelName": "modelPredictSaleByTemperatureAdvertisingDiscountFit",
        "description": "Example model predicting sales based on temperature, advertising, and discount.",
        "modelType": "sm.OLS",  # Example model type (replace with actual type)
        "inputs": [
            {"name": "Temperature", "type": "float"},
            {"name": "Advertising", "type": "float"},
            {"name": "Discount", "type": "float"}
        ],
        "transformer": add_constant_columnTransformer,  # Use your transformation function here
        "outputs": [
            {
                "name": "Sales",
                "type": "float"
            }
        ]
    }
)

testData = [
    {
        "Temperature": 42,
        "Advertising": 15,
        "Discount": 5
    }
]

# Create a DataFrame from the dictionary
testDf = pd.DataFrame(testData)

# this will be your model without export
modelPredictSaleByTemperatureAdvertisingDiscountFit.predict( add_constant_column(testDf)

# result 0    19590.46727 \n dtype: float64

# this will be result of predict with exported model
loadedModel.mlModelSavePredict(testDf)
#result [{'Sales': 19590.467270313893}]

Supported Models

Current supported models by jrjModelRegistry:

supportedModels = {
    "sm.OLS": {
        "supported": True,
        "normalPredictorFunction": "predict"
    },
    "sm.Logit": {
        "supported": True,
        "normalPredictorFunction": "predict"
    },
    "sklearn.neighbors.KNeighborsClassifier": {
        "supported": True,
        "normalPredictorFunction": "predict_proba"
    },
    "sklearn.tree.DecisionTreeClassifier": {
        "supported": True,
        "normalPredictorFunction": "predict_proba"
    },
}

Next Steps

• [-] Support More Models WIP: Extend jrjModelRegistry to support various types of models beyond simple linear regression, such as decision trees, neural networks, and ensemble methods.

• [-] Additional Examples: Provide diverse examples demonstrating the use of jrjModelRegistry with different machine learning models and data preprocessing techniques.

• [] Video Tutorial: Create a comprehensive video tutorial demonstrating how to train models, use jrjModelRegistry for saving and deployment, and integrate with popular frameworks like Flask and FastAPI.

• [] Save Models to S3: Implement functionality to save models directly to Amazon S3 for scalable and reliable storage, ensuring robust deployment in cloud environments.

Support More Models

Expand jrjModelRegistry to handle a variety of machine learning models beyond simple linear regression. Example models could include decision trees, support vector machines, and deep learning models. Ensure that each model type integrates seamlessly with jrjModelRegistry's saving and deployment functionalities.

Save Models to S3

Enhance jrjModelRegistry to include options for saving models directly to Amazon S3. This feature ensures that models are stored securely and can be accessed from any location, facilitating deployment across distributed systems and cloud environments.

Additional Examples

Include a range of examples demonstrating jrjModelRegistry's capabilities across different use cases. Examples should cover various scenarios such as regression, classification, and time series forecasting. Provide clear instructions and code snippets for each example, showcasing how to prepare data, train models, and deploy them using jrjModelRegistry.

Video Tutorial

Produce a video tutorial that guides users through the entire process of using jrjModelRegistry. The tutorial should include steps for training a model, integrating with jrjModelRegistry for saving and loading, and deploying the model using popular web frameworks like Flask or FastAPI. Emphasize best practices and common pitfalls to help users maximize efficiency and reliability in their machine learning projects.

These next steps will enhance jrjModelRegistry's usability and scalability, enabling users to leverage advanced machine learning models effectively in production environments.