The MVP of Machine Learning Models

[Thoughts on machine learning – 4]

The lean startup MVP applies nicely to Machine Learning software products, when we start we try to find our MVP of the models, which is the simplest, the most understandable, and the most economically built, we call it the baseline, we learn from that, then we either preserve or pivot.

ML Model in a Container

In this post, I will describe a recipe for ML Software Engineers who want to publish their ML model built with Python using FastAPI and containerize it using Docker. This will be one of the steps toward making the model ready to be deployed to the production environment whether it is on-premise or on the cloud.

FastAPI is a modern, high-performance Python web framework that’s perfect for building RESTful APIs. It can handle both synchronous and asynchronous requests and has built-in support for data validation, JSON serialization, authentication and authorization, and OpenAPI.

At the same time, Docker is a containerization technology that has changed the way we ship our applications forever. Now, we deploy containers that reuse most of the underlying host operating system and adds only the needed layers for our software to run!

Containers also help us to horizontally scale our application by starting and stopping additional instances depending on the current demand.

Create ML Model

We will not start from scratch but rather will use one of the datasets provided with the scikit-learn library and build a simple regression model on top of it.
We will take the Housing dataset which contains information about different houses in Boston. This data was originally a part of the UCI Machine Learning Repository. There are 506 samples and 13 feature variables in this dataset. The objective is to predict the value of prices of the house using the given features. Here is our simple linear regression to achieve that:

import pickle
import numpy as np
import sklearn.datasets as datasets
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split

def fit():
    house = datasets.load_boston()
    train_x, test_x, train_y, test_y = train_test_split(house.data, house.target, test_size=0.2, random_state=1)
    lr = LinearRegression()
    lr.fit(train_x, train_y)
    pickle.dump(lr, open('./models/lr_reg_boston.h5', 'wb'))
    return

def predict(x_to_predict):
    lm_pickled = pickle.load(open("./models/lr_reg_boston.h5", "rb"))
    result = lm_pickled.predict(np.reshape(x_to_predict, (1, -1)))
    return result

if __name__ == "__main__":
    fit()
    house_to_evaluate = [0.62739, 0., 8.14, 0., 0.538, 5.834, 56.5, 4.4986, 4., 307., 21., 395.62, 8.47]
    price = predict(house_to_evaluate)
    print(f"result of predciton of {house_to_evaluate} is {str(price[0])} 1000s USD")

Our code have two methods, fit and predict, fit method loads the dataset uses scikit-learn to split the data into train and test sets then it trains linear regression algorithm and saves the resulting model to the disk. The Predict method starts by loading the trained model and then pass in a sample house vector of features values to predict its price. Save this code to boston_housing.py and run it with python boston_housing.py to train and generate the model.

Publishing Linear Regression Model  with FastAPI

FastAPI is our API server of choice for the reasons we explained earlier. Here is the code:

import uvicorn
from fastapi import FastAPI
import boston_housing
#
app = FastAPI()
#
@app.get("/")
def index():
    return 'ML Model API is alive!'
#
@app.post("/predict")
def predict(house_to_evalute: list):
    prediction = boston_housing.predict(house_to_evalute)
    return str(prediction)

if __name__ == "__main__":
    uvicorn.run(app, host="0.0.0.0", port=8000)

The predict method basically handles http post type of requests, it expects the payload of the request to contain a vector of feature values, it then uses the model we trained and generate before to predict the price.

Containerize your FastAPI ML Model

Now, let’s create a Dockerfile for our model.  If our model runs correctly then all what we have to do is to write Dockerfile for it, this way we can use to build a docker image and then spin a container out of it.

We will keep our Dockerfile in the build context which is the folder containing the application and dependencies and it should read as the following:

FROM python:3.7.6
WORKDIR .
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY ./ .
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

Here we simply start from base image pyhon:3.7.6, on top of that we install requirements using pip install, we then copy all our files from current directory to the image root directory, we tell docker that we want to expose port 8000 from our image. Finally we run our application using CMD command.

One comment here is that in production you would like to use gunicorn instead of unvicorn, concequently we need to change the CMD to look like the following

Build and Run

Now we have created our Dockerfile we can build an Image for our model using docker build command as the following, we start your Docker desktop and on the root folder of our model we issue the following command from the termainal:

docker build -t housing_price_pred_model .

If we ls the docker images we find:

(venv) G:\0Yaser\repos\ml_model_containerized>docker image ls
REPOSITORY                        TAG                 IMAGE ID            CREATED             SIZE
housing_price_pred_model          latest              cc6b1383c057        52 seconds ago      1.29GB

And to run a container based on our image we use the following command:

docker run -d --name housing_price_pred_model_container -p 8000:8000 housing_price_pred_model

(venv) G:\0Yaser\repos\ml_model_containerized>docker container ls
CONTAINER ID        IMAGE                      COMMAND             CREATED             STATUS              PORTS                    NAMES
a181597082ed        housing_price_pred_model   "python app.py"     7 seconds ago       Up 6 seconds        0.0.0.0:8000->8000/tcp   housing_price_pred_model_container

Now we can go to http://localhost:8000 to check if our API server is alive:

We point to the automatically generated swagger documentation for our API’s on http://localhost:8000/docs

Finally we can test our predict API like:

Full code of this post is here: https://github.com/YaserMarey/ml_model_containerized

Salam,

Full-Text Search vs Question-Answering

[Thoughts on machine learning – 3]

You are successful in building an NLP Question-Answering system when you find it exhibiting the behavior of a full-text search that takes into account the meaning of the words in your query and the documents you’re searching.

Scrum 2020

Last November, Ken Schwaber and Jeff Sutherland issued a new and major update to Scrum Guide.

This version has a number of noticeable changes from Scrum 2017, here the most important changes in my opinion:

Less is More

Scrum is a framework, and this is emphasized in this version removing any implementation details and softening prescriptive language. One of the first things you will notice is the removal of Daily Scrum questions.

PO is Part of the Team

The Product Owner is not the new Requirements Engineer or the System Analyst. He or She is part of the team, this is stressed in this version of the guide and consequently, PO shares the responsibilities with developers and the scrum master.

New Concept of Product Goal

Set by the Product Owner, the Product Goal is another manifestation for the idea of Scrum as a framework whose objective is to give the team the focus and let the team figure out the details. Each sprint should bring the team closer to achieving the Product Goal.

2020-Scrum-Guide-ARDownload

Commitments

More transparency into progress through a set of commitments, one for each of the Scrum artifacts. In Scrum 2020 Guide, the Product Backlog has a Product Goal as a commitment, the Sprint Backlog has Sprint Goal as a commitment, and the Product Increment must fulfill its commitment which is the Definition of Done.

Self-Managing Team

Scrum self-organizing teams have the authority to choose who and how to do the work. This version of the Scrum Guide adds choosing “what” to work on from the product backlog each sprint to these authorities.

I have also just finished my translation to Arabic for this new version of the Scrum Guide. You can download it from here and soon it will be published on scrumguides.org

In this translation, with help from a number of interested colleagues, I put extra effort to make the Arabic text readable and understandable as much as possible.

Salam,

Humble machines

[Thoughts on machine learning – 2]

“All men by nature desire to know…” Aristotle. To be wise is to admit that you don’t know, you are not sure, that is to take probabilities into account! If we want machines to be wise, that is to exhibit intelligence, then machines should admit its ignorance! For me, computer programs written with a fixed heuristic set of rules are only stupid, those which send their feedback with a shy confidence are way more intelligent. 

The unreasonable effectiveness

[Thoughts on machine learning – 1]

“There’s something magical about Recurrent Neural Networks (RNNs)” Andrej Karapthy. Such statement is not very settling for me. How for a software engineer to write a test case for something magical! I don’t think we can afford to treat a ML model as a black box. Machine Learning Interpretability is crucial.

MNIST: The Neural Network Hello World Dataset

The great OCR was one of the earliest Neural Network applications. The utility of converting hand or machine written text from scanned documents to machine-encoded text format was and still huge.

In 1998 Yann LeCun, currently Chief AI Scientist at Facebook, was working at Bell Labs on an OCR that recognizes the set of digits from 0 to 9 to be deployed and used in banks, and post offices to recognize check numbers and zip codes.

The application was a success and as a by-product of their work LeCun, who later led the invention of Convolutional Neural Network (LeNet-5), he and his team produced MINST dataset.

The dataset, which contains 60,000 training and 10,000 testing different images of 0-9 handwritten digits, then led a life on its own and became the benchmark for supervised learning algorithms.

source (https://www.mdpi.com/2076-3417/9/15/3169)

The MNIST is created by modifying samples from NIST or The National Institute of Standards and Technology original datasets and hence the M in the name.

Almost 20 years later and in 2017 a group of researchers collected a drop-in replacement they called Fashion-MNIST which is a dataset of Zalando’s fashion articles ( shoes, bags, pants .. and who cares) images consisting again of a 60,000 training and a 10,000 testing images classified into 10 classes.

The Fashion-MNIST creators were bold enough to propose this dataset as a new benchmark for Computer Vision image classification algorithms claiming that MNIST is a solved problem and no longer an enough challenge, which I think true!

Extended MNIST or EMNIST is also introduced the same year 2017. This time the dataset includes both digits and letters.

Yet another variation following MNIST tradition is Kaggle American Sign Language MNIST presented also 2017. The dataset contains thousands of images of hand gestures represent for 24 English letters (excluding J and Z which require motion).

Experimenting with MNIST dataset is like the Hello World of Computer Vision, Image Classification and  Neural Networks, honoring this tradition I am collecting and experimenting with MINST, Fashion MNIST, EMINST and Sign Language MNIST mainly using TensoFlow but sometimes comparing implementations and performances with other frameworks as well.

Stay safe,

Salam

Survey of Three Chatbot Engines: Part – 4

This is the fourth post in a series of posts discussing the results of my survey of a three of the well known chatbots engines and my observations while trying to build a conversational sentiment-aware chatbot using them.

Wit.ai

Wit.ai was acquired by Facebook in 2015. Wit.ai is an NLP based chatbot development platform that supports many languages and is offered as a hosted service.

Labeeb Wit.ai Architecture

Labeeb Wit.ai Architecture

Since Wit.ai is proprietary technology there is not much published information about its architecture. Also, Wit.ai provides only NLU and it doesn’t provide any of the features of Dialog Management. Programming interfaces for Wit.ai are available in JavaScript and Python.

Labeeb built on top of Wit.ai is composed of:

1. Set of training examples for different entities that include all intents as well because Wit.ai doesn’t distinguish intents from entities.
2. Http Service running on Express HTTP Server listening to call back from Facebook Messenger input channel.
3. Custom Dialog Management which figures out what is the best response out of a set of utterances based on where we are in the dialog and what is recognized intent. It also, when required, investigates the sentiment analysis sent along with the request and customizes the response accordingly.
4. Facebook connector: responsible for registering webhooks, verifying request signature to make sure it is coming from an authentic source and for processing incoming messages from Facebook Messenger.

Machine Learning

Not much is published about Machine Learning algorithms employed by Wit.ai to perform NLP/NLU.

Sentiment Analysis: again and similar to RASA Wit.ai doesn’t provide sentiment analysis, however, using Facebook Messenger as an input channel, which obviously very well supported, avails NLU service from Facebook AI, in such case an additional sentiment data node in the request message is sent by Facebook Messenger through webhook back to Labeeb Wit.ai based chatbot, I can then use this information to customize the response the bot is sending back.

Tooling

Wit.ai has a very nice web interface through which chatbot developer can easily train the NLU engine by providing examples and highlighting the entities and their values.

Wit.ai User Interface

Developers Support

According to Wit.ai website, Wit.ai service is used by 200,000 developers worldwide. Documentation is reasonable, searching “Wit.ai” + tutorial results in 10,900 hits compared to 6000 hits for a similar search for Rasa and approx. 96000 for searching for Dialogflow + Tutorial.

 Integration

Wit.ai doesn’t provide ready connectors to any of the input channels however it provides Python, Node.JS, Ruby API’s in addition to Http endpoints.

Salam,

Survey of Three Chatbot Engines: Part – 3

This is the third post in a series of posts discussing the results of my survey of a three of the well known chatbots engines and my observations while trying to build a conversational sentiment-aware chatbot using them.

Dialogflow

Dialogflow is a platform owned by Google and offered as a hosted service to allow developers to build a conversational user interface for application and devices.

Labeeb Dialogflow Architecture

Labeeb Dialogflow Architecture

Dialogflow provides NLP, Dialog Management, and a wide spectrum of ready connectors to Google Assistant, and most of the well know messaging platforms. Dialogflow integration with Google assistant enables voice interaction with the developed chatbot out of the box. Integration with Google Assistant is provided with a single click and only minimal configuration is required.

Labeeb built using Dialogflow is composed of:

1. Set of intents, entities and the associated training examples for each of them. Dialogflow documentation recommends 10-15 training examples per intent, however, I used an amount of training examples similar to other implementations to try to have a fair comparison between the bots.
2. Set of conversation contexts, which helps the engine track the dialog and select the right response.
3. Fulfillment, which is the custom code that is plugged into each intent and executed by the engine before sending the response back to the input channel. This fulfillment is a JavaScript code that in case of Labeeb is defined inline within Dialogflow, it can be also run on a separate server and then called when needed. The fulfillment code is also responsible for investigating the sentiment analysis sent along with the request and customizes the response accordingly.
4. Integration with Google Assistant.

Again and similar to Wit.ai, not much is published about NLP/NLU algorithms used by Dialogflow.

Sentiment Analysis: Dialogflow offers sentiment analysis readily available within the request itself however it is enabled only for enterprise plans, not on the free one. Also when testing the Labeeb Dialogflow chatbot integrated with Google Assistant I found that sentiment analysis data is not passed, I opened a case with Dialogflow support but till the time of writing this report I didn’t receive their feedback yet.

Tooling

Dialogflow is by far the richest environment with tools. For example test console is panel is available in the main page itself, diagnosis tool that developer can use to look into the request and response JSON is also there and inline code editor for fulfillment. Tooling also extends to cover analytics and a rich set of prebuilt bots that developer can install and customize.

Dialogflow User Interface

Developers Support

Dialogflow has the largest community, over 800,000 developers, also a huge number of online resources, approx. 96000 hits when searching for Dialogflow + Tutorial, and books.

Integration

Dialogflow offers a comprehensive set of integration connectors with messaging platforms including a unique connector for telephony and also integration with Amazon Lex.

Large set of Connectors

Salam,

Research Papers Nouveau

Recently I encountered another interesting idea when it comes to writing a research paper, distill.pub provides a new way to write a research peer-reviewed paper which instead of using rigid PDF, it actually uses HTML and encourages researchers to present their ideas as an interactive document where you can see graphs and equations and possibly change them and see results inside the paper itself!

This is, in fact, the second flavor of “writing a research paper as a Jupiter Notebook” where instead of writing about your experiment what is better than putting your experiment code and allowing the readers to run the code and generate the graphs inline within the text of your paper.