Basics

Create app.py

from fastapi import FastAPI

app = FastAPI()

@app.get('/')
async def root():
    return {'message': 'Hello World!'} 

Add model and create an endpoint

from transformers import pipeline

model = pipeline('sentiment-analysis')

@app.get('/predict')
async def predict(q:str):
    return model(q)

Test endpoints

• If testing in Jupyter, you will need to run this first:

import nest_asyncio
nest_asyncio.apply()

• Then test using the TestClient

from fastapi.testclient import TestClient

def test_sentiment(q):
    with TestClient(app) as client:
        r = client.get(f'/predict?q={q}')
        sentiment = r.json()[0]['label']
        return sentiment
    
assert test_sentiment('Ice cream is delicious') == 'POSITIVE'
assert test_sentiment('Papaya is gross') == 'NEGATIVE'

• You can also spin up the actual server to test from the browser via jupyter:

import uvicorn
uvicorn.run(app)

• ...or shell:

  uvicorn app:app --reload
  

Deploy

Create docker image

1. In Dockerfile:

   FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7
   
    COPY ./app /app
   

2. Create a directory called app and add the app.py from above

   mkdir app
    mv app.py app/app.py
   

3. Build the container

   docker build -t model_app
   

4. Test the container

   docker run -d --name my_model_app -p 80:80 model_app
   

Deploy with Compose

# docker-compose.yml
version: 3.8
services:
    fastapi:
        ports:
            - 80:80

Deploy with Kubernetes

Generated from kompose

# fastapi-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.22.0 (955b78124)
creationTimestamp: null
labels:
    io.kompose.service: fastapi
name: fastapi
spec:
replicas: 1
selector:
    matchLabels:
    io.kompose.service: fastapi
strategy: {}
template:
    metadata:
    annotations:
        kompose.cmd: kompose convert
        kompose.version: 1.22.0 (955b78124)
    creationTimestamp: null
    labels:
        io.kompose.service: fastapi
    spec:
    containers:
        - image: fastapi
        name: fastapi
        ports:
            - containerPort: 80
        resources: {}
    restartPolicy: Always
status: {}

# fastapi-service.yaml
apiVersion: v1
kind: Service
metadata:
annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.22.0 (955b78124)
creationTimestamp: null
labels:
    io.kompose.service: fastapi
name: fastapi
spec:
ports:
    - name: "80"
    port: 80
    targetPort: 80
selector:
    io.kompose.service: fastapi
status:
loadBalancer: {}