Implementing a Python microframework from scratch - Part 1

In this series of BlogPosts we'll be making our very own python microframework from scratch. We'll take baby steps and build our framework step by step, making it better and adding more features at each step.

In the part 1 of this series, we'll first see what things are already available for us to write a very basic HTTP Handler. What python provides and how we can used it for this purpose. We'll also look into what are the shortcomings of these tools and how we can solve them while building our framework.

Inspirations

The framework that we'll end up building will be inspired from the good things borrowed from 2 of the most famous frameworks:

  1. Django (we will incline towards DRF instead of pure Django)
  2. Flask
  3. Tornado

What to expect

  • We'll start with a bare-bones version of code using only what is available in python. We'll not be relying on any external packages.
  • Next we'll provide better api's around our code better suited for use as library.
  • Next we'll implement a routing library with styles inspired from flask, and injecting URL params into our handler functions.
  • Request context and query params parsing and injecting it inside our handler function.
  • Proper mechanism for writing back responses by introducing a Response class.
  • Make our framework production deploy-able by introducing a mild redesign and providing a plug-and-play support with uWSGI and gunicorn

Let's dive in

For the purpose of this blog post, we'll be relying on 2 classes for implementing a simple HTTP endpoint.

  1. HTTPServer
  2. BaseHTTPRequestHandler

If you have written API's using either Django/flask/Tornado, let me tell you - both of the above have a very obscure API that in my opinion, no programmer would want to work with. Also this API is not very extensible, hence arises the need of frameworks.

So lets start.

We'll start by using the HTTPServer class – refering to documentation, this is how the call signature of this class looks like

HTTPServer(server_address, RequestHandlerClass)

  • The server_address needs to be a tuple containing a string (the address) and an integer (the port).
  • The RequestHandlerClass is the class which we will extend to implement our handler function (this will use the BaseHTTPRequestHandler)

Let's go ahead and create a file called main.py and add the following content into it:

from http.server import HTTPServer, BaseHTTPRequestHandler

s = HTTPServer(('0.0.0.0', 8000), BaseHTTPRequestHandler)
s.serve_forever()

This will make our HTTP server bind to localhost:8000 i.e. localhost on port 8000. If we hit http://localhost:8000/ right now, we'll get 501

If you look at what HTTP 501 error states:

indicates that the server does not support the functionality required to fulfill the request. This is the appropriate response when the server does not recognize the request method and is not capable of supporting it for any resource.

This error came up because we didn't implement any GET handler for our server. We will do that now.

Our first handler

Lets create a package named rip with the following structure:

./rip
├── __init__.py
└── server.py

We'll add our code to server.py:

from http.server import BaseHTTPRequestHandler
from http import HTTPStatus


class API(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(HTTPStatus(200))
        self.end_headers()
        self.wfile.write(b'Hello World!')

The methods used can be looked up on the python documentation:

  1. send_response
  2. end_headers
  3. wfile

Notice the order of methods

  • First we set the response status
  • Then we end the headers
  • Then we write body

Some people may find this a bit obscure (like setting response code first) but this totally aligns with the http spec. A typical http response would look like this (refer here for spec):

$ http GET http://httpbin.org/get
HTTP/1.1 200 OK                                                 # <---------- first the status code (send_response)
Access-Control-Allow-Credentials: true                          # -----------
Access-Control-Allow-Origin: *                                  #           |
Connection: keep-alive                                          #           |
Content-Encoding: gzip                                          #           |
Content-Length: 176                                             #           |- HTTP Headers
Content-Type: application/json                                  #           |
Date: Tue, 09 Apr 2019 09:37:23 GMT                             #           |
Server: nginx                                                   # -----------
                                                                # <---------- Blank like indicating end of headers (end_headers)
{                                                               #_
    "args": {},                                                 # \
    "headers": {                                                #  \
        "Accept": "*/*",                                        #   \
        "Accept-Encoding": "gzip, deflate",                     #    \
        "Host": "httpbin.org",                                  #     \____ HTTP Body (wfile.write())
        "User-Agent": "HTTPie/1.0.2"                            #     /
    },                                                          #    /
    "origin": "118.185.3.65, 118.185.3.65",                     #   /
    "url": "https://httpbin.org/get"                            #  /
}                                                               #_/

Back in our main file we import the above defined handler and pass it to our server: main.py

from http.server import HTTPServer
from rip.server import API


s = HTTPServer(('0.0.0.0', 8000), API)

s.serve_forever()

Let's test this now

$ http GET localhost:8000
HTTP/1.0 200 OK
Date: Tue, 09 Apr 2019 10:19:30 GMT
Server: BaseHTTP/0.6 Python/3.7.3

Hello World!

You can also try to see this in a browser by going to localhost:8000

These were very basic steps, in the next post we'll raise the bar and look into implementing a better API for our handlers.