Tutorial: Link Shortener¶

Note

This document continues from where the first part left off. As in the first part, we proceed by developing an example application step by step. We suggest that you code along and try out the various stages of the application. In this manner, completing it should take about an hour.

The first part of the tutorial covered some basic topics like routing, form handling, static assets, and JSON endpoints. This second part will show examples for resource handling, redirection, errors, and middleware usage.

The example application will be a link shortener. There will be an option for letting shortened links expire, based on time or on the number of clicks. Users can select the shortened names (aliases) themselves, or let the application generate one. Expired aliases will not be reusable.

A screenshot of the application is shown below:

Application screenshot showing link addition and listing form.

The user can fill in a form to create a new link, or view recorded links. The first and last recorded links are autogenerated, whereas the second one is user-supplied.

For the sake of simplicity, we’ll use the shelve module in the Python standard library as our storage backend. A stored link entry will consist of the target URL, the alias, the time when the link will expire, the maximum number of clicks, and the current number of clicks. The alias will be the key, and the full link data will be the value. Below is a simple implementation (file storage.py), without alias generation and link expiration features:

import os
import shelve
import time


class LinkDB:
    def __init__(self, db_path):
        self.db_path = db_path
        if not os.path.exists(self.db_path):
            with shelve.open(self.db_path, writeback=True) as db:
                db["last_id"] = 41660
                db["entries"] = {}

    def add_link(self, target_url, alias=None, expiry_time=0, max_count=0):
        with shelve.open(self.db_path, writeback=True) as db:
            now = time.time()
            entry = {
                "target": target_url,
                "alias": alias,
                "expires": now + expiry_time if expiry_time > 0 else 0,
                "max_count": max_count,
                "count": 0,
            }
            db["entries"][alias] = entry
        return entry

    def get_links(self):
        with shelve.open(self.db_path) as db:
            entries = db["entries"].values()
        return entries

    def use_link(self, alias):
        with shelve.open(self.db_path, writeback=True) as db:
            entry = db["entries"].get(alias)
            if entry is not None:
                entry["count"] += 1
        return entry

The expiry time is given in seconds. Expiry values of zero for both time and clicks means that the link will not expire based on that property. It’s also worth noting that the .add_link() method returns the newly added link. Since alias generation isn’t implemented yet, the users will have to enter aliases themselves.

Getting started
Resources
Redirection
Named path segments
Errors
Using middleware
Cookies

Getting started ¶

Let’s jump right in and start with the following template:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <title>Erosion</title>
    <link rel="stylesheet" href="/static/style.css">
  </head>
  <body>
    <main class="content">
      <h1>Erosion</h1>
      <p class="tagline">Exogenic linkrot for limited sharing.</p>

      <section class="box">
        <h2>Create a URL</h2>
        <form method="POST" action="/submit" class="new">
          <p class="target">
            <label for="target_url">Web URL:</label>
            <input type="text" name="target_url">
          </p>

          <p>
            <label for="new_alias">Shortened as:</label>
            <span class="input-prefix">{host_url}</span>
            <input type="text" name="new_alias">
            <span class="note">(optional)</span>
          </p>

          <p>
            <label for="expiry_time" class="date-expiry-l">Time expiration:</label>
            <input type="radio" name="expiry_time" value="300"> five minutes
            <input type="radio" name="expiry_time" value="3600"> one hour
            <input type="radio" name="expiry_time" value="86400"> one day
            <input type="radio" name="expiry_time" value="2592000"> one month
            <input type="radio" name="expiry_time" value="0" checked> never
          </p>

          <p>
            <label for="max_count">Click expiration:</label>
            <input type="number" name="max_count" size="3" value="1">
          </p>

          <button type="submit">Submit</button>
        </form>
      </section>

      {?entries}
      <section>
        <h2>Recorded URLs</h2>
        <ul>
          {#entries}
          <li>
            <a href="{host_url}{.alias}">{host_url}{.alias}</a> &raquo; {.target} -
            <span class="click-count"> ({.count} / {.max_count} clicks)</span>
          </li>
          {/entries}
        </ul>
      </section>
      {/entries}
    </main>
  </body>
</html>

This template consists of two major sections: one for adding a new entry, and one for listing recorded entries. It expects two items in the render context:

host_url for the base URL of the application
entries for the shortened links stored in the application

And now for the application code:

import os

from clastic import Application
from clastic.render import AshesRenderFactory
from clastic.static import StaticApplication


CUR_PATH = os.path.dirname(os.path.abspath(__file__))
STATIC_PATH = os.path.join(CUR_PATH, "static")


def home():
    return {"host_url": "http://localhost:5000", "entries": []}


def create_app():
    static_app = StaticApplication(STATIC_PATH)
    routes = [
        ("/", home, "home.html"),
        ("/static", static_app),
    ]
    render_factory = AshesRenderFactory(CUR_PATH)
    return Application(routes, render_factory=render_factory)


app = create_app()

if __name__ == "__main__":
    app.serve()

This is a very simple application that doesn’t do anything that wasn’t covered in the first part of the tutorial. Apart from the static assets, the application has only one route. and its endpoint provides an initial context for the given template.

Resources ¶

The first issue we want to solve is that of passing the host URL to the template because the application will not run on localhost in production. To achieve this, we need a way of letting the endpoint function get the host URL, so that it can put it into the render context. Clastic lets us register resources with the application; these will be made available to endpoint functions when requested.

Let’s start by adding a simple, ini-style configuration file named erosion.ini, with the following contents:

[erosion]
host_url = http://localhost:5000

Now we can read this file during application creation:

from configparser import ConfigParser


def create_app():
    static_app = StaticApplication(STATIC_PATH)
    routes = [
        ("/", home, "home.html"),
        ("/static", static_app),
    ]

    config_path = os.path.join(CUR_PATH, "erosion.ini")
    config = ConfigParser()
    config.read(config_path)

    host_url = config["erosion"]["host_url"].rstrip("/") + "/"
    resources = {"host_url": host_url}

    render_factory = AshesRenderFactory(CUR_PATH)
    return Application(routes, resources=resources, render_factory=render_factory)

The application resources are kept as items in a dictionary (resources in the example). After getting the host URL from the configuration file, we put it into this dictionary, which then gets registered with the application during application instantiation.

Endpoint functions can access application resources simply by listing their dictionary keys as parameters:

def home(host_url):
    return {"host_url": host_url}

Let’s apply a similar solution for passing the entries to the template. First, add an option to the configuration file:

[erosion]
host_url = http://localhost:5000
db_path = erosion.db

Next, add the database connection to the application resources:

from storage import LinkDB


def create_app():
    static_app = StaticApplication(STATIC_PATH)
    routes = [
        ("/", home, "home.html"),
        ("/static", static_app),
    ]

    config_path = os.path.join(CUR_PATH, "erosion.ini")
    config = ConfigParser()
    config.read(config_path)

    host_url = config["erosion"]["host_url"].rstrip('/') + '/'
    db_path = config["erosion"]["db_path"]
    if not os.path.isabs(db_path):
        db_path = os.path.join(os.path.dirname(config_path), db_path)
    resources = {"host_url": host_url, "db": LinkDB(db_path)}

    render_factory = AshesRenderFactory(CUR_PATH)
    return Application(routes, resources=resources, render_factory=render_factory)

And finally, use the database resource in the endpoint function:

def home(host_url, db):
    entries = db.get_links()
    return {"host_url": host_url, "entries": entries}

Redirection ¶

Let’s continue with creating new shortened links. The new link form submits its data to the /submit path. The endpoint function for this path has to receive the data, and add the new entry to the database. Once this is done, we don’t want to display another page, we want to redirect the visitor back to the home page. Since the home page lists all entries, we should be able to see our newly created entry there. We use the redirect() function for this:

from clastic import redirect
from http import HTTPStatus


def add_entry(request, db):
    target_url = request.values.get("target_url")
    new_alias = request.values.get("new_alias")
    expiry_time = int(request.values.get("expiry_time"))
    max_count = int(request.values.get("max_count"))
    entry = db.add_link(
        target_url=target_url,
        alias=new_alias,
        expiry_time=expiry_time,
        max_count=max_count,
    )
    return redirect("/", code=HTTPStatus.SEE_OTHER)

What’s left is adding this route to the application. If an endpoint function directly generates a response -as our example does via redirection- there is no need for a renderer:

from clastic import POST


def create_app():
    static_app = StaticApplication(STATIC_PATH)
    routes = [
        ("/", home, "home.html"),
        POST("/submit", add_entry),
        ("/static", static_app),
    ]

    ...

We add this route as a POST route. This makes sure that other HTTP methods will not be allowed for this path. You can try typing the address http://localhost:5000/submit into the location bar of your browser, and you should see a MethodNotAllowed error. There are also other method-restricted routes, like GET, PUT, and DELETE.

Named path segments ¶

Now let’s turn to using the shortened links. Any path other than the home page, the form submission path /submit, and static asset paths under /static will be treated as an alias, and we’ll redirect the browser to its target URL. [1] It makes sense to make this a GET-only route:

from clastic import GET


routes = [
    ("/", home, "home.html"),
    POST("/submit", add_entry),
    ("/static", static_app),
    GET("/<alias>", use_entry),
]

Important

Note that the ordering of the routes is significant. Clastic will try dispatch a request to an endpoint function in the given order of routes.

Angular brackets in route paths are used to name segments. The part of the path that matches the segment will then be available to the endpoint function as a parameter by the same name:

def use_entry(alias, db):
    entry = db.use_link(alias)
    return redirect(entry["target"], code=HTTPStatus.MOVED_PERMANENTLY)

Errors ¶

But what if there is no such alias recorded? A sensible thing to do would be to return a NotFound error:

from clastic.errors import NotFound


def use_entry(alias, db):
    entry = db.use_link(alias)
    if entry is None:
        return NotFound()
    return redirect(entry["target"], code=HTTPStatus.MOVED_PERMANENTLY)

Using middleware ¶

Clastic allows us to use middleware to keep endpoint functions from having to deal with routine tasks such as serialization, logging, database connection management, and the like. For example, the PostDataMiddleware can be used to convert submitted form data into appropriate types and make them available to endpoint functions as parameters:

from clastic.middleware.form import PostDataMiddleware


def create_app():
    new_link_mw = PostDataMiddleware(
        {"target_url": str, "new_alias": str, "expiry_time": int, "max_count": int}
    )

    static_app = StaticApplication(STATIC_PATH)
    routes = [
        ("/", home, "home.html"),
        POST("/submit", add_entry, middlewares=[new_link_mw]),
        ("/static", static_app),
        GET("/<alias>", use_entry),
    ]

    ...

The endpoint function doesn’t need to get the data from request.values anymore:

def add_entry(db, target_url, new_alias, expiry_time, max_count):
    entry = db.add_link(
        target_url=target_url,
        alias=new_alias,
        expiry_time=expiry_time,
        max_count=max_count,
    )
    return redirect("/", code=HTTPStatus.SEE_OTHER)

Cookies ¶

At the moment, after adding a new entry, the endpoint function only redirects to the home page. Say we want to display a notice to the user indicating that the entry was successfully added. This requires passing the new entry data from the add_entry() endpoint function to the home() endpoint function. But redirection means a new HTTP request and we need a way of passing data over this new request. One way to achieve this would be using a cookie: the add_entry() function places the data in a cookie, and the home() function picks it up from there.

Cookies can be accessed through request.cookies, but in this example we want to use a signed cookie. Clastic includes a SignedCookieMiddleware for this purpose. This time we’re going to register the middleware at the application level rather than for just one route. The secret key for signing the cookie will be read from the configuration file:

from clastic.middleware.cookie import SignedCookieMiddleware


def create_app():
    ...

    cookie_secret = config["erosion"]["cookie_secret"]
    cookie_mw = SignedCookieMiddleware(secret_key=cookie_secret)

    render_factory = AshesRenderFactory(CUR_PATH)
    return Application(
        routes,
        resources=resources,
        middlewares=[cookie_mw],
        render_factory=render_factory,
    )

If a function wants to access this cookie, it just has to declare a parameter named cookie.

Here’s how the first endpoint function stores the new alias in the cookie:

def add_entry(db, cookie, target_url, new_alias, expiry_time, max_count):
    entry = db.add_link(
        alias=new_alias,
        target_url=target_url,
        expiry_time=expiry_time,
        max_count=max_count,
    )
    cookie["new_entry_alias"] = new_alias
    return redirect("/", code=HTTPStatus.SEE_OTHER)

And here’s how the second endpoint function gets the alias from the cookie, and puts it into the render context:

def home(host_url, db, cookie):
    entries = db.get_links()
    new_entry_alias = cookie.pop("new_entry_alias", None)
    return {
        "host_url": host_url,
        "entries": entries,
        "new_entry_alias": new_entry_alias,
    }

And a piece of markup is needed in the template to display the notice:

<h1>Erosion</h1>
<p class="tagline">Exogenic linkrot for limited sharing.</p>

{#new_entry_alias}
<p class="message">
  Successfully created <a href="{host_url}{.}">{host_url}{.}</a>.
</p>
{/new_entry_alias}

For the alias generation and link expiration features, you can refer to the full application code in the repo. To make this example into a real-world application, the storage module must be modified to handle concurrent requests.

[1]	You should remember that a browser can make an automatic request for the site’s favicon at an address like `/favicon.ico`. Our code will treat this as a missing alias.

Tutorial: Link Shortener¶

Getting started¶

Resources¶

Redirection¶

Named path segments¶

Errors¶

Using middleware¶

Cookies¶