Chat App step by step

00 -> 01 - changes in app.py

connect to a SQL (sqlite) database

Let's start with empowering our Flask app with a SQL database.

This is only boilerplate code, but it's central to the rest of the steps.

globals

• We import SQLAlchemy, that provides a full SQL toolkit and Object Relational Mapper (ORM) for Python.
• We configure Flask to use the DB located at sqlite:///chat.db
• We create a new database object, and its session attribute, that will allow us to interact with said DB

01 -> 01b - changes in app.py

first endpoint to check if the database is alive

endpoints

• We create a new endpoint /db/alive that will return a 200 OK response if the DB is alive

This triggers the minimal SQL code, just to make sure we can reach the DB; think of it as a "Hello world" for the DB

to try it out

in all the rest we assume you run the Flask server on port 5001

http :5001/db/alive

Note for windows users

It seems that the http command, when run on "Git Bash" for Windows, triggers an error:

Request body (from stdin, –raw or a file) and request data (key=value) cannot be mixed. Pass –ignore-stdin to let key/value take priority

In that case, take the message at face value, and add the -I (shorthand for --ignore-stdin) option to the command:

http -I :5001/db/alive

Tip: to continue using the same sentence (i.e. without the -I), you might be able to trick bash and define an alias

alias http='\http -I'

01b -> 01c - changes in app.py

a /api/version endpoint to get the app version

endpoint

• We create a new endpoint /api/version that will return the version of the code (we will increment the global VERSION variable as we go along the steps)

Note while we're talking versions, note that in a production environment, the API endpoints would rather be versioned like /api/v1/...
so that one can define a breaking change in the API, and still support the previous one

to try it out

http :5001/api/version

01c -> 02 - changes in app.py

define a table in the DB with SQLalchemy

the User table

thanks to SQLAlchemy, and specifically its ORM (Object Relational Mapper), we can define a table in the database as a Python class.

• the class must inherit db.Model, which is the base class for all models in that database
• also, it contains a primary key - this is standard practice in SQL databases
• it also contains as many columns as we need to model a user, in our case name, email and nickname
• note how each column is defined with db.Column(), and thus typed (here we mostly have strings, but the primary key is an integer)

database actual creation

note the call to db.create_all() as the first instruction executed in the server life cycle.

this is crucial if we want to actually create the table (in DB words, this means we apply the schema to the database)

02 -> 02b - changes in app.py

a POST endpoint to create a new user

rows creation

Look at the code for the new /api/users endpoint; it is totally idiomatic of how to use SQLAlchemy to create a new row in the database:

• you create a User object, passing the values for each column
• you than add it to the session
• and finally you commit the session to the database (this is the actual writing to the DB)

(does the add / commit thing ring any bell ?)

to try it out

here again see the inline comments in the code for how to trigger the new endpoint

02b -> 03 - changes in app.py

an endpoint to GET the list of all users

this is rather straightforward; now that we have a means to create users, we need a way to get them back

• we use the same /api/users endpoint
• except that this time we use the GET method

how to retrive stuff with SQLAlchemy

on a SQLAlchemy class - here User - and specifically on its query attribute, we can use methods like

• all() to get all the rows
• first() to get the first row
• get() to get a row by its primary key
• filter_by() to filter the rows
• and other similar words...

in this case a call to all() will return an iterable of User objects

now, we cannot unfortunately return a list of these objets as-is in the Flask route function

why is that ? because these User objects are not JSON serializable !
and Flask would automatically try to convert them to JSON, and fail miserably...

this is the reason why we need to re-create a regular Python dictionary (with obviously the same keys as the User class)

Note There are ways to deal with this a bit more concisely, but for educational purposes, and for the sake of clarity, we will stick to this admittedly rather tedious way of doing things

to try it out

you know what to do

03 -> 04 - changes in app.py

a basic frontend web page to display users

In this move we add in the mix the first seed of the app frontend

new files

a HTML template, and a CSS file, see below

new endpoint

And we also add a dedicated endpoint /front/users that will serve this page; the way it is intended to work is:

1. you direct your browser to http://localhost:5001/front/users
2. which will call the /api/users endpoint
3. which in turn will retrieve all users from the DB
4. and pass that list to the new Jinja2 template (the .j2 file)
5. that will create one custom HTML element per user
6. and return the full HTML page - with all users - back to the browser

new imports

We need:

• render_template to render the Jinja2 template;
• requests to call the /api/users endpoint

keeping the app modular

Step #3 deserves a few more words; to retrieve all user details, we have a choice between:

• asking the database directly
• or forwarding the request to the /api/users endpoint

We have gone for the latter option, as it is more in line with the micro-services philosophy
The idea is that even though our current deployment runs in a single Flask app, we want to be able to deploy it in more distributed way, with the services for

• the database
• the /front/
• and the /api/ endpoints

all running in different containers/computers

Also note that this way of doing things is SSR (Server-Side rendering); relying on the API to implement this endpointmakes it more likely for us move to CSR (Client-Side rendering) in the future if need be.

04 : new file style.css

some style

just some style to be applied in the frontend

this is a plain static file, it won't be subject to any processing, hence it goes in the static folder

04 : new file users.html.j2

the HTML template

this new file is a Jinja2 template, which is a templating engine for Python; it allows you to create HTML pages dynamically by embedding Python code within the HTML.

the basics for Jinja2 templating is:

• you compute one or several variables on the Python side - here e.g. we have users and VERSION
• you pass them to the template with render_template()
  render_template('users.html.j2', users=users, version=VERSION)

• and from then on you can access the variables in the template with the corresponding names - here users and version
• for example, to display the version in the template, we use:
  <h1>Welcome to the chat app (version {{ version }})</h1>

• Jinja also offers more advanced features, like loops and conditionals; observe how the template iterates over the users variable to create one HTML element per user
    {% for user in users %}
        <div class="user">
        <h2>{{ user.name }}</h2>
        <p>{{ user.email }}</p>
        <p>{{ user.nickname }}</p>
        </div>
    {% endfor %}

04 -> 05 - changes in app.py

how to redirect

Nothing crucial here, but an opportunity to show how to redirect HTTP traffic

what is a redirect ?

There are many reasons why you would want to redirect a request to another URL:

• the URL has changed
• the resource is now located on another server
• ...

how to redirect

In Flask you can use the redirect() function to redirect a request to another URL; it's dead simple; in our case we just redirect the /' URL to the /front/users` URL:

see also

in particular about the HTTP codes, see:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Redirections

05 -> 06 - changes in app.py

the /api/users/id endpoint retrieves a user details

A new endpoint to get the details of one specific user from their ID

Nothing new indeed, very similar to the /api/users endpoint, except that we use the get() method to retrieve the user by its primary key

(and we still need to convert the User object to a regular Python dictionary)

06 -> 07 - changes in app.py

a new table for storing messages

In this very rustic app, a message is simply a text sent from one user (the author) to another one (the recipient)

Nothing exactly new here; we use the same approach as for the users, but this time we deal with messages

using primary keys to model relationships

Worth being noted though, in the Message class we need to refer to the User class (for the author and recipient)
the standard approach here is to use the primary keys in the messages table

hence the author_id and recipient_id columns, that are both Integer, and declares as ForeignKey to the id column of the users table

This is an intermediary step, and in a future step down the road we will improve this a bit, but it's helpful to understand how to deal with foreign keys in SQLAlchemy;

07 -> 07b - changes in app.py

a POST endpoint to create a message

nothing new here as compared to users; except the minor point that the date column is not provided by the caller, but computed by the API at creation time

try it out

Here's a trick that can help you populate the DB with some users and messages, from scratch; especially as we might have to do this over and over again

create a bash file

copy and paste the following in a file named aliases.sh (or whatever you want)

# put this in aliases.sh
function list-users() {
    http :5001/api/users
}

function create-users() {
    http :5001/api/users name="Alice Caroll" email="alice@foo.com" nickname="alice"
    http :5001/api/users name="Bob Morane" email="bob@foo.com" nickname="bob"
    http :5001/api/users name="Charlie Chaplin" email="charlie@foo.com" nickname="charlie"
}

function create-messages() {
    http :5001/api/messages author_id=1 recipient_id=2 content="trois petits chats"
    http :5001/api/messages author_id=2 recipient_id=1 content="chapeau de paille"
    http :5001/api/messages author_id=2 recipient_id=3 content="not visible by 1"
}

function create-all() {
    create_users
    create_messages
}

function message-to-alice() {
    http :5001/api/messages author_id=3 recipient_id=1 content="from the API"
}

load the file

By running the following, you will load the file in your current shell session

source aliases.sh

run the commands

After you've source'd aliases.sh your shell knows about the functions defined in the file; so you can do

create-users
create-messages
list-users

07b -> 08 - changes in app.py

a GET endpoint to retrieve all messages

super straightforward step, just retrieve all messages

not very useful actually, but only for completeness.

08 -> 09 - changes in app.py

/api/users/id/messages : v0 for a user

more useful now, we need an API endpoint to retrieve messages for a given user
like always we will refer to the user by its primary key, so that is the expected parameter in the URL

in other words, /api/users/1/messages will return all messages for the user with id 1

a first implementation

in this first naive approach:

• we return messages with only user ids (i.e. without their nickname)
• also we will only return messages where the user is the recipient

this is not very useful, but again it will help us understand how to properly deal with relationships

nothing out of the ordinary

we can use the exact same approach as for the users:

• we run a query on the messages table
• and we simply filter it by the recipient_id column

something missing

in a practical app, we would probably need to return BOTH messsages that have the user as recipient and as author

but let's not get ahead of ourselves, we will improve all this later on

09 -> 10 - changes in app.py

the messages of interest for a user: also the messages from the user

first improvement to the previous step, we will now return messages with the user and not only for the user
meaning that we need to write a little more elaborate filter, and look for messages that have the user as either the author or the recipient

which in logical terms amounts to doing a OR between the two conditions
hence the import of the or_ function from SQLAlchemy, that will allow us to combine the two conditions

10 -> 11 - changes in app.py

retrieving more details, like nicknames, with messages

this now is interesting;

imagine you write the frontend, and you need to display messages; so typically alice has sent a message to bob
with the current code, you would get something like this:

{
    "id": 1,
    "author_id": 1,
    "recipient_id": 2,
    "content": "trois petits chats"
}

which is not very useful, as you need to display the name of the author and the recipient, not just their ids !

relationships

the trick here is to extend our ORM and to make it explicit that, in addition to the author_id and recipient_id columns, we also want to leverage the User class to get the author and recipient details for each message

that is the purpose of these lines here in the Message class:

author = relationship("User", foreign_keys=[author_id])
recipient = relationship("User", foreign_keys=[recipient_id])

impact on the endpoint

as we have decided earlier, we stick to the strategy that we explicitly build the dicts returned by the API
so in the /api/users/<id>/messages endpoint, we add a author and a receiver keys to the returned dict (and thus remove the author_id and recipient_id keys that can now be retrieved through the author and recipient relationships)

11 -> 12 - changes in app.py

a web page to display the messages for a given user

just like we did earlier for users, it's time to build a frontend to display messages
this comes in two parts:

• a new frontend endpoint /front/messages/<id> that will be the main page for one user
• an html template that will display the messages relevant for a given user

accessing the DB

like for the earlier step, we refrain from accessing the DB directly, and use the /api/users/<id>/messages endpoint
the arguments we had for users - the fact that we want the app to remain modular - still holds as well of course
plus, on top of that, the approach here makes even more sense, in that the logic for retrieval this time is a little more complex, and should not be duplicated

other caveats

at this point, the main default is, we don't see incoming messages as they are posted; the user needs to refresh the page to see new messages; a bit of patience...

12 : new file messages.html.j2

the template

the template is created with 2 variables:

• user: the details about the user, that we get .. wait for it .. from the /api/users/<id> endpoint
• messages: the messages for the user, that we get from the /api/users/<id>/messages endpoint

in this first version we just display the messages, we'll add a prompt area later on

12 -> 13 - changes in app.py

the html page has a dialog and can now also send messages

it's nice to see the messages; but better still, we'd need to be able to send messages as well

in order to achieve this, we need:

• more information made available in the frontend page: the list of users, so that the user can select a recipient
• in the frontend as well, a new form area to send messages
• a new frontend script - stored in static/script.js - that will actually send the message - by talking back to the API

the /front/ endpoint

• simply makes a 3rd request to the API, to get the list of users

still missing

and fixed in the next step:

the page does not refresh after sending a message; i.e. the new message is not displayed in the list of messages

12 -> 13 - changes in messages.html.j2

adding a form element in the template

this new version of the template:

• injects the new JS script
• adds a new form area to send messages, that has as many <input> fields as there are fields in a message, namely
  • a hidden input with the author_id (the user id) - that won't change, but is required as part of the newly created message
  • a <select> tag for the recipient
  • a <input> tag for the content, typed as text

13 : new file script.js

there is a new JavaScript file

• as we've learned in the frontend courses, the script arranges to execute itself when the page is loaded
• at that time it binds the 'submit' event of the form to a function that will
  • prevent the default behavior of the form (which would be to reload the page)
  • get the values of the fields in the form
  • and send them to the API, using a POST request

Note: there actually is a default behaviour for a form submission event, which uses the action and method attributes of the form to determine where to send the data. However that default behaviour does not support JSON encoding, this is the only reason why we need to bother with the JS script at that point.
(we will take advantage of our custom script in the next steps anyway, so it's no regret)

13 -> 14 - changes in app.py

no change in the app itself

just the version number

the caveats

at this point in time, the data returned by the API about the newly created message only contains user ids, not the user details like nicknames

13 -> 14 - changes in script.js

also display newly created message

the messages page displays the message it just sent

upon successful creation of a message, the API returns the message details so, we can use that to display the message in the frontend

in the mix, we can also take care of any error that may occur, and inform the user (actually in this rustic app we just log in the console)

we just need to extend the JS script to handle the response from the API; for that

• we improve on the sequence of .then() calls
• if the response is a success, we create a new row in the messages table
• and if not we use a .catch() to log the error in the console

14 -> 15 - changes in app.py

return richer data from POST /api/messages endpoint

to be able to display the nicknames of the users in the newly created messages, we need to slightly tweak our /api/messages creation endpoint
instead of returning a author_id and a recipient_id, we will return the author and recipient objects with full details

for that the endpoint does 2 requests to the DB

Note in real life, one tends to optimize the number of requests to the DB;
it would be possible to get the author and recipient objects in one go, but here we are not in a performance context, and we want to keep things simple

caveats

now the app works reasonably fine as far as sending messages is concerned
but it still sucks in terms of receiving messages; we have no simple way to be made aware of incoming messages

without the addition of websockets, the front would need to poll the API for that...

14 -> 15 - changes in script.js

display nicknames in newly created messages

in this version, we now receive full user info, so we need to tweak the JS accordingly

15 -> 16 - changes in app.py

enable & connect to SocketIO: the messages page connects to the backend

to make the app more reactive, we will use SocketIO, so the backend can notify the frontend of incoming messages

in this first - admittedly quite limited - step, we just lay the groundwork for adding SocketIO in the picture, even though it is not actually used for anything useful yet

SocketIO basics

SocketIO is a library that enables real-time, bidirectional communication between clients and servers. It is built on top of WebSockets, which is a protocol that allows for full-duplex communication channels over a single TCP connection.

SocketIO provides a higher-level abstraction over WebSockets, making it easier to work with. It handles many of the complexities of WebSockets, such as reconnection, event handling, and broadcasting messages to multiple clients.

It exposes the notion of channels (or rooms) that kind of like act as multicast groups: a message sent to a channel reaches all clients subscribed to that channel. Exactly what we need here then.

SocketIO implementations

in our context, we actually need two SocketIO implementations:

• one for the backend - so, in Python, and more specifically for Flask
• and one for the frontend - so, in JS
• the backend will be responsible for sending messages to the frontend each time a message gets created
• and the frontend will be responsible for
  • connecting the server in the first place
  • subscribing to the right channel (since our app has only DMs and no room, we will use one channel per user)
  • and reacting to incoming messages - typically by displaying them in the UI

So, what does all that take actually, in terms of code changes ?

boilerplate

in app.py

• we need to import (and pip install if needed) the flask-socketio package
• and to create a SocketIO instance called socketio - it will in some context act as a replacement for the Flask app

a socketIO callback (kind-of like an endpoint)

still in app.py

• we create a function called connect_ack
• the syntax for creating it looks a bit like the one for a regular Flask endpoint
  except the decorator is @socketio.on instead of @app.route
• the name used for creating it - here connect-ack is the name of a channel

so what we're saying here is, each time a message occurs on the connect-ack channel, the backend will print a message about that
we could have gone entirely without the connect-ack channel, but it is a good way to test that the connection is working

expected behaviour

all this means that, when the web page is loaded:

• the frontend connects to the backend
• once connected it says so on the console
• and sends an acknowledgment to the backend
• which prints a message on the console as well

of course all this traffic is purely for educational purposes, and is not required for the app to work properly

15 -> 16 - changes in script.js

the frontend connects to the backend socketio server

upon document loading, the JS code will

• create a connection to the server using socket = io()
• and then define a callback function
• that will trigger upon a connect event (this time this is a builtin name, as opposed to the connect-ack channel we created earlier)
• and so this is how the frontend will send an acknowledgment to the backend upon successful connection

15 -> 16 - changes in messages.html.j2

the HTML template

this is the place where we add the SocketIO client-side code into the mix; as always we use a <script> tag in <head> to do that; we picked a CDN for that

16 -> 17 - changes in app.py

subscribe to the nickname channel

our next move is to subscribe to the nickname channel
but wait, the JS code currently is static, and has no access to the nickname

we have 2 options here:

1. make the JS code a template, and instantiate it with the nickname
2. pass the nickname to the JS code through the HTML tree

of course this version is not functional yet, as nobody writes on that channel yet... but we'll fix it later

16 -> 17 - changes in script.js

retrieve nickname and subscribe to the channel

from then on, the nickname becomes accessible in the JS code, thanks to this line, that inspects the data- attributes in the body tag of the HTML page:

    // retrieve the nickname from the body tag
    const nickname = document.body.dataset.nickname

and from there, we can simply subscribe to the nickname channel, using the socket.on() method like before

16 -> 17 - changes in messages.html.j2

store nickname in the HTML tree

actually, the first option is a little awkward, there is no need to make the JS code a template;

instead we can simply attach the nickname in the HTML tree; and there is a standard practice for that, which is to add a data attribute to the HTML element; and since this is of interest to the whole tree, we pick the <body> element for that purpose

hence this line in the HTML template:

<body data-nickname="{{ user.nickname }}">

17 -> 18 - changes in app.py

the backend writes on the nickname channel

almost done now: in this last-but-one step, we will have the backend write on the nickname channel

so the /api/messages endpoint just goes on doing this once it is done:

        socketio.emit(recipient.nickname, json.dumps(parameters, default=str))

the only thing remaining is for the frontend to use that data to reresh its page

18 -> 19 - changes in app.py

finish it off

the last ring of changes is concentrated in the JS code

and this time the app is totally functional !

18 -> 19 - changes in script.js

properly display incoming messages in the frontend

we need to do 2 things:

• the code that used to add a newly sent message to the table is no longer relevant (we will receive this information through the channel)
• but on the other hand we need to add messages to the table when we receive them through the channel

so the changes are

• to factor out a function that we call display_new_message() - based on the previous code
• and to call this function when we receive a message through the channel, like so

    socket.on(nickname, (str) => display_new_message(JSON.parse(str)))