In this post we will discuss about a very high level overview of implementing a long running connection between two services with the use of WebSocket and then writing unit tests for the functionality.
The WebSocket protocol allows a two-way communication between two channels. Most developers would know it as a fast, near real-time communication medium between the client and the server. In the Phoenix and Elixir world, many would recognize it from LiveView or Channels which are abstractions built on top of fast realtime transports, usually a WebSocket.
But another area that WebSocket is a great communication medium for is an app built on the Microservices architecture. Let’s say we have an e-commerce application that uses (among others) two services:
Orders
- Create and manage the user’s ordersPayments
- Processing the payments.When a user makes a purchase, the Orders
service would create an order corresponding to the purchase and send it to the Payments
service.
This service could then respond with a payment page URL which the user then needs to visit to complete his payment.
After the completion, the Orders
service needs to create an invoice for the order and send it to the user, so it will need a way to watch for the completion of payments.
For such an architecture, a long running connection between Orders
and Payments
through a WebSocket is a great choice as both services can interchange messages and notify the other service about updates or events.
WebSockex is a library to implement a WebSocket client in Elixir. A very simplified implementation of the client as it could look on the Orders
service can be found here.
The client behaves as follows:
Payments
WebSocket Server and some details about the order
it needs to process.Payments
service to initiate the payment process.Payments
service responds with a list of available payment methods for the order.Payments
to initiate payment with that method.Payments
sends a message with the payment page url that the Orders
service can use to initiate a payment on the web app.Payments
sends a message to the client informing about the state of the transaction, e.g. FULFILL
, CANCEL
etc.FULFILL
, fulfill the order. If the payment is CANCEL
, cancel the order and close the connection.Now that we have a working WebSocket client in place, the next question is how do we test it?
It connects to an external service (even though this service is ultimately controlled by us, but this is still external in terms of the Orders
service).
There are several solutions for writing tests that interact with external services:
All these methods and their pros and cons have been covered in detail in several posts in the community. One that I particularly like and recommend is Testing External Web Requests in Elixir?.
We will discuss how to write tests for our above client by rolling out our own server during the tests.
To test out the client, we first need to create a Mock Server that will be able to provide a connection to this client and interact with it. We will use plug and cowboy to create the server and control the socket.
First, since we might need many of these servers running in parallel during the tests, we will need a way to generate ports so that the servers don’t collide.
This can be done by starting an Agent
that starts randomly at a port and then assigns us a port incrementally during the tests
defp get_port do
unless Process.whereis(__MODULE__), do: start_ports_agent()
Agent.get_and_update(__MODULE__, fn port -> {port, port + 1} end)
end
defp start_ports_agent do
Agent.start(fn -> Enum.random(50_000..63_000) end, name: __MODULE__)
end
The Mock Server now needs to obtain a port, start listening on that port for connections. On each new client connection, we will open a socket (which we will see later). This is the server side socket that is linked with the client side socket being opened from our WebSocket client implementation. This socket is running in its own process, so in order for our test code to interact with it, we will need to obtain its PID inside the tests. Since the tests only know about the HTTP Server and not the socket, we will send the Socket PID back to the server so we can retrieve it from the test code. Here is how the Mock Server code looks (I know it is a bit complex, so don’t be alarmed, it is easy to understand once we see the socket).
The usage of the above server is very simple:
Start the server:
{:ok, {server_ref, url}} = Commerce.Orders.MockWebsocketServer.start(self())
on_exit(fn -> Commerce.Orders.MockWebsocketServer.shutdown(server_ref) end)
If you need to send messages to the client, first obtain the server pid
server_pid = Commerce.Orders.MockWebsocketServer.receive_socket_pid()
Send Messages to the server that will relay them to the client. Check Commerce.Orders.TestSocket.websocket_info/2
for all possible clauses
send(server_pid, {:send, {:text, frame}})
This server also sends all messages that it receives to the owner process. This means that to verify that the server has received a message, you can use
assert_receive on the owner:
assert_receive("SOME MESSAGE")
The next thing after creating the mock server is to create the actual server side socket that will handle the connections.
This is the Commerce.Orders.TestSocket
to which our mock HTTP server dispatches the initial connection to.
The full code for the test socket can be found here. Let’s break this down.
The init
is what is called after the MockWebsocketServer
is started and it receives a new connection request from our WebSocket client (that we want to test).
You can check out the full cowboy_websocket
documentation for more details about the supported responses.
[{test_pid, agent_pid}]
is the initial state that we sent out from the MockWebsocketServer.dispatch
and we will keep track of that in state to manage sending of frames to the socket.
websocket_init
is called once the connection is upgraded to WebSocket and this is where we send own (i.e. the Server Socket’s) pid
to the MockWebsocketServer
which the test code can then retrieve using receive_socket_pid
.
websocket_handle
is called for every frame received.
This is where we further process the received frame to verify messages from our client.
If this differs between different WebSocket clients that we want to test, you could skip the call to handle_websocket_message
and it’s implementations and just use send(state.pid, to_string(msg))
to send all messages to the test process.
The test process can then verify the frames as required.
websocket_info
is called for every Erlang message that the process receives.
This is what we will be using to receive messages from the test code to trigger certain events from the Server Socket.
The first argument to this function could be anything that the test code sends, so you can add as many clauses as you would like to support to simulate certain events.
Here we are interested in only initiating a close
from the test code or sending a particular frame to the client socket.
Now that we have the WebSocket client and the Mock Server and Mock Socket out of the way, we can get to the actual test code that will test the WebSocket client using the Mocks.
To start the server in our tests, we will use MockWebsocketServer.start
:
setup do
{:ok, {server_ref, url}} = MockWebsocketServer.start(self())
on_exit(fn -> MockWebsocketServer.shutdown(server_ref) end)
%{url: url, server_ref: server_ref}
end
Let’s test out our client with the mock server in place. Instead of several smaller tests that test a single part of the client, I am writing a big test case that describes the full interaction for the sake of describing all available testing options without too much boilerplate code.
test "interacts with the server", %{url: url, order: order} do
{:ok, pid} = WebSocketClient.start_link(%{url: url})
server_pid = MockWebsocketServer.receive_socket_pid()
# A payment is initiated immediately after a connection
assert_receive Jason.encode!(%{initiate_payment: true})
# WebSocket client receives the payment methods from the server (from the hard coded message response). We do not need any code for this since we generalized it directly in the socket.
# The payment is initiated with credit card
assert_receive Jason.encode!(%{initiate_payment: "credit_card"})
# Send a FULFILL state from the server. This is the alternative to hard coded messages in the server side socket.
send(server_pid, {:send, {:text, Jason.encode!(%{state: "FULFILL"})}})
# Confirm that the order was fulfilled
assert {:ok, %Order{state: "FULFILL"}} = Orders.get_order(order.id)
end
In this post we touched briefly on a sample Microservices architecture, how we could use WebSockets to communicate between different services and finally saw a very simplified implementation of a WebSocket client using WebSockex
.
If you are looking for a more complex WebSockex
client example, I recommend this post which walks through building a complete STOMP client with WebSocket.
While it took some work to set up our mock server and socket for testing our WebSocket clients, it makes our testing really smooth and simple. If we add another WebSocket client later, we could utilize the same server to unit-test the new client without any complex setup or mocking.
In addition to this, we now have tests that do not depend on any implementation details of the client.
We are just testing business logic rather than implementation or tying our test code to specific libraries being used in the implementation.
So if you were to replace the WebSocket client implementation to use gun
instead of WebSockex
, we wouldn’t need to change anything in our tests.
In fact, our tests would serve as an additional validation point for this migration to see that everything keeps working like it did before.
This article was originally posted on AppSignal Blog