Testing a chat server

After my feeble fumblings towards what is quite possibly the most minimal HTTP server designed (minimal as in functionality rather than in minimal but complete according to specification), it was clear that part of the problem was trying to use HTTP preformance tools to test the chat server. To rectify this, it made sense that I write my own little test utility that would allow me to hammer the chat server and test it in a more appropriate way.

Aside

It is still clear that there are some issues with the original chat server since the shutdowns shouldn’t be occurring regardless of what kind of mad clients try to connect. However, the hope is that with a more focused test utility, the problems can be resolved more easily.

The plan

Rather conveniently, there is a corresponding simple TCP client for the original simple TCP server. Once again, this is where we shall begin our quest.

The idea is to convert the client from a command line user-interface to an automated tool that will connect to a server, send some input, receive some output, close the connection and then return the output received. In this way we will be able to test that we receive from the server what we expect.

The implementation

Rather suprisingly, implementation went relatively smoothly. I rapidly got a command line utility that takes a host, port and number of connections to make. It then forks a new thread for each one and each writes a single line, consisting of its number. Each then listens for a second beforewriting a final string and then closing the connection.

While this is happening, I am also listening to replies from the server. Any replies are tagged with the client id and added to a giant map. When the terminating string is recognised, that client is removed from a large list and when all clients have completed, this in turn returns. The value returned is a map of client id to the strings received from the server by that client. This should now allow some analysis to ensure all are working as they should.

Initial testing

First testing was against the minimal HTTP server. And it worked. But being keen to test it aggressively, I immediately cranked up the number of connections to make to 2000. At that point I got this rather worrying error.

getProtocolByName: does not exist (no such protocol name: tcp)

Thinking that it was likely that I had deleted the TCP protocol, I suspected an issue with open sockets once more. This was confirmed by a quick search over the number of connections. The maximum I was allowed without getting the delightful error above was in fact 1021. This looked fair enough when the output from socklist included the following TCP sockets

type  port      inode     uid    pid   fd  name
tcp   9999    2284399    1000  15202    3  minimal
tcp    631    2245789       0      0    0
tcp     25      99077       0      0    0

and ulimit -a provided the information

open files (-n) 1024

which implied that there would be 1021 sockets available.

Interestingly enough…

Doing a Google search for “getprotocolbyname does not exist no such protocol name tcp” turns up several Haskell-related links. Could this be an indicator that Haskell is often being used for large-scale socket manipulation? Or that Haskell programmers are more likely to be doing strange things with large numbers of sockets? Or even that Haskell programmers don’t know what they are doing and do crazy things like that?

Further testing

Since I was flush with my success, I moved on to testing the original chat server. Here, as expected, things did not go as well.

Small number of connections were fine. But when I rolled out the big guns and set 500 connections at the server, it promptly fell over (gracefully as ever).

Although disappointing that the chat server is somehow broken, the fact that the test utility reproduces the problem is reassuring as I slowly add to my minimal HTTP server making it more fully-featured.

Scary code

The thing about Haskell is that it lets you do some frightening stuff in a single line. Case in point:

sequence_ $ map forkIO $ zipWith ($) (zipWith ($) (repeat (start host portStr chan)) [1..n]) $ map (:[]) $ map show [1..n]

which is just scary. Of course there is probably a much simpler way to do something like this, but if it works, don’t mess with it, at least unless there is some sort of reliable test framework to rely on.

Conclusion

Getting a basic test framework was easy, but the proof of the pudding will be in the eating. Or, in other words, I shall wait and see how much use I can make of it as I progress. The final state of the code is reproduced below – for all its pretty colours, it is really quite messy!

The code

> module Main where
 
> import Prelude hiding (catch)
 
> import Network (connectTo, withSocketsDo, PortID(..))
 
> import System.IO
> import System.IO.Error (isEOFError)
> import System.Environment (getArgs)
> import Control.Exception (finally, catch, Exception(..))
> import Control.Concurrent
> import Control.Concurrent.STM
> import qualified Data.List as L
> import qualified Data.Map as M
> import Control.Monad
 
> main = withSocketsDo $ do
>     [host, portStr, nStr] <- getArgs
>     let n = (read nStr :: Int)
>     chan <- atomically newTChan
>     sequence_ $ map forkIO $ zipWith ($) (zipWith ($) (repeat (start host portStr chan)) [1..n]) $ map (:[]) $ map show [1..n]
>     output <- mainLoop chan  [1..n] $ M.fromList $ map (\i -> (i, [])) [1..n]
>     putStrLn $ (show output)
 
> start :: String -> String -> TChan (Int, String) -> Int -> [String] -> IO ()
> start host portStr chan i ss = do
>     let port = fromIntegral (read portStr :: Int)
>     sock <- connectTo host $ PortNumber port
>
>     netTID   <- spawn $ listenLoop i (hGetLine sock) chan
>     sequence_ $ map (hPutStrLn sock) ss
>     threadDelay 1000000
>     atomically $ writeTChan chan (i, "COMPLETE")
>     hClose sock

No related posts.

Tagged as: chat server, Haskell, testing