My previous discussion on Inversion of Control raised some questions so I want to take a step back and discuss Dependency Injection. Dependency Injection is a pattern where software components (classes, methods or functions) are given their dependencies as parameters and these software components do not instantiate their dependencies directly.

Dependency Injection is an important pattern to use when you wan to create classes that are easier to reuse and unit test. Since the dependencies are external, they can be configured and maintained outside class or function and there’s no need to change the code in order to reuse it. Also, when dependencies are injected into a class or a function it is possible to substitute a mock implementation of the dependency. In unit tests, mock objects are used as replacement for the real implementation, this helps isolate the functionality being tested.

A simple example

The following function has it’s dependencies, a list of numbers, passed to it, and since calculate_average only talks to a list of numbers it is easy to reuse.

def calculate_average(list_of_numbers):
  x = sum(list_of_numbers)
  return x/len(list_of_numbers)

Imagine a similar function which uses a database connection, retrieves a list of numbers then calculates the average.

def calculate_average():
  sql_query = "select numbers from table"
  list_of_numbers = db.query(sql_query)
  x = sum(list_of_numbers)
  return x/len(list_of_numbers)

This function is difficult to reuse because it depends on the database, future users of this function will have to either modify the function or create and maintain a database just to use it. More importantly it’s difficult to test this function. A test would require a database connection and a fixture to populate the database with data.

You should be thinking, but Shey, I would never do this with a simple function like this. You’re right, you wouldn’t do this with a simple function, but as soon as calculate_average turns into a more complicated calculation such as calculate_value_at_risk we start instantiating or dependencies.

A slightly less contrived example

The BillingService class is responsible for charging your customers a fee.

class BillingService
  def initialize(credit_card_processor)
    self.processor = processor

  def charge(account, amount):
    if account.has_balance:
      result = processor.charge(account.cc_number, amount)
    if result == "success":
      account.deduct(amount)

Lets assume that Authorize.net is offering better rates than PayPal. If the class accepts an implementation of a credit card processor as a dependency it is possible to switch to the authorize.net implementation of the credit card processor class without changing the BillingService class. Similarly during unit testing, a mock implementation of a credit card processor can be used to alwasy return “success” and isolate the charge functionality of the BillingService.

class mock_credit_card_processor:
  def charge(cc_number, amount):
    return "success"

    def test_charge():
      bs = BillingService(mock_credit_card_processor)
      a = Account(cc_number="411111111111111", balance=300.00)
      bs.charge(a, 300.00)
      assert(a.balance == 0)

Coming from a .NET background and having applied SOLID principles to software development, I was surprised by the lack of inversion of control containers for Python.

The few discussions I read online implied that Python doesn’t need an IoC framework because it is a dynamically typed language.  Dynamically typed languages eliminate the need to use interfaces, they do not do away with the need for inversion of control.

IoC is used to decouple components of an application, remove direct dependencies so that replacing a component will not have a side effects on the rest of the system.  As Dave Thomas explains

a DI application is written as a set of loosely coupled components. These components contain no knitting code: nothing in the application code itself is responsible, for example, for making sure that the necessary objects somehow get an instance of the database connection. Instead, the components all run in a container. This container is given a description of the knitting to be done (typically using an XML file). The container then instantiates objects and sets them into components that need them

I’m new to the language and there maybe a more “pythonic” way to handle inversion of control than using a container. Here is small and contrived example of inversion of control using Pinsor, a IoC container in Python. Those coming from a .NET or Java background will find Pinsor easy to use, but slightly lacking in features.

Introduction
As stock markets open, order management systems are flooded with new orders and it’s crucial that an OMS not miss prices or drop orders during this period.  The purpose of this initial benchmark is to simulate a flood of orders hitting the queue.

Benchmark
The benchmarking script sets up 10 queues, 10 producers and 10 consumers, each producer writes to a single queue and each consumer reads from that queue ie: p1 writes to q1 and c1 reads from q1 as fast at it can.  The time measured is the time it takes a message to travel from p1 to the queue and to be read by c1 and it was calculated by appending the timestamp to the message itself.

I chose to run a series of tests with message size varying from 35 bytes to 400 bytes and the message count varying from 50 messages to 100 messages in an attempt to visualize the broker’s behavior under varying conditions.

The benchmark was performed using a server running 2.8GHz Pentium D with 903464 kB RAM running on Kernel 2.6.18-128.1.10.el5 using RabbitMQ 1.5.5 and a machine with similar specifications as the client—both the producer and consumer run on the same machine.  The network is a typical Fast Ethernet with two switches separating the server and client machines.

Conclusion
Inconclusive! The numbers are very surprising I wasn’t expecting the message transfer times to be greater than .5 seconds under this load.  It’s possible that the consumer may not be able to keep up with flood of messages, if that’s the case than a consumer written in C# is needed.  What is obvious is that more tests  with a constant message size and a varying number of messages  are needed.

The standard deviation is also greater than I expected, so I plan on creating longer running tests and see if the broker falls into a steady state.  Finally, I also want to test with 50 messages per producer using 20 queues/producers instead of 10  and see how RabbitMQ responds.

I’m wondering what sort of performance others are experiencing—any and all feedback is greatly appreciated.

Postscript
I intended on using C# to benchmark RabbitMQ but for want of time  I used a modified version of Colin Surprenant’s bunnypunisher ruby script.  There’s a slightly outdated (and dirty) version of the script available on github.  I will upload the latest version along with the “launcher” tomorrow morning.

Kyle Burton at Asymmetrical View has a fairly comprehensive comparison of RabbitMQ and Qpid.