What benefits are there to employing agile software development methodologies for quants?

Question

Perusing the other SE network sites, particularly Programmers, I often find vigorous support for various agile software development methodologies, particularly the various values known as Extreme Programming (I'm thinking of unit testing, coding standards, etc. and also SOLID principles, variable naming consistency). Yet none of the quantitative researchers I know employ these methods and design principles anywhere near as rigorously as it would appear some software developers do. In fact, many of these concepts are virtually unheard of outside the IT departments of many major financial institutions.

Would it be worth my while to learn and adapt some of these principles to my work, or is it a waste of time? I would mostly be doing this on my own, since there is little desire on the part of my boss and colleagues to change the way we do things now. Most of my code gets thrown away anyhow, and often the qualitative conclusions I reach are more important than re-usability. That is, I am typically testing some hypothesis, and once I have reached a conclusion, that code is not really needed any more. Occasionally I have to re-write large chunks of code to automate the recalculation of proprietary indicators I have developed, but I usually have no idea what will end up in production until I am nearly done with the research. Besides, there may even be value in reviewing all the code as it is re-written now that I have an end goal in mind.

Answer 1

"Extreme programming" is a buzzword that has received a lot of hype in the past few years. However it's important to note that it's only one item in the long list of SW development philosophies and that it's not - contrary to its proponents' claims - a panacea.

On the other side it's very beneficial to follow a few simple rules while writing even small pieces of code. To the fundamental ones you already mentioned (sticking to a coding standard and a consistent naming convention) I'd add extensive commenting.

For your usage model ("most of my code gets thrown away" and "conclusions I reach are more important than re-usability") I'd suggest setting up a simple repository and revision control. This may sound like an overkill but IMO you'll benefit from it in the long term:

1. All pieces of your code are in one place and are easily accessible. My experience tells me that sooner or later you'll want to at least have a look at some code written in the past - even if you thought that it wouldn't be needed anymore.
2. Change tracking is trivial. The need to look at / go back to a previous version happens surprisingly often. Having a source repository at hand is then a godsend. For the purposes listed above you don't need all the fancy features of modern revision control systems - just a central repository and linear history. I'd suggest going with Mercurial - there is no need for setting up a server, it's cross-platform and simple to use. It also has a nice GUI named TortoiseHg. To avoid having separate repositories for separate pieces of code you may use directories inside one big repository. The whole repository and Mercurial binaries may be put on a pen drive - there is no need to install anything if you don't want to.

Unit testing is extremely useful and beneficial in regular software development. I'm not sure if it'll help you in your work (research) apart from adding extra burden. But definitely start thinking of it as soon as you notice that you're transforming some of your code into e.g. a library of functions (or anything that is intended to be re-usable). And of course keep your unit test in your repository.

For your type of work I'd also stick to YAGNI and KISS principles. (Wikipedia articles on them are sufficient to get a good grasp of them).

PS For bs-free view on SW development I recommend works by Frederick Brooks.

Answer 2

I have only seen one framework that works in a research oriented development environment which is the spiral model. Using try agile methodologies is impossible because the frontier of tasks is not known. Agile is very useful for building/maintaining known applications with known functionality and problem spaces. It is not useful for research oriented development since agile requires good estimates to plan releases (you could say this about almost any methodology but it is very important to agile).

In a team I used to work in the spiral model went something like this:

1. Three weeks of research
2. Three weeks of building
3. Three weeks of integration, testing and deployment

I thought it worked very well. In a larger team than I have now I would use a variant of this again.

All that being said software development methodologies are a religious war and I think anything can work. Honestly I think that until you have several people working on the same component and a larger team agile or any other methodology is going to have very low returns as all approaches are attempting to solve the problem of communication overhead in a large group of people.

In our 3 man trading shop we use the following:

• No formal software development methodology
• Pivotal Tracker for task tracking
• Github for source control
• Central build server with static code analysis for enforcing coding standards, especially things like functions that we don't allow
• Code reviews. We read each others code. In my experience code reviews more than anything else lead to better code.

What works for me may not work for you. Remember if you are serious measure the before/after productivity with the introduction of any methodology. If you can't see an improvement in either code quality/features shipped you are spitting in the wind.

Answer 3

There are some Agile benefits that you will reap, even if you are the sole programmer.

• You may feel silly doing a scrum by yourself in the morning. But you may find it to be a benefit to plan what you would like to work on that day, and to think about what you might need that day (especially if you need to read about solving a quant problem).
• Planning out what will be released in the next (short) release cycle is a valuable skill that can help you shape the project as a whole. Estimating the time to release a feature and measuring actual time will help you to avoid fantasy estimates and help you gauge your abilities as a developers. It helps you think globally, about software strategy (think Field Marshall); your tendency may be to think locally, in the trenches.
• Building unit tests as you go will server you wonderfully in two ways. First, it documents what you have done and how to use your classes under test. (This works well when you're going back several weeks or months and can't remember how to use the class.) Second, and more important, it helps you think about the software contract, that is, what different objects are responsible to do.

A word about general design principles: if you find yourself throwing away large pieces of code, you may need to abstract-out your classes. For example, if there are methods that every fixed-income equity needs to respond to, such as nominal interest or par value, then perhaps you need an abstract fixed-income object. If there are fields that every equity object has, such as a CUSIP or a company name, then consider an abstract equity object with those fields.

Answer 4

As an agile developer and quant finance programmer, I think that unit testing is invaluable. Because you really never know if your code is doing what it is supposed to do without tests. How do you know that your code is calculating your proprietary indicators correctly?

You probably ran your new code and checked the result against some other code or system that is known to be correct. That is precisely what a test is. The only difference is that instead of doing it once manually, you save the expected output as part of a test.

This has several benefits

1. You can check your code at any time in a few minutes with virtually no effort.
2. You can refactor your code and be confident that it is still correct.
3. You can improve the quality of the test. Instead of doing a quick test once, you can save a large amount of the expected output. This will minimize the possibility of small bugs that only show up occasionally. For example, instead of checking that a few rows of the indicator data is correct, you can verify that it is correct for thousands of rows and dozens of assets.
4. You can create new code from scratch for production, and use the tests to verify that it is correct.

I think that unit testing is particularly important for quantitative finance because the intermediate results are never really checked. For example, you can create custom indicators but the results are used by an algorithm.

I found The Art of Unit Testing easy to use and understand. It's for .NET programmers.

Answer 5

An important part of research is reproducibility of results. There is no point of drawing conclusions, if you can't reproduce the data to back them up. For this you need at least an organized way of storing your code, so that you can find what was the algorithm you used to produce that graph. A source code repositore is an ideal way to do it.

By the same token, unit tests are useful in research because they give you a better guarantee that your code actually does what you think it does.