A filter system, for retrieving data from a List

From WikiJava

In the article Downloading stock market quotes from Yahoo! finance I showed you how to download stock quotations using Yahoo's Ichart service, and to save them in a Product Object, that owns a collection of DataRanges. This Collection contains several DataRanges, for example it contains the List of StandardQuote objects representing the daily opening values of a stock quotation, it contains the closes, and it can contain quite a few other type of data.

The reason why Product is designed in such a way is because it can also contain other types of Data, as long as they extend the DataRange Interface. For example I may decide that a moving average, is in facts representable as a DataRange, and I can store it there as well.

Once you dump in the Product all the DataRanges, it may become quite a nasty exercise to get them back. Especially if you are developing with an Agile methodology, you risk to have to adapt the Product Class, every time you extend your software by creating a new DataRange type.

In this article I will introduce a mechanism to retrieve and select specific DataRanges from Product, in a very dynamic way. We'll do it using a filtering mechanism, that has several benefits:

• It's very versatile, you can extend it as you go, even if you introduce new DataRange types.
• It is fast
• You don't need to know a thing about what you have already in the Product. In facts, all you need to know is the characteristics of the DataRange you are looking for.
• It can retrieve several DataRanges, depending on what you are looking for.
• you can create filters that you can reuse to retrieve similar data from any Product.

In the article I will explain the whole concept using also some class diagrams.

The model

In extreme synthesis the class diagram of my domain model is like in the image below, which shows what the Product contains, and what's the structure of a DataRange:

Where MovingAverage and AnyRange classes are just examples of what could be easily stored in a Product object.

This is very versatile, because as long as my new DataRange types implement the DataRange interface, I can create as many implementations as I want, and I will always be able to store them with the Product they belong to.

The problem (which I solve in this article) is that once a DataRange object is saved in the Product, it becomes tricky to get it out. For example: say you created and saved in the Product a new moving average calculated in 5 periods. You now want to retrieve it. How do you do that?

I'll analyse the problem in the next section.

How to retrieve the data

As a matter of facts, to solve this problem we will probably need to redesign my DataRange implementations to be more retrieval friendly.

We could start thinking about a solution like:

You could put in each option a String (or similar) where, by convention, you encode the information necessary to uniquely identify the data. In this way when you want to retrieve, you just formulate a matching String and search in the ArrayList for a DataRange matching it.

This would work, unfortunately the convention needs to be very well designed, so to be able to deal with evolutions of the software, which we can't foresee now. Additionally the convention is an additional complexity to the system, and we don't want to go there.

The best solution here should not require any modification to the DataRange implementations, this is the only way to guarantee extendibility and simplicity.

You could keep somehow a list of what you have stored in the Product, Like an index so that you can retrieve the data when you need it.

If you think carefully to it this solution is not much different from the previous. It just prevents you from saving data inside the DataRange implementations. But it introduces a new whole set of problems with keeping the list and the index aligned. It doesn't really give much of a benefit.

You could do a data traversal (i.e. iterating through the List) and check each DataRange against the properties you need to retrieve, for instance you could use the instanceof native operator to find out instances of a specific implementation.

This is indeed a good idea, as it does not require any addition to the implementations of the DataRange or the Product. But implementing a whole data traversal each time you need to retrieve some data may become quite wordy, and you'd get a lot of duplicate code, with all the consequent maintenance troubles.

My solution is very similar to this last, the difference is that it centralizes the selection process of the implementation in a way that it improves drastically the maintanability, reduces at the most the quantity of code to write eacn time you need to retrieve the data, and maintains extremely well the expandibility of the software.

In the next sections I'll explain in detail how the solution works and provide parts of the code that implements it.

The solution

The solution is shown in the following class diagram (again in it's most essential form):

The earth of everything is the DataRangeFilter class and the Filter interface.

I'll start from the Filter interface and with example implementations of it.

The filters

The filters are very simple classes that implement the Filter interface, which is defined as:

package org.wikijava.stockfriend.model.quotations.filters;
 
import org.wikijava.stockfriend.model.quotations.DataRange;
 
public interface Filter {
 
	boolean matches(DataRange datarange);
 
}

For example a simple filter on the type is:

package org.wikijava.stockfriend.model.quotations.filters;
 
import org.wikijava.stockfriend.model.quotations.DataRange;
import org.wikijava.stockfriend.model.quotations.DataRangeType;
 
public class TypeFilter implements Filter {
 
	private DataRangeType type;
 
	public TypeFilter(DataRangeType type) {
		this.type = type;
	}
 
	@Override
	public boolean matches(DataRange datarange) {
		return datarange.getType() == this.type;
	}
 
}

It just returns true if the type matches the one the filter was initialized with. As you can see this is extremely simple, and extremely reusable.

The fun comes with the next component: the DataRangeFilter class, explained and shown in the next section.

One level of abstraction up, the DataRangeFilter

The DataRangeFilter is the earth of the filtering system. It's a very simple class though, all it does is store a List of Filter and when the matches method is called applies them in order, to return true if all the filters match the DataRange.

It's really all as simple as:

	public boolean matches(DataRange datarange) {
 
		for (Filter curFilter : filters) {
			if (!curFilter.matches(datarange)) {
				return false;
			}
		}
		return true;
	}

This simple yet powerful mechanism assures that we can arbitrarily retrieve any of the DataRanges stored, with great versatility, and being able to reuse the code. We can even generate a new Class, containing as public static final a list of the most used filters, not to have to regenerate them all the time.

Before going to the next section I just would like to show you a couple more methods from this class, that are also very powerful and I'll use in the next section.

The first is a constructor:

	public DataRangeFilter(Filter... filter) {
		filters = new ArrayList<Filter>();
		if (filter == null) {
			return;
		}
		for (int i = 0; i < filter.length; i++) {
			filters.add(filter[i]);
		}
	}

Which basically adds any filter passed in the constructor to the DataRangeFilter being created.

The second method is the addFilter:

	public DataRangeFilter addFilter(Filter filter) {
		if (this.filters.size() == 0) {
			this.filters.add(filter);
			return this;
		}
 
		DataRangeFilter result = new DataRangeFilter(this);
		result.filters.add(filter);
		return result;
	}

Which maintains the immutability of the class, by returning a new object (generated using the copy constructor, omitted in this article).

Defining the default filters

Said all the above, we can think of several, most commonly used, DataRangeFilters. We can predefine them somewhere, as static final variables, so that we can reuse them when we like.

Here's the (very simple) class diagram:

Here's the content of a possible DefaultFilters class:

package org.wikijava.stockfriend.model.quotations.filters;
 
import org.wikijava.stockfriend.backend.calculations.indicators.LowCrossingIndicator;
import org.wikijava.stockfriend.backend.calculations.indicators.LowOscillatorIndicator;
import org.wikijava.stockfriend.backend.calculations.indicators.MovingAverage;
import org.wikijava.stockfriend.backend.calculations.indicators.RSI;
import org.wikijava.stockfriend.model.quotations.DataRangeType;
import org.wikijava.stockfriend.model.quotations.OriginalDataRange;
 
public class DefaultFilters {
 
	/**
	 * filters the close Dataranges
	 */
	public final static DataRangeFilter closes = new DataRangeFilter(
			new ClassFilter(OriginalDataRange.class), new TypeFilter(
					DataRangeType.Close));
	/**
	 * filters all the moving averages
	 */
	public final static DataRangeFilter movingAverage = new DataRangeFilter(
			new ClassFilter(MovingAverage.class), new TypeFilter(
					DataRangeType.MovingAverage));
 
	/**
	 * filters all the RSI Objects
	 */
	public static final DataRangeFilter RSI = new DataRangeFilter(
			new ClassFilter(RSI.class), new TypeFilter(DataRangeType.Momentum));
 
	/**
	 * filters all the LowOscillatorIndicator Objects
	 */
	public static final DataRangeFilter LowOscillatorIndicator = new DataRangeFilter(
			new ClassFilter(LowOscillatorIndicator.class), new TypeFilter(
					DataRangeType.Signal));
 
	/**
	 * filters all the LowCrossingIndicator Objects
	 */
	public static final DataRangeFilter LowCrossingIndicator = new DataRangeFilter(
			new ClassFilter(LowCrossingIndicator.class), new TypeFilter(
					DataRangeType.Signal));
 
}

As you can see all these DataRangeFilters are generated using the constructor mentioned above. These default DataRangeFilters can also be espanded anytime, for example by:

DataRangeFilter filter = DefaultFilters.RSI
				.addFilter(new PeriodsFilter(periods));

Which thanks to the immutability of the DataRangeFilter class works just fine, without influencing other parts of the program, that maybe use the same DefaultFilter.

Using the whole filtering system

At this point, the whole system is ready to be used, and there's nothing to add to it. To use it it is very simple. The following spike puts it all together:

• downloads the Yahoo's quotations starting 300 solar days ago, till today. (see Downloading stock market quotes from Yahoo! finance for details on how this part works)
• uses a DefaultFilter to extract one DataRange from the downloaded ones.
• prints the DataRange found using it's toString method.

package org.wikijava.stockfriend.backend.controller;
 
import org.wikijava.stockfriend.backend.data.YahooDownloader;
import org.wikijava.stockfriend.model.DownloaderException;
import org.wikijava.stockfriend.model.calendar.TradingDay;
import org.wikijava.stockfriend.model.calendar.TradingDaysRange;
import org.wikijava.stockfriend.model.product.BaseProduct;
import org.wikijava.stockfriend.model.quotations.filters.DefaultFilters;
 
/**
 * <Replace this with a short description of the class.>
 * 
 * @author Giulio Giraldi
 */
public class DownloadDataAndRetrieveOpens {
 
	public static void main(String[] args) {
		YahooDownloader yahooDownloader = new YahooDownloader();
 
		BaseProduct product = new BaseProduct("YHOO", "Yahoo! inc.");
		TradingDay start = TradingDay.getSingleton(-300);
		TradingDay end = TradingDay.getSingleton();
 
		TradingDaysRange range = new TradingDaysRange();
		range.setStart(start);
		range.setEnd(end);
 
		try {
			yahooDownloader.getQuotationRange(product, range);
		} catch (DownloaderException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
 
		System.out.println(product.getRanges(DefaultFilters.closes));
 
	}
}

Final notes

In this article I showed you a very simple way to extract data from a List in a very versatile way. I showed it applied to the financial context, but it can be easily reapplied to many other contexts.

Seen carefully the mechanism implemented here looks quite similar to the design pattern Chain of responsibility, this is because the action is passed, one by one to each of the filters, who have the responsibility of refusing a DataRange if it doesn't match.

I know you won't be able to execute the code in this article, because I omitted quite some code, I omitted it for conciseness. Though, all I omitted is trivial, and it should not be a problem rewriting the required code.

Comments from the users

To be notified via mail on the updates of this discussion you can login and click on watch at the top of the page