Slow day parametrising a simulation

Parametrising models is really dull sometimes. Its not so bad if the model runs quickly so that you can crank through it but this one is a bit of a slow one. I just need one more bit of data and then I can do a big run and it should be all done… slow day.

Text Message Alerts

I have long been interested in setting up my computer so that it is possible for it to tell me what is going on via sms messaging. This would be useful for both my day job and also to alert me to interesting things that it might detect. So I finally got round to setting it up.

Using an old Huawie E220 GSM modem and the linux package called gammu-smsd all of my software can now send me texts. I used to use the modem for mobile broadband but I upgraded to MiFi so I got an ordinary pay as you go sim card for the modem and set it up. These modems are perfectly capable of sending texts. I chose the gammu-smsd because it has the option of using a mysql backend. One tip is that the mysql tables are provided in a file which is included in the distributed source code. Also I ran tail on /var/log/messages (Opensuse – do the same for your version of linux) to see which device the modem is attached too. You can force it use the same one if you like. I might end up doing this if it causes a problem.

Using mysql is handy because you can send a message by directly writing the messsage details in the the outbox table in the database. Easy. I wrote a very simple Java interface to the mysql database that all of my programs can connect to. Any program can use the Java library to inject a message into the database and then it is promptly sent. The computer set up to send the sms messages doesn’t even have to be the computer that is running the simulation. One machine with an accessible database can provide a messaging service for any number of computers. There are alternative approaches but this seemed the easiest to set up quickly.

This is helpful because often simulations take a long time to run and it is useful to be able to leave them to it knowing that I will be notified when the task has completed, or if the task failed for some reason (there is nothing more annoying than finding out that a simulation stopped for some reason the minute you turned your back). I can then log back into the machine and either start the next simulation or whatever. I can also set it to alert me to interesting market events.

Full Trading Platform Built

I have finished building a virtual trading platform. The idea is that this can be used to learn trading algorithms for shares on UK and US markets, and then later be coupled to a real trading platform using an API provided by a company such as Interactive Brokers. For now it just takes virtual positions, so no real money at risk.

The algorithm currently in use is pretty naive and needs to be replace. The question is with what. In many ways I would like the system to learn its own method for trading, and in true complex systems style, perhaps I don’t even have to be able to understand what it is doing. I intend to take an agent based approach by having a population of agents running different algorithms or the same algorithms with different learnt parameters. Successful agents will be allowed to duplicate and persist, unsuccessful with be removed from the population. Essentially a evolutionary approach not a million miles away from a genetic algorithm.

Multiple sources of information will be able to the agents to inform their trading. From sentiment analysis (twitter etc), to a large database of historical share prices  and realtime market data. The intention is that the learning part of the system will learn what subset of the available information is useful for trading share. The system only needs to be able to be slightly better than random chance.

Finally sorted my data…

So I have had programs saving data in a raw form for months. Almost years in some cases. I have finally got my act together and written the programs to actually turn this raw data into nicely formatted data that is stored in mysql tables. I will of course keep the old data, just in case.

I now have enough to try a number of different things. Sentiment analysis, clustering analysis of price movement, and perhaps enough to try to built some networks. Things are finally taking shape.

Sentiment Analysis

So, I have been thinking of dipping my toe into the pool of sentiment analysis for a while now. I made the first positive steps the other day. I found a list of positive and negative words on the internet. I have a Java library that works as a basic dictionary and I have created mini dictionaries of these words. The idea is that I can make some assessment of the sentiment of what people are saying about a share for example and then use this as part of a trading algorithm. People have been doing this for years, but I would like to try it myself. I want to see what the problems, pitfalls, limitations are firsthand. I suspect the key will be making the connection between share price movements and certain key individual’s or group’s sentiment. This is similar to that throw away line in the film The Social Network. Something like, “..he made $300,000 trading oil based on the weather…”. If weather is a key indicator of oil price changes I need to find the equivalent key persons for shares that I am interested in trading. Classic old problem of finding the signal in the noise, and there is a lot of noise.