Random Geeking
There are sometimes things that I want to do, and have to spend a while figuring out. So I am going to store that knowledge on here.
Loading R libraries at start… (works in Opensuse linux)
Easy, create a file in /usr/lib64/R/etc called ‘Rprofile.site’. Put in that file the libraries you want to load.
library(ggplot2)
When you load R those libraries with be loaded.
Right, next time I reinstall my system I will know where to look if I have forgotten…
I have finished building a virtual trading platform. The idea is that this can be used to learn trading algorithms for shares on UK and US markets, and then later be coupled to a real trading platform using an API provided by a company such as Interactive Brokers. For now it just takes virtual positions, so no real money at risk.
The algorithm currently in use is pretty naive and needs to be replace. The question is with what. In many ways I would like the system to learn its own method for trading, and in true complex systems style, perhaps I don’t even have to be able to understand what it is doing. I intend to take an agent based approach by having a population of agents running different algorithms or the same algorithms with different learnt parameters. Successful agents will be allowed to duplicate and persist, unsuccessful with be removed from the population. Essentially a evolutionary approach not a million miles away from a genetic algorithm.
Multiple sources of information will be able to the agents to inform their trading. From sentiment analysis (twitter etc), to a large database of historical share prices and realtime market data. The intention is that the learning part of the system will learn what subset of the available information is useful for trading share. The system only needs to be able to be slightly better than random chance.
Recently I have been working hard at my day job and not really felt like doing much out of work. This is partly due to the work I’m doing being really interesting. So its by no means a bad thing. Its not helping me on the quest to making enough money to buy a house without a mortgage. Which I think is the basic general plan. I haven’t really put a time scale on my adventure but I theory I would like to achieve this goal in about 4 years. I better get going I suppose.
So I have had programs saving data in a raw form for months. Almost years in some cases. I have finally got my act together and written the programs to actually turn this raw data into nicely formatted data that is stored in mysql tables. I will of course keep the old data, just in case.
I now have enough to try a number of different things. Sentiment analysis, clustering analysis of price movement, and perhaps enough to try to built some networks. Things are finally taking shape.
So, I have been thinking of dipping my toe into the pool of sentiment analysis for a while now. I made the first positive steps the other day. I found a list of positive and negative words on the internet. I have a Java library that works as a basic dictionary and I have created mini dictionaries of these words. The idea is that I can make some assessment of the sentiment of what people are saying about a share for example and then use this as part of a trading algorithm. People have been doing this for years, but I would like to try it myself. I want to see what the problems, pitfalls, limitations are firsthand. I suspect the key will be making the connection between share price movements and certain key individual’s or group’s sentiment. This is similar to that throw away line in the film The Social Network. Something like, “..he made $300,000 trading oil based on the weather…”. If weather is a key indicator of oil price changes I need to find the equivalent key persons for shares that I am interested in trading. Classic old problem of finding the signal in the noise, and there is a lot of noise.
So after leaving it to the very last second (well maybe hours) America avoids a voluntary default. Most people didn’t seem to think it was going to happen. Although I suspect that it was more hope than insight. Not only has a default been avoided, it looks that for now at least there will be no downgrade of US treasures. Odd that I think; the US looks in a bad way. The economy is slowing. Growth slowed for the last quarter and the previous quarter was revised down. Manufacturing is down too. However this new data might serve to support a watered down deficit reduction plan. The new debt deal will cut spending by about $2.5T, about half what the rating agencies were calling for. if the economy is slowing then perhaps this isn’t the time for tax increases and spending cuts. In some ways the US and UK are following different paths. Who will fair better? On the face of it the UK is getting support from the markets. With interest rates on it borrowing dropping to an all time low, 2.75%. I can’t thinking that perhaps the world is just looking the other way at the minute. Problems in the US, and further crises brewing in Europe (Spain and Italy have seen there borrowing costs increase sharply) mean that no-one is paying close attention to the UK right now. How long will that last I wonder.
So are exchange traded funds (ETFs) the next CDO? That is, are they a likely cause of the next financial crise? Well, its possible that is for sure. Particularly as they are linked in some ways. First of all it is worth pointing something out. Lets imagine I want to set up an ETF that tracks the FTSE 100 stock market. I have two possible options. One is to set up a fund that buys an equal amount of shares in all the FTSE 100 companies. That fund will then track the movement of the FTSE 100 exactly. People can give me money I buy the shares and everyone knows what they are getting. The fund will go up and down with the FTSE 100. This is very simple and its is easy to understand how it works. This is a standard ETF.
Now, lets muddy the waters. Enter synthetic ETFs. An organisation can offer a synthetic ETF that tracks the FTSE 100. The provider of the fund grantees that its value will change in line with the FTSE100. However, the provider can take the money that people invest and put it where ever they like. The idea is that the provider thinks its really smart, and that it can take your money and invest it in such away that it beats the FTSE 100. It has only guaranteed you what the FTS100 makes and therefore it can keep any additional profit.
So in theory everything is great. You get your relatively safe investment fund. The clever people in the bank get more money to invest and make prophet on. The problems come when the bank makes a loss on the investments that they really make with your money. In theory they have guaranteed it, so they have to pay up. This is where things get even more complicated. In order for the bank to take your money elsewhere to invest it they have to put up some collateral in case those other investments do loose money. This collateral is then used to fore fill any obligation to you. Ok, fine. Its complicated but it sounds sort of ok. However it is the quality of hat collateral that is what scares me and it is scaring other people too.
So lets say that instead of buy FTSE100 shares with your money in my EFT. I setup a synthetic ETF and buy something else. I then put collateral up to guarantee the fund in case I go bust. Lets say I do go bust. You don’t get your share of FTSE 100 companies, you get a share of whatever I put up as collateral. That could be a real mixed bag of sovereign debt, company bonds, whatever really. How would you feel if I had put Greek sovereign debt up as collateral for my FTSE100 EFT?
Things get ever more murky and hard to follow as that collateral could be wrapped up in a nightmare of collateralized debt obligations (CDOs), credit default swaps (CDS), over the counter (OTC) swaps. What would all that mean? Where would you go for your money should I fail? It does start to look like the makes of another credit crisis where know one knows who owes what to who and how much. There are calls to increase the transparency of synthetic EFTs, particularly what the collateral is. I think we need that.
That error up there has been causing me some head scratching. I couldn’t figure out what the problem was, and it causes issues with different parts of WordPress. I was seeing it on the Dashboard but it stops things like upgrades and plugins installing. I tried different solutions, adding .htaccess files etc etc. None of it looked that convincing, and none of it worked. Then one evening, this evening in fact, as I uploaded the newest version of WordPress (3.2) because the automatic update wouldn’t work it occured to me that I hadn’t tried poking the firewall.
It was the bloody firewall! Blocking all outgoing connections. Although this is good for security its not good for WordPress. So if you are seeing this error and have tried everything else. Take a look at your firewall. I wish I had weeks ago.
I changed the banner picture to one I took of my computer screen. It’s some of the programs/data/output from the programs I have written to gather share price data. I finally got round to moving the scrubbing programs onto my rackpsace server. Saves me running computers all day, and rackspace is more reliable . I had problems with my internet connection dropping out etc etc. The basic cloud server isn’t expensive either, about $10 plus traffic a month.
Getting data is crucial to doing any sort of modelling on shares/markets. The way that data is tightly controlled by financial and trading companies is a very good indication of what it is worth. However, I often think that it might not be worth as much as they think. The raw price data alone is not that helpful. I think more information is required if you want to predict future market movements. Its all about the people involved, that it was you need the data on. Markets are unpredictable; large groups of people however, they are very predictable.
I sold my Copper ETFS shares today (COPA:LSE). Exchange traded funds are just a vehicle for trading commodities on markets. You don’t buy the commodity itself, rather you buy into a fund which is supposed to buy the commodity. Some say that ETFS are the next derivative bubble. The next collateralized debt obligations (CDO) if you will. I’m not so sure about that, I will give it some thought some other time.
Anyway, I made $480 on the sale. Not at good as I was hoping. I did much better on the rise of crude (ETFS OILD:LSE) after the crisis, I got into copper too late. Commodities have been fairly volatile recently and there are worries and rumours that they are, in general, over priced. So I decided to sell. It means that I have about £700 to invest now and I think my portfolio could do with some more stable stocks. I might think about some food companies. If commodity prices do fall then we might see their margins increase over the next year. Could be a good time to buy…