Time Series of Special Kind
Most of the time, in finance we work with time series that closely resemble random walks. Standard ML methods are not designed to use this information and, therefore, are not suited well for our purposes.Back to our AI Platform
METHODS WE APPLY TO SOLVE IT:
Proper Prior Estimation of Correlations between Strategies
The mains driving force of our work on extension of ML approach is the need to explicitly address peculiarities of the financial data as well as peculiarities of the final goal, which is a data-driven automated generation of trading strategies.
One of the peculiarities of financial data is that financial time series resemble very closely random walks. We can benefit from this fact significantly, if we directly encode this prior knowledge about the data into our ML approach instead of hopping that a model can learn this property itself from the data.
Intuitively we understand that two given strategies can be strongly correlated not because of some special pattern in the data but because of their intrinsic similarities. For example, the two strategies can operate on similar features and use similar function on top of the input features. We can very accurately estimate the part of the correlation conditioned by similarities between the two given strategies by running these strategies on a very large synthetic random walk resembling statistical properties of the real time series. Such an approach gives us a very good prior estimation of correlations between sub-strategies, which is definitely better than any other naïve priors such as either no correlation between the sub-strategies or a constant universal strategies independent correlation.
After a good prior estimation of correlations, we switch to the real data set with patters to further improve the prior estimations if there a statistical evidence that those corrections should be made. This approach allows us to compose a very accurate and smooth (noise-free) correlation matrix and, as a result, construct a larger and reliable portfolio of sub-strategies.