TeraMarv has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I am looking for some advice on how to generate a realistic stock price series. I have hacked together the following but it relies on hardcoded ceiling values for daily changes and price ranges. Obviously I will need some kind of controls to limit the movement of prices but I would also like my market simulation to have the potential to 'crash'.

Currently my price series looks too random when plotted, I need to inject a little bit of human irrationality into the series.

Any pointers on how this may be done would be gratefully received. 

use strict;
use warnings;

my $start     = sprintf("%.2f",(1 + rand(100)));
my $direction = sub { return rand(1) > 0.5 ? 1 : -1; };
my $varience  = sub { return rand(0.05); };
my $range     = sub { return rand(0.08); };
my $high;
my $low;
my $close;
my @prices;

for(1..1012){
    
    if(!defined $high){
        $high = sprintf("%.2f",( $direction->() * $varience->() * $sta
+rt ) + $start);
        $low   = sprintf("%.2f",( $high - ($high * $range->())));
        $close = sprintf("%.2f",(rand($high - $low) + $low));
    }
    else {
        $high = sprintf("%.2f",( $direction->() * $varience->() * $hig
+h ) + $high);
        $low   = sprintf("%.2f",( $high - ($high * $range->())));
        $close = sprintf("%.2f",(rand($high - $low) + $low));
    }
    
    print "$high,$low,$close\n";
    
}

exit;

[download]

Many Thanks, TeraMarv

Replies are listed 'Best First'.
Re: Generating Stock Price Data
by dHarry (Abbot) on Sep 09, 2009 at 15:09 UTC

    I bet you want to test your trading system;)

    If you want realistic stock price series this is not so easy. You basically have to run Monte Carlo simulations using probability distribution(s). Of course there are already tools that do just that. I'm not familiar with any Perl tools though. I use stuff from sourceforge. I was positively surprised by the amount of tools and APIs available. I encountered this link on MonteCarlo stock pricing, which may be of help to you.

    Harry

[reply]
      Harry, how did you know! ;)

      Thanks for the links, I'll take a look.

      It would be nice if I could implement this in Perl though so I can easily hook it up to all my groovy indicators.

      Thanks again,

      TeraMarv.
[reply]
Re: Generating Stock Price Data
by BrowserUk (Patriarch) on Sep 09, 2009 at 17:23 UTC

    How about something like this? 

    #! perl -slw
use strict;
use GD::Graph::lines;
use GD::Graph::colour qw(:colours :lists :files :convert);


our $STOCKS ||= 5;
our $CHANGE ||= 10;
our $TREND ||= int rand 3;

my $graph = GD::Graph::lines->new( 1024, 768 );

$graph->set(
    title             => 'Price over day',
    'bgclr' => 'white',
    'transparent' => 0,
    'interlaced' => 1,

    x_label           => 'Time (hours)',
    x_max_value       => 24,
    x_tick_number     => 12,
    x_label_skip      => 2,

    y_label           => 'Price',
    y_tick_number     => 10,
    y_label_skip      => 2,

) or die $graph->error;

my @tracks = map int( rand 100 ), 0 .. $STOCKS;
my @data = map [], 1 .. $STOCKS;

for my $interval ( 0 .. ( 24 * 6 - 1 ) ) { ## every 10 minutes
    push @{ $data[ 0 ] }, ( $interval * 10 ) / 60; ## time in hours

    for my $stock ( 1 .. $STOCKS ) {
        my $trend = rand() < .2 ? $TREND : int( rand 3 );
        my $change = int( rand $CHANGE ) * ( -1 + $trend );

        push @{ $data[ $stock ] }, $tracks[ $stock ] += $change;
    }
};

my $gd = $graph->plot( \@data ) or die $graph->error;

open(IMG, '>graph.png') or die $!;
binmode IMG;
print IMG $gd->png;
close IMG;

system 'graph.png';

    [download]

    A few settings that produce interesting examples: 

    c:\test>794374 -STOCKS=5 -CHANGE=20 -TREND=-2
c:\test>794374 -STOCKS=5 -CHANGE=20 -TREND=1
c:\test>794374 -STOCKS=5 -CHANGE=20 -TREND=0
c:\test>794374 -STOCKS=5 -CHANGE=20 -TREND=-1
c:\test>794374 -STOCKS=5 -CHANGE=10 -TREND=-1
c:\test>794374 -STOCKS=5 -CHANGE=10 -TREND=1
c:\test>794374 -STOCKS=5 -CHANGE=10 -TREND=2
c:\test>794374 -STOCKS=5 -CHANGE=10 -TREND=0

    [download]

    You can obviously get more sophisticated by having the trend alter trhough the day (week/month/year) etc.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]
      Thank you for your input. I'll see what I can 'steal' that fits my purpose.
[reply]
Re: Generating Stock Price Data
by ELISHEVA (Prior) on Sep 09, 2009 at 18:29 UTC

    If you are serious about testing your model, I strongly encourage you do so some academic quality reading on financial engineering. Each of the suggestions above have significant pitfalls that it would behoove you to be aware of.

    Monte Carlo simulation is a very popular technique for modelling prices, but it is important to remember that the modelling is always done in conjunction with an underlying pricing model. It can only simulate market behavior if the underlying pricing model is, in fact, a reasonably accurate model of the market. It is often used in financial modeling because market pricing models often involve extremely complex differential equations that are hard to evaluate analytically.

    Historical data also has difficulties. The past is not the future and the past is not consistent. Over the last 100 years there have been periods of slow steady rise, sudden falls and jumps. Volatility, short term moving averages (7 years or so), and many other measures of market behavior change in statistically significant ways over time. Models based on historical data often are very sensitive to the time range of data studied. For example, the 30 year moving average is far more steady than the 7 year moving average. The longer time period takes into account one or more full business cycles, whereas shorter time frames might catch just a small part of a single cycle.

    I'd also be especially careful about intuitive theories like "markets rise half as fast as they fall" or "we've reached a psychological price point and the market (or stock) will hold steady". I used to hear a lot of that sort of thing on CNBC during the height of the dot.com boom. The problem with those models is that the human eye is a phenomenal pattern maker. Random processes are a lot like coastlines - they seem to go in a defined direction when you look at them up close. Only when you step back and take the aerial view can you see that the raggedness is random. To emphasize this point, one of my biz school professors began his introductory lecture by showing a small bit of a randomly generated curve. Nearly everyone assumed some sort of pattern. Then he flashed the entire series on the board was it clear that the overall picture was simply a bunch of noise.

    To discern genuine patterns you'll need to apply some fancy statistics and mathematics to rule out the noise. The researchers that have done this do find strong evidence that markets are responsive to both rational judgments and collective emotions. However, the exact relationship is a matter of much debate among researchers and theorists. Also there are some interesting timing issues. For example, the well-publicized financial report announcements for large companies generally have a much smaller impact on stock prices than do reports for small public companies. It is theorized that the market already knows so much about the Fortune 500 companies that they don't really consider the reports as news. On the other hand, small companies only get attention when on announcement day so their reports have a disproportionate impact on stock prices.

    Finally, markets aren't uniform in their behavior even at a single point in history. Each stock in the market can be categorized according to its betas - the real world factors (for example, weather, interest rates, unemployment, commodity prices) whose changes correlate with changes in stock prices. Your model may need to take that into account as well.

    Best, beth

[reply]
      Wow,

      Thank you for the comprehensive reply. Looks like this may exceed my mathematical capabilities.

      I was hoping that someone had tried something similar before that I could leverage.

      Another thought has crossed my mind though. If I am trying to test a trading system that uses a data series (price) as input does the nature of the input really have a any bearing on the performance of the system?

      On the subject of 'real world factors' the study of technical analysis assumes that all relevant information is already priced in and that price is all you need to analyze. A simplification of the market for sure but the simplest ideas are often the best.

      Thanks again for your reply, it has raised a few more questions in my mind which can only be a good thing.

[reply]
Re: Generating Stock Price Data
by rcaputo (Chaplain) on Sep 09, 2009 at 17:10 UTC
[reply]
Re: Generating Stock Price Data
by tmharish (Friar) on Sep 09, 2009 at 15:36 UTC
    One reason that the price series may apeare too random is that complete randomness is inherent to the series that you are generating.

    Like you have mentioned, I think what one expects when looking at something like a price series is some order.

    The one way, that I can think of, to get around this problem would be to actually keep track of history. Your "direction" and "variance" functions are totally independent of whats been happening prior to now.

    I dont know too much about how the markets work but one thing I can think of, off hand, is the following
    • If the price starts off by growing or (dropping) maybe one could increase the chances of subsequent growth (or drop) in the 'direction' function (that way one can avoid continuous "ups and downs".
    • To balance this out one could start to increase the range of variance as the chances of moving in a particular direction increases.
    This should also take care of the "crash" or "boom" as the price will grow or drop more drastically as it moves in a particular direction.

    I like to think of this as the market getting more unstable as it moves in a particular direction for too long. Of course this could be a fundamentally wrong assumption but like I said I understand less of the markets than I would like to.

    Of course, so far the assumption is that both the variance and the direction get reset once the direction changes. This need not be the case. It might be better to start with a high probability of moving in a particular direction and then reduce that as the price moves along that direction for too long. Also instead of a complete reset one might want to increase the variance exponentially and reduce it linearly (or the other way round).

    Finally - and I find this to be useful when trying to simulate real word phenomenon - one could actually base this fictitious price index on an actual one. It need not be a direct correlation, it could be inverse or a combination of multiple real word stock indices. Of course the simulation can even draw from historical data!!



[reply]
      Some interesting points.

      Maybe I need a 'Bull' and 'Bear' buffer which gets populated or not based on the previous directional movement which could then influence the next price in the series.

      I also like the idea of harvesting the real world price series for 'example' movements on which to base my simulated movements. Suggesting this also has me thinking I can divide previous real world series into bull, bear and sideways periods and randomly mix these periods into my simulation.

[reply]
Re: Generating Stock Price Data
by whakka (Hermit) on Sep 09, 2009 at 17:08 UTC
    Define "realistic." Your series is fairly autocorrelated - one may question whether it's random enough. Looking at the price picture you can see this - each day's price is largely the result of the previous day's momentum.

    I agree with tmharish that you should look at real stock data for a realistic series. Google finance lets you look up historical data on a stock and exports to .csv (ex.). I'm sure there are other sources out there.

[reply]

      I was hoping that I could at least partly simulate the 'Greed' and 'Fear' that drives the market. Mostly price movement in my opinion is driven by human psychology and is not truly random in that we are prone to irrational decisions. When you look at stock price movements the decline in prices is nearly always faster that the previous increase by at least a factor of two and usually greater. This is one example of where a purely random system is not representative of the real world.

      While using real stock data does provide 'realistic' data for testing it is limited to historical movements which may or may not be repeated in the future. To test the robustness of a trading system it needs to be tested on all scenarios not just those which have occurred.

[reply]
        "When you look at stock price movements the decline in prices is nearly always faster that the previous increase by at least a factor of two and usually greater."

        Do you have the source for this? It sounds like an interesting read. I can see where that might be the case for bubbles, but I didn't know such a thing was so prevalent.

        As to your last point, I would argue that the sheer volume of existing historical data probably encompasses more possible trends than one can invent. Especially if you think prices are psychologically driven, then you'd expect the trends to repeat themselves over and over again, no?

[reply]

Back to Seekers of Perl Wisdom