Testing Prophet Time Series Prediction On Bitcoin – Jason Bowling
It’s pretty easy to do, but how well does it work?
This experiment started when I saw a graph that stopped me in my tracks. It was a graph showing the breakdown of the history of the Bitcoin price history showing the seasonal component of the time series. It looked like this.
OK. That means Bitcoin fluctuates with the calendar year. That’s really interesting. I’ve looked at the time series of stocks before, and even retailers didn’t exhibit anywhere near that level of seasonal fluctuation.
I’m pretty convinced that most stock values are on a random walk, but that kind of seasonal variation indicates there may be something different about what’s driving Bitcoin’s fluctuations. So can we use that?
There are a number of published articles describing how to use Prophet to forecast stocks and Bitcoin, but they tend to be light on details about how well it actually performed. It was time to try it.
Test Approach
I won’t rehash what Prophet is, how it works, or why it’s dangerous to make investment decisions based solely on something like this. Your investment risk is yours alone.
There are a number of articles detailing how to use it — I found this one based on predicting avocado prices to be quite good. Bitcoin is way, way more chaotic than avocado prices, so it’s a serious stretch of any forecasting tool. I’m going to focus on how well it worked and observations during testing.
Essentially, I pretended it was early 2019, and trained a series of Prophet models looking out sixty days each. I then compared the predictions to what it actually did, for all of 2019 and January of 2020. Here’s what the backtest code looked like.
First, grab the daily price history and load it up into a Pandas dataframe. We define where we want the training data to end, and how far out we want to try to predict.
Next, we make two copies of our historical data. One will be trimmed down prior to training — we make a training dataset and drop everything after our defined end of training date. We want to make very sure that the data later than that date doesn’t sneak into our model’s training. which are the upper and lower limits of what the model considers the most likely range. If the model works well, the actual value should fall within that range.Maintaining two copies is inefficient in terms of memory, but easy to code and understand.
We now have two copies of the historical closing value by day. While we’re at it, we convert the string representations of the dates to a Python datetime object so that we can manipulate the data based on the dates.
Now we can declare a Prophet object and fit the model to the dataset. We request a forecast for our previously defined sixty day window. If you’d like to get a look at the time series components that make up the model, including the trend, 7 day pattern, and 30 day pattern, you can use the code shown in the snippet below.
Now we merge the forecast dataframe with the actual results dataframe. and trim it to only contain data for the time frame we are looking at.
Finally, we can graph it. I ran into a problem plotting the results that was fixed with the help of this article.
I chose to look at a bit over the last year, in sixty day increments. This was pretty arbitrary — shorter time windows might introduce less uncertainty.
Results
As you look at the following graphs, the red line represents the upper bounds of the uncertainty interval (yhat_upper), and the green line represents the lower bound (yhat_lower). The yellow line represents Prophet’s forecast (yhat), and the blue line indicates what Bitcoin did during that interval (actual).
Keep in mind that each test below is a different model — each model in succession was trained on 2 months more data than the model before it.
January 2019 to March 2019
That’s not bad. The actual value remained quite close to the uncertainty interval. It forecast a modest drop that never materialized.
March 2019 to May 2019
That’s not so great. The actual value quickly separates from the model’s uncertainty interval and stays well outside the band as it continues to rebound.
May 2019 to July 2019
Here we see an even more rapid separation from the uncertainty band. If you’d relied on the Prophet model alone, you’d have missed out on some serious growth over the short term.
July 2019 to September 2019
We’re not even close this time…
Sept 2019 to November 2019
OK, things are settling down now, and the actual value tracks nicely within the band of uncertainty the vast majority of the time. I consider these results to be quite satisfactory.
November 2019 to Jan 2020
Here we once again separate from the band of uncertainty by a substantial margin. Had you invested solely based on the model’s predication of a rise, you’d have lost a significant amount.
January 2020 to February 2020 (time of writing)
This is only a thirty day prediction, and it improves later in the test period.
Conclusions
2019 was a pretty wild year for Bitcoin, as it rebounded sharply in the first six months. Though the seasonal component of Bitcoin pricing is strong, Prophet alone would make a poor trading indicator during periods of exceptional volatility. Later in the year, the models performed better, but the actual values did not reliably remain in the uncertainty interval.
It might be useful when combined with other technical indicators, and Prophet is certainly an excellent tool for more predictable time series. With Bitcoin in particular, and investing in general, it’s important not to place too much importance on any one tool, technique, or indicator.
I leave you with the Prophet forecast of Bitcoin for the next year, at time of writing. You can see the blue line, which represents the model’s best fit to the data. The black dots are previous values which don’t fit well with the model — outliers. The light blue band shows the uncertainty interval.
Have fun, and be careful out there.
Published at Sun, 09 Feb 2020 04:05:47 +0000
{flickr|100|campaign}
