Forecasting Weekly Book Sales
Classical, machine-learning and hybrid models on Nielsen BookScan data
Summary
This report forecasts weekly sales for The Alchemist and The Very Hungry Caterpillar 32 weeks ahead, from 12 years of Nielsen BookScan data. We wanted the best weekly forecast, so we compared classical, machine-learning and hybrid models. A classical seasonal model (SARIMA) was the most reliable on both titles. XGBoost was competitive on The Alchemist on one error measure but weaker on the Caterpillar, and a standalone LSTM was the worst, most likely from too little data.
The two titles behave differently: The Alchemist leans on its Christmas spike, while the Caterpillar sells more steadily through the year. Every model under-forecasts the Christmas peak, so we also priced that error. Under a sale-or-return trade, ordering a little above the forecast at the peak is the safer mistake, and the choice of error measure recovers a small part of the rest.
1. Data and task
The data is Nielsen BookScan UK weekly sales, 2001 to 2024 (227,224 rows across 500 books). A week with no sale has no row, so we first put every book on a weekly calendar and set the silent weeks to zero. We then kept the 61 titles still selling after July 2024 and focused on two with different shapes: a sharp festive title and a higher-volume year-round staple. Each was trimmed to 2012 onward, about 655 weeks, which drops the noisy launch years.
The task is to forecast each title 32 weeks ahead. We held out the final 32 weeks as a test set the models never saw, scored on MAE and MAPE, and also report RMSE, which weights the big Christmas misses more heavily. Figure 1 shows the whole workflow.
2. What the data shows
The 61 active titles cover a wide market. Figure 2 places each on three axes at once: how fast it sells (copies per week), where it sits on price (average selling price) and how big it is overall, with bubble area set to total revenue. The two titles modelled here are the two biggest earners, and they sit at opposite corners of the speed-against-price trade: the Caterpillar sells fastest and cheapest, The Alchemist slower but dearer. The quick sellers are mostly children’s books and popular fiction; the priciest are illustrated or reference titles that sell slowly. Figure 3 ranks the top five on each measure, and an interactive version lets you hover every title.
The series themselves share no exact shape, but a common one recurs: a Christmas spike on a slower run of year-round sales. That shape settles over a title’s life, from a noisy launch into a steadier backlist rhythm after about a decade, the period a backlist forecast needs, so we model from 2012 onward.
For the two titles, the clearest signal is the yearly cycle: the autocorrelation (ACF) spikes cleanly at a lag of 52 weeks, with the partial autocorrelation (PACF) agreeing, fixing the seasonal period at one year (m = 52). ADF and KPSS disagree on the raw series, which is the sign of a series that only settles after differencing, so the model has to handle the trend. Testing the pre-, during- and post-COVID stretches separately showed the lockdown weeks behaving unlike ordinary demand, so we replaced the 27 zero-sale lockdown weeks per book with the same week’s average from normal years; left in, they teach the model a yearly dip that was really just a shutdown.
The festive swing grows when the book sells more overall, which suggested a log or Box-Cox transform, but both scored worse on the test weeks, so we forecast on the raw scale; only the LSTM needed its inputs rescaled. The residuals differ by title. The Alchemist is stable apart from its sharp festive spikes, while the Caterpillar’s level has roughly doubled and its errors widen in the later years, more a rising-baseline shift than a pure Christmas miss.
3. Models and accuracy
Table 1 gives each model’s accuracy on the 32-week test set. SARIMA and its sequential hybrid (an LSTM trained on what SARIMA gets wrong) are the most reliable on both titles; the hybrid wins a couple of columns by so little that SARIMA stays the sensible default. Auto-ARIMA set the orders: both titles difference once (d = 1) for the drifting level and use no seasonal difference (D = 0), handling the yearly cycle through seasonal terms at the 52-week lag.1 After fitting, the leftover error sits mainly on the December weeks for The Alchemist; the Caterpillar also under-calls across the whole year.
XGBoost edges SARIMA on RMSE for The Alchemist but not on MAE, and is clearly behind on the Caterpillar. The LSTM is weakest on both, and far worse on the Caterpillar: its 26-week window cannot see the yearly cycle, and the 32-week recursive forecast lets early errors snowball off the rising baseline. Tuning the parallel SARIMA/LSTM blend put all the weight on SARIMA, so the neural model adds nothing the seasonal model misses. Forecasting on monthly totals did not beat the weekly models, and the stocking decision is weekly, so we kept weekly. Table 2 lists the configurations, and Figure 4 plots the forecasts against actual sales: both follow the level and shape but fall short of the sharp December peak (SARIMA forecasts carry confidence intervals).
4. The cost of under-forecasting
To tie the forecast to a stocking decision, we priced its error at each week’s actual selling price. Ordering exactly to SARIMA’s forecast would fall short by about £22,900 for The Alchemist and £30,100 for the Caterpillar over the test period (Figure 5). This is a value-left-on-the-table estimate, not a profit model: we have no margin or returns data. The two totals are close but sit very differently. Over half of The Alchemist’s gap is in the six festive weeks; only 9% of the Caterpillar’s is. One is a Christmas-peak problem, the other a year-round level problem.
5. Recommendation
Use SARIMA, with the per-title orders in Table 2, as the working forecast for both titles, and read it as a guide to how much to stock and reprint. Keep the sequential hybrid only where its small extra accuracy is worth the work, and drop the standalone LSTM and parallel hybrid for these two books.
The error is asymmetric: under sale-or-return a missed sale is gone for good, while surplus stock is largely recoverable, so under-ordering is the dearer mistake. Treat the Christmas weeks as a deliberate over-order, sized by how fast that stock would clear. The response differs by title: the Caterpillar can take a steady lift across the year, while The Alchemist needs a sharper but more careful festive over-order, where scoring on RMSE rather than MAE buys back a little of the peak accuracy.
Footnotes
A wider
auto_arimasearch improved neither title’s test set, so the simpler orders were kept.↩︎