Is Our Building Using Less Energy this Year?-Part 2

This is the second part of a two part post that shows how to use monthly data to figure out whether or not your building is using less energy now than in a baseline period.

Our client asked whether their hospital building was using less energy in 2013 than in previousyears, as a result of retro-commissioning.  Using a seven step method, the answer is “yes” and we can estimate the savings.

We covered steps 1-3 in the part 1 post.   This post will go through steps 4-7 and assumes you’ve got a little bit of knowledge about something called regression analysis and an acquaintance with control charts.   People in your organization who have quantitative skills—improvement advisors, black belts, statisticians—can help you out.  We’ll focus on the electricity use, measured in kilowatt-hours, kwh, to illustrate the steps.

Method to Check Whole Building Changes in Energy Use

1. Get the right data.
2. Set a baseline period.  
3. Plot the data to understand patterns and unusual values.
4. Model the energy use as a function of temperature or degree days.
5. Predict the energy use beyond the baseline period.
6. Compare the actual energy use and predicted use. Use a control chart to judge if there are savings.
7. If step 6 gives you a signal of savings, estimate avoided energy use and costs.

In part 1, review of the data suggested that electricity use not only went up and down with the outside temperature but also may have been increasing over the 36 month baseline period.

Step 4. Model the Energy Use as a Function of Temperature or Degree Days

We used the statistical package R to describe the relationship between electric energy use in kilowatt-hours (kwh) and average mean monthly temperature.   To allow for constant increase in kwh month by month, we used a variable that starts at 1 in January 2010 and ends at 36 in December 2012.   If you want to do this kind of equation building in Excel, you need to use the Analysis ToolPak and then use the Regression function.   

Here’s the equation:

Predicted kwh = 1,341,783 + 5596 x Month Time Step – 20455 x Average Monthly Mean Temperature + 

402 x (Average Monthly Mean Temperature)²

 With a bit of high-school algebra, the equation predicts a minimum use of electric energy at about 25° F.

How Good is the Model?

Whenever you create a regression equation, you can compare the predicted values to the actual values.  In this case, we look at the difference month by month of the actual kwh value and the predicted value from our equation.  These 36 differences are the “residuals” or left-overs, comparing the model equation predicted values to the original kwh values.

There are many ways to judge the performance of our prediction equation; one important way is to plot the residuals in time order and check for striking patterns--if our model has done a good job in capturing the important factors driving kwh, the residuals should bounce around the zero line in a random way.
 

Comparing the original data plot to the residual plot, our model doesn’t seem to fit 2010 very well—the first six months of residuals are below the zero line and the next six months of residuals are above, a pattern that is unlikely to appear in a sequence of independent random values.

We decided to use this model in the next steps of our method despite the 2010 issue. The scatter of residuals in 2011 and 2012 has no obvious patterns, compared to a random set of 24 values.   Skip down to step 6 for a brief discussion of an alternate model that uses Cooling Degree Days.

Step 5. Predict the Energy Use beyond the Baseline Period

With the prediction equation, we can now estimate kwh for January to September 2013, the months the facilities staff believe will show lower energy use.   To get the predicted values, just plug in the time steps, starting at 37 for January 2013. Then look up the average monthly temperatures for these months.

Here’s the graph, showing our predicted and actual values.   In 2013, the predicted kwh are consistently above the actual kwh:  the blue dots are above the red dots.

We see that the dramatically lower summer peak use in 2013 shows up in both the predicted and actual series.  The lower summer peak in 2013 in both series is related to a much cooler July 2013, which was 4 to 6 degrees cooler on average than the preceding three years.

Step 6. Compare the Actual Energy Use and Predicted Use. Use a Control Chart to Judge if There Are Savings

Let’s use the residuals to compare the actual and predicted use, through September 2013.  The points to the right of the green dashed line show the residuals after we've extended our equation beyond the baseline.

This residual plot magnifies the pattern seen in the preceding plot:   the 2013 residuals are consistently below the reference line of 0, which means actual energy use is consistently below predicted use. We can add control limits to the plot.  Control limits require an estimate of variation; in our example, we can use either an estimate from the point-to point differences in the first 36 points (the median moving range).  Or we can use the residual standard deviation from our regression fit.   The control limits in this example are essentially the same using either estimate; we used the median moving range.
 

When we use control charts, we want to detect unusual points or patterns of points that we can associate with specific events, known as “special" or "assignable" causes. The facilities managers took action in late 2012 to retro-commission the hospital’s controls and control points.   Does the chart support a belief that the retro-commissioning made a difference? 

While there are no points beyond the upper or lower control limits, we also look for patterns of points that are relatively unusual, compared to a reference series of independent random values.   One of the patterns is “eight consecutive values on one side of the reference line”, which shows up here (circled in green.) 

So, it looks like there is a signal in the control chart that matches the timing of the retro-commissioning action.

A Refined Model

I redid steps 4-6 using Cooling Degree Days with a base of 25° F.   We get a similar model with a trend in time and a curved relationship in temperature, represented by Cooling Degree Days, which accounts for most of the variation in monthly kwh:

Predicted kwh = 1,093,467 + 5340 x Month Time Step + 0.41225 x (Cooling Degree Days Base 25°F)².

Remember that use of degree days adjusts for the number of days in each month and so reduces the residual variation.  

With less residual variation, the control limits are narrower.  In the control chart derived from the model fitted to Cooling Degree Days, we see a signal of special cause in December 2010 and a special cause signal in 2013 we saw before—the pattern of eight consecutive points below the reference line.  The April 2013 residual is also below the lower control limit.   Our refined control chart provides relatively strong evidence that the actual energy use is lower than predicted in 2013.

7. If Step 6 Gives You a Signal of Savings, Estimate Avoided Energy Use and Costs

Let’s start with a simple estimate of savings:  multiply the estimated monthly 2013 savings in kwh by the monthly cost in cents per kwh .  Here’s a summary table with savings estimated from the model based on Cooling Degree Days, base 25° F.

Summing across the months in 2013, the actual energy use is 820,000 kwh less than the predicted use; the average monthly reduction is 91,100 kwh. In dollars, the charges are $69,400 lower than predicted, using the monthly average price per kwh, $7700 per month. 

Summary of Steps 4-7

The retro-commissioning carried out in late 2012 appears to have reduced electric energy use in 2013 relative to predictions derived from a model of electricity use, 2010 through 2012. Based on the model predictions, the hospital avoided utility charges of about $70,000 in the first nine months of 2013.  Our client should continue to track monthly electricity and gas use, noting interventions.  A move to daily energy collection will yield even greater insights.

Conclusion

While daily energy tracking and predictions gives you a great platform for improvement, most facilities do not yet have daily energy records.  You can get started with monthly data, even aggregated at the building meter level.  We described seven steps to determine whether or not there are savings.  If there are savings, then you can estimate the amount of savings.  The method accounts for the effect of outside air temperature, which is a major factor in energy use.   You may need some help with to build the regression model and control chart.   Manyorganizations have quality improvement specialists who know how to do this work.   Invite a QI specialist to a cup of coffee and ask for help!

Technical Notes

Model Issues

We used adjusted values for energy use rather than direct observations of monthly energy.   The adjustments, likely to induce slight negative autocorrelation, have an uncertain impact on the analysis.   The best fix we know is to move from monthly data to daily data.   Then the relationship between energy use and (daily) temperature is straightforward.   More importantly, daily data gives a30-fold increase in sensitivity as we seek to detect the impact of changes to improve performance.

The model we used has a strong assumption built in:  every month electricity use increases by more than 5000 kwh.   In 12 months, monthly electricity use would be higher by 60,000 kwh.   Future savings, relative to the model, will continue to compare actual use to predicted use that is increasing month by month.   At some point, we should discount this trend so we don’t over-estimate the savings.   It’s not clear how to make this discount.

The model does not fit 2010 very well and there seems to be an additional anomaly in December 2010.

Control Chart Issues

We used the median moving range to estimate the control limits; this calculation is not affected by one unusual point like December 2010.    The residuals from a regression model are positively correlated and so the control limits are narrower than if the values were uncorrelated.  I looked at the month to month correlation, known as “lag 1” for the model based on average mean temperature.   The residuals have modest non-zero correlations imposed by the model fit.  The largest lag 1 correlation between observations is about 0.20, with most of the lag 1 correlations less than 0.10 in absolute value.  Higher lag correlations drop off as the lag increases.   Following the discussion in Wheeler (1995) Advanced Topics in Statistical Process Control, SPC Press Inc., Knoxville TN, chapter 12, I used the ordinary limits, without adjustment. 

Uncertainty in the Estimate

From the control chart calculation, we can build a rough estimate of uncertainty for total energy reduction in 2013.   The control limits are 3 "sigma-hat" units from the center line, so σ-hat is 54552 kwh.  This gives an approximate 95% confidence interval for the energy savings as 820.000 +/-  327,000 kwh.   If we use the lowest monthly unit cost of kwh, we get an approximate 95% confidence interval of dollar savings as $67,200 +/- $26,800