Non-technical barriers to energy model sharing and reuse

An energy model generated during the design phase of a building could - in principal - be converted into a calibrated energy model and used to improve the building's operational performance. However, this rarely happens in practice. Through a survey of 306 building professionals, this research investigates whether this model reuse is technically feasible, based on today’s design-phase models, and what non-technical barriers might stand in the way of its implementation. An important finding is that 75% of the engineers/energy modelers surveyed believed that their models could be used by a third party in building commissioning and operation. At the same time, many modelers voiced various reservations that might prevent them from sharing their energy models with the owner or design team. These reservations varied from a desire to protect their intellectual property, to liability concerns, to the fear of incurring additional unpaid work. In response to these findings, this paper provides suggestions for overcoming these non-technical challenges and includes references for contract precedents.


Introduction
Today's commercial buildings are complicated machines, and, unlike cars or electronics, rarely are two buildings alike, making quality assurance difficult. As a result, problems such as installation errors, malfunctioning equipment, equipment degradation, and inefficient control strategies often go unnoticed [1]. Fortunately, strategies exist for addressing operational problems in buildings. For example, Liu et al [2] suggested that the energy consumption of commercial buildings in the US can typically be reduced by about 20% with improved operation and maintenance. Claridge et al [3] demonstrated, in a study of 34 academic buildings in Texas, USA, that payback times of one to two years could be realized using ongoing-commissioning in these buildings.
Research has found that computerized building energy simulation can significantly aid this process [4]. Computerized building energy simulation involves entering a custom data model of a whole building or specific building components and their physical properties into a computer program which then performs a calculation of all relevant heat and mass flows in and around the building, typically over the course of a year. Common model outcomes are energy loads, use, and occupant comfort predictors. Today, practitioners mainly use building energy simulation as a tool to compare various options during the design phase, to demonstrate code compliance or, occasionally, in measurement and verification to evaluate the performance of specific energy conservation measures [5].
However, in addition to these more common uses, energy simulation can be helpful in evaluating and improving operational buildings in the following processes: in building commissioning, or the process of verifying that building systems operate as intended [6], in on-going commissioning, i.e. commissioning that occurs repeatedly over time, to aid in fault detection and diagnostics [7], in commissioning or on-going commissioning to help refine building control strategies by testing various options [8,9], in model predictive control, to aid in real-time control decisions, based on dynamic variables such as weather [10], and to predict the savings from these measures. Perhaps surprisingly, practitioners rarely utilize energy simulation in these ways. What are the reasons for this?
Obvious obstacles to the use of energy simulation during commissioning previously were the effort and cost required to prepare an energy model. However, this cost argument is becoming less relevant since energy models are increasingly available anyway from the aforementioned design and code compliance modeling efforts, such as the UK's Building Regulations Part L [11] or for voluntary green building rating systems such as the US Green Building Council's LEED system [12]. Compliance with a program such as LEED typically requires a detailed, whole building energy model. Yet, these models are typically ignored after the design phase. In theory, this growing supply of design-phase energy models could be repurposed and reused later to improve building operation as described above. However, it is well understood that design energy models need to undergo a calibration procedure to become suitable for commissioning.
Calibration involves tuning key simulation inputs such as temperature set point and occupancy schedules to actual building use rather than to assumptions made during design.
To highlight the potential magnitude of the effect, Figure 1 shows annual simulated and measured energy consumption for 98 LEED-NC version 2 certified buildings. The figure is based on a dataset [13,14,15] of buildings completed from 2000-2006 and for which the New Buildings Institute collected various information including simulated and measured building energy use intensity (EUI), or the measure of energy consumption per unit floor area. Figure   1 shows a scatterplot of measured EUI against simulated EUI 1 . If design-phase energy models were perfect predictors of actual energy use, all data points in Figure 1 would lie on the ideal model line shown. However, the figure shows a fair amount of scatter (R 2 =0.4 relative to the ideal model) 2 .
How much of an effort is it to calibrate design-phase energy models and how close should measurements and calibrated model predictions lie? It has been shown that carefully calibrated energy simulation models are able to model the annual energy use of commercial buildings to within a 5% error from measured consumption [16] or 5% mean bias error (MBE) and 15% coefficient of variation of the root mean squared error (CV(RMSE)) with monthly data [17]. Traditionally, the calibration process relied on expertise and potentially timeconsuming trial and error [4]. Today, a number of researchers are working on improved workflows to improve this process [7,18]. Logic suggests that such technical innovations coupled with the growing supply of energy models create a context that is ever more conducive to the use of energy simulation throughout the life of a building.
However, there are also non-technical challenges to reusing design-phase models postdesign. These are the focus of this paper. They include the potential unwillingness of modelers to share their digital files which may be necessary, especially for longer-term uses of the model such as on-going commissioning. To fully appreciate the significance of these non-technical topics, the reader may consider the history of two other digital file precedents: Computer Aided Design (CAD) and Building Information Modeling (BIM). Through the 1990's and early 2000's in the United States, design professionals were very wary of sharing digital 1 The natural logarithm is a transformation used here, and commonly in statistical practice, to make a curved relationship more linear. This transformation was found to be appropriate here through an analysis of normal probability plots using Stata Release 10 software. As logic suggests, buildings with greater EUI tend to have a greater divergence between simulated and measured data. The data transformation helps to highlight the relative rather than absolute error. CAD files due to fears of inaccuracy, misuse or liability [19]. In cases when architects and other design professionals did share digital files they were often accompanied by, in the words of the American Institute of Architects (AIA), "draconian disclaimer notices" which forbid reliance on the digital information [19,20]. With the advent of modern digital file sharing agreements the situation has improved [19]. However, problems persist even today when, in theory, BIM offers a platform for architects, engineers, and contractors to contribute to or extract information from one coordinated digital file. Barriers such as concerns about liability, intellectual property [21], unresolved issues of model ownership/maintenance, and insufficient legal contracts [22] significantly impede both the adoption of BIM and its implementation to its fullest capability. It is likely that these same types of barriers also impact the shared use of energy models. To understand whether this is indeed the case, the authors prepared an online survey investigating these and other barriers to energy model sharing and reuse. The results of this survey, along with some possible solutions, are presented in the following. Project Delivery, and post-occupancy evaluation; however, these topics are outside the scope of this paper.

Methodology
The focus groups for the survey were building owners (preferably informed owners of multiple buildings), architects, HVAC engineers, and energy modelers. The authors primarily recruited participants through email invitation to selected mailing lists. Table 1 identifies the email lists used to recruit the majority of survey participants.
The answer format included multiple-choice and write-in responses. Participants were asked to specify which of the focus groups (owners, architects, design engineers, or energy modelers) they belonged. The last two categories were presented with the same set of questions which is why the results are sometimes presented herein for owners, architects and engineers/energy modelers. In this paper terms such as "owner", "architect", etc. may refer to a person or an entity; e.g. an owner may be an institution. Because of the survey's branching format and the option to skip questions, not every participant answered each question. The authors tallied the multiple choice responses and performed some basic statistical analysis as presented in the Results section below. The participants provided extensive information via write-in responses throughout the questionnaire. The authors discovered popular themes apparent across these responses. These themes included commonly perceived barriers to both the feasibility of energy model reuse and the willingness to share digital models. Some of the concerns revolved around legal issues. In response to these concerns, the authors performed a literature review and found that standard contracts have already begun to address these issues in related fields. In the discussion section, the authors present these precedents and provide other suggestions for overcoming the barriers to energy model sharing and reuse uncovered in this survey. The following presents the most relevant survey questions and answers. The complete survey questionnaire is available from the authors.

Participants' Background
A total of 306 individuals participated, identifying themselves as 116 energy modelers/energy consultants, 38 design engineers, 34 real estate owners, and 118 architects. Responses came from 31 countries across the globe with the majority from the US (60%) and Canada (13%). The modelers/engineers were a more international group, with 54% coming from outside of the US. The architects' firms varied in size from less than 5 employees (26%) to more than 50 employees (47%). As hoped, the participating real estate owners held relatively large portfolios with a median size of 160,000 m 2 (1.75 million ft 2 ).

Current Practice
The survey explored the current practices of each participant group related to understanding and improving building energy consumption. Two results are presented below.

Evaluating the Energy Performance of Designs
First, the questionnaire asked architects if they evaluate the energy performance of their projects. A total of 100 architects responded. Via multiple choice, 33 indicated that they

A Problem with Energy Modeling
Next, the questionnaire asked the modelers/engineers to list the software they primarily use. The 116 respondents listed 14 different energy simulation programs, none of which was listed by more than roughly 20% of the respondents. Later, the survey asked modelers/engineers to list up to three major problems that they frequently encounter during energy modeling that they think prevent its more widespread and cost-effective implementation, and "software interoperability issues" was listed as an example. This was the most frequently cited issue as 40 of the 98 respondents mentioned it in some way. Most elaborated on the difficulty in transferring information between CAD, BIM, or other analysis software and energy modeling programs. Many respondents mentioned that universal file formats, such as gbXML [23] and IFC [24], exist but that transfer workflows are still troublesome.

Client Interest
After this initial probe into current practices, the authors sought to gauge the readiness of the industry to extend the use of building energy models beyond the building's design phase.
Therefore, the survey asked owners whether they were interested in the following scenario:

If properly calibrated, an energy model can help you… to closely monitor and often substantially lower the energy use of your buildings as well as alert you if parts of your HVAC systems fail or become less efficient over time. To make such use of an energy model you need the help of a trained building modeler.
The owners surveyed showed almost unanimous positive interest, 91% (21), in providing some form of compensation for in-house or outsourced services to use energy models for ongoing-commissioning, at least in the best-case scenario described.

Technical Feasibility
The survey asked modelers/engineers if they think that their energy models could -in principle-be used by the owner or another member of the design team during commissioning and operation. As illustrated in Figure 3, the majority (88 of 118) responded with 'yes'.
The remaining 30 respondents who did not believe that their energy models could be feasibly reused by another party were asked why. In the authors' judgment, all the write-in responses fell into four categories of concern, paraphrased by the authors in Figure 3.
The survey gave the example, "the model is too specific or complicated for somebody else's use." This example was then reiterated in some way by 18

Willingness to Adapt Model
As a follow-up question, the respondents who did not believe that their energy models feasibly could be reused were asked if they would be willing to adapt their energy model if an important client made this a firm contract requirement. The 28 respondents indicated that they would: not adapt their model (4), adapt and share their model under certain conditions (12), change the simulation program and share their model under certain conditions (13), or not change the simulation program (6). Only 14% of all modelers (4) indicated that they would not adapt their model in any way to accommodate ongoing commissioning.

Willingness to Share
Then, the survey asked, under what circumstances respondents would be willing to share their energy models with the owner or the rest of the design team. Figure 4 shows the results. Respondents who chose at least one of the right three choices indicated an unwillingness to share their model. Of the 112 respondents, 37% (41) indicated that they would NOT share their models with the owner or design team. As Figure 4 shows, concerns over intellectual property and liability acted as barriers preventing model sharing for 18% (20) and 12% (13) of respondents. Respondents previously mentioned similar concerns, as noted in 3.3.2, without any prompting by the survey wording.
As an alternative workflow, 34% (38) of respondents would potentially offer the owner, as an additional paid service, calibration of the model rather than the model itself to help the owner with building commissioning. Some respondents would include stipulations in order to share their models. Specifically, 47% (53) would require the recipient to sign a disclaimer indemnifying the modeler against responsibility for the conclusions/actions of others based on the model; 32% (36) would prohibit unpaid modeling support, 30% (34) would contractually prohibit the owner from sharing the model with a competitor; and 25% (28) would request an additional fee for sharing the model. Finally, 38%, (32 of the 84 offered this answer choice) indicated that they already share their models. In the write-in responses, five respondents described their liability or intellectual property concerns in greater detail, and one respondent explained that his willingness to share the model depended on its eventual use. He welcomed interaction with a sophisticated user but was not willing to train someone else to use the model.

Additional Fee for Sharing
As a follow-up, those who indicated that they would request an additional fee for sharing their energy model were asked what that fee would be. The example given was 25% of the regularly paid fee for service. A total of 28 modelers/engineers provided write-in responses.
The answers varied greatly and included both "hourly" and 5-50% of the modeling fee. Some of the respondents answered with a percentage, but did not specify to which fee this percentage would be applied. The authors used their judgment in interpreting these responses. Generally, the answers gravitated around 20% of the energy-modeling fee. Some of the respondents qualified their answers by explaining that the fee would be necessary to pay for additional work such as gathering background documentation or switching the model to a more user friendly format.

Model Ownership
Finally, the survey asked each professional group about energy model ownership. 3 As illustrated in Figure 5, this question sparked a great deal of disagreement. In fact, the respondents tended to believe that, in the absence of specific contract language, their own group owned the model. Local regulations may affect model ownership; however, even responses from the same locales disagreed over model ownership. Respondents offered additional write-in responses that confirmed the state of confusion, as some respondents stated that whoever built the model owns it (5), while others stated that whoever paid for the work owns it (12).

Discussion of Survey Results
As noted, 116 energy modelers/consultants completed this survey. For reference, this sample represents a significant portion, almost 6%, of the 1992 individuals subscribed in 2010 to the email list, onebuilding.org [25], and the authors believe that representatives of the overwhelming majority of expert energy modeling firms in the world subscribe to this list. For building owners and architects the sample percentage is of course much smaller, and while these limited samples do not characterize the entire population, they do provide evidence of the existence of certain viewpoints.
The fact that specialized energy models have been used for commissioning, on-going commissioning, and other operational purposes in semi-controlled research settings, as described in the introduction, does not necessarily mean that the body of today's typical design-phase models would be appropriate for the task. Therefore, the fact that 75% of the participating energy modelers, who constituted a relatively sizeable sample of the industry, believed that their models could be used in commissioning and operation was the central outcome of this survey.
The survey results emphasized that the energy simulation software market has not yet consolidated and problems of interoperability exist. This is an issue of potential importance if models are to be shared between individuals and potentially used over the lifetime of a building. However, 46% of modelers indicated a willingness to change their modeling software if it were a contract requirement. Similarly, human interoperability, or the ability of the future user to understand the original author's work, also emerged as a potentially important barrier to model reuse. Fortunately, 43% of modelers indicated a willingness to adapt their models.
Although the participating owners may not constitute a representative sample, they do confirm that a subset of large-portfolio building owners exist with an interest in providing compensation for the use of energy models for on-going commissioning. This means that there is both a product and a potential consumer. As the survey revealed, one barrier separating them is an unwillingness of some professionals to share their energy models.
Meanwhile, confusion abounds over who owns the model and thus the right to make that decision.
Concerns over intellectual property and liability impede sharing practices for some modelers, and others desire protection against unpaid services and exposing trade secrets to their competitors. Fortunately for them, the building industry already has experience in dealing with these issues for other digital file types such as CAD and BIM. While the solutions are still evolving, they may offer precedents for concerned professionals in the energy modeling community. Therefore, some examples are presented below. 4

Digital File Sharing References
In the United States, the US Copyright Act [26] and the Architectural Works Copyright Protection Act [27] govern the issue of model ownership. However, both laws show a lack of sensitivity to post-digital age works; therefore, the design and construction industry usually interprets these laws to mean that the creator of content owns that content [28], and owners often establish a license for the intended model uses. The American Institute of Architects (AIA) Digital Data Licensing Agreement [29] and other references listed herein [30,31,32] include example licensure language for digital files.
The survey responses revealed a fear among some expert energy modelers of losing their competitive edge if beginners were to gain access to their custom tools and modeling techniques. Although not foolproof, other industries have used non-disclosure agreements (NDAs) to help protect such trade secrets. A number of free NDA templates are readily available via internet search. More specific to the building industry, AIA documents C106 [29] and E201 [30] include a clause limiting the disclosure of confidential information.
Some respondents described fears of being held accountable for changes made to the model by others or for decisions made based on the model outside of its original intent. Sections 5.1 and 5.6 of the ConsensusDOCS BIM Addendum address such liability issues [31].

Advice for Model Reuse
In light of this investigation, the authors offer the following advice for building owners or others interested in reusing design-phase energy models later in the life of the building. First, convey the end-goals for the model up-front. Although not written specifically for energy models, the State of Ohio Building Information Modeling Protocol [33] offers a precedent for defining end-uses, specifying levels of detail and accuracy, and requiring as-built updates.
Next, require the documentation and submission of modeling assumptions, conduct a handoff meeting to orient the next user, specify the desired software format, and expect to pay an additional fee for additional services. Finally, design-phase models should most importantly support the design process, and as the survey confirmed best practices include implementing simulation early in the process.

Future Research
This paper dealt with the non-technical challenges in design-phase energy model reuse. In response to the question of feasibility of reuse, one survey participant summarized the technical challenge: most energy modeling "is a comparative exercise, not a predictive exercise. Although energy models can be converted to more predictive types of models by calibrating them against a building's actual historical energy usage, this is a whole other exercise that is potentially even more involved than the original design phase type energy modeling exercise itself." The authors agree with that generalization of today's models.
However, the important question is not whether calibration is more difficult than creation of the original model. The question to be answered is whether the benefit of reusing the model can outweigh the cost. The poor performance of our commercial buildings leaves a large margin for improvement. Carefully documented case studies are now required to quantify the financial and environmental benefits of using calibrated energy models for operational and financial decision-making throughout the life of the building.

Conclusion
This paper presented results from an online survey on the potential use of calibrated designphase energy models in building commissioning and operation. The 306 responses reflected a sizable interest, especially in the energy modeling community, in this topic. Considering the enormous energy savings potential of commissioning and ongoing-commissioning, the question investigated was whether utilizing the energy model in these processes is technically feasible and whether professionals are willing to engage in the process. The survey results indicated that 75% of modelers believed it was, in principle, feasible with their models, and a group of large-portfolio building owners was potentially interested in using energy models in this way. Some energy modelers expressed concern over sharing their digital files. In response the paper referenced contract precedents from other digital file types. The authors conclude that a framework exists and expect energy model reuse beyond the design phase to increase in the near future. Both technical and non-technical challenges surround model calibration and reuse. Nevertheless, energy simulation is a powerful and relatively young tool, and the frontiers of its utility deserve more exploration.