The trouble with Post-Occupancy Evaluation
Most studies into the performance of buildings take the form of structured post-occupancy evaluation (POE) investigations. Usually they concentrate on the evaluation of technical performance issues such as the effectiveness of the heating, daylight, acoustics, energy use and overall occupant comfort. Normally a survey of occupants to elicit views on building perception will also be included.
Predictably, where these studies find failures researchers stand on presentation podiums to cite poor construction information, design inadequacies or "tail wagging dog" controls complexity as the root causes. Alas, this information does little to excite the enthusiasm of owners or occupiers towards wanting to do POEs in the first place, let alone follow-up on the improvement of the building or it's performance. Indeed most POE findings being technical in nature go completely over users heads, and only serve to the frustration with the industry.
Critical research of the now almost standard approaches to POE have highlighted that most methods overlook the fact that users require a different value proposition than that typically on offer from the industry. It is apparent that at the interfaces between buildings and users, there are complex sociological, even psychological and cultural, as well as technical issues at play, none of which are being successfully addressed by modern building feedback methodologies. If POE is to be of benefit it needs to research the things users want to know about, which are invariably social and operational rather than technical and procedural.
A starting place in the search for more user-centric outcomes might be to look at how other industries have addressed similar problems, and how by using similar design and review feedback techniques to them, the property industry might replicate their success in meeting users’ needs to hit the defined “performance sweet spot” with certainty.
Investigations indicate that the hospitality, technology media and telecommunications (TMT), aeronautics, and automotive industries each invest large sums annually in researching customer feedback and experience of their products. Within the TMT industry, a specialist sub-sector called “UX” an acronym for user experience has matured around what are known as Agile Project Management (APM) techniques. Rather than waiting until the launch of the product, APM looks for user feedback not at the end but at the beginning of the project and constantly samples it to completion. So successful is this approach in mitigating the risk of building the wrong thing, APM has become a project management methodology in its own right. One that has overtaken PRojects IN Controlled Environments (PRINCE2) as the preferred design tool methodology in these user savvy sectors over the last three decades.
APM grew out of the post-war United States, European and Japanese war rebuilding programmes where automation and efficiency were needed to offset the loss of working-age labour. Over the next fifty years, Agile Methods (unfortunate naming terminology sometimes confused with Lean and Kanban or agile office design, which are somewhat different) was recognised as the solution to solve the high risk of customer rejection inherent in conventional Taylorism PRINCE2 software design management approaches.
Continuing the development of APM into the 1970’s Winston Royston coined the term “waterfall”, as an almost derogatory descriptor for Taylorism in project management, i.e. following a linear process and falling over weirs or milestones to the next stage. Inferring that once the process flow had commenced, and with little user feedback and plenty of designers assumptions and a few rounds of value engineering along the way, it became difficult to halt the momentum generated, and the project, with its inbuilt and undiscovered inadequacies, went with the flow to completion and handover, as is the case in construction today.
Royston contended that with waterfall, certainty of satisfaction with the outcome is not determined until the process is complete. Cost and time estimation is just that, an estimation and is invariably wrong.
Eric Reis is more scathing of Taylorism’s waterfall process listing several defects with it;
Scope Bloat. - Cram in as much as the budget can afford. Aspirations usually exceed budget eventually leading to cost overrun and “de-value engineering”.
Blind Investment – cost risk remains and is only mitigated on completion.
Architectural Risk - Design risks come to surface too late
Functional Risk – Users don’t get to know if it works until the end.
Change Risk – Cost risk supersedes change risk, meaning it is never adequately addressed.
Uncertain Cost and Duration – Large numbers of change requests are commonplace.
Resource levelling/availability –Windows of availability never align. Availability of the right people input at the right time is difficult to align, which reduces quality.
Team motivation – When projects go wrong commercially, interest wains, quality declines further, user outcomes drop to the bottom of the priority list.
All the above being factors that affect the production of buildings, and which ultimately lead to uncertainty and user dissatisfaction with the out-turn product. Elements which led Bill Bordass to conclude;
Wherever one looks in building-performance studies, one tends to see under-achievement. This is not because researchers are unduly gloomy, but because this is what they find in reality."
So why is project failure almost a norm for users? Expanding the work of Royston, Edwards Deming with the experience of several years troubleshooting projects in post second world war Japan, proposed his 14 principles for project or process management success.
From his post project review experiences, Edwards Deming observed that where failure of a project was identified, it is usually the product of a management process failure, rather than a failure of execution or the competence of the operatives implementing it. He noted that managers of the failed projects also diagnosed the causes of their inability by incorrectly attributing the reasons for project failures on technical non-compliance or operative incompetence. All of which sounds familiar to those of us who have critiqued modern post occupancy evaluation processes.
Where projects failed, Edwards Deming observed, the failure had its origins in the generation, management and care taken over the quality of information arriving or leaving the design table, and usually had little to do with the execution of the planned procedure at site level. Suggesting that when a project goes wrong for users, we should be examining the process, not the procedures. Edwards Deming proposed that projects should have the core purpose of increasing quality for the customer from the outset. Where quality becomes the focus, the project team will find better ways to reduce time and cost by default.
The 14 principles led Deming to the corollary;
(a) When people and organisations focus primarily on quality (of information), defined by the following ratio, quality (of the product) tends to increase, and costs fall over time.
(b) However, when people and organisations focus primarily on costs, costs tend to rise and quality declines over time.
This movement towards improving quality and a focus on how information flows within the design process became Edwards Deming’s theory for advocating a separate design management process, splitting and separating the management of the design from the management of the construction process.
If the indications are that the construction industry is failing to meet current expectations of its customers now, the situation can only exacerbate with time. The industry must find ways to understand the needs of its end user customers thus defining and fixing quality more meaningfully into design processes from the very beginning. The current fixation of the industry with legal and technical criteria as benchmarks of successful delivery does little to achieving quality, and only serves to compound these constraints to the detriment of customer outcomes.
Lessons learned from other sectors such as TMT indicate that the construction’s current pre-occupation with cost and contracts, over quality and effectiveness, is ultimately unsustainable, serving only to initiate a “spiral to the bottom” as argued by the industry’s own seminal report “Modernise or Die”. Despite this, most in the industry agree the finish and quality of the built environment product is high, but users through POE studies still find fault and complain. The industry is invariably building the wrong product by placing value in the wrong areas. Users feel the design is being done “to them” not “with them or for them” which is what ultimately lies at the heart of the “performance sweet spot” issue. It seems the logical solution to this situation is to involve users as customers in design and to do so in significantly more immersive ways. For us this means adopting an Agile Methods APM style of project management, where users are naturally integrated into the development of the new work or living place.
As for POE, as a concept it is predicated to measure failure within an industry where revelation of the product and its cost to the customer happens on the last day rather than the first. For this reason it will invariably measure technical achievement, just because that's what those conducting it understand best. What the property industry really needs to get its head around is how we understand what users want from the beginning so that we understand what is the right thing and the right quality to build before we fixate on building and then measuring the wrong thing at the end.
#postoccupancyevaluation #BREEAM2018 #POE #usercentricdesign #agilemethods #UX #buildings #propertydesign #APM #ModerniseorDie