Financial reporting on public cultural and scientific heritage assets (government-held collections) has been a requirement of Australian state governments for several years now. As a result, collection valuations are becoming familiar tasks for many museums, art galleries and other holders of public collections. Accounting for these assets can be a complex task from two angles.
Firstly, determining the value of a collection item in a monetary figure depends on many factors with varying levels of subjectivity and logic (Barton 2005; Carnegie & West 2005; Hooper et al. 2005). Secondly, many organisations have collections that number in the hundreds right through to millions of items, which constitute a wide array of item types, stored and organised in a wide array of methods. Most organisations required to undertake a collection valuation will not be able to simply obtain a monetary value for each individual item and add up the total. Instead a sampling methodology needs to be employed to calculate an accurate estimate of the total value and this can be a difficult process when conducted for the first time.
Introduction
Financial reporting on public cultural and scientific heritage assets (government-held collections) has been a requirement of Australian state governments for several years now. As a result, collection valuations are becoming familiar tasks for many museums, art galleries and other holders of public collections. Accounting for these assets can be a complex task from two angles. Firstly, determining the value of a collection item in a monetary figure depends on many factors with varying levels of subjectivity and logic (Barton 2005; Carnegie & West 2005; Hooper et al. 2005). Secondly, many organisations have collections that number in the hundreds right through to millions of items, which constitute a wide array of item types, stored and organised in a wide array of methods. Most organisations required to undertake a collection valuation will not be able to simply obtain a monetary value for each individual item and add up the total. Instead a sampling methodology needs to be employed to calculate an accurate estimate of the total value and this can be a difficult process when conducted for the first time.
As a government organisation holding public collections, Museum Victoria (MV) has undertaken four collection valuations since 1995 to fulfil the requirements outlined by the state accounting regulations. For each valuation, MV has contracted the Australian Valuation Office (AVO) to oversee the project as an external party. Therefore, AVO is responsible for sub-contracting all personnel required, including statistical consultation, ensuring all accounting requirements are met and that the methodology is approved by the appropriate regulatory bodies. This removes a large part of the reporting responsibility from the museum and as such, these particular details will not be covered in this paper. Instead I will focus on MV’s most recent collection valuation as a case study discussing the procedure and documentation of the project.
Context
Museum Victoria holds approximately 16 million state cultural and scientific assets plus a substantial collection of books, journals and other documents as part of the MV Library (~38,000 items). These collections are housed or exhibited across five MV campuses in Melbourne, Victoria. The state and library collections are extremely diverse in nature and scope, with minute through to very large objects, artefacts and specimens plus a wide assortment of images, documents, books and artworks.
The first valuation of MV’s state and library collections took two years to complete from 1995 to 1997. It was the first time this had been attempted by MV and AVO and it required the development of the statistical methodology, which was tested with a pilot study. Until 2011, this was the only full financial assessment of the collections. Deprival value was used as the basis for determining the value of collection items with a market-based value and re-collection costs were calculated for items without an existing market (e.g. most natural sciences items). The two collection valuations that followed in 2002 and 2005 simply were indexed revisions of the previous total values adjusted for changes in current value estimates plus the addition of new acquisition values. The basis of how heritage assets in Victoria were valued also changed from Deprival Value to Fair Value.
In the 14 years since the first collection valuation at MV, several things had changed: key staff involved in the first valuation were no longer available; consolidation of electronic collection data into a single database (KE EMu) and and increased amount of collection records on this system; and all collections had moved campus and storage arrangement at least once since the first valuation. We had significantly less than two years to conduct the 2011 collection valuation but we were fortunate to have enough documentation from the 1995-1997 project to draw upon. This documentation included the final reports (AVO & MV, 1996 & 1997) and two filing cabinet drawers of notes, data, diagrams and results. From this we had sufficient information to be able to generate a framework for proceeding with the valuation but it was clear that having good documentation could improve the efficiency and progress of repeated projects such as this.
Documentation tools
1. Collection Information Management Systems.
MV uses KE EMu database for its state collections and Ex Libris Voyager for the library. With KE EMu, valuation data and metadata were recorded for collection items through the Valuation Module and Record Activity Tab. MV has also developed a Collection Location System that is integrated with KE EMu called MVWISE. The software can be operated through hand held personal digital assistants (PDAs) and is primarily used to manage item locations. However, for this project MV developed a workflow that enabled direct entry of valuation data into collection item catalogue records.
At the time of the valuation, the library database did not have a specific valuation field in use, but we were able to use a similar, unused field for our valuation figures.
2. Electronic Document and Records Management Systems.
MV uses TRIM for recording electronic and hardcopy documents and emails. This has been the primary tool for retaining the documentation relating to how we undertook the collection valuation. This included emails, drafts of sampling frameworks, spreadsheets containing primary sampling data and notes from meetings and discussions.
3. Reports.
Although not written by MV, the official report produced by AVO, (including the statistical report) documents the necessary compliance with state and federal regulations, provides detail on the methodology and statistical process, and provides the overall final results.
Valuation methodology
The valuation methodology (as opposed to the sampling methodology) was developed by AVO and followed the methodology undertaken in MV’s 1995-1997 valuation. Fair Value was used as the basis for determining values, which was undertaken by professional valuers (‘Approved Valuers’ for the Federal Cultural Gifts Program). The only exceptions were for a small percentage of sensitive indigenous collection items that were completely excluded from the valuation and most natural science and archaeological items that do not carry market values. These latter collections were assigned a re-collection cost based on average price per catalogue unit at different ‘distance’ tiers (e.g. a frog collected from central Victoria received a ‘Local’ value, while a sea star collected from Antarctic waters received a ‘Remote’ value, which is substantially more) (AVO, 2011). This method of accounting for most scientific items and other objects that have no market-based value is still flawed but is probably the most objective methodology currently in use.
Statistical methodology
Due to the number and diversity of items held by MV, we worked with a statistical consultation team for both the 1995-1997 and 2011 valuations. The random sampling methodology that was developed and applied to the collections in 1995-1997 was broadly replicated for our recent valuation by a team from the University of Western Australia’s Centre for Applied Statistics. This sampling methodology enables the collections to be divided into sampling units that contain items that are broadly similar in nature and value. From there, each sampling unit is either valued completely (as a census if there are few items) or a sample is taken to estimate the total value. Each sampling unit will have its own sampling strategy, which depends on the type of items in the unit and how variable in value they may be.
Documenting the process
The reporting requirements relating to the sampling methodology were the responsibility of the statistical consulting team. They provided detailed explanations of the methodology and the relevant statistical information relating to the sampling units so the information can be used as reference for future valuations (AVO, 2011). This included information relating to the number of items in each sampling unit, how many tiers of sampling there were – if stratified sampling was applied, what percentage of items were physically valued and the percentage of statistical standard error1. However, MV has retained the drafts, notes and other contextual documentation relating to the sampling methodology that could prove beneficial when undertaking the next valuation.
Fortunately as this was our second full valuation of the collections, the establishment of the statistical framework was simplified significantly for two primary reasons. Firstly, sampling units had already been identified in 1995 for all items held in the state collections and library. While many of these units had changed composition or no longer existed, we were still able to broadly follow the former sampling units and reassess them to ensure they accounted for all items held as state or library collections. This required extensive consultation with the relevant collection managers and curators right across the museum to ensure we had captured all our collection assets that had been accessioned at the time we conducted the valuation. This included items that were not entered on the database or even registered, the majority of which are natural science specimens as they compose more than 90% of the state collections at MV. Secondly, we were able to consolidate some sampling units and rapidly account for the total number of items in some groups due to the increase in electronic catalogue records. This removed some of the need to account for item location when defining the sampling units.
The first and one of the most important sampling units identified were the items of high value (in a financial sense), which were all valued individually. From a statistical point of view, these items are placed in their own sampling unit because they have the potential to skew the accuracy of a total value calculated from a sample. For example, if a high value item is randomly selected within a sampling group of middle-low value items, it will skew the value up. Vice versa, if it is missed within the sample, the value of that individual item will not be accurately reflected in the calculated value of that sampling group. From the point of view of the museum, this was an opportunity to take stock of items that are deemed high value and to obtain current estimates of value, which can be useful for insurance purposes or security reasons (for MV the high value threshold is $50,000 but for other organisations this may vary depending on the type of items held). The items identified as high value items in the 1995-1997 valuation, could be grouped electronically in the database. As curators and collection managers identified items that were missing from the list, they could easily be added. After accounting for the high value items, the remainder of the collections were subject to sampling with the exception of some groups that contained few enough items to warrant individual valuation (similar to the high value items). For units that required sampling, there were two ways this could be done: physically or electronically.
Physical sampling
Physical sampling was the only method for selecting items during the first MV valuation in 1995-1997 and it remains the only way to sample the parts of the collections that are not on the database. Physical sampling usually requires the collection items to broadly be located in the same place and in a similar fashion with clear, identifiable ‘tiers’ of storage that enables rapid quantification and estimation of the items within each stratum of the sampling unit. For example, for a sampling unit that contains items stored in drawers within cabinets, the cabinets would form the first stratum of sampling, and the drawers the second before selecting the items to be valued. The statisticians calculated how many things within each stratum (be they cabinets, drawers, shelves, pallets etc.) needed to be selected at random (e.g. every second cabinet is selected and two drawers are randomly selected from each, followed by one item randomly selected from each of those drawers, resulting in two items being individually valued from every second cabinet within that sampling unit). Although it is not necessary to know how many items there are exactly within the sampling unit, the total number of things in each stratum (e.g. the cabinets) do need to be quantified (i.e. all the cabinets need to counted followed by the number of drawers in each cabinet selected, followed by the number of items within each of the randomly selected drawers).
In some instances, in order to reduce the amount of statistical error within some sampling units, the statisticians identified the base stratum (the level at which the valuation takes place) to be a whole drawer or shelf or similar, rather than individual items. This evened out the variation in value for the sampling unit, but meant individual items were not valued.
Statistical sampling
Statistical sampling was used for some sampling units in the 2011 valuation. This method was statistically simpler but required items to have been entered on the database and most importantly, they needed to be retrievable from the database into discrete sampling units. For some parts of the collections, this was relatively easy (e.g. all mineralogy specimens had been entered and could be retrieved in a simple search to form a sampling unit), while others required a slightly different approach based on location (e.g. all the large social history items located within a certain store could be retrieved electronically by searching on the location but not on a particular category that unites them all). Once lists were generated for each unit, we could identify exactly how many items there were and the statisticians would randomly select the items to be valued from the list. With this sampling approach, great care needed to be taken to avoid duplication of items across different sampling units. This could have inflated the number of items held in the collection and items may have been accidentally selected for valuation twice, both of which would have affected the statistical accuracy. It also became slightly tricky using a combination of physical sampling for some units alongside electronic sampling for others. This meant we had to ensure the lists of items produced for an electronic sampling unit were not located within an area that was to be physically sampled.
One of the main problems encountered with electronically sampled units related to difficulties in locating or accessing the items selected for valuation. In some cases the items selected were physically too difficult to access within a reasonable time frame (e.g. stored in crates or on the top level of pallet racks). In others it was due to poor location control with items not where they were supposed to be. For the units with more than half the items unlocatable or inaccessible, we had to re-sample physically to ensure accuracy and coverage of all items.
Documenting the results
One of the primary concerns for MV during this project was to maximise the amount and speed of the results that were data captured. The results documented by AVO only list the cumulative totals, not the individual values, which are of more use to the museum. For the 2011 valuation project we created a new data entry workflow for MVWISE (MV’s Collection Location System) that enabled direct entry of valuation data into the KE EMu database in real time. The data that could be entered included the item value, the name of the valuer and whether the item was physically sighted. At the same time, metadata on who was entering the information, when it was entered and what project it related to, was simultaneously recorded through the Record Activity Tab in the Catalogue Module. The advantage here was that we could enter valuation data into the database in real time while objects were being valued, cutting out the need to manually interpret and enter data at a later stage and also to automatically record data on who was involved and when it happened.
As MV holds a wide diversity of collection items, we required the services of several different professional valuers to cover the scope of expertise needed. We also attempted to provide as much data on the items to be valued to the valuers prior to their visits to the museum. However, even then we found many valuers are unable or prefer not to place a value on an item in ‘real time’, which invalidated the direct data entry. Fortunately this was not the case for all valuers, and in those instances, the MVWISE workflow was successful and efficient, enabling the export of results to the statisticians the next day. Results that were provided sometime after the valuer had seen the items were still entered through the MVWISE workflow as it automatically captured relevant metadata.
Despite trying to avoid having loose hard copies of valuation results (as they could get lost or be misinterpreted), we still printed hard copy notes for each group of items that were to be valued. These were provided to the collection managers who had to locate the items and to the valuers to provide some background information that may not necessarily be on labels with the item (e.g. age, provenance, past valuation figures). As printed item lists were being made anyway, we decided to provide a space to enter any comments from the valuers and the item value. Many valuers did provide written results on these printed sheets and although the data is now also stored within the database these have been retained as the hard copy results from the valuation and archived in TRIM. In the instances when the valuers provided results at a later date, many sent their results electronically which have also been retained in TRIM.
One of the main issues we encountered with the recording of results was in relation to the ‘whole drawer valuing’. These were the cases when it was a whole drawer or shelf containing one through to hundreds of items that was valued rather than an individual item. As the valuers only provided an estimate of the value of everything they saw, we could not record the results in the database, so we just documented them through hand written results and spreadsheets that have all been retained in TRIM. These group results have little use to the museum and it is highly likely that the composition of the particular drawer, shelf or pallet that was selected for valuation will change in future.
For the majority of MV’s natural sciences collections (exceptions being mineralogy, meteorites and tektites, and high value zoology and palaeontology specimens), re-collection costs based on the provenance of the specimens were applied as the specimen value. This meant that we did not require a valuer to view the selected items and also meant that for the parts of the collections that have been entered on the database completely, we could apply a value to every individual item as no sampling was required. This was applicable to all our vertebrate zoology collections, which encompass more than 100,000 specimens combined. As these were all on the database, we could upload the valuation data into each record and archive the original upload spreadsheets.
Conclusions
During the 2011 MV collection valuation we attempted to improve the project documentation and increase our data capture. We found there were certainly many advantages to having an improved electronic collection database, including having access to more comprehensive data on collections (e.g. size of collections, where located) and on individual items. The method of electronic sampling has the potential to be much more efficient than physical sampling, but until all items have an electronic catalogue record, physical sampling will also have to be undertaken for some sampling units. Improvements in data entry methods (including the direct entry through MVWISE) have also ensured that all valuation results were added to the database before the completion of the project (with the exception of some natural sciences items that have not yet been electronically catalogued), and we have archived all the valuation results in TRIM.
While the reporting requirements were taken care of by AVO and the statistics team, the documentation that MV has retained will provide further details and reasons behind many of the decisions made and solutions found to issues and problems that were encountered. As the reporting requirements currently stand, MV and all the Arts agencies in Victoria will be re-valuing their collections every five years. It is anticipated that some of the improvements and new methods MV has tried to implement and document during the 2011 re-valuation will make the next one slightly easier and more straightforward.
By Karen Roberts, Collection Manager, Sciences (Museum Victoria)
Presented at the 2013 Museums Australia National Conference
———
Footnotes
1A unit for which all items were valued would have a 0% standard error, with the percentage rising to reflect an increasing variation in values obtained for the items in that particular group. To reduce standard error (and improve accuracy), the number of items sampled for valuation can be increased, or the unit can be divided further to lower the variation in value that may be encountered in a single group of items.
References
- Australian Valuation Office and Museum Victoria. 1996. Museum Victoria valuation of collections Stage 1.
- Australian Valuation Office and Museum Victoria. 1997. Museum Victoria valuation of collections Stage 2.
- Australian Valuation Office, under instruction from Museum Victoria. 2011. Valuation of Museum Collection for financial reporting purposes as at 30 June 2011.
- Barton A. 2005. The conceptual arguments concerning accounting for public heritage assets: a note. Accounting, Auditing & Accountability Journal, 18: 3, 434-440.
- Carnegie G.D. and West B.P. 2005. Making accounting accountable in the public sector. Critical Perspectives on Accounting, 16, 905-928.
- Hooper K., Kearins K. and Green R. 2005. Knowing “the price of everything and the value of nothing”: accounting for heritage assets. Accounting, Auditing & Accountability Journal, 18: 3, 410-433.