Since it is Sunday morning, perhaps I should begin with a text. Romans 12:3 comes to mind: "For by the grace given to me, I say to every one of you: Do not think of yourself more highly than you ought, but rather evaluate yourself with sober judgement, in accordance with the measure of faith God has given you." But since this is the end of the conference, perhaps you would prefer if I used John 13: 27 - "What you would do, do quickly."
A friend of mine once told me that he would always remember the address at his college commencement. The speaker said, "There are two questions that are going to be important in your working life. The first question is the one you are asking yourself right now. You are looking at corporations and government agencies and academic departments and theater companies and a hundred other types of organizations and you see their imposing facades and the glow of purposeful, well directed energy and you are asking yourself, How will I ever get inside?' Well, don't worry about it. One way or another you will get inside. And then, after you have been inside about five years, you will ask yourself the second question. As you look around at the organization and your superiors and your co-workers and perhaps at yourself and you will ask yourself, What on earth holds this thing together!?'" Today I will give you a brief description of the efforts of the staff of my own archives to discover what held us together.
The Billy Graham Center was begun by Wheaton College and the Billy Graham Evangelistic Association (the BGEA) in 1974. Its purpose was to serve as a resource for the church in organizing evangelism efforts. It did this by setting up institutes, such as the Institute of Evangelism, by hosting conferences of Christian workers, by creating training programs, by publishing and by collecting resources on the history of evangelism. The resource component of the Center consisted of three parts - a Library for published materials, a Museum for artifacts and the Archives for the records of the BGEA. The Archives was the first department to move from plans to reality. Staff was hired in June 1975 and the work began.
Our first decade saw a variety of significant developments, significant in our microcosm anyway. Our collecting policy expanded to include material beyond the records of the BGEA. We would in addition collect papers and records that deal with nondenominational North American Protestant efforts to spread the Christian Gospel - evangelism in other words. This was an area of the documentation universe no one else was systematically going after. We became the repository for about three dozen existing evangelistic organizations. We also acquired the private papers of hundreds of people involved in evangelism in one way or another. We helped to plan the archival component of the Graham Center building on Wheaton's campus and we moved into the completed building in 1980. After five years of living in a warehouse with a tin roof, it was like entering the Promised Land. Our collections were opened to the public. Our staff expanded to four full time workers. So with this, that and the other thing, a few years passed and we began to wonder what we should be thinking about ourselves.
In nineteen eighty-four, our tenth year, we began to think of doing a self study for a number of reasons. Our archives had formed in a few years more or less out of nothing and now had coalesced into a solid shape. We were starting to develop institutional traditions and shibboleths and regulations by the bucket full. It seemed like this was a good point to take a few steps back and think about the way we were developing, to consider whether the twig was bent in the directions we wanted before it became an unyielding oak. Then too Wheaton College had a policy of each department doing some kind of evaluation at periodic intervals. This was not something that was enforced in any serious way, but it did encourage us to go in the direction we wanted to go in anyway. I think too that we were influenced by the fact that self-study was much discussed among archivists at the time. The SAA Task Force on Institutional Evaluation had published their booklet on the Evaluation of Archival Institutions and the advantages were discussed in various forums. I think our isolation played a part too. You have to remember that this was back in the long ago days before the archives listserv. Now you can ask in the morning what is the best color for plastiklips and have three dozen answers before you go home at night. The situation was different then. We attended professional meetings and interacted there with other archivists about theoretical concerns, but we did not have much of a chance to interact on a day to day basis with other archivists besides ourselves on the nitty-gritty of applying professional lore and practice to the actuality of our own experience. We wanted some outside critics to critique what we had created so far. And we wanted to measure ourselves against other, similar archives.
The SAA, as part of the task force's work, had set up an agency for assigning archivists to visit other institutions and make exactly this sort of judgment. So we commenced a self-study/outside evaluation in 1984, using the SAA model.
The first phase was to create written documentation, using the questions in the evaluation booklet, on our organizational and institutional setting, our profession practices, our users, our reference services. This involved compiling a history of our brief existence, our mission statement, organizational chart, resumes of staff, policy statements, forms, facility plans, procedures manuals, conservation practices, researcher lists, annual reports and so forth. For each set of factual background questions there was also a set of questions intended to make us think about the how and why of what we were doing in our day to day professional activities. Thus, after we answered the factual questions listed under financial resources about the current budget, the part the staff played in the development of the budget, etc, we answered a series of questions asking us whether we thought the funds available were adequate, did the budget provide the necessary degree of flexibility and autonomy, are certain areas of the archives operation budgetarily deprived, etc. All together there were 155 questions to be answered. Each full-time staff person (the three archivists and the office manager) took a section and answered all the questions of that section. Then the four parts that would make up the total report were put together for all the staff to read and comment on and suggest changes, additions and deletions. Then we met together at several staff meetings over the period of months to go through the final report sentence by sentence. By the end of that time, of course, we were so sick of the whole self study process that we never wanted to hear about it again, but we had arrived at a final report and developed from it significant goals to work on over the next several years.
Next was the evaluation by archivists from outside our institution. The Task Force was arranging such visits for institutions that had done a self study using the evaluation booklet and I contacted them to make arrangements. They suggested the team of Maynard Brichford, of the University of Illinois - Urbana-Champaign and David Klaassen, of the Social Welfare Archives, and better choices there could not have been. We bundled up our report and attachments and sent them to the evaluators. Then on August 13 and 14, 1984 Maynard and David arrived for their investigation.
Besides sessions with the archives staff, individually and as a group, they talked with a representative from the BGEA, the College vice president who oversaw the Center, the director of the Center, the heads of other departments of the Center. After their visit, Maynard and Dave prepared a preliminary report by the beginning of October which they sent to the staff for review. We had only one or two minor comments and the final report was in our hands by the middle of December. The comments and recommendations of the outside evaluation were combined with our own list of goals coming out of the self study, we developed schedules for each goal and for the next several years these goals provided the model for our development. I have some samples of our yearly goals lists in 1990 and 1996, both of which grew out of our self studies in 1984 and 1995 respectively.
Among the achieved goals that resulted from the 1984 evaluation was the preparation of a catalog to our holdings in machine-readable form, processing procedures, a formal evaluation process for our collecting policy, a kind of super-guide to allow greater intellectual control over our BGEA collections, an increased program of outreach to potential users, a conference to evaluate the archival needs of the Evangelical community, and a more comprehensive system of statistical reporting. In fact, looking back, I would say that it was the outside evaluators report, along with articles on cost analysis by Tom Wilsted, that developed in our archives on the concern for statistical measurement that, for better or worse, is part of our corporate character.
Our 1994-1995 self study experience was very similar to that of 1984. As the ten year mark approached since our last study, the staff talked about doing a new one. There were a variety of reasons: The college policy of ten year department reviews had become more firmly rooted; the previous experience had been positive and productive. Plus the Center was going through a transition at the time and it seemed like a good point to do some serious thinking about our goals and methods. Then too, the profession was being affected by the acquisition, preservation and service possibilities opened up by technology, particular the computer and the web. All in all, a good time to reflect and plan.
The self-study process was the same as in 1984. We used the evaluation booklet and the more recent Archives Assessment and Planning Workbook edited by Paul McCarthy in 1989 to develop our list of 163 questions, the main differences with the 1984 list being that there was more questions about our use of computer and more analysis of strengths and weaknesses. As before, each staff member did a section which was critiqued by the rest of us and much discussion went into developing the final written responses and a summary of strengths and weaknesses. We had completed the self study by the spring of 1995 and were ready for an outside evaluation. The SAA task Force no longer involved itself in setting up evaluation teams, so we needed to do that on our own. Following the model of 1984, we decided a two person team would be adequate and within our budget. We wanted to have someone who had been involved in the last evaluation in order to have some continuity in the process and new person who was knowledgeable about religious archives. Either Dave or Maynard would have been an excellent choice for the old hand. I contacted David and he agreed. For the second person our choice was Elizabeth Yakel because of her own experience managing religious archives as well as her two years of serving as a resource person to religious archives in the New York. She also agreed to be part of the team.
Dave and Beth came in July 1995 and spoke to BGEA staff, the president of Wheaton, the senior vice president, the head of the College library, faculty members who regularly used the archives. They also spent a lot of time with us, including hours talking to the staff as a group and meetings with individuals. We received their final report by the end of 1995 and used it and our self study to again set ourselves some long term goals. As in 1984, The evaluators report, as well as the conclusions of our self study was passed on to the director of the Center and as in 1984 was a polite acknowledgment of receiving the report. We did use portions of the report ourselves when we were pressing for funds or support for various projects and the executive summary was included in our annual report, which went to the Center's board of trustees.
Within the archives, we are still struggling with some of the concerns that grew out of the 1995 evaluation. Goals so far achieved include development of a web site and the posting of a substantial portion of our guides on the web, increased input from Wheaton College faculty on the Archives' program. We have also begun charging the organizations whose records we preserve for the services we provide.
So that is the factual background, the bones and outline of the experience. What about the heart, the meaning of the experience for us? What might be meaningful from our experience for other archives that are considering a self study or a self-study and outside evaluation?
A few blocks from here, at the Midland Hotel, the Midwest Archives Conference held its 1997 spring meeting. It was a very innovative concept for an archival conference, involving extensive use of discussion groups rather than the usual seminars and workshops. In the debriefing at the last session of that conference, Beth Yakel pointed out that the process of interaction in the groups and the plenary sessions was much more important than any final statement the groups might make. As soon as she said that, I saw that for us too, in our self study and outside evaluation, the process was as important as our final goal lists.
The process is one that should involve everyone on staff - head of departments, the volunteers, the freshly minted processor, the leathery old reference supervisor. Gathering the information about the archives and having all critique it makes sure that the newer employees and maybe some of the older one gets a crash course in the history, structure and purpose of the organization. But of course, most staff are not going to learn many facts here. The value of involving everyone comes from the variety of perspective and interest that will be involved. Your facts give you a common base to start with but the real value added comes from the staff and what they make of those facts - what can be done, what should be feared, how problems can be solved, how problems can be met. The process helps you bring out areas of general agreement or disagreement and these areas can be helpful in shaping plans for the future. The self studies we have done have played an important part in developing and strengthening a common departmental culture among the staff.
And how do you involve all staff in the process? By talking it all out. Sitting around a table and going through the responses to 155 questions or 166 can be an excruciating experience, a shared trauma like the Second Punic War was for the Romans. But it is also the place where the staff perspective I mentioned are brought to bear (if you are committed to bringing them to bear) . It is where new solutions, new possibilities, the things you thought couldn't be done, the things you never even thought of can be brought up and examined. This kind of talking cure is not something you want to go through often, but by the time you have taken the archives apart and put it back together again, it will belong to the whole staff and that sense of ownership will bring a new energy and commitment to the work of the department
Okay, okay, that is almost ridiculously idealistic. But if it is even 10% true after your self study, the work will have been worth it.
Of course, talking should lead to something concrete. Or, to put it another way, since we are archivists, it is not enough that we do something, we have to have a record that we have done it and detailing what comes next. One of the greatest benefits for us was that the process concentrated our thinking on what we wanted to do over the next few years and what was needed to reach our goals. Of course, you do not have to do a self-study to make a list of goals, but I would say from our experience it is very helpful in the goal setting process to step out of your daily grind and look at your work in a longer perspective and with the participation of the entire staff.
The participation of professionals from outside the Archives was both an important and enjoyable experience. I do not think that the weaknesses the outside evaluators pointed out to us were ones of which we were unaware, but their comments helped to underline perceived problems and motivate us to be more active in trying to find solutions. In each case, in 1984 and 1995, it was helpful to have to respond to questions about our archives from people who knew what they were talking about. Of course, for an outside evaluation to work, the evaluators need more preparatory information from the archives staff than a few pages of vague generalities. The self study needs to be completed months before the evaluation, so that the evaluators have a chance to study it. Both you and the evaluators need to know what you expect from them for an end product (format, deadline). And of course they need major input in setting up their visit - who they will talk to, what kind of contact they will have with the staff, what kind of information they will need other than the self study. This kind of advance preparation and meshing of expectations will help ensure a final report you can really use, both internally in your Archives and with your bosses.
What would I do differently if I had to do over again (or do again in a decade)? A few things. For one, we did not make as much use as we should of McCarthy's workbook. We relied mainly on the 1984 booklet, which is not as good a stimulus for group discussion. I think the 1989 workbook is simpler to use and makes the actual mechanics of the study less of a burden so you can get on to the real burdens. And very importantly, the McCarthy's book includes the means for statistical comparison with other archives. This was very revealing and stimulating in many cases, the way a kick in the rear end often is. This was true even though the data was some six years old by the time we did our study. Sadly, the data in the workbook is still the most current information for comparison purposes for American archives.
I would strongly encourage anyone doing a self study to modify whatever model they use to include questions that fit the peculiarities of their own situation and recent developments in the field of archival administration.
I would try to keep to a tighter schedule. As you can tell, for me the heart of the whole process is the talking cure, and for that to be of maximum value, I think it is better if the preparation and critique of the sections of the study, as well as the staff discussion of it occur in a brief space of time, rather than spread out over such a long stretch that people forget what comments were made five months ago or feel they are in a never ending purgatory.
I would try and find ways to get significant input from superiors, users, donors during the self study. During the outside evaluation, the evaluators did talk with users, administrators, colleagues, but we did not seek any input from them during our self study. In a way, that makes sense, since this is a SELF study and we are trying to get a clearer picture of the staff's understandings, expectations and view of the future. But some input from the significant groups that govern, benefit from and influence the staff could have a bracing effect and their comments can also suggest lines of inquiry for the outside evaluators. If your archives has an outside advisory group, involving them in the self study might be beneficial. (Incidentally, as one of the goals of our last self study, we are setting up such a group.)
A point that I should mention for your consideration. The staff at the BGC Archives is very small and was very stable for the period under consideration. In 1984 and 1995, there were four full time staff members. Three of the four staff people involved were the same people during both study periods. Larger staff would doubtless have to structure the self study differently, since such complete participation by everyone might not be possible. However, I do think that you are depriving yourself if you do not involve everyone to some extent so that you can benefit form their frustrations and anxieties as well as their thoughts and suggestions.
In conclusion, our experience shows a self study is a very useful tool, when applied at long intervals, for re-examining methods and goals and for combining and focusing staff energies. Its major drawback is the time and effort involved, its major advantage is the new perspectives that can guide you as you attack old problems. As the title proclaims, what you are studying is yourself - the nature of your organization, how and why it does what it does, what holds it together. And the benefits of achieving a better self-understanding can be great. As the old proverb says, to discover the treasures of the Indies, you must carry the Indies within.