Feature Articles
Education Center
Marketplace
Subscriptions
Career Center
Your Profile
About Us

The CMA Examination: Focus on Quality

by Roland L. Madison, Ph.D., CPA

Since 1990, I have made the trek 11 times from Cleveland, Ohio, to Montvale, New Jersey (where the Institute of Management Accountants is located), in order to participate in grading the CMA examination.
Upon reflection, I have seen a number of changes made in the CMA program, intending to stimulate interest in becoming a Certified Management Accountant and to enhance the quality of the CMA examination.

The Examination: Format and Content
The CMA exam is given semiannually in the months of June and December. It is divided into the following four parts: Economics, Finance, and Management; Financial Accounting and Reporting; Management Reporting, Analysis, and Behavioral Issues; and Decision Analysis and Information Systems.
Currently, each part of the exam contains 25% objective response-type questions, with the remainder having either computational, narrative-discussion, or both types of response formats. Each of the latter-type questions (non-objective) may vary from 12 to 21 points in value. Candidates are permitted to use calculators, and may readily obtain booklets containing copies of prior CMA examinations, including the unofficial answers, from the IMA's Publication Department.

Finally, while the CMA exam is presently an open exam, the ICMA Board of Regents has approved (May 1995) making the examination non-disclosed, effective with the June 1997 exam. This action means that, after the December 1996 CMA Examination, candidates can no longer receive their exam booklets after the test has been administered. This also means that the publication of the Question and Unofficial Answer booklet ended with the December 1996 examination.

The Regents give several good reasons for this action, including cost savings and permitting certain basic concepts to be repeatedly tested without requiring the development of totally new questions. However, this author is not a proponent of their actions. First, the ICMA will lose the opportunity to provide potential CMA candidates, their employers, and the general financial community with a full appreciation for the real quality of the Institute's professional examination program. Providing interested parties with a booklet containing a few sample questions simply will not convey the true merits of this professional examination. Also, the members of the academic community, who contribute to the CMA program in a variety of manners, will not receive the same degree of recognition for their efforts when the Question and Unofficial Answer booklet ceases to be published. This may decrease the incentive to participate in the development of exam materials, unless the fee structure for this service is modified.

Quality Control: Exam Preparation
The process to create an examination is quite extensive. First, an outline of the exam that is consistent with the Content Specification Outline is prepared by the ICMA staff. Next, a question-concept (idea) outline is developed for the specific questions on each part of the exam. These are then sent to the Board of Regents (twelve members mainly at the executive level in management accounting) for review and comment at least a year prior to the scheduled exam date. If the general outline and question-concept outline are approved, the ICMA staff begins the detailed preparation of questions.

To digress briefly, both the concept for questions and fully developed, detailed questions have three potential origins: they may be ICMA staff-prepared; unsolicited questions may be submitted to the Institute; or the development of questions, based on concepts approved by the Trustees, may be privately contracted by the managerial staff of the ICMA. In recent years, greater than half of the exam questions have come from sources beyond the ICMA staff.

The contributor submits a question along with a suggested solution and sources for the answer. The ICMA staff evaluates the question; and, if it is selected for potential use, edits the question and assigns it to one of the four examination parts. Next, the question is taken to an Examination Review Committee (ERC), consisting of 3 regents and 3 current senior-level management accountants in industry. The question is evaluated at a detailed level for inclusion in the examination. After this detailed review, the questions for each exam part are tested on a small sample of confidential persons. After the test responses to the sample are evaluated, the ICMA staff prepares a revision of the part for an additional--and hopefully final--evaluation by the ERC. When a question successfully passes this point, it will be scheduled for use on a future exam, where it meets one of the specific requirements of the Content Specification Outline.

A Question is Accepted for the Exam
At some point in this evaluation process by the ERC (the exact point is confidential), contributors normally receive a stipend and a letter of thanks from Ms. Terri Funk, director of the CMA Examination. With the exception of contracted questions, this tentative acceptance could be a year or more after the question was submitted. In no case are contributors told if or when their questions will appear on a future exam.
Prior to printing an examination, a question is sent to one or more outside reviewers, in addition to an ICMA staff person, for one last review for contemporary appropriateness and the reduction of any final ambiguities.

Quality in Exam Grading
Before production grading of the non-objective response-type questions begin, both a narrative-solution outline of each question, along with a point-specific grading guide must be developed. Using the solution prepared by the contributor of the question as well as the solutions prepared by the blind referees of the question, the ICMA management team develops a grading guide and narrative solution for each question. Next, a group of senior grades and grading supervisors use the staff-compiled solutions to evaluate a sample of 100 papers from the current exam. The purpose of this sample grading is to deal with any potential problem areas, and to finalize the narrative answer sheets and grading guides to be used by the production graders.

To become a CMA grader, a person must complete an application that includes proof of their academic credentials, work experience, and professional certifications held. Production graders are then placed into three levels of experience--it takes about four years to reach the senior level. Graders usually work in three- or four-person teams to deal with the increasing number of papers to be evaluated. The ICMA staff tries to balance the experience levels of each team, giving consideration to the complexity of the question and the number of papers to be graded. Finally, every effort is made to assign a grader to working in their first or second preference area of expertise.

Prior to starting production grading, each grading team is introduced and is given an orientation to the grading program by the Director. Next, all graders, regardless of seniority level, review all procedural guidelines (e.g. confidentiality, secure handling of papers, scoring, and reconciling differences in point evaluations) before proceeding to the quality-control pre-grading process.

Grading-Team Communications Ensures Consistency and Quality
The latter item is a very important change, recently implemented by the ICMA, to ensure consistency and quality in the evaluation of all exam papers. Each member of the grading team independently evaluates and scores a selected group of exam papers. The team then meets to discuss their evaluations in relation to the grading guide and to reconcile any differences. Next, the entire team meets with part supervisors to discuss the results of their evaluations. The supervisor compares their composite results with the projected results that were developed during the sample grading process. It is during these two meetings that a consensus is to be reached among the graders and the supervisor that will ensure the uniform and consistent evaluation of all CMA candidates' papers.

After all of this preparatory effort toward quality has been applied, it is time for production grading to commence. During the grading process, consistent quality is maintained by the supervisors, who continuously review the work of graders. In addition to grader reviews by the supervisors, the ICMA also carefully monitors the grading patterns of all co-graders to see that a statistically significant difference does not develop during the examination. If one does begin to develop, it will quickly be detected, and a meeting with a supervisor, project manager, and the grading team (or grader) will be held to analyze the cause of the disparity. The ICMA may require a regrading of some group of papers if a disparity of more than one point between graders persists during the first 30% of the exams graded. By this point in the process, most co-graders have developed a solid and consistent rapport in terms of the credit to be granted under the majority of possible circumstances.

Serious Consideration Given to Marginal Papers
Much like the CPA exam, the ICMA follows a practice of not assigning marginal scores within a certain point range. (Could we state what this range is?) For this reason, every single question on each exam that received a grader score of ZERO is regraded for the potential oversight of credit. Finally, and perhaps of greatest significance, all papers with an exam-part score of 50 percent, but less than 70 percent ( a passing score), are regraded at least once, and potentially two times, before a final score is assigned. Every effort is made to give the candidate complete credit in the grading process for knowledge demonstrated.

Thoughts for Potential Graders to Consider

To efficiently process candidate's examinations, the ICMA staff requires that production grading is completed during a period of less than a month. For both graders and staff, this requires a significant commitment of time and involves a lot of effort and concentration. While time commitment and fatigue may be a factor for some persons doing production grading, I found the ICMA staff to be most supportive of graders and cooperative in setting schedules, allowing for personal breaks to relax, and creating an overall comfortable working atmosphere.
From a personal standpoint, serving as a grader provides an opportunity to make new friends and learn about our profession, as well as make a commitment to serve it in some different fashion. Also, working with the IMA provides one the opportunity to become more aware of the programs the Institute has to serve its members.
Economically, graders from the local area may fare rather well if their schedules are carefully planned. Even those experienced graders from out of town may fare reasonably well if they work as a team. While not part of the formal agenda, graders often do some sightseeing while visiting the Big Apple area.

As potential CMA candidates, you should feel extremely confident that the Institute takes great pride in the quality preparation of an examination that is designed to evaluate your fundamental skills as a management accountant. CMA candidates should feel most comfortable to know that the ICMA utilizes all of the quality controls described herein to ensure the accurate evaluation of your skills.

Conclusion: Take the Opportunity to Make a Commitment to Quality
I have a very positive view about the quality of the CMA program sponsored by the Institute of Management Accountants. I would urge both practitioners and academics to consider preparing a written contribution to the exam, applying to serve as a CMA exam grader, or sitting for the CMA examination as a candidate. Persons who hold the CPA certification are automatically granted credit for Part 2 of the exam, and those who are employed in academia may enjoy other benefits from involvement with the CMA program.*

Roland L. Madison is a professor of Accountancy at John Carroll University, University Heights, Ohio. A member of the Cleveland East Chapter of the IMA, Dr. Madison has served as a consultant, grader, and contributor to the CMA examination program since 1987. This author gratefully acknowledges the technical contributions that Terri L. Funk, CMA, Director--Examinations, has made to this article.

Footnote, to go along with asterisk
* Persons interested in the CMA program may contact the ICMA at its toll-free number, 1-800-638-4427, to obtain a CMA Information Kit, which contains registration materials and the Content Specification Outline with a list of suggested reading materials (e.g. texts, authoritative bodies, and periodicals), to help the candidate prepare for the exam.

Feature Articles | Education Center | Marketplace| Subscriptions | Career Center | Your Profile

Home | Customer Support | About Us | Advertising