Association of Academic Support Educators Bar Advocacy Committee Best Practices for Online Bar Examination

The Association of Academic Support Educators (“AASE”) is a non-profit professional organization for law school academic support educators. AASE members collaborate to develop research-based teaching methods and enhancement programs that empower students to succeed in law school, on the bar exam, and in the practice of law. AASE members are subject matter experts in assessment delivery, learning strategy, performance application, and licensure requirements.

Through its standing committee for Bar Advocacy, AASE channels the collective expertise of its members to provide the following “best practices.” These practices are designed to maximize outcomes for bar applicants and to ensure a testing environment that is both secure and reasonably accommodating for applicants. By adopting these Best Practices, licensing authorities can provide further assurance to bar applicants that those who test via remote online procedures, will have no less comfort, advantage, opportunity, or convenience than bar applicants who test in-person. These practices recognize the importance of exam security and integrity for the authorities involved in all stages of the examination process including, design, drafting, preparation, delivery, administration, identity verification, execution, collection, scoring, and score reporting.

Best Practices for Online Bar Exam Adoption, Delivery, and Administration

Those charged with creating, delivering, and administering an online exam must first acknowledge that an online exam is different from an in-person exam. Traditional classroom instruction and bar preparation modules have not been oriented towards an online bar exam.  As such, applicants may be at a disadvantage when the format of the exam does not parallel the method of preparation. The following best practices should be adopted to ensure an equitable administration of a remotely administered online bar exam.

1. Providing Notice and Communicating Policies Regarding Online Examination

• Jurisdictions should decide as early as possible if they will administer an online exam and communicate the exam options to registered applicants and on any websites hosted by or affiliated with the state or local bar examiners.
• Jurisdictions should provide ample advance notice of intent to offer online exams to allow applicants time to sufficiently prepare for the testing modality. Changes implemented on short notice create stress and unfairly disadvantage applicants.
• Any such website postings and notice to registered applicants should provide clear rules and a list of frequently asked questions about the terms of the online administration including the technology requirements and rules for exam submission.
• Applicants should be reasonably informed of the passing score range and the general method of scoring before the deadline for exam application submission. Licensing authorities should provide information about online exam scoring while the registration period for the exam remains open. At a minimum, such information should include the proportionate weight of each exam component, the minimum and maximum raw score available for each component, and the method of scaling the exam.
• Applicants who do not wish to take an online exam should be allowed an option to take an in-person exam if, and when, such an exam is available at no additional cost to the applicant.
• Jurisdictions who have adopted the Uniform Bar Exam (“UBE”) should not be put to an election to choose either print or remotely administered version of the exam, especially when public health, weather conditions, or natural disasters would limit the applicants’ ability to prepare for the exam, travel to and from an exam site, or have internet connectivity and remote testing conditions sufficient for exam completion.
• The National Conference of Bar Examiners (“NCBE”) should make equally available to all jurisdictions both print and online versions of the exam.
• Licensing authorities should provide information regarding online exam administration and scoring to the deans of all law schools in the jurisdiction. The law school deans should share the information received with the academic support faculty and staff and publish the same to all enrolled law students and recently graduated alumni.
• Applicants should be given multiple opportunities to practice with the online software including completing multiple-choice questions, essays, and performance tasks.

1. Delivery and Administration of Online Examination

• To every extent practicable, an online exam delivered for remote administration should be open book in format.
• To ensure equity, jurisdictions must provide sites with reliable internet access at no additional cost to applicants.
• Licensing authorities should inquire whether every applicant has reliable internet access sufficient for completing all portions of the online exam in the time and manner set by the licensing authority. State licensing authorities should present an approved alternative path to licensure to applicants who cannot complete an online examination due to lack of reliable internet access.
• Unless prohibited by state law, local ordinance, or institutional policy, a law school should make physical space and/or internet access available to applicants who:

1. are graduates of the law school;
2. are located within the general locality of the law school;
3. desire to complete an online examination; and
4. lack reliable internet access during the examination period.

• Online exam platforms should mimic, as closely as possible, the in-person exam delivery in terms of notetaking, ease of view, and multitasking abilities.
• Jurisdictions should permit applicants to have and use scratch paper during the administration of all components of the exam.

1. Applicants who test in-person have access to scratch paper and/or the written test materials at all times during exam administration. In-person applicants have the opportunity to make notes, outlines, or calculations on scratch paper or the exam booklets. Remote online applicants should have the same access to note making, outlining, or calculating as do in-person applicants.
2. Jurisdictions should continue the use of exam delivery technology that enables online notetaking during the exam.

• At all times during the administration of online exams, applicants should be allowed to highlight or underline on-screen text from all components of the exam. Exam components include, but are not limited to, text from the instructions, questions, File or Library of the Multistate Bar Exam (“MBE”), the Multistate Essay Exam (“MEE”), and the Multistate Performance Exam (“MPT”) and any successor variant of these exams and/or any state equivalent to these exams.
• Jurisdictions should provide split screen technology to allow applicants to see the essay question facts and prompts and or performance test instructions, file, and library while writing their answers. Depriving applicants of the ability to simultaneously read and type would place an unfair and unreasonable burden on applicants.

1. Technical Support

• Applicants should be provided with written documentation on how to use the software and what features are available within it. The software should allow for ease of reading and use of the written testing material in an accessible and natural format.
• Applicants should not experience long delays in reaching tech support during the exam.
• Licensing authorities must ensure that any software, monitoring, and technology contractors have provided evidence of a tech support staff that is adequately trained, and sufficient in number, available to respond to calls and requests for assistance, at all times, during the exam administration.

1. Exigent Circumstances

• During public health emergencies, inclement weather, natural disasters, times of war, and other acts of god or man, that prevent applicants from travel, congregation, access to water, food, power, shelter, or quiet study, bar examiners and state licensing authorities should make every reasonable accommodation putting the welfare of the applicants as the primary consideration. Reasonable accommodations shall include, but are not limited to, delay or postponement of the exam, reduction in exam length or content, modified scoring (where modifications lower or waive the cut score range), waiving the exam, substituting the exam with approved licensure alternatives.
• During any situation of local or national emergency, licensing authorities, who use any one or more of the multistate exams, the UBE in its current form, or any successor version of these exams under other names, should be reasonably prepared to deliver and administer a non-NCBE exam that affords the jurisdiction the flexibility to change exam dates, exam forms, and other aspects of the online exam that will provide the greatest protection and accommodation to applicants.
• Applicants who applied to take the UBE, but who are not able to do so because of an unavoidable emergency should not be denied the benefits of score transfer. Under exigent circumstances, all jurisdictions should freely and readily enter into transfer agreements to allow admission reciprocity or score transfer procedures.
• Licensing authorities should provide an alternate pathway to licensure for any applicant who applied to take an in-person exam, but because of public health, weather conditions, or natural disasters, or a jurisdictional inability to offer an in-person exam, would otherwise only have the choice to take an online exam or to delay licensure.

1. Respecting the Dignity of Applicants

• Jurisdictions should allow applicants to remove themselves from the test screen as their basic human needs dictate.
• Applicants should not fear adverse exam consequences for natural body functions, like yawning, stretching, sneezing, lip reading, scratching, twitching, or making contorted facial gestures. Applicants must not be stripped of their dignity and freedom to be human and should not have to test in fear that an involuntary bodily movement, function, or sound could cost them the ability to practice law.
• Applicants who take a remote or proctored online exam should have no greater restrictions for restroom use than applicants who test in-person. Under no circumstances should an applicant be at risk of disqualification or exam failure for solely using the restroom during the exam.
• If circumstances or exam length prohibit voluntary restroom access, jurisdictions should ensure that no uninterrupted exam period runs longer than 90 minutes, which shall include the time allotted for written, verbal, or recorded instructions.
• The need for, use, and choice of menstrual hygiene products should not be questioned or dictated by any licensing authority. Menstrual hygiene products should not be touched or handled by any licensing authority or agent thereof. Applicants should not have to demonstrate, display, explain, or reveal menstrual hygiene products.
• Lactating applicants taking a remote or online exam should be afforded the same privacy and courtesy offered to lactating applicants taking an in-person exam.

1. Protecting the Privacy of Applicants

• The NCBE and licensing authorities must identify alternative methods of identity verification that do not employ facial recognition technology. Facial recognition softwares are unreliable at best, and disproportionately disadvantage applicants with medium and darker skin tones.
• The use of remote proctoring software should be limited. Any risk of data breach, releasing of private information, or damage to personal laptops, must be fully disclosed to applicants in advance of the exam start date. The NCBE and licensing authorities should take precautions to ensure that agreements with software providers limit the access to and use of applicants’ personal information only to bar exam security concerns.
• Licensing authorities should disclose the use of Artificial Intelligence (“AI”) systems and the scope and objective of the AI system.
• Licensing authorities should inquire whether or ensure that the AI systems to be used have been audited or are auditable by independent third parties.
• To the extent that the NCBE or any licensing jurisdiction requires the use of AI software and/or remote exam proctoring, the licensing authorities must take demonstrable steps to mitigate the risk of implicit and racial biases in the review, screening, and/or flagging of applicant behavior and actions while testing.
• Licensing jurisdictions that employ AI protocols for remote proctoring and identity verification should develop and provide an administrative procedure for the appeal of any decision made as a result of AI monitoring or reporting. All applicants taking an online bar examination should be advised of the right to appeal.

1. Competency in Scoring and Reporting

• The NCBE or the jurisdiction that will conduct the scoring and scaling of the exam should provide, to applicants, data on scaling and equating a remote, non-paper examination in multiple locations as compared to the written in-person testing in a controlled environment.
• The NCBE or the jurisdiction that will conduct the scoring and scaling of the exam should provide, to the deans of all law schools within a particular jurisdiction, data on scaling and equating a remote, non-paper examination in multiple locations as compared to the written in-person testing in a controlled environment.
• Exam scoring errors should be resolved in a light most favorable to the applicant. Any applicant who is falsely notified that their exam score is within the passing score range, should not under any circumstances other than applicant misconduct, or dishonesty during the exam, be made to repeat the exam as a result of the error or incompetence of an agent or employee of the NCBE or a state or local licensing authority, regardless of the source of or reason for the error.
• As minimal competence is the aimed hallmark of our current licensure process, all parties involved in the process must be held to the same standards of competence. The highest judicial authority within a licensing jurisdiction should send a written letter of apology to any applicant who is falsely notified that their exam score is not within the passing score range, but who actually passed the exam.
• Licensing authorities should publicly disclose the score appeal process for applicants who fail to receive a passing score. If appeals are not permitted, licensing authorities should publicly disclose the internal review or regrading policies employed, if any.

1. Transparency and Full Disclosure

• Applicants should have access to information that clearly delineates the scope of personal information that can be collected, the manner of collection, the length of time any collected information can be maintained, and any additional purposes for which the information collected can and will be used. Agreements between the licensing authorities and software and technology providers should be made publicly available on each jurisdictional law examiners’ website. Barring only proprietary data, applicants and the public should be made aware of public monies paid to software and technology providers for exam administration, data collection, and remote proctoring services.
• Applicants should have access to information that clearly delineates the authority of state or local bar examiners and the authority reserved by the NCBE. Agreements between the NCBE and licensing authorities should be made publicly available on each jurisdictional law examiners’ website. Barring only proprietary data, applicants and the public should be made aware of public monies paid to NCBE for bar exam delivery and scoring services.

Submitted on behalf of the Association of Academic Support Educators by:

Deshun Harris                                Melissa Hale                                    Marsha Griggs

President                                         President-Elect                               Bar Advocacy Chair