Assessment Systems Corporation :: Software :: Classical Test Theory :: Lertap 5 - Classical item and test analysis with Microsoft Excel

Lertap 5 - Classical item and test analysis with Microsoft Excel

Lertap, the Laboratory of Educational Research and Test Analysis Package, has been around since 1973. Lertap 1 and Lertap 2 ran on mainframe computers. Lertap 3 and 4 ran on the first batch of microcomputers, including the Apple ][, the Osborne I, and the original IBM PC.

Now well into its fifth generation, Lertap runs as an Excel application. There are editions for Windows computers, and for Macintoshes. It's a solid, time-tested package used for the conventional classical analysis and scoring of cognitive tests and affective scales, with a variety of extra features, many of them graphics-based. Some user testimonials may be seen here.

Here's how it looks when used with Excel 2010:



Lertap 5 is a self-contained system; no tools other than Excel are required in order to use it. Test results are recorded in an Excel worksheet called "Data". Analysis specifications go into the "CCs" worksheet using the simple Lertap control language. Results are obtained by clicking on options found on the Lertap Excel tab, such as "Interpret", "Elmillon", and "Histograms". As options are selected and run, tables and graphs are created in a series of reports. Each report is an Excel worksheet.

By and large Lertap's reports are concise. On-screen links to context-sensitive help are available to provide detailed, web-based explanations of the terms used in the reports. Those who like supporting documents will find that Lertap has a rich variety, from its printed manual, to an extensive list of technical papers and sample datasets.

Data may be entered in a variety of ways. For example, they may be "typed" directly into the Data worksheet, or they may be brought in from a file created by a scanner. Lertap has a converter for one of the data input formats used by ASC's Iteman and Xcalibre programs, and it's able to write output files ready for subsequent input to Xcalibre, Iteman, and Bilog-MG. Of course, with its Excel base, the data in Lertap's worksheets may be directly read by programs such as SPSS and SAS.

Links to examples of Lertap in action:

Features of special note:

  • It's Excel. It's alive. Interactive. Colorful. Add your own charts and graphs. Copy and paste to Word and PowerPoint. Can even be fun for a rainy day.
  • The "Full" version allows data for up to 16,300 items and over 1,000,000 students (standard Excel 2010 column and row dimensions).
  • The free "Mini" (N=50) version is limited to processing 50 students, but is otherwise fully functional.
  • Handles dichotomous and polytomous items; permits complex scoring patterns.
  • Creates concise reports, many ready for direct printing. See, for example, the series of Stats reports.
  • Two criteria may be used in item analysis: internal and external.
  • Superior at flagging faulty item options.
  • Includes elementary but effective data integrity check ("Freqs").
  • Produces extensive graphics, including unique "packed plots".
  • "DIF", differential item functioning, fully supported, with graphs.
  • Group differences analyses feature ANOVAs, response plots, and box plots.
  • Experimental features include 2PL IRT estimates.
  • Forms complete interitem R matrix (SMCs, tetrachorics, eigenvalues).
  • Easy export to Xcalibre, now one of the best IRT programs (also exports to Bilog-MG).
  • Allows data sampling (to "halve and hold").
  • Incorporates a cheat-checker: "response similarity analysis".
  • Is programmable, allowing users to patch in their own code modules.
  • Complies with all AERA/APA/NCME standards for testing, including those pertaining to the use of cut scores.
  • Satisfies third-party accreditation program needs, including ANSI and NCCA.
  • Support for instructors and classes. (Free class sets of Lertap often available on a limited-term basis.)
  • A more complete data and item analysis system than similar products, backed by years of application in various fields and countries, including large-scale achievement assessments.
  • Ever-friendly support desk 24/7 (subject to brief temporary closures when fishing).
  • Inexpensive perpetual licenses. (In fact, all licenses are perpetually perpetual.)

Author's notes on Lertap 5.10

The 10th version of Lertap 5 was introduced in mid 2012. Comments on its major features are noted below. Lertap 5.10.2, released in mid 2013, addresses a number of security and stability issues, and has enhanced support for mastery and certification tests. Meant for use with Excel 2010 and 2013, 5.10.2's "off the shelf" configuration is a free, fully-functional "Mini" edition for use by instructors, students, and researchers with small sample sizes (N=50 or less). It may easily be upgraded to a "Full" version via the purchase of a license.

There were three things I wanted to specially address in the 10th release: providing access to more immediate on-line help for users, getting Lertap to provide more warning flags for items which may have problems, and to make "packed plots" easier to get.

All of Lertap's reports now have a small h at the top which provides a link to context-sensitive, web-based help. Thus, for example, for someone looking at a "Stats_b" report, and wondering about the meaning of its ? column, a click on the h will get the appropriate help topic to display straight away (assuming internet access; otherwise the standard in-built help may be accessed for the same information, but it's a bit more cumbersome to use). Speaking of the Stats_b report, it now includes the value of coefficient alpha at the base of the item summary lines, just above the scatterplot of item difficulty by discrimination. (You can see it here.)

I have over the years tried to make Lertap's output easier to understand. Back when we first started large achievement assessment programs, it seemed that most everyone could understand and make use of the statistics produced by Frank Baker's FORTAP program from Wisconsin, a program which provided the impetus for "DIEitem", the Spanish-language forerunner of Lertap written in 1972. For years we lived with reports which closely resembled today's "Stats_f" output. As time passed and more Lertappers emerged, it became quite clear that a more concise report would be well received. This became the "Stats_b" report. More time passed. Excel became more powerful, its charts were easier to program, and the "Stats_ul" reports, with companion quintile-like plots, emerged as an alternative, or a complement, to Stats_b. These have become extremely popular, even Iteman now has them.

Now I think there may be too much information. The output available presently may be good, and often pretty, but gee, do I (or you) really need to look at all of it?

I think perhaps not. Often I work with people who want to know what factors have served to produce a reliability figure less than expected or desired. I'm of the opinion that they do not need or even want to look at the stats for each item, especially not if it means taking in each and every item table in Stats_f, and/or the detailed response endorsement proportions seen in Stats_ul's item option tables. No. Consequently I have now trained Lertap to be a better flagger. There's a flag man or woman lurking behind each Stats report. His/her job is to add something to the report for those items which seem to have problems. The idea is that a user will now be able to scan a report, any report, be it Stats_f, Stats_b, or Stats_ul, and there will be little flags and/or special colours attached to items with potential problems.

Stats_b has always been good at raising and waving flags; it has always had that ? mark column just for this very purpose. Now Stats_f reports have flags too, and, in Stats-ul, items with bad discrimination are also flagged. Now a user can quickly spot items which may be bringing reliability down, there's not a need to necessarily study each of the item tables.

In this same vein, sort of a "gestalt" vein, Lertap's ability to make "packed" quantile plots has been considerably improved, and items with bad discrimination are even flagged in the plots (red coloring is used). Have a wee look:



Packing such as this has been available since Edition 9 (5.9.1 and later); even Edition 8 can do it. But now, in Edition 10, these packed plots are much easier to create. Not only that, but "bad" items will use red flagging. In the samples above, one of the twelve items was mis-keyed, and has been flagged. Can you spot it? (It's item 11.)

Next time someone brings some test output to look at, and two cups, glasses, or bottles of something to plonk on my desk, I'll sit with them, going through Stats_b, looking especially at its scatterplot of item difficulty and discrimination, then moving into packed plots. This output is generally easy to understand; I'll print it (in color), and use the results as a way-in to thinking about how test reliability might be improved -- with some practice, the packed plots (in particular) greatly help to hone in on item problems. The packing brings out the gestalt -- you can pretty well tell right away whether or not it's going to be possible to "fix" the test (when only a few bad response patterns are seen in the plots), or whether it'll be back to the drawing board, making recourse to texts and other references on good item writing techniques.

Downloads

 

Lertap software and documentation are exclusively distributed worldwide on behalf of Curtin University of Technology by Assessment Systems Corporation

 

  • 30-Day Trial Copy for use with Windows Excel 2002 and 2003

    A fully-functioning 30-day trial copy is available for users of Windows Excel 2002 and 2003. If you decide to purchase a Lertap license, we can unlock your copy by email.

    • After downloading the file, unzip it and then double-click on SETUP.EXE.
    • If you do not have a program to unzip files, PC users can obtain one at  www.winzip.com.
    • WARNING - Lertap may not work properly unless it is installed by the installation program (SETUP.EXE) that is supplied as part of the download.

    Download the 30-day trial copy for use with Windows Excel 2002 & 2003 (15.1 MB).


    Note 1: please check these notes on Excel 2003 and 2007 if you’re not sure which Windows version of Excel you have.
    Note 2: the Excel 2003 version of Lertap does not support DIF, differential item functioning. Users interested in Lertap's DIF support should download either the Excel 2007 or the Excel 2010 version
    Note 3: this version will not work with Calc, the OpenOffice spreadsheet program.


  • 30-Day Trial Copy for use with Windows Excel 2007

    A fully-functioning 30-day trial copy is available for users of Windows Excel 2007. If you decide to purchase a Lertap license, we can unlock your copy by email.

    • After downloading the file, unzip it and then double-click on SETUP.EXE.
    • If you do not have a program to unzip files, PC users can obtain one at www.winzip.com.
    • WARNING - Lertap may not work properly unless it is installed by the installation program (SETUP.EXE) that is supplied as part of the download.

    Download the 30-day trial copy for use with Windows Excel 2007 (13.9 MB).


    Note 1: please check these notes on Excel 2003 and 2007 if you’re not sure which Windows version of Excel you have.
    Note 2: this version will not work with Calc, the OpenOffice spreadsheet program.


  • Version 5.10 for use with Windows Excel 2010 and Excel 2013

  • A fully-functional N=50 "Mini" copy is available for users of Windows Excel 2010 and Excel 2013. Should you decide to purchase a Lertap license, we can eliminate the N=50 limitation and upgrade you to the "Full" version by email.

    • After downloading the file, unzip it and then double-click on SETUP.EXE.
    • If you do not have a program to unzip files, PC users can obtain one at www.winzip.com.
    • WARNING - Lertap may not work properly unless it is installed by the installation program (SETUP.EXE) that is supplied as part of the download.

    Download Version 5.10 for use with Windows Excel 2010 & 2013 (27.7 MB).


    Note 1: please check these notes on Excel 2003 and 2007 if you’re not sure which Windows version of Excel you have. Excel 2010 and 2013 are much like Excel 2007 for comparison purposes.
    Note 2: this version will not work with Calc, the OpenOffice spreadsheet program.


  • 30-Day Trial Copy for use with Macintosh Excel X and Excel 2011

    Please write directly to the author (larry@lertap.com) to obtain a copy of the 30-day version of Lertap for use with Macintosh Excel 2011


Licenses

Licenses are determined by the use of the program. All versions (2003, 2007, 2010, and Mac) are the same price, please let us know which one you need when unlocking.

Student research license ($249 one-time cost) - Grants are provided to cover costs of our software, allowing students to purchase a full academic license; click here for more information and an application..
Academic ($249 one-time cost) - The program will be used at an accredited college or university for research or teaching purposes only.
Academic lab ($99 each, minimum 5) - The program will be used in a student computer lab for teaching and research purposes.
Government/Non-profit ($449 one-time cost) - The program will be used at a governmental or a non-profit organization, or by a school district or college or university for analyzing the data from its testing programs. The use of the program must also be non-profit; that is, if the organization is non-profit but conducts consulting activities for other organizations, the use is commercial.
Government/Non-profit institution license (contact us for price) Multiple copies of the software will be installed throughout a governmental or a non-profit organization, or by a school district or college or university. The use of the program must also be non-profit; that is, if the organization is non-profit but conducts consulting activities for other organizations, the use is commercial. Please tell us how many licenses you need.
Commercial ($999 one-time cost) - The program will be used for commercial purposes, for example, in the course of consulting in test development and psychometrics.
Upgrades - Lertap5 upgrades are usually required when users have switched from one version of Excel to another, as, for example, when changing from Excel 2007 to Excel 2010, or when moving from a Macintosh version of Excel to a Windows version. In such cases, purchasers will receive credit towards the Lertap5 upgrade (the amount of credit will depend on the type of license). Please order the appropriate license below and then contact us with information regarding the previous purchase, so we can validate the existing license.

Lertap is provided via download. If you wish to have a backup on a CD, you can simply burn the downloaded file on to your own CD. Volume discounts are available for purchases of 10 or more licenses. Contact us for more information.
SKU LERTAP5-Ac  
License/OS
 
Our price: $249.00
Quantity