Version 5 for Windows and Macintosh
Compare All Analysis Programs
Reviews of Lertap, and replies to the reviews by Lertap's developer are at www.lertap.curtin.edu.au/Reviews.htm.
Lertap 5, an Excel-based classical item, test, and survey analysis system is now compatible with all versions of Excel for Windows, including Excel 2007 and Excel 2010. It is also compatible with two Macintosh versions of Excel, Excel X, and Excel 2004.
The version of Lertap 5 for Excel 2007 and Excel 2010 operates from a tailor-made ribbon of controls, making it easier to access and apply Lertap’s options.
The most recent enhancements to all versions of Lertap 5 (along with selected other features) include:
- Differential item functioning graphs and Mantel-Haenszel statistics
- The ability to analyze ITAP-formatted input files, to make use of Lertap’s graphing capabilities.
- Enhanced support for users of mastery, licensing, and certification tests. Reports now include a table and graph of conditional standard errors of measurement, complementing estimates of classification consistency and dependability. A supporting document, with several examples, is available at: http://www.lertap.curtin.edu.au/Documentation/JERM2007d.doc
- A new graphical summary has been added to one of the standard item analysis reports: a scatterplot of item difficulty and discrimination indices is now produced for each cognitive subtest.
- A new menu, 'Macs', lets users connect locally-made macros to the
Lertap toolbar. A page on the Curtin Lertap website gives an example,
involving a special macro for processing data exported from the Angel
Learning System. This page is found at: http://www.lertap.curtin.edu.au/Documentation/AngelLearningLertapMacros1.doc
- A new paper, "Visual Item Analysis", suggesting an ocular
approach to item analysis, and exemplifying some of the graphics made by
Lertap 5, is available as a PDF file at: http://www.lertap.curtin.edu.au/Documentation/UsingLertapQuintilePlots.pdf
- Capabilities for processing missing data have been considerably
enhanced, involving adjustments to item statistics when item sampling has
been used, and when some students have seen just a subset of items.
- An analysis of variance table has been added to the groups breakout
report, complete with an effect-size estimator. A menu option has been
added to make it easy to re-code the demographic variables typically found
in the analysis of group differences.
- Extended support for "response similarity analysis" (cheat checking) is now
in Lertap. An inferential test statistic, "sigma", has been added to help detect cheaters
(see Sample Screenshots).
Read more about it at: www.lertap.curtin.edu.au/HTMLHelp/HTML/responsesimilarityanalysis.htm
- Test scores may now be tabulated by groups, making it possible, for example, to quickly compare results by regions, gender, course study code,
and so on. Each scores breakout is accompainied by a plot of group means.
Get the scoop at: www.lertap.curtin.edu.au/HTMLHelp/HTML/breakouts.htm
- An option to produce a boxplot (box and whisker plot) of group results has been added.
Please refer to: www.lertap.curtin.edu.au/HTMLHelp/HTML/box_and_whiskers.htm
- A "production mode" capability is now available, making it
possible to set options which will have Lertap automatically output selected
reports and graphs with just a single mouse click. More information
about production mode options is at:
www.lertap.curtin.edu.au/HTMLHelp/HTML/productionmode.htm
- Items may now use as many as thirty (30) options, making it possible to
process extended-matching questions, "EMQs", with Lertap.
- A new setting allows data tables to be automatically attached to
all quintile plots.
- Quintile Plots. Quintile plots are a powerful item analysis
tool, turning response frequencies into graphs with a click of your mouse.
Lertap's quintile plots display item option performance based on:
- More Features:
The Laboratory of Educational Research Test Analysis Package,
"LERTAP", is a classical item and
test analysis system. Lertap also analyzes surveys and mastery tests.
Lertap uses a special control
language that makes analyses simple and efficient. In Version 5, Lertap's
reports have been packaged in a new style, and numerous new features have been
added. You shouldn't be without it -- here's why:
It's an Excel Application
Lertap 5 has been developed as an application that runs within Microsoft's Excel system.
- Lertap 5 will work with any computer capable of running English-language
versions of Excel 97 and upward. It operates under English-language versions
of Windows 95/98/NT/ME/2000/XP/VIsta/7 for all versions of Excel and Macintosh OS 8/9/10 (X) for versions of Excel through Excel 2004. (Excel 2008 for Macintosh cannot run Lertap.)
- Lertap's input
files are Excel worksheets. In many cases, the output sheets generated by
Lertap are carefully formatted and labeled Excel worksheet tables, styled as
reports suitable for immediate printing, or for copying to a word
processor.
- Standard Excel functions and charts are used as much as possible,
resulting in a computationally efficient and statistically precise application
having an interface familiar to many users.
Classical item analysis
Lertap processes results from cognitive tests, and from surveys. For
cognitive test data, Lertap's default mode of operation is based on classical
test theory.
- Lertap uses both correlational and upper-lower groups methods to index
item functioning. It produces three reports for each cognitive subtest - two
based on correlation coefficients (point-biserial and biserial, corrected for
part-whole inflation), and one on upper-lower
proportions.
- One of the correlational reports includes a section with item
difficulty and discrimination data, summarized in a series of ten bands,
making it possible to easily identify items whose performance differs from the
subtest norm. This
report also provides descriptive statistics related to test scores, such
as their mean, median, and standard deviation. Test reliability is estimated
using coefficient alpha. The index of reliability, and the estimated standard
error of measurement, are output. A unique feature of Lertap's reliability
analysis is a table
that shows how a test's alpha value would change should an item be removed
from the test.
- Another report provides a
simple summary of item functioning, using one line of results for each
item. This report provides a succinct analysis of item difficulty,
discrimination, and distractor performance. Or, you can view complete
classical item analysis results for each item in your test.
- An item's response weights may have any value, positive or negative,
making Lertap one of the most flexible test scoring systems available. And, of
course, Lertap provides scores
on numerous scales/subtests and score
summary statistics within a test, plus a score intercorrelation matrix.
- Items may have different numbers of alternatives, up to a maximum of
thirty (30). The alternatives themselves may be either letters, such as
{A,B,C,D,E}, {T,F}, {a,b,c,d,e}, or digits, such as {1,2,3,4,5}. Support for
this many item options means that extended multiple-choice questions, or
"EMQs", may be processed (EMQ items typically employ from 15 to 26
options). Note: prior to the release of version 5.4.6, the maximum number of
item options was ten.
- Lertap can produce an Excel chart
of item response frequencies. Other Excel chart functions are used to make
histograms
and scatterplots
- Correction for chance, or "formula scoring", is supported. An external
criterion measure may be used in Lertap's item analyses, and, when requested,
full matrices of item intercorrelations and item weights may be produced for
subsequent analyses by other packages, such as MicroFACT. Item
correlations may be tetrachorics.
Mastery testing
Lertap's support for cognitive tests includes criterion-referenced
and mastery test analysis. Its default mastery cut-off score of 70% is
easily reset by users. In terms of statistical reporting, in the mastery case,
Lertap:
- Computes Brennan's generalized index of item discrimination.
- Produces the statistics associated with a Brennan-Kane variance components
analysis, including the index of dependability (in some cases referred to as
the "generalizability coefficient").
- Derives an index of classification consistency using the method advocated
by Peng and Subkoviak.
- Outputs the value of Cohen's kappa coefficient, another indicator of
classification success.
- Computes two conditional standard error of measurement estimates using procedures based on the binomial error model.
A special document for users of mastery, licensing, and certification tests is available at: http://www.lertap.curtin.edu.au/Documentation/JERM2007d.doc
Survey analysis
Lertap analyzes results from affective instruments, such as surveys and
questionnaires, including Likert and semantic-differential questions.
- Lertap produces two statistical reports for each affective subtest, one
with detailed results, another
with a single-line summary of item performance, made for easy reading.
- Lertap's affective item statistics include the mean, standard deviation,
and correlation with either an internal or an external criterion, corrected
for part-whole inflation when needed. An option in Lertap 5.4.6 and above will adjust
item's N's so that records with missing data are excluded from calculations.
- An affective item's responses may have any weight. Item weights are easily
and quickly reversed.
- A full item intercorrelation matrix may be produced for any affective
subtest, ready for further processing by other statistical packages, such as
MicroFACT.
- Lertap's affective
test statistics includes measures of central tendency (mean and median),
measures of dispersion (standard deviation, population variance, and sample
variance), and measures of reliability (coefficient alpha). As is the case of cognitive
tests, summary bands of selected item statistics enable users to quickly see
which items are performing differently from the others. A
table is provided to show how a test's alpha value would change should an item
be removed from the test.
- Item
response charts, score
histograms, and bivariate
scatterplots may be obtained by clicking on appropriate icons on Lertap's
toolbar.
Quintile Plots Based on Total Score
Item responses may now be summarized for up to five groups of respondents
(prior to version 5.4, the limit was two). The groups may be defined on the
basis of scores on either an internal or external
criterion; the external criterion may be a categorical variable, such as gender,
ethnic group, or geographical region.
Two graphical summaries may be made of group responses. One plots the
performance of each item option by groups, as seen below:
This example traces the functioning of each item option over total score
groups, from "lower" to "upper". Groups such as those seen here would often be
referred to as quintiles – there are five of them, each with 20% of the total N.
There are six lines in the plot; five of them correspond to the item’s
options, while the sixth, "other," corresponds to those test takers who did not
respond to the item.
The keyed-correct answer for this item is "C" (the yellow line), as shown in
the legend under the X-axis. We can readily see that the proportion of test
takers able to identify the correct option steadily increases as we go from left
to right, that is, as total score on the test increases. In the lowest score
quintile, only about 10% got this item right, rising to about 75% in the upper
group.
We can also see that the distractors seemed to function well. In general, as
we go from left to right, from low score to high, the proportion of test takers
selecting each distractor tends to decrease.
It is easy to obtain the table corresponding to the graph. This is done by
right-clicking on the graph, selecting chart options, and then selecting "show
data table". Note: versions 5.4.6 and above have an option which, when
selected, automatically sees that this table is attached to the graph. The table
displays as seen below:
Quintile Plots Based on Item Response
Lertap swaps the axes on the total score quintile graphs, resulting in a
plot such as the one shown below:
In this graph we’ve got the five quintile groups traced over the item’s
options, providing another way of capturing the action. Now it’s possible to
quickly detect the voting patterns of each group – the strongest group, "upper"
(the dark blue line), selected the correct option more frequently than the
distractors, while in the lowest group (the dark red line) the most popular
option was distractor A, followed by E.
Quintile Plots Based on External Variables
In the graph below, two groups have been defined on the basis of an "external
criterion," in this case a categorical variable indicating whether or not test
takers had had prior coaching. EC=2 (the blue line) corresponds to the coached
group, while EC=1 represents the non-coached students (the red line).
The coached students (the blue line) did better on the item – the correct
option was 1, and about 35% of them identified it, compared to about 20% in the
other group.
What is perhaps more noteworthy, however, is the difference in the groups’
propensity to leave item A49mc unanswered.
Omitted items are denoted as "other." The graph shows that almost 40% of the
non-coached group omitted the item (or failed to reach it), compared to only
about 8% of the coached students. Of itself this may not be a telling result,
but the pattern seen here was repeated on the twenty items which followed this
one. This was a speeded aptitude test, with no penalty for guessing. The plots
suggested that coached students seemed to have an advantage as they left far
fewer items unanswered, increasing their chances of obtaining a higher test
score.
How much work is required to get these plots? Extremely little. If the
criterion used to define the groups is internal, that is, the test score itself,
the graphs result from a single mouse click on a toolbar icon. Just one click
gets Lertap to plot results for every item – there are no dialog boxes to
complete. What about swapping the axes – surely users have to do something to
have Lertap invert the axes? Yes, true: they have to click on the same toolbar
icon again. That’s all; that’s it.
What? Lertap plots results for all items? That would surely take a fair
amount of time. Is 24 seconds for seventy plots a long time? (That’s what it
took on a late-model Pentium 4 / 2 GHz computer manufactured in 2002.)
The type of graph made on the first mouse click is a user option set in
Lertap’s System worksheet. The default gets Lertap to make a classic quintile
plot, with groups along the X-axis, but this is easy to change. If the groups
are defined by an external criterion, Lertap’s Run Menu is used to define the
external criterion, after which a single click on the same toolbar icon produces
the plots.
The Plot Thickens
Experienced Excel users may recognize what Lertap does to get quintile plots:
it simply passes a data table to Excel, and gets Excel to insert the right type
of chart. Excel has powerful graphing capabilities, and Lertap leans on them
often.
In version 5.4, we increased access to Excel’s plotting capabilities, adding
a new drop-down menu to the toolbar called "Shorts". One of the shortcuts
available from this menu is a simple line plotter.
Have a look at part of a normal Lertap item scores matrix, one that’s been
augmented with eigenvalues (better known to some as "latent roots"):
We’ve selected the row with eigenvalues, Row 370 above. To get a "scree
test", a plot of the eigenvalues, we go to the new Shorts menu, and click on
"Make a line graph". This is what we get:
Granted, this plot is too condensed as it appears here. Can it be enlarged?
But of course. This is standard GUI (graphical user interface) stuff: click on
the graph, and tug at the corners.
The point we want to make: Here we have another example of how easy it is to
plot things. In this case, three mouse clicks were required to get the plot (not
bad, especially when you think of those people who require a whole lifetime to
get the plot).
Lertap’s "Shorts" shortcut lets users make line graphs from a variety of its
reports. For example, columns in the brief item stats report can be highlighted,
and plots quickly made of item difficulties and discriminations.
These graphs can be truly handy. You might not think that a plot of item
difficulties or point-biserial correlations would be too exciting, but you ought
to try it: a graph of such statistics provides a colorful, information-rich
resource, capable of quickly revealing trends, or problems.
This discussion will undoubtedly remind many readers of the old adage: A
graph is worth a thousand tables. What you probably do not realize is that
this proverb was spawned by our Lertap-using forbearers.
Response Similarity Analysis (Checking For Cheaters)
RSA is new in version 5.5 and above. Users may direct Lertap to create a data file for input to Wesolowsky's
(2000) SCheck program, a rigorous routine designed to ferret out students
whose responses are statistically "excessively similar", or users may
get Lertap to develop reports which involve its own similarity measures, ones
based on the work of Harpp,
Hogan, and Jennings (1996).
Screen shots from Lertap's RSA reports:
To read more about Lertap and response frequency analysis just click
here.
New Statistics and Functions
New statistics introduced in version 5.4 include tetrachoric correlation
coefficients and SMCs (squared multiple correlation coefficients). Lertap 5.4
provided support for extracting eigenvalues from its inter-item correlation
matrix by using a free third-party Excel add-in available over the Internet. In
version 5.4.6, the eigenvalue reduction routines have been built into Lertap,
making it unnecessary to download the add-in.
Lertap 5.4 also added a new analysis option, "To halve and hold", working
from the Run Menu. It will randomly divide a data set into halves, useful for
calibration and validation efforts.
We’ve now provided support for users who want to feed Lertap data into two
popular IRT programs, XCALIBRE and BILOG-MG.
The screen snapshots below provide an idea of what Lertap 5.4 does; the first
shot is a worksheet ready to use XCALIBRE, while the
second is one destined for BILOG-MG:
To cater to BILOG-MG users wanting to carry multiple group-definition
variables in their analyses, Lertap 5.4 has added an option to its Move Menu
which copies selected data columns, reformatting them if required so that
they’re BILOG-ready. It then updates the FORTRAN format statement (the first row
in the worksheet above).
Banded Results Tables
The first personal-computer version of Lertap, released in 1983, featured the
use of small tables with ten rows, or "bands", to quickly summarize selected
item statistics, such as item difficulty and discrimination. Version 5.4.6 has
two new banded results tables, one for item SMC values, and another for item
correlations with the first principal component (or first principal factor). An
example is shown below.
This example is from a cognitive test with 150 items. It "plots" results for
a selected 37-item subtest, indicating, in a handy at-a-glance manner, the
distribution of item SMCs, and item correlations with the first principal
component. The tables readily reveal that 24 of the 37 items had correlations of
.30 and above with the first principal component, with 9 of these 24 having an
SMC value at or above .30.
Furthermore
Lertap's online help system, "Lelp", has been improved, and continues to be
available in three formats: PDF, Windows Help, and HTML. Improvements include
comments on how to export Lertap results to the FastTEST 2.0 Test Development System.
More information may be found at http://www.lertap.curtin.edu.au/
The Lertap Control Language
A special free-field job control language was developed in 1973 for a former
large-computer version of Lertap, Lertap 2. This language has now been adapted
for use within Excel, and provides the means for setting up a Lertap 5 test
analysis.
The language's syntax is simple, but powerful. For example, the following two
lines set up a classical item analysis for a chemistry test having ten
items:
*columns (c2-c11)
*key DAABC
CBBBD
In this case, the user has recorded item responses in columns two through
eleven of an Excel worksheet. The *key line gives the keyed-correct answer for
each item. If this were a mastery test, with 80% set as the mastery level,
appropriate control lines would be:
*columns (c2-c11)
*subtest
mastery=80
*key DAABC CBBBD
The three lines below set up a Lertap analysis for a Likert-style survey
having ten items, with every other item to be reverse-scored:
*columns (c2-c11)
*subtest name=(Milk
survey), affective
*pol +-+-+ -+-+-
The lines following indicate how cognitive and affective tests may be
processed simultaneously (Lertap will compute the correlation between the two
tests as part of its standard output; if desired, the "ProtKnow" score could be
used as an external criterion for the "MilkSur" items):
*columns (c2-c11)
*subtest name=(Protein
knowledge), title=(ProtKnow)
*key DAABC CBBBD
*columns
(c12-c21)
*subtest affective, name=(Milk survey), title=(MilkSur)
*pol
+-+-+ -+-+-
System Requirements
Lertap 5 will work with any computer capable of running English-language
versions of Excel 97 and upward. It operates under English-language versions
of Windows 95/98/NT/ME/2000/XP/VIsta/7 for all versions of Excel and Macintosh OS 8/9/10 (X) for versions of Excel through Excel 2004. (Excel 2008 for Macintosh cannot run Lertap.)
To determine whether Lertap 5 will function properly under any other versions
of Excel or Windows, we recommend that you download and install the 30-day trial
copy.
Technical Resources
- Lertap 5 is the product of over 30 years of development by Professor Larry R.
Nelson and colleagues. Dr. Nelson is at the Curtin University of Technology in
Australia. Curtin supports Lertap users via email, and has a
comprehensive Lertap website (http://www.lertap.curtin.edu.au/).
The site includes many examples of the use of Lertap, plus a "Known Problems"
page that addresses commonly asked questions regarding system implementation on
a variety of computer configurations.
- The Lertap website has numerous examples based on applications of Lertap
in
authentic test development situations:
- Lertap in action in test validity and reliability studies:
- Scree tests, factor analysis, and coefficient alpha:
- For users of mastery, licensing, and certification tests:
- Lertap's general documentation section:
Lertap 5 Manual
- Lertap's 235-page manual is provided as a Microsoft Word 97 file with both
the commercial and demo/student versions.
- The Lertap manual is also available for download:
- A companion to the manual, an interactive help file known as "Lelp", is
available at:
Student and 30-Day Trial Copy Downloads
- Demo/Student Version
- A limited Demo/Student version is available. This version implements all
the functions of the commercial version, but is limited to the analysis of 40 examinees. Note that this is different from the Student License of the full Lertap, which is in the purchase menu below.
These are available for both PC and Mac, and may be obtained at:
http://lertap.curtin.edu.au/Software.htm
30-Day Trial Copy for use with Windows Excel 2002 and 2003
- A fully-functioning 30-day trial copy is available for users of Windows Excel 2002 and 2003. If you decide to purchase a Lertap license, we can unlock your copy by email.
- After downloading the file, unzip it and then double-click on SETUP.EXE.
- If you do not have a program to unzip files, PC users can obtain one at
www.winzip.com.
- WARNING - Lertap may not work
properly unless it is installed by the installation program (SETUP.EXE)
that is supplied as part of the download.
- Download the 30-day trial copy for use with Windows Excel 2002 & 2003 (15.1 MB).
Note 1: please check these notes on Excel 2003 and 2007 if you’re not sure which Windows version of Excel you have.
Note 2: this version will not work with Calc, the OpenOffice spreadsheet program.
30-Day Trial Copy for use with Windows Excel 2007 or Excel 2010
- A fully-functioning 30-day trial copy is available for users of Windows Excel 2007 or Excel 2010. If you decide to purchase a Lertap license, we can unlock your copy by email.
- After downloading the file, unzip it and then double-click on SETUP.EXE.
- If you do not have a program to unzip files, PC users can obtain one at
www.winzip.com.
- WARNING - Lertap may not work
properly unless it is installed by the installation program (SETUP.EXE)
that is supplied as part of the download.
- Download the 30-day trial copy for use with Windows Excel 2007 or Excel 2010 (11.2 MB).
Note 1: please check these notes on Excel 2003 and 2007 if you’re not sure which Windows version of Excel you have.
Note 2: this version will not work with Calc, the OpenOffice spreadsheet program.
30-Day Trial Copy for use with Macintosh Excel X and Excel 2004
Please write directly to the author ( larry@lertap.com ) to obtain a copy of the 30-day version of Lertap for use with Macintosh versions of Excel (Excel versions X and 2004; will not work with Excel 2008).
Purchasing Lertap: Download or CD
You have the option of having Lertap shipped to you on a CD, or to download from the link above to save shipping costs. When you check out using our Shopping Cart, shipping cost will be estimated. If you wish to download the software, please note this in the "Comments" field. Your invoice will be adjusted by our sales associates, and you will not be charged for shipping.
Books on Test Construction and Item Analysis
|
Order Options and Details
|
 |
| |
|
Upgrades
|
|
|
Lertap for Excel 2003 (Windows) to Lertap for Excel 2007 (Windows)
|
| |
Products
Your Account
Your cart
