Happy birthday to us! We’re 40 years old! To celebrate our 40th birthday, we thought we’d do something big. But before we cheers to the Big 4-0, we want to travel back to 1979. 

In the ‘70s, Dr. David J. Weiss and his team at the University of Minnesota were tasked to design and develop the first Computerized Adaptive Test (CAT) for the U.S. Department of Defense. Dr. Weiss, who we lovingly refer to as the Grandfather of Computerized Adaptive Testing, brought this innovation to life. CAT revolutionized testing by making it smarter, faster, and fairer. Ensuring tests are smart, fast, and fair continues to be our mission at Assessment Systems. 

40 years later, Dr. Weiss, Dr. Nathan Thompson, and the rest of us here at Assessment Systems continue to push the field of psychometrics forward. To celebrate the last 40 years, we want to share two big Assessment Systems updates. 

Have You Met Ada?

Ada is AI-Driven Assessment and the future of psychometrics. We’ve been working hard on Ada for months and we’re ready for you to check it out. 

Ada is built by assessment professionals for assessment professionals. Ada incorporates a number assessment necessities: Item Authoring, Item Banking, 50+ Item Types, and Examinee Management, to name a few. Plus some things you’ve never seen before like Automated Item Generation. Long story short, Ada has been designed specifically to guide you along best practices and sound psychometrics, enabling you to make strong, defensible, and engaging assessments.

Ada is currently in its most basic form; we’re in the process of building additional powerful features and microservices. Click here to get an early look and start using Ada.

40 Has Never Looked This Good

After 40 years, we’ve decided to update our visual identity! This includes a new logo, brand colors, fonts, and a new website. You’ll see these changes on our website, social sites like LinkedIn, and our products.

Assess logos through the years

Our new logo is an evolution of the logos that precede it; anchored with an interpretation of the IRT curve in orange, our new logo represents a more modern, simplified, and accessible visualization of who we are and what we do: we assess. As a data-driven company, we’re relentless in our approach of using data and assessment internally, and passionate about our products and consulting to empower our partners to assess with precision and ease.

We’re proud to release Ada and our new visual identity into the world. We’ve spent 40 years innovating and building stronger and more efficient assessment tools, and we’re looking forward to seeing what we will accomplish in the next 40 years. Thank you for being part of the Assess community. Whether you just read the blogs or you’ve been a partner for years, we’re grateful to celebrate our 40th birthday with you. 

We share a belief at Assessment Systems that we have one life – not a separate work life and a separate home life – one life that integrates and intersects with all of who we are. As a fully virtual company, home and work blend all day every day. Our community at Assessment Systems is home to single parents, parents with new babies, parents who sometimes feel more like a taxi driver for their kids – you get the idea. What I’m trying to say is, as a community, we’re a bunch of individuals with commitments outside of our work at Assessment Systems – and as the head of Human Resources, I love that.

When we look at our benefits at Assessment Systems, we’ve always tried to think outside of the box, as most progressive, people-focused companies do. Wellness and self-care have been something we’ve constantly tried to integrate, embrace, and elevate in our one life. What we’ve realized over time is wellness and self-care look different for each of us. For one of our employees, acupuncture is their go-to for wellness. For others, like myself, it’s a nice haircut. Because wellness looks different to each of us, and may look different month-to-month, we wanted to find a solution that would support that.

Progressive, people-focused companies put wellness front and center. At Assessment Systems, wellness and self-care have become a company wide conversation and something we’re always trying to improve. I stumbled upon Emburse while researching ways to improve our wellness initiatives. What immediately caught my attention was that it was solving the exact problem I was facing: empowering my employees to have access to wellness without being a burden on their time, wallet, or other commitments.

What is Emburse? you ask – great question! Simply put: it’s a credit card. But to me, the HR pro at Assessment Systems, it is much more; Emburse is the answer to how I can design a personalized wellness program for my incredibly hard-working employees.

You’re probably wondering why I don’t just add money to their payroll or issue company cards. Another great question, smarty pants! I chose not to do that because having an Emburse card at Assessment Systems has one purpose: treat yo’ self. No need to budget a paycheck, no need to worry about expense reporting… Just take your card and go take care of yourself.

For my fellow HR professionals and executive leaders reading this, please know there’s one thing Emburse couldn’t do for me or my team. Emburse couldn’t give us more time in our day for wellness. That’s something we, as a leadership team, had to enforce on our own. Because we live one life, and our work life and our home life are so intertwined, we strongly encourage our employees to use their Emburse card during the work day when their kids are at daycare of school and they can sneak away from their desk for a massage, a manicure, a haircut, or whatever wellness may be for them that month.

To learn more about Emburse, visit www.emburse.com.

Well, I made it! Twenty-one days away from my wonderful family. I do want to thank my family for being patient with me for being gone and my awesome wife for holding the fort down at home by herself with two boys and Tucker (Coon Hound). Don’t worry for those reading, I am going home with presents for everyone, including my wife! Although I am ready to go home and see my family, I will miss this view from the 29th floor of my apartment.
As the trip is winding down, things for Assessment Systems are heating up (no pun intended) here in the Middle East. I had a meeting today about some very interesting opportunities elsewhere in the region. This was the icing on top of the cake in terms of all of the things we accomplished on the trip.
I hope you enjoyed taking this ‘virtual trip’ with me! Don’t worry, there will be more trips, and who knows where Assessments Systems will end up in the search for Smarter, Faster, and Fairer assessments!
I am going into my second weekend in the Middle East now – time to take a little time for me! But, work and improvement (personal and professional) is never too far away. Have you ever wondered how we can get better as people or professionals without data? Well funny you say that, I did in the most peculiar of places. I rescheduled my drag racing experience at the Yas Marina Circuit from last weekend due to rain! When I finish my first of 10 runs, I could have been satisfied that I took a 500hp Camaro down the quarter mile in just over 13 seconds at 112 MPH. But once I looked at the data, the car is a 13 second car and I went 13.25 seconds on my first run, I was not too thrilled. Using data is key to who I am and who we are at Assessment Systems. Armed with that data, I went to work. I started to analyze my reaction time, RPMs at launch, and shift points. Suddenly, runs 2-7 started get faster. On run 8, I went 12.60 at 114.3 MPH. On run 9, I went 12.63 at 114.1 MPH. Now, I have found something to create and sustain my performance. This all occurred due to data. I could have easily been satisfied with making it down the track in one piece and going over 100 MPH, but the data showed me (as it did not lie) that there was more performance to be had. After I was finished, I found out that in ‘perfect’ conditions with a ‘professional’ driver, the car can do 12.55. Without the data, I would never have come close. I think I’ve found a new hobby!

 

All fun aside, data is key to everything we do at Assessment Systems. It is so ingrained into us, that it is a part of our everyday lives. I D.R.E.A.M. every day! (Data Rules Everything Around Me).

 

This trip is starting to wind down, but there is still more fun to be had!
This trip is going by quickly! I had a great week; I’ve been spending quality time with our key partner here in the Middle East. One of our core beliefs at Assessment Systems is that we don’t want bad tests to hurt good people. This week I assisted our key partner in running technical trials on their local test sites and on our system in order to make sure all candidates receive a world-class experience. I wish more testing organizations would take cues from our friends here in the Middle East; their attention to detail and process for rolling out their assessments is nothing short of world class!
As Thursday is the end of the work week here, I am getting ready to go into the weekend.  But the work does not stop! We have so many positive and exciting assessment updates coming up that I must power through and keep helping people create and deliver Smarter, Faster, and Fairer assessments!

Whether you are a newly-launched credentialing program or a mature certification body, it is important to perform frequent “checkups” on your assessments, to ensure that they’re not only accurate, but also legally defensible.  The primary component of this process is a psychometric performance report, which provides important statistics on the test like reliability, and item statistics like difficulty and discrimination.  This work is primarily done by a psychometrician, though particular items flagged for poor performance should be reviewed by Subject Matter Experts (SMEs).  However, checkups should also sometimes include Job Task Analysis studies (JTAs) and Cutscore studies. This is where your SMEs really come in.  The frequency depends on how quickly your field is evolving, but a cycle of 5 years is often recommended. JTAs are sometimes called job analysis, practice analysis, or role delineation studies.

Your SMEs play a pivotal role in getting new assessments off the ground and keeping existing assessments fair and accurate. Whether they keep your program abreast with current innovations and industry standards or help you quantify the knowledge and various skills measured in your assessment, your SMEs work side-by-side with your psychometric experts through the job task analysis and cutscore process to ensure fair and accurate decisions are made.

If your program or assessment is in its infant stages, you will need to perform a Job Task Analysis to kick things off. The JTA is all about surveying on-the-job tasks, creating a list of tasks, and then devising a blueprint of what knowledge, skills, and abilities (KSAs) are required for certification in a given role or field.

The Basics of Job Task Analysis

  • Observe— Typically the psychometrician (that’s us) shadows a representative sample of people who perform the job in question (chosen through Panel Composition) to observe and take notes. After the day(s) of observation, the SMEs sit down with the observer so that he or she may ask any clarifying questions. The goal is to avoid doing this during the observation so that the observer has an untainted view of the job.  Alternatively, your SMEs can observe job incumbents – which is often the case when the SMEs are supervisors.

  • Generate— The SMEs now have a corpus of information on what is involved with the job, and generate a list of tasks that describe the most important job-related components.  Not all job analysis uses tasks, but this is the most common approach in certification testing, hence you will often hear the term job task analysis as a general term.
  • Survey— Now that we have a list of tasks, we send a survey out to a larger group of SMEs and ask them to rate various features of each task. How important is the task? How often is it performed? What larger category of tasks does it fall into?

  • Analyze— Next, we crunch the data and quantitatively evaluate the SMEs’ subjective ratings to determine which of the tasks and categories are most important.

  • Review— As a non-SME, the psychometrician needs to take their findings back to the SME panel to review the recommendation and make sure it makes sense.

  • Report— We put together a comprehensive report that outlines what the most important tasks/categories are for the given job.  This in turn serves as the foundation for a test blueprint, because more important content deserves more weight on the test.  This connection is one of the fundamental links in the validity argument for an assessment.

Cutscore studies after job task analysis

When the JTA is completed, we have to determine who should pass the assessment, and who should fail. This is most often done using the modified Angoff process, where the SMEs conceptualize a minimally competent candidate (MCC) and then set pass/fail point so that the MCC would just barely pass.  There are other methods too, such as Bookmark or Contrasting Groups.

For newly-launching certification programs, these processes go hand-in-hand with item writing and review. The use of evidence-based practices in conducting the job task analysis, test design, writing items, and setting a cutscore serve as the basis for a good certification program.  Moreover, if you are seeking to achieve accreditation – a third part stamp of approval that your credential is high quality – documentation that you completed all these steps is required.

Performing these tasks with a trained psychometrician inherently checks a lot of boxes on the accreditation to-do list, which can position your organization well for the future. When it comes to accreditation— the psychometricians and measurement specialists at Assessment Systems have been around the block a time or two. We can walk you through the lengthy process of becoming accredited, or we can help you perform these tasks a la carte.

Well, it has been an amazing trip so far. I have been able to get a lot of work done and look forward to continuing to work towards accomplishing all of the goals I set out on this adventure. Yesterday, I went to the Yas Marina Circuit (Formula 1) and drag race a 500 hp Camaro. Now I know there are a few of you out there thinking, they are letting him do what!? He can hardly drive a rental car without incident. Well, those of you who think that got lucky. We got some light rain showers in the drag strip that shut that down. So, they decided to fix that by giving me an even faster car for the F1 circuit (Mercedes AMG GT C), since I can do that safer with a damp track. Then it started pouring rain in Abu Dhabi! Really!  Just when they were going to give me a $140k car and an open race track. I think they was a conspiracy theory concocted by my co-workers to keep me out of fast cars! The good thing is I rescheduled, so you will get to know how this story ends.
All fun aside, this is an amazing place! It reminds me that life is about more than just things. It is about doing the right thing everyday alongside ‘ride or die’ friends and colleagues and having the opportunity to have amazing experiences.
Hello everyone!  I am almost halfway through my trip.  So far, my trip has been very productive.  The last couple of days, I have been in Dubai, as opposed to Abu Dhabi.  If you have not been to this part of the world, even though the cities are in the same country and only about 100 miles apart, they are very much different from each other. On top of that, if you get a chance to drive between the two cities, hold on to your hat – it is definitely an experience. I had to channel my inner Cole Trickle (Days Of Thunder reference) just to make it in one piece!
All kidding aside, my meetings in Dubai were very productive. I love working with individuals who are intelligent and are willing and able to innovate. The status quo is boring. Our universe is expanding and improving at a staggering rate. Unlike our competitors, I want to be able to provide the Smartest, Fastest, and Fairest assessment we can to our partners. Just because an outdated business model has created billions of dollars of profits for some in the industry by staying in the past just to make money and not advance assessment, does not mean we need to do things that way. One of our core values is to Do The Right Thing by our people, planet, and our partners. I take that seriously. To do right by our partners, we should be assisting them in developing and delivering the smartest, fastest, and fairest assessment possible. And that is why I am in the Middle East for these three weeks; the amazing people here have an incredible appetite to do just that.
As I am writing this in 80+ degree Fahrenheit weather, my family back in Minnesota is getting pounded with snow. I am definitely not going to tell my wife how awesome the weather is here in Abu Dhabi right now. I think I will use Celsius to describe the weather as opposed to Fahrenheit!
When I went into the office of one of our partners here in the UAE, I found out some incredible news. They had just won a prestigious international award for the work they are doing in the assessment of speaking and learning Arabic. Like I said the other day, this is why I work. Knowing Assessment Systems had a small part in this partner doing the great work that they are performing is very satisfying.
I have been in the Abu Dhabi for almost a week.  Now it is time to have a little fun. On Saturday, I am going to go to the Yas Marina Circuit (Formula 1) and go drag race a 500hp Camaro! If you know me, doing this makes perfect sense. There is nothing like the smell of burnt rubber after smoking the tires in preparation for going fast! Look for some awesome pictures soon (as long as I don’t put the Camaro into the wall!)

On Sunday and Monday, I had the opportunity to attend the Assessment in Middle East conference hosted at the prestigious Abu Dhabi University. The conference brought together higher education professionals across the Middle East to discuss how the assessment of students and faculty can contribute to quality improvement through the use of data. It was fascinating to see talk with these professionals on how they use data to make themselves better.

Within our own business, we like to D.R.E.A.M. (insert Wu Tang Clan reference here… RIP ODB). D.R.E.A.M stands for Data rules everything around me, and at Assessment Systems it really does. It is refreshing to have been surrounded by professionals for two days who thrive off of continual improvement using data as opposed to doing things like they always had because ‘than is just how we do it here’. These are the types of professionals I love working with.

I have a lot of great takeaways from my time at the conference and look forward to sharing them with my team when I return.

Until next time!