It's been a few years (OK a few decades) since the C'reer team has taken the PSAT, SAT or ACT, but as deadlines for those tests loom (see below) we got to thinking of the best tips we received when we were prepping for the tests.Read More
We completed development of the career assessment that’s in C’reer a few months ago. In fact, it was probably the biggest undertaking in the first year of the company – to develop and prove a vocational assessment could be mobile friendly and scientifically accurate. With the app now in the market, we finally got some time to sit down with one of our lead researchers on the project – Dr. Jane Swanson – to reflect on the process and methodology that went into developing our assessment.
C’reer: What were your initial thoughts when we proposed the idea of developing a vocational assessment specifically for mobile?
JS: The underlying idea was intriguing, for a couple of reasons. First, there were the technological considerations. Other than the use of high-speed computers to expedite data processing, little has changed in measuring interests over the past 100 years. We had to consider how that longstanding approach would work in a mobile environment. Second, access to traditional career assessments is limited due to their control by mostly educational institutions, expense, and dependence on trained professionals to interpret the results. An assessment on a mobile platform changes that dynamic and expands the range of individuals that can take advantage of career assistance.
C’reer: You’ve worked on the Strong Interest Inventory in the past — how was the process in developing the assessment in C’reer different or similar?
JS: The work on the Strong Interest Inventory focused on gathering data from individuals in occupational criterion groups – the samples that are used to build scales measuring interest and similarity to people already in occupations. For C’reer, we started at the beginning, in terms of choosing, developing and evaluating items to use in the assessment. We also were working on a much shorter time interval for C’reer, from initial concept to viable product and targeting pre-career individuals as target and test groups.
C’reer: Were there any surprises or a-ha moments in this for you?
JS: We knew going into the design process that we could not utilize the large set of items that are typically found in normal career inventories. The C’reer assessment needed to be engaging, relatively quick to complete, and provide a smooth user experience. That meant showing one question at a time and balancing the right size rigor with the requirements of a mobile application. User feedback thus far indicates we’ve successfully achieved that.
C’reer: What do you say to the person who gets an assessment recommendation and they say, ‘that’s not me’?
JS: I’d want to know more about the person’s experience and in what context they took the assessment. What type of exposure have they had to career information or knowledge about the variety of careers available? In other words, is the assessment based on some real-world knowledge about the world of work? If the person does not have much hands-on experience, how might they go about obtaining this experience to augment or pursue additional information? Also, distraction can be factor when evaluating yourself for career potential – individuals should take care to respond quickly, going with their gut response, but not carelessly.
C’reer: What do you think the biggest surprise will be for the person taking our assessment inside C’reer?
JS: If they are used to traditional 100+ text based assessments, they will be surprised by how quickly they can engage with the questions and obtain results that can help them better understand their own career interests. They may have the same quick and relatively fun experience of taking a Buzzfeed quiz, but the outcome will be something far more useful and based on science!
C’reer: Was the process in testing the various stages of the C’reer assessment different because it was intended solely for mobile?
JS: Despite the mobile application, the development was still rooted in tried-and-true scientific methods. The same basic processes were followed in each stage of developing the assessment, including basing it on well-established theoretical constructs, writing high quality items using content experts, multiple pilot testing, choosing appropriate samples, refining items, and estimating reliability.
C’reer: Did you have to rethink how you asked specific questions due to space constraints?
JS: Yes, but that was the nature of the challenge. The items themselves needed to be short and to the point, and the number of items needed to be reduced, all to conform to the mobile platform. We also had to identify outdated questions in some of the older inventories and eliminate them.
C’reer: What do you think the biggest challenges our assessment will get from educators and your fellow researchers?
JS: The brevity of the assessment feature itself. Traditional psychometric methods build scales using considerable more items, which is a standard tenet of reliability – all things being equal, longer scales are more reliable. And reliability helps establish validity. In other words, larger assessments can be viewed as typically being more reliable. We put C’reer through multiple pilot tests and are confident that the length of the assessment in the C’reer app strikes the right balance between usability in a mobile platform and reliability.
How do you think we can make the C’reer assessment better or more predictive?
JS: Any assessment can be improved with additional analysis of data from new samples, which we have committed to doing over time with C’reer as more data are collected. Making it more predictive can best be accomplished by collecting data that links individuals’ assessment results to important and valued criteria such as satisfaction with the choices presented, and clarifying the purpose of the assessment
Vireo Labs hired the entire Alpha Omega Associates team of researchers to develop the vocational assessment that is running inside of C’reer. The team included:
- Jane L. Swanson, Ph.D., Professor Emeritus
- Joel T. Nadler, Ph.D., Associate Professor
- Meghan R. Lowery, Ph.D., Organizational Consultant
As you probably know by now, the idea behind C'reer is to make the career exploration process available to all students in this country, not just those who benefit from attending a wealthier high school with the best software and reasonable school counselor-to-student ratios.Read More
Much like like a teenager, a good mobile app goes through phases and changes.
But when your app targets teens as its user base, it's even that much imperative that you iterate and pivot (that's geek speak for making changes.)Read More
With nearly 1,500 people having downloaded and taken personality assessment we thought it would be a good time to take a look an early look at the trends and where are users are landing in terms of their personalities.Read More
As our CTO Wally Holt very eloquently put it in our first live demo of C’reer, we’re not trying to be the Yelp for careers and colleges. We’re trying to add more value than simply giving you results based on what is immediately nearby but rather we’re sourcing the best performing programs that just happens to be regional.Read More
Today’s high school counselors are doing much more than simply giving ‘guidance.’ That's why the American School Counselor Association (ASCA) and 2017 school counselor of the year Terri Tchorzynski are celebrating National School Counseling Week by telling everyone that the job title ‘guidance counselor’ is old-school, today is the day of the school counselor.Read More
Today’s teens are all about immediate gratification and communication. Whether’s it’s a tweet, snap or text, the way today’s teens communicate has become faster and most of it is occurring on their phones.Read More
Where should I go to school? It’s a question high-schoolers have been asking for decades. And an extremely important factor in coming up with an answer to that question is the actual physical location of a school.Read More
I had my first editorial published when I was 12. I’ve been writing ever since. I took that creative bent and passion for writing into an undergrad degree in journalism and a graduate degree in screenwriting.Read More
I always liked puzzles and problem-solving. When I laid my hands on an early personal computer, I knew that I had found something that I loved. That first computer was a TRS-80 that used a cassette tape player to store programs. That is probably an idea that most kids today can't even start to understand. It was a couple years later before I finally had the first computer that I owned. It was an Apple IIe that I still have today.Read More
As a kid, I wanted to grow up and be Joe Morgan. I couldn’t hit a curve ball so then I figured I’d be a dentist but this thing called organic chemistry ended that delusion. Turns out I am wired for business – specifically to be an entrepreneur, not so much an employeeRead More