Integrating Theory, Research, and Practice: A Viewpoint from a Member of the Editorial Board of the Journal of Career Assessment
The Hebrew University of Jerusalem
This chapter appears in J. P. Sampson, E. Bullock-Yowell, V. C. Dozier, D. S. Osborn, & J. G. Lenz (Eds.), Integrating theory, research, and practice in vocational psychology: Current status and future directions (pp. 141-147). Tallahassee, FL: Florida State University. doi:10.17125/svp2016.ch14
The chapter is based on a paper presented at the conference of the Society of Vocational Psychology, May 16–17, 2016, Florida State University, Tallahassee, FL. A video presentation of a preliminary version of this paper may be retrieved from https://www.youtube.com/watch?v=haL6ccTHo20
Appreciation is expressed to Naomi Goldblum, Shachar Hechtlinger, Nimrod Levin, Yuliya Lipshits-Braziler, Zehava Masuri, Jim Sampson, Dana Vertsberger, and W. Bruce Walsh, the editor of the Journal of Career Assessment, for their comments on an earlier draft of this chapter.
Correspondence should be addressed to Itamar Gati, Department of Psychology, The Hebrew University, Jerusalem 91905, Israel. E-mail: Itamar.firstname.lastname@example.org
Keywords: career assessment, Information and Communication Technology (ICT), multidimensional career assessment, computerized assessment, interpretation of assessments, incremental validity, career interventions, effectiveness of interventions
The Journal of Career Assessment (established in 1993) is devoted to research on career issues, with particular attention to the assessment of individuals' characteristics relevant to career counseling and vocational behavior. According to the journal’s homepage, it
provides methodologically sound, empirically based studies focusing on the process and techniques by which counselors gain understanding of the individual faced with making informed career decisions. The journal covers the various techniques, tests, inventories, rating scales, interview schedules, surveys, and direct observational methods used in scientifically based practice to provide an improved understanding of career decision-making. (http://jca.sagepub.com)
Compared to the Journal of Career Development and the Career Development Quarterly (see Duffy, 2017; Flores, 2017), which are more practice-oriented, the Journal of Career Assessment focuses on the development of theories and the publication of research, like the Journal of Vocational Behavior and the Journal of Employment Counseling (see Furbish & Smith, 2017; Schultheiss, 2017). In the spirit of the Journal of Career Assessment, this chapter will focus on the challenges of integrating theory, research, and practice, and particularly issues of career assessment. I will explore how the integration of theory, research, and practice can be facilitated by Information and Communication-based Technology (ICT; see also a discussion of challenges in the translation of research into practical applications; Schultheiss, 2017). Detailed information regarding the fundamentals of research planning and execution can be found in Duffy’s (2017) concise section on the technicalities involved in writing papers.
The opinions expressed in this chapter derive from my experience as an editorial board member of the Journal of Career Assessment (JCA) for more than 10 years, as the author of 15 JCA articles, as well as my involvement as an author, editorial board member, or ad-hoc reviewer for the Journal of Vocational Behavior, Journal of Counseling Psychology, Career Development Quarterly, The Counseling Psychologist, British Journal of Guidance and Counselling, Journal of Career Development, and the International Journal of Educational and Vocational Guidance. Thus my views do not necessarily [End Page 141] represent those of Professor W. Bruce Walsh, the editor of JCA, or any other editorial board member.
The Interplay Among Theory, Research, and Practice
In their annual review for the Career Development Quarterly, Sampson et al. (2014) highlighted the need to facilitate the integration of career development theory, research, and practice. The goal of integrating these three factors is producing and disseminating professional knowledge relevant for researchers and practitioners, which is, of course, consistent with the editorial policy of the JCA. To achieve this goal, it is important to promote theories and research that can inform and elevate practice. While there are many ways to do so, the focus here is on a few core features. The criteria for high-quality theories include (a) testable (and refutable) predictions, (b) stimulating research, and (c) potential for practical implications. For research, they include (a) appropriate participants, (b) replicable and generalizable results, and (c) informative conclusions (with implications for both theory and practice). Finally, for high-quality practice, the criteria for quality include (a) client satisfaction, (b) transferable interventions, and (c) evidence-based practices.
Promoting relevant theories, research, and practice is often achieved by publishing journal articles. Focusing on the JCA, the papers in JCA represent all possible combinations of focused, combined, and comprehensive papers. The majority of the papers published in the JCA are research-focused. Only a few are purely theoretical (mostly in special issues) and seldom purely practice-oriented (e.g., descriptions of interventions). Papers combining theory and research are quite frequent, while papers combining theory and practice are rare. There are only a small number of comprehensive papers integrating theory, research, and practice—perhaps because such papers are always challenging to write, although they have a greater chance of being published.
Types of career-related assessments. Most studies published in the JCA involve career assessments directly or indirectly. Thus, it could be argued that JCA is oriented towards the integration of theory and research. I argue that career assessments in themselves can be considered practice-oriented as well. When assessments are developed in the field through collaboration with practitioners, they allow for testing the interactions among theories, research, and individuals seeking career counseling services. Finally, it is important to acknowledge that assessment too is a facet of practice, since any assessment is in fact an intervention, especially if the client receives feedback.
Career assessments can be unidimensional, focusing on the measurement of a single construct, such as the Occupational Self-Efficacy Scale (Rigotti et al., 2008), job satisfaction (Todorova, Bear, & Weingart, 2014), or the Vocational Identity Scale (Holland, Johnston, & Asama, 1993). The integrated responses of an individual to a unidimensional measure is represented by a single score. However, most career assessments are multidimensional and assign the individual a set of scores. There are three kinds of multidimensional assessments:
those with an informative total score (e.g., the Career Decision-Making Self Efficacy Scale [Taylor & Betz, 1983], the Career Thoughts Inventory [CTI; Sampson & Peterson, 1996; Sampson, Peterson, Lenz, Reardon, & Saunders, 1998], the Career Decision-making Difficulties Questionnaire [CDDQ; Gati, Krausz, & Osipow, 1996]),
those without a total score (e.g., the Self-Directed Search [SDS; Holland, 1997] , the NEO-Big 5 [Costa & McCrae, 1985]),
and those with an informative partial score that is an aggregate of only some of the scales (e.g., the Career Decision-Making Adaptability score in the Career Decision-Making Profile [Gati & Levin, 2014]).
Opportunities for Integrating Theory, Research, and Practice
Information and communication technology (ICT) has been used to administer and score career assessments, provide career information, and facilitate guidance for more than 40 years (Harris‐Bowlsbey, 2013; Sampson & Osborn, 2015). Nowadays, ICT is typically used to provide career assessments, career information, and interventions not only via computers and laptops, but also through tablets and smartphones, which allow access to career-related sites anytime and [End Page 142] anywhere. This section describes the contribution of ICT-based assessments and interventions in facilitating the interplay between theory, research, and practice. Specifically, I review studies that demonstrate the fruitful use of ICTs to facilitate the development and testing of theories, improve assessments and individualized feedback, promote quality research using innovative designs, and implement effective interventions.
The contribution of ICT to facilitating the integration of theory, research, and practice. Career counseling may be viewed as decision counseling, which aims at helping clients make better decisions (Gati, & Levin, 2015; Spokane & Oliver, 1983). The challenge is to evaluate how to use ICT effectively to achieve these goals in individualized career counseling as well as stand-alone interventions. As opposed to flexible, face-to-face individualized career counseling and guidance interventions, ICT-based career guidance and planning systems are highly structured. There are three types of ICT-based career tools: (a) computerized administration of assessments (either with scoring alone or with scoring followed by immediate personalized interpretive feedback [e.g., SDS, CDDQ]), (b) career information databases (e.g., Occupational Outlook Handbook, Bureau of Labor Statistics), and (c) interactive career guidance/planning systems (e.g., SIGI3, EUREKA).
Integrating theory and practice. Transforming a theory into structured ICT-based interventions is challenging, as it requires the meticulous operationalization of theoretical constructs. This promotes clarifying and elaborating the theory as well as designing a testable practice that is a consistent, structured treatment (unlike individualized counseling). For example, theory guides the way Holland's RIASEC typology-based vocational interest inventories are interpreted. Holland (1997) suggested that the three letter Holland code, based on the three types with the highest scores, best represents the individual's vocational personality. However, Gati and Blumberg (1991) proposed an alternative method of interpretation based on career counselors’ expert judgments. They found that the client's vocational personality should be represented by a flexible number of types, based on the individual's particular six-scale score profile (e.g., RI rather than RIA for a score profile of 12, 11, 3, 2, 1, 0, for RIASEC, respectively).
Integrating theory and research. One advantage of using ICT in research is access to actual clients—those who are deliberating and about to make a career-related decision and who choose to use a career planning system or career assessment on their own initiative (in contrast to those who participate in research for course credit or for payment as Mechanical Turk participants). Another advantage is that ICT-based systems allow monitoring the interaction and saving a record of it accurately and automatically (the dream of all researchers). Such monitored interactions can then be used to test theories. For instance, Gati and Tikotzki (1989) used such records to test the descriptive validity and the theoretical adequacy of the sequential elimination model for career decision making (Gati, 1986) by analyzing the sequence of pages visited by the system's users. The theoretical rationale of the sequential elimination model for prescreening was also empirically tested by Gati, Gadassi, and Shemesh (2006) in a six-year predictive validity study of the recommendations of Making Better Career Decisions (Gati, 1996), an interactive ICT-based career planning system (Gati & Asher, 2001).
Integrating research and practice. One of the potential interfaces between practice and research is evaluating the effectiveness of career interventions. Such evaluations should be tested with several criteria. These may include noting changes in the client’s career decision status (Gati, Kleiman, Saka, & Zakai, 2003), assessing any decrease in the client’s career decision-making difficulties (Gati, Ryzhik, & Vertsberger, 2013; Gati, Saka, & Krausz, 2001), evaluating the clients' perceptions of the benefit from the intervention (e.g., Gati et al., 2003), and evaluating the client’s satisfaction from the chosen field of study or career (Gati et al., 2006).
One frequent use of ICT in career assessments involves administration, scoring, and interpretation. When assessments are administered, research has shown that clients' responses to inventories and questionnaires using ICT-based assessment are equivalent to responses to paper-and-pencil assessments (e.g., SDS; Lumsden, Sampson, Reardon, Lenz, & Peterson, 2004; CDDQ, Gati & Saka, 2001; Kleiman & Gati, 2004). However, ICTs have a significant role in testing the validity and benefit of computerized interpretations of assessments. From the viewpoint of practice, the use of ICT in [End Page 143] assessment amplifies the importance of the question "How do we know that we can rely on the client's responses? Are they reliable and trustworthy?" In the case of assessing individuals’ career decision-making difficulties, these questions have led to the development of a systematic four-step procedure (Amir, Gati, & Kleiman, 2008) that was tested and validated with career counselors’ expert judgments:
ascertaining credibility (using validity items and the time required to fill out the questionnaire);
estimating the degree of differentiation among the ten CDDQ scales scores;
locating the salient, moderate, or negligible difficulties (based on the individual's absolute and relative scale scores);
and determining the need to add reservations to the feedback provided (based on doubtful credibility, partial differentiation, or low informativeness).
As Furbish and Smith (2017) noted, such an approach can demonstrate how the integration of theoretically-based interpretive procedures and empirical research can help improve interpretations.
The Challenges of Integrating Theory, Research, and Practice
While the above examples demonstrate the mutual benefits of integrating theory, research, and practice, it is also important to be aware of its challenges for career assessments. On the theory level, the challenges may include explicating the theoretical rationale of assessments, and promoting multidimensional assessments. On the research level, the challenges may include (a) testing the rationale and informativeness of the total score (over and above those of the scale scores), (b) using multiple approaches to assess the quality of measures, and (c) being meticulous about translated or adapted assessments. Lastly, on the practice level, interpretation and meaning are more complex in multidimensional assessments than in unidimensional ones, for both career counselors and clients; thus, effectively incorporating multidimensional assessments into the career counseling practice can be a challenge.
In addition, there are challenges involving the design of ICT-based systems, namely, how to elicit career counselors’ expert knowledge and transform it into computerized interpretations of assessments. Finally, there are challenges involving the effective integration of ICT-based systems and ICT-based assessments (with or without interpretations) into face-to-face career counseling.
Regardless of these challenges, there are unlimited opportunities for using career assessments to facilitate the integration of theory, research, and practice. Assessments can be used to test and refine theories, identify the critical components that contribute to the effectiveness of interventions, and improve practice by providing pertinent information about our clients. Incorporating ICT can help us integrate theory, research, and practice by increasing our access to various groups and appropriately modifying our assessments of their needs (e.g., students trying to choose a major, college seniors looking for a job, adults considering a career change, cross-cultural groups). These groups can then be compared to find their similarities and differences in the assessments used (e.g., Willner, Gati, & Guan, 2015). Also, experts’ knowledge can be translated into structured interventions (e.g., dealing with dysfunctional career thoughts found in the CTI responses [Sampson et al., 1998] and providing recommendations for dealing with specific career decision-making difficulties that emerged in the CDDQ [Amir, Gati, & Kleiman, 2008]). However, theory-based ICT interventions should always be tested for empirical validity as well as practical effectiveness (e.g., which types of difficulties can be reduced by what interventions to which individuals), and the conclusions can and should be used to upgrade interventions and refine the theory.
Recommendations for Journal Editorial Policy
Like the other remarks in this chapter, the following suggestions and recommendations do not necessarily reflect the views of the other JCA editorial board members. First, we should encourage refuting aspects of theories, not only confirm them. The utility of a good study that reports disconfirming results is much greater than that of an additional study supporting a theory with another group of participants having some different characteristics. Second, we should encourage [End Page 144] studies that compare and contrast theories (e.g., the cognitive information processing theory [Sampson, Reardon, Peterson, & Lenz, 2004] and the prescreening, in-depth exploration, and choice model [Gati & Asher, 2001]) in terms of their common and distinct features. For example, Rounds and Tracey’s meta-analysis (1996), which compared predictions derived from the hexagonal structure of Holland's (1997) RIASEC typology with those derived from the hierarchical-classificatory structure of interests proposed by Gati (1979, 1991), yielded unexpected results. Specifically, although Holland's hexagon fit the structure of vocational interests better in 73 USA samples, the data of 76 samples from 18 other countries fit a hierarchical structure better.
Second, with respect to research, in today's complex world of work, the design and use of multidimensional assessments should be encouraged. ICT makes it possible to design innovative research as well as potentially provide access to appropriate groups of participants. Furthermore, research that tests the incremental value of a new measure over and above existing measures should be preferred to studies that only report the associations between the new measure and some vocational or career-relevant variables. Finally, research that reports multiple studies supporting or disconfirming a new measure’s theoretical rationale; development; psychometric properties; and construct, concurrent, and (if available) incremental and predictive validity should be encouraged.
Third, with respect to practice, we should not be satisfied with a mere positive effect of the new intervention. Rather, it is important to compare the effects of the new intervention to those of previous ones and to test the incremental value of the new intervention over and above previous interventions. In addition, it is important to pinpoint the specific components of interventions that make the difference (e.g., Brown & Ryan Krane, 2000). Finally, journals should also encourage dialogue—discussions, comments, reactions, and rejoinders in response to published articles. Special issues can facilitate such fruitful interactions, as can conferences, such as the 2016 Society of Vocational Psychology conference at Florida State University.
Amir, T., Gati, I., & Kleiman, T. (2008). Understanding and interpreting career decision-making difficulties. Journal of Career Assessment, 16, 281–309. http://doi.org/10.1177/1069072708317367
Brown, S. D., & Ryan Krane, N. E. (2000). Four (or five) sessions and a cloud of dust: Old assumptions and new observations about career counseling. In R. W. Lent & S. D. Brown (Eds.), Handbook of counseling psychology (pp. 740–766). New York: Wiley.
Costa, P. T., & McCrae, R. R. (1985). The NEO personality inventory: Manual, form S and form R. Lutz, FL: Psychological Assessment Resources.
Duffy, R. D. (2017). Ensuring quality in theory, research, and practice: The Career Development Quarterly. In J. P. Sampson, E. Bullock-Yowell, V. C. Dozier, D. S. Osborn, & J. G. Lenz (Eds.), Integrating theory, research, and practice in vocational psychology: Current status and future directions (pp. 137). Tallahassee, FL: Florida State University. Retrieved from https://doi.org/10.17125/svp2016.ch13
Flores, L. Y. (2017). Journal of Career Development: Recommendations for future journal practices for integrating career theory, research, and practice. In J. P. Sampson, E. Bullock-Yowell, V. C. Dozier, D. S. Osborn, & J. G. Lenz (Eds.), Integrating theory, research, and practice in vocational psychology: Current status and future directions (pp. 148). Tallahassee, FL: Florida State University. Retrieved from https://doi.org/10.17125/svp2016.ch15
Furbish, D., & Smith, A. (2017). A view from the editor’s desk: Ensuring quality in theory, research, and practice. In J. P. Sampson, E. Bullock-Yowell, V. C. Dozier, D. S. Osborn, & J. G. Lenz (Eds.), Integrating theory, research, and practice in vocational psychology: Current status and future directions (pp. 155). Tallahassee, FL: Florida State University. Retrieved from https://doi.org/10.17125/svp2016.ch16
Gati, I. (1979). A hierarchical model for the structure of vocational interests. Journal of Vocational Behavior, 15, 90–106. http://dx.doi.org/10.1016/0001-8791(79)90021-6
Gati, I. (1986). Making career decisions: A sequential elimination approach. Journal of Counseling Psychology, 33, 408–417. http://doi.org/10.1037//0022-0184.108.40.2068
Gati, I. (1991). The structure of vocational interests. Psychological Bulletin, 109, 309–324. http://doi.org/10.1037/0033-2909.109.2.309
Gati, I. (1996). Computer-assisted career counseling: Challenges and prospects. In Savickas, M. L. and Walsh, B. W. (Eds.) Handbook of career counseling theory and practice (pp. 169–190). Palo Alto, CA: Davies-Black Publishing.
Gati, I., & Asher, I. (2001). Prescreening, In‐Depth Exploration, and Choice: From decision theory to career counseling practice. The Career Development Quarterly, 50, 140–157. http://doi.org/10.1002/j.2161-0045.2001.tb00979.x
Gati, I., & Blumberg, D. (1991). Computer versus counselor interpretation of interest inventories: The case of the self-directed search. Journal of Counseling Psychology, 38, 350–366. http://doi.org/10.1037//0022-0220.127.116.110
Gati, I., Gadassi, R., & Shemesh, N. (2006). The predictive validity of a computer-assisted career decision-making system: A six-year follow-up. Journal of Vocational Behavior, 68, 205–219. http://doi.org/10.1016/j.jvb.2005.08.002
Gati, I., Kleiman, T., Saka, N., & Zakai, A. (2003). Perceived benefits of using an internet-based interactive career planning system. Journal of Vocational Behavior, 62, 272–286. http://dx.doi.org/10.1016/S0001-8791(02)00049-0
Gati, I., Krausz, M., & Osipow, S. H. (1996). A taxonomy of difficulties in career decision making. Journal of Counseling Psychology, 43, 510–526. http://dx.doi.org/10.1037/0022-018.104.22.1680
Gati, I., & Levin, N. (2014). Counseling for career decision‐making difficulties: Measures and methods. The Career Development Quarterly, 62, 98–113. http://doi.org/10.1002/j.2161-0045.2014.00073.x
Gati, I., & Levin, N. (2015). Making better career decision. In Paul J. Hartung, Mark L. Savickas, and Bruce W. Walsh (Editors-in-Chief), Handbook of career intervention: Vol. 2. Applications (pp. 193–207). Washington, DC: American Psychological Association.
Gati, I., Ryzhik, T., & Vertsberger, D. (2013). Preparing young veterans for civilian life: The effects of a workshop on career decision-making difficulties and self-efficacy. Journal of Vocational Behavior, 83, 373–385. http://doi.org/10.1016/j.jvb.2013.06.001
Gati, I., & Saka, N. (2001). Internet-based versus paper-and-pencil assessment: Measuring career decision-making difficulties. Journal of Career Assessment, 9, 397–416. http://doi.org/10.1177/106907270100900406
Gati, I., Saka, N., & Krausz, M. (2001). “Should I use a computer-assisted career guidance system?” It depends on where your career decision-making difficulties lie. British Journal of Guidance and Counselling, 29, 301–321. http://doi.org/10.1080/03069880124945
Gati, I., & Tikotzki, Y. (1989). Strategies for collection and processing of occupational information in making career decisions. Journal of Counseling Psychology, 36(4), 430–439. http://doi.org/10.1037/0022-022.214.171.1240
Harris‐Bowlsbey, J. (2013). Computer‐Assisted Career Guidance Systems: A Part of NCDA History. The Career Development Quarterly, 61, 181–185. http://doi.org/10.1002/j.2161-0045.2013.00047.x
Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments. Odessa, FL: Psychological Assessment Resources.
Holland, J. L., Johnston, J. A., & Asama, N. F. (1993). The Vocational Identity Scale: A diagnostic and treatment tool. Journal of Career Assessment, 1, 1–12. http://doi.org/10.1177/106907279300100102
Kleiman, T., & Gati, I. (2004). Challenges of Internet-based assessment: Measuring career decision-making difficulties. Measurement and Evaluation in Counseling and Development, 37, 41–55.
Lumsden, J. A., Sampson Jr, J. P., Reardon, R. C., Lenz, J. G., & Peterson, G. W. (2004). A comparison study of the paper-and-pencil, personal computer, and Internet versions of Holland's Self-Directed Search. Measurement and Evaluation in Counseling and Development, 37, 85–94.
Rigotti, T., Schyns, B., & Mohr, G. (2008). A short version of the Occupational Self-Efficacy Scale: Structural and construct validity across five countries. Journal of Career Assessment, 16, 238–255. http://doi.org/10.1177/1069072707305763
Sampson, J. P., Hou, P-C., Kronholz, J., Dozier, C., McClain, M. C., Buzzetta, M., Pawley, E., Finklea, T., Peterson, G. P., Lenz, J. G., Reardon, R. C., Osborn, D. S., Hayden, S. C. W., Colvin, G. P., & Kennelly, E. L. (2014). Annual review: A content analysis of career development theory, research, and practice - 2013. The Career Development Quarterly, 62, 290–326. http://doi.org/10.1002/j.2161-0045.2014.00085.x
Sampson, J. P., & Osborn, D. S. (2015). Using information and communication technology in delivering career interventions. In Paul J. Hartung, Mark L. Savickas, and Bruce W. Walsh (Editors-in-Chief), Handbook of career intervention: Vol. 2. Applications (pp. 57–70). Washington, DC: American Psychological Association.
Sampson, J. P., & Peterson, G. W. (1996). Career Thoughts Inventory. Lutz, FL: Psychological Assessment Resources.
Sampson, J. P., Peterson, G. W., Lenz, J. G., Reardon, R. C., & Saunders, D. E. (1998). The design and use of a measure of dysfunctional career thoughts among adults, college students, and high school students: The Career Thoughts Inventory. Journal of Career Assessment, 6, 115–134. http://doi.org/10.1177/106907279800600201
Sampson, J. P., Jr., Reardon, R. C., Peterson, G. W., & Lenz, J. G. (2004). Career counseling and services: A cognitive information processing approach. Pacific Grove, CA: Brooks/Cole.
Schultheiss, D. E. (2017). Integrating theory, research, and practice in vocational psychology: Perspectives of a journal of vocational behavior editorial board member. In J. P. Sampson, E. Bullock-Yowell, V. C. Dozier, D. S. Osborn, & J. G. Lenz (Eds.), Integrating theory, research, and practice in vocational psychology: Current status and future directions (pp. 166). Tallahassee, FL: Florida State University. Retrieved from https://doi.org/10.17125/svp2016.ch17
Spokane, A. R., & Oliver, L. W. (1983). The outcomes of vocational interventions. In W. B. Walsh, & S. H. Osipow (Eds.), Handbook of vocational psychology (Vol. II, pp. 99–136). Hillsdale, NJ: Erlbaum.
Taylor, K. M., & Betz, N. E. (1983). Applications of self-efficacy theory to the understanding and treatment of career indecision. Journal of Vocational Behavior, 22, 63–81. http://doi.org/10.1016/0001-8791(83)90006-4
Todorova, G., Bear, J. B., & Weingart, L. R. (2014). Can conflict be energizing? A study of task conflict, positive emotions, and job satisfaction. Journal of Applied Psychology, 99, 451–467. http://doi.org/10.1037/a0035134
Willner, T., Gati, I., & Guan, Y. (2015). Career Decision-Making Profiles and Career Decision-Making Difficulties: A cross-cultural comparison among US, Israeli, and Chinese samples. Journal of Vocational Behavior, 88, 143–351. http://doi.org/10.1016/j.jvb.2015.03.007