Notes on Sort-and-Score and Youth Team Progress for the W.K. Kellogg Foundation
Summary of the YES Proposal
Notes: youth drop-out rates, skill dimensions, information aspects, ontology
Sketches for YES components
Outline for Deliverables, 3/31/08-9/30/08
References
Summary of the YES Proposal
Today, one in five youths in the US does not secure a high school diploma. Indications are that this trend is worsening.
In previous decades, the US manufacturing sector offered a route to decent jobs for workers with limited formal education. This is no longer the case.
Growth in health care and the services sector has compensated for some of the job loss associated with a declining manufacturing sector. But good fits are not the rule since (i) job requirements are heterogeneous on cognitive, emotional, and social skills, and (2) information on heterogeneous youths’ capabilities is difficult to credibly signal and costly to screen for. In this environment, even a high school diploma no longer offers assurance of a decent entry-level job and a good career path.
The New Options Initiative (NOI) at the Kellogg Foundation aims to address some of these problems. The first phase of NOI will generate expert-vetted testable prototypes that begin testing in the field starting Fall 2008.
NOI prototypes are expected to satisfy the following five requirements:
- They tackle the problem of high school drop-outs not through school reform but via creation of paths toward rewarding and productive careers. (NOI prototypes focus on managing opportunities and the environment facing drop-outs rather than mitigating/fixing the occurrence of the drop-out event itself).
- They draw on clear co-creative elements, directly engaging youth as well as employers for shaping and validation, and subsequent monitor-learning-inform-based feedback.
- Their design and roll-out plan is vetted and approved by relevant experts. Besides demonstrating good promise for success, prototypes must satisfy the experts on potential for scalability and security.
- They reach testability by October 2008.
- They identify clearly the value propositions for private sector acceptance.
Among the different avenues under exploration during this first phase of NOI is an information gathering-scoring-matching mechanism that aims to efficiently link out-of-school youth to entry-level openings in the labor market. The proposed youth employment system, YES hereafter, addresses both incentives and information asymmetries. YES can generate good fits that prove mutually beneficial, sustainable, and empowering to youth as well as employers.
YES showcases components that can be driven separately through design-test-refine cycles, which loose coupling provides a low-risk deployment. Moreover, even if the full system does not achieve broad rollout, the successful components can be independently scaled to realize value.
0.1 Components of YES
The YES proposal features five major components:
- A credential platform that generates a dynamic portable “passport” for youth ready to take productive jobs. Through a series of information-gathering steps, which will include public/verifiable data (such as credit scores, driver records, jobs, training school performance), on-line tests, and semi-structured privately provided data (such as references, relevant compelling histories, information is obtained that enables a passport to accurately and credibly communicates a young person’s multi-dimensional skills relevant to the labor market. Dimensions include cognitive skills, people skills, and emotional skills. The platform allows youth to edit and enter and submit relevant information. The platform creates an interactive forum whereby youth can understand their strengths and weaknesses relative to peers and to the current demands of the labor markets. The YES passport is akin to the high school diploma in terms of being universally understandable, but superior in attractiveness to youth both because of the user interface and its predictive potential for success in matching with desired jobs. In this regard the YES passport is the credential.
- Second is supportive handholds to the network of self-directed pathways for ongoing acquisition of tools and training and development which enable youth to succeed across a range of workplaces and jobs. In this regard the YES handholds are the training pathways.
- Third is a job postings platform that updates, integrates, and normalizes available positions. It accommodates employer preferences and granular wage-training offerings, that can be proportional It clears the market for youth employment in the resulting dynamic system. It is comprised of a passport and the ‘acquisitional’ network where the passport is accepted. The chief characteristic is that is leverages generative agency for NOI kids. In this regard the postings are the employers’ commitments.
- Fourth is a youth↔jobs matching algorithm that identifies mutually beneficial fits between individual capabilities and specific job demands. It incorporates important preferences and constraints of youth and employers. It supports user-friendly querying by matched youth and employers about rationales for the matches that builds trust and participation in YES. In this regard the matching algorithms are the market.
- Fifth is the monitor-learn-inform feedback loop. Not only does YES facilitate the two-sided youth labor market to reach satisfactory scale and to clear efficiently, it surfaces market-tested valuations on different skills. When these valuations are combined with a feedback loop on job-performance outcomes of operationalized matches, the YES system can generate clear bottom-line results to all stakeholders. The valuations and feedback information can also offer strong signals to youth regarding the promising training and development to undertake. This information can guide the creation and refinement of training and development programs for youth. In this regard the feedback loops are the governance.
In sum, the aim of YES is to develop a practical passport-network-market structure that substantially improves the job market outcomes of youth at risk. It is superior to a GED-type program in two key respects: it communicates multi-dimensional youth capacities with high accuracy, and it fits youth to varied job openings. It accounts for the positive value of a completed high school education without rendering out of school youth without a compensatory pathway of equal value. With a successful YES, youth are encouraged to invest and own their futures. The YES platform is thus both GENERATIVE AND INDICATIVE.
0.2 Roll out
The YES solution will be developed in three phases.
- In phase 1 spanning March-September 2008 (Summer+ 2008), a prototype will be designed and component mock-ups will be refined via focus groups and small-scale deployments to youth and selected employers. The phase will establish close partnership initially with two youth-training sites (such as Year Up in Boston or Youthworks in Santa Fe or Living Classrooms in Baltimore or Youth Radio in Oakland).
- In phase 2, spanning October 2008 to Summer 2009, the components will be connected and the system will be launched in two locations (most likely the initial partner-sites of phase 1). Testing and experimentation will be conducted in the field with youth, youth-trainers, and employers. Findings from the testing and experiments will determine the final design of a large-scale roll-out.
- In phase 3, starting October 2009, YES will be deployed on a large-scale with a monitor-learn-inform feedback loop in place. Strong reporting and robust performance to drive value and trust will be front and center.
The rest of this proposal is organized as follows. In section 1, YES is framed in the context of best current literature and market lessons. In section 2, schematic diagrams that put more details on YES are sketched. Section 3 lists key deliverables relevant for the first phase of YES (through September 2008).
Notes: youth drop-out rates, skill dimensions, information aspects, ontology
Notes in this section review information on high-school dropout rates in the US, and on understanding of the skills of high-school dropouts that are valued in the labor markets. The references find dimensions beyond the cognitive ones, specifically for people skills (such as listeningcommunicating and being a team player) and emotional skills (such as commitment, diligence, selflearning, self-motivation).
Additional notes make the case for YES based on the informational problems in the youth labor market. Notes on the two-sided aspect of the youth labor market raise the challenge of setting appropriate pricing and incentives for the YES market to achieve efficiency. Finally, notes from the ontology perspective for YES are listed.
1.1 High-school dropout rates
Heckman & LaFontaine (2007), correcting for biases in previous calculations, establish that (a) the true high school graduation rate is substantially lower than the official rate issued by the National Center for Educational Statistics; (b) the rate has been declining over the past 40 years; (c) majority/minority graduation rate differentials are substantial and have not converged over the past 35 years; and (d) the decline in high school graduation rates occurs among native populations and is not solely a consequence of increasing proportions of immigrants and minorities in American society. Their key figure is reproduced below.
Figure 1. US Graduation Rates For Birth Cohorts Across Data Sources
(Figure XII from Heckman & LaFontaine, 2007)
The trend of declining high school graduating rates over time is clear, regardless of the data source used for the computation. In the most recent cohort, a worrisome 23% of youth that could have graduated from high school did not. The labor market prospects for a high-school dropout in recent years have been very poor. The proposed Youth Employment System (YES) aims to fix the labor market breakdown for such youth (a veritable “market for lemons”) that arises from unfavorably costly screening by employers on one side and unaffordable signaling by youth on the other. YES creates a two-sided market that enables employers to presume a favorable result when hiring youth who lack a high school diploma. If YES succeeds, it also encourages higher investment by youth in communicable skills that will get rewarded over their careers which then establishes a virtuous circle.
1.2 Employer-relevant skills
Cognitive skills, measured by Armed Services Vocational Aptitude Battery (ASVAB), the Armed Forces Qualifications Test (AFQT), are widely regarded to predict success in lower-status jobs.
That identifiable non-cognitive skills are equally and possibly more valued in the labor market is established by:
- In Christopher Jencks (1979) pioneering work, controlling for standard human capital variable, he makes the case for industriousness and perseverance in explaining success in jobs.
- Bowles et al (2001), who provide a useful broad survey/summary. Surveys of employers routinely emphasize attitude as the most important factor affecting their selection of entry-level or less skilled positions.
- Heckman, Stixrud & Urzua (2006), who provide a state-of-the-art econometric analysis that finds a non-cognitive factor to matters as much if not more than the cognitive factor in explaining earnings. The same non-cognitive factor can also explain risky social behavior associated with criminality, alcohol/drug abuse, and teen pregnancy. There appear to be significant gender differences.
- Cunha & Heckman (2007) argue for public policy that recognizes multi-dimensional skill development with emphasis on a rich range of non-cognitive skills. Also see Howard Gardner on multiple intelligences.
Also, see:
- Skills USA PDP/Career Skills Education Program and Workforce Ready System
- SCANS Competencies (http://wdr.doleta.gov/SCANS/teaching/)
1.2.1 Examples of non-cognitive skills that affect labor market success
The degree of a person’s fatalism, measured for example by the Rotter test, is widely implicated in poor success in labor markets.
Similarly, propensity to behaviors that involve manipulating others for one’s own interest, “Machiavellian intelligence” measured by well specified tests for “Mach scores”, have been adversely linked to labor market success.
Particular personality dimensions of integrity and conscientiousness appear to be predictors of occupational success.
Osborne (2005) implicates measurable aggression for women with adverse labor market outcomes. In earlier work, Osborne finds adverse labor market success for lower status jobs for men who score high on “withdrawal”.
1.3 Existing internet solutions for the labor market don’t serve youth
Useful references on internet solutions for the labor market are:
- Moon (2007), Hadass (2004)
- Websites of Unicru, Kronos, American Staffing Association, Connectededu.net.
The internet recruiting markets to date focus on “skilled” jobs, i.e., not ones that are the scope of YES. The fee per month of job posting on the existing systems is $150-$400. The existing systems appear to mostly enable search with convenient broadcast of job openings. The match is mostly left to job seekers. So far, no measurable effect on macro unemployment has been observed from the advent of these sites (perhaps because skilled workers are already fully employed). The sites are best seen as lowering search costs (esp. by enabling cross-geographic reach relative to print postings).
1.4 Information incompleteness on youth capabilities
Useful summaries of the key informational problem for job markets are in Spence (2001) and Stiglitz (2001) and references therein. There is heterogeneity in skills across youth, and which skills the employer cannot easily observe but values. The employer does not incur the cost of screening (assuming that can be done) if the expected payoff is low, perhaps because the selected youth will leave. The youth may not be able to credibly signal skills without an authenticating party, and the market for authenticators may need scale and trust that may not be worthwhile for private investors to try and develop. Then, without a non-profit YES helping with the informational problem, the youth labor market can behave as if all youth have low skills (-- structurally this case is similar to that for used cars as discussed by Akerlof (1971) path-breaking discussion of the market for “lemons”). I.e., we have a severe adverse selection problem.
Note that it is reasonable to assume that the size of the lower skilled youth is a significant fraction and thus that any pooling equilibrium is quite inefficient. The outcomes gets worse if the higher skilled youth feel that they are unfairly not rewarded and drop out from those seeking jobs that can’t pay initially for actually valuable skills. And, youth under-invest in skills that they are unable to communicate and get compensated for. What YES then offers is a way for youth to credibly signal with low deadweight cost. If the YES “handholds” increase the quality of the signal while furthering investments in skills, we achieve even better outcomes. The possible payoffs from YES increase further if there is heterogeneity in youth-suitable jobs and YES enables better matching and thus employers to invest in match-appropriate training.
1.5 Two-sided market for youth labor
Rochet & Tirole (2004) define a market to be two-sided if the platforms can affect the volume of transactions by charging more to one side of the market and reducing the price paid by the other side by an equal amount; in other words, the price structure matters in a 2-sided market, and platforms must design it so as to efficiently bring both sides on board.
In the YES setting, the Coase theorem does not apply because of the significant real and psychological transaction costs among end-users (especially youth but also some for employers). Further, the problem of externalities bites. YES’s matching algorithm needs cumulated post-hiring outcomes data to be well tuned. Thus, employers need to be incented to contribute post-hiring outcomes. The employer’s price to use YES needs to reflect their past contribution of post-match information (though it is possible to not tie the two and create a separate reward/rebate for contributing post-match information). Because of privacy concerns, the platform likely needs to regulate interactions between end-users too.
Quite generally, YES must perform the balancing act between the two sides along multiple policy dimensions. It may often regulate the terms of the transactions between end-users, screen members in non-price related ways and monitor intra-side competition. Even if break-even is a requirement, YES can sensibly sacrifice profit by constraining one side in order to boost attractiveness for and recoup losses on the other side.
The Platform must be able to convey “probable acquisition of aspiration” in some meaningful way to both sides. Youth who have dropped out have little capacity to absorb additional blows to their self esteems that an ill-conceived interface could impose. OSY entering the system must be able to package their accomplishments and aspirations into a communicable message discernible by business.
The platform must also be able to convey to business they are likely to obtain new hires that meet their own specific criteria for ready-willing-able, and do so at reduced expense on the front as well as the back end.
1.6 Ontology of YES
In computer science, an ontology is a data model that represents a set of concepts within a domain and the relationships between those concepts. It is used to reason about the objects within that domain. Common components of ontologies in this sense include:
- Individuals: the basic or "ground level" objects
- Classes: sets, collections, or types of objects[1]
- Attributes: properties, features, characteristics, or parameters that objects can have and share
- Relations: ways that objects can be related to one another
- Function terms: complex structures formed from certain relations that can be used in place of an individual term in a statement
- Restrictions: formally stated descriptions of what must be true in order for some assertion to be accepted as input
- Rules: statements in the form of an if-then (antecedent-consequent)sentence that describe the logical inferences that can be drawn from an assertion in a particular form. More generally, these may be probabilistic.
- Axiom: assertions (including Rules) in a logical/statistical form that together comprise the overall theory that the ontology describes for its domain of application.
- Events: the changing of attributes or relations
We will flesh out the above features in the YES context and manage them.
Other features of YES:
- Many data sources.
- Security/privacy. (Compliance?)
- Evolving algorithms that are best expressed on meta-attributes
- Interactions by role (review, inputs, edit, query/search, monitor, assess, adapt)
- Constraints (region for job, wage schedule, …)
- 2(n?)-sided heterogeneous preferences
- Nature of firm: location, size, credential-building/reputation, …
Sketches for YES components
This section gathers sketches on YES components in order to facilitate discussion for specifying the prototype.
2.1 Youth platform
Enlarge this image
Enlarge this image
| Public Information Zones (candidate permission not required) |
Private Information Zones (candidate permission required) |
Assessment Information Zones (candidate participation required) |
Enhanced Information Zones (candidate participation required) |
| Criminal record Source: National Criminal Database |
Work ethic record Source: Self-reported 3rd-part verified (e.g. performance review, refs) |
Skills assesment Source: Basic Achievement Skills Inventory |
Creative Aptitude Source: MyNewOption.com |
| Civil Court record Source: Federal and County & Court Database |
Reliability record Source: Self-reported, 3rd-party verified ( e.g. skills training, volunteer) |
Aptitude assessment Source: General Ability Measure for Adults GAMA |
Communications Aptitude Source: MyNewOption.com |
| Driving record Source: Bureau of Motor Vehicles (State-by-state) |
Motivation record Source: Self-reported 3rd-party verified (e.g. skills training, voluteer) |
Personality assessment Source: Million Index of Personality Styles (MIPS) |
? Source: MyNewOption.com |
| Credit record Source: FICO& |
Interpersonal record Source: Self-reported, 3rd-party verified (e.g. performance review, refs) |
Interest assessment* Source: Interest Determination, Exploration and Assessment System |
? Source: MyNewOption.com |
| Drop out Circumstances Source: Self-reported, 3rd-party verified (e.g. references) |
*for sorting purposes | ||
| If Score = Accept ---------> If Score = Accept, ---------> If Score = Accept, Then Continue Then Continue Then Continue |
|||
2.2 Employer platform
2.3 Match algorithm
For possible model of assortative matching, see Becker (1991) on marriages. If strategic complementarity of skills is sufficiently strong between youth at jobs, then lack of assortative matching could induce poor equilibria with low levels of human capital investment – see model of Michael Kremer (1993).
2.4 Monitoring, learning and assessment
Feedback loops are critical for at least three reasons:
- Allows tuning of algorithms and platform structures in response to data on inputs and outputs. Also allows for dynamic relations.
- Allows evaluation of YES. Follow outcomes on job after match, measuring satisfaction of youth and of employers. For fundamentals of evaluation design, see Manski and Garfinkel (1992; especially Part II, The Design of Evaluations).
- Allows inference of key skills dimensions and their evolving market value. This information can inform decisions by both youth (what to invest in and how much) and youth trainers (what skills training to provide).
Outline
It is assumed that two training sites will be partnered for development of YES as soon as possible. (Possible partners include Year Up (Boston) or Youth Radio (Oakland), or Youthworks (Santa Fe)).
Deliverables are sketched for current phase by module below. Two asterisks [**] flag areas where other NOI groups may be actively engaged. The likely group is noted if possible
3.1 “Passport” Module
- List of skills that we need to measure
- Mechanism to measure skills [Group 3?]
- Includes tests known from literature to be associated with successful employment
- Includes recommender system
- Data sources and contracts to get data that measure skills (public, private)
- ** Identify types of youth job interests, constraints, preferences (provide information on career paths) [Group 2]
- Develop a SCORE
- From public data
- Combine the component tests
- Consider legal issues
- Platform to deploy mechanism and collect data (high security, check legal issues)
- Get youth feedback (play with it, focus groups)
- Involve training partners (ie. Year Up)
- Take feedback and lessons from partners and finalize prototype
3.2 **“Handholds” module
- Take information from 3.1.1 and identify ways in which those skills can be further developed or credentialized
- Identify explicit contacts or organizations in the training network to support 3.2.1
- Create website which makes 3.2.2. available (can be friendly to query and update)
- Validate website information (see 3.1.7)
- Take feedback from 3.2.4 and lessons and finalize prototype
3.3 **“Employers” module
- Have employer group sign off on 3.1.1. and perhaps expand list
- Data sources and contracts to get employer data (public firm info, job openings and characteristics)
- Identify employer interests, constraints, preferences
- Deploy platform to collect 3.3.2. and 3.3.3.
- Get employer feedback and revise platform to finalize prototype
3.4 Algorithms: Attributization and Matching
- Convert employer and youth data into latent scores for matching process
- Develop a candidate matching algorithm which clears the market, simulate characteristics and efficiency of algorithm
3.5 Other Areas
A. **Marketing [Group 4 and/or 5?]
- Explicit partnership with CBOs who serve youth to promote YES
- Explicit buy in from employers who will use system
- Attract youth
B. Incentives and behavior:
- Participation incentives (youth, employers, training-partners)
- Upgrading of skills & taking jobs for credentialization (not end in themselves, virtuous circle) [Group 2]
C. Miscellenea
- Ontology engine preparation: Use cases. Review of design. Focus groups (youth, employers).
- UI: Mock-ups. Focus groups (youth, employers).
- On harnessing process effects (Hawthorne, etc.)?
- Key behavioral considerations?
References
G. Akerlof, 1970. “The Market for `Lemons’: Quality Uncertainty and the Market Mechanism”, Quarterly
G. Becker, 1991. A Treatise on the Family, expanded edition (Cambridge, MA: Harvard University Press).
R. Benabou, J. Tirole, 2007. “Identity, Dignity and Taboos: Beliefs as Assets”, IZA Discussion Paper 2583.
S. Bowles, H. Gintis, M. Osborne, 2001. “The Determinants of Earnings: A Behavioral Approach”,
F. Cunha, J. Heckman, 2007. “The Technology of Skill Formation”, Institute for the Study of Labor, IZA Discussion Paper 2550.
Y. Hadass, 2004. “The Effect of Internet Recruiting on the Matching of Workers and Employers”, Harvard University, working paper.
R. Halpern, 2006. The Challenge of System Building in the After-School Field: Lessons from Experience.
J. Heckman, P. LaFontaine, 2007. “The American High School Graduation Rate: Trends and Levels”, NBER working paper 13670.
C. Jencks, 1979. Who gets ahead? (New York, NY: Basic Books).
J. Heckman, J. Stixrud & S. Urzua, 2006. “The Effects of Cognitive and Noncognitive Abilities on Labor
M. Kremer, 1993. "The O-Ring theory of economic development," Quarterly Journal of Economics, 108: 551-575.
C. Manski & I. Garfinkel, eds., 1992. Evaluating Welfare and Training Programs (Cambridge, MA: Harvard University Press).
M. Osborne-Groves, 2005. “How important is your personality? Labor market returns to personality for women in the US and UK,”
J. Rochet, J. Tirole, 2005. “Defining Two-Sided Markets”, RAND Journal of Economics, forthcoming.
A. M. Spence, 2001. “Signaling in Retrospect and the Informational Structure of Markets”, Nobel Prize Lecture.
J. E. Stiglitz, 2001. “Information and the Change in the Paradigm in Economics,” Nobel Prize Lecture.
W. Moon, 2007. “The Dynamics of Internet Recruiting”, Issues in Political Economy, v. 16, August.
We are seasoned community developers focused on achievable, measurable impacts.
